Impact of API Rate Limit on Reliability of Microservices-Based Architectures

Many API patterns and best practices have been developed around microservices-based architectures, such as Rate Limiting and Circuit Breaking, to increase quality properties such as reliability, availability, scalability, and performance. Even though estimates on such properties would be beneficial, especially during the early design of such architectures, the real impact of the patterns on these properties has not been rigorously studied yet. This paper focuses on API Rate Limit and its impact on reliability properties from the perspective of API clients. We present an analytical model that considers specific workload configurations and predefined rate limits and then accurately predicts the success and failure rates of the back-end services. The model also presents a method for adaptively fine-tuning rate limits. We performed two extensive data experiments to validate the model and measured Rate Limiting impacts, firstly on a private cloud to minimize latency and other biases, and secondly on the Google Cloud Platform to test our model in a realistic cloud environment. In both experiments, we observed a low percentage of prediction errors. Thus, we conclude that our model can provide distributed system engineers and architects with insights into an acceptable value for the rate limits to choose for a given workload. Very few works empirically studied the impact of Rate Limit or similar API-related patterns on reliability.