LETHE: Combined Time-to-Live Caching and Load Balancing on the Network Data Plane

Abstract—Load balancers in distributed caching systems face a fundamental trade-off between networking and caching performance metrics. The first comprises how the network traffic of object requests and replies is balanced on the different links to/from the cache servers while the second denotes cache hit rates and response times. In a nutshell, the root of this trade-off lies in the combination of the skewed popularity and dynamic nature of incoming object requests as well as whether the load balancing function is agnostic to the caching application or not. In this paper, we present Lethe, a network data-plane load balancer for distributed Time-to-Live (TTL) caching. We consider TTL caching as it separates the object dynamics but still performs equivalently to many classical caching algorithms (e.g. LRU) under appropriate TTL parametrization. Lethe segregates the cache objects based on the pattern of the incoming requests and efficiently places objects in cache servers to balance the
network traffic without sacrificing the caching system hit rate. We implement Lethe in P4 and experimentally show that it improves the average response time and cache hit rate as compared to application-agnostic load balancing even for a skewed and dynamically changing workload.