Nightcore: Efficient and Scalable Serverless Computing for Latency-Sensitive, Interactive Microservices (Artifacts)

This is the artifacts of paper "Nightcore: Efficient and Scalable Serverless Computing for Latency-Sensitive, Interactive Microservices", which will appear in ASPLOS '21.

Abstract:

The microservice architecture is a popular software engineering approach for building flexible, large-scale online services. Serverless functions, or function as a service (FaaS), provide a simple programming model of stateless functions which are a natural substrate for implementing the stateless RPC handlers of microservices, as an alternative to containerized RPC servers. However, current serverless platforms have millisecond-scale runtime overheads, making them unable to meet the strict sub-millisecond latency targets required by existing interactive microservices.

We present Nightcore, a serverless function runtime with microsecond-scale overheads that provides strong, container-based isolation between functions. Nightcore's design carefully considers various factors having microsecond-scale overheads, including scheduling of function requests, communication primitives, threading models for I/O, and concurrent function executions. Nightcore currently supports serverless functions written in C/C++, Go, Node.js, and Python. Our evaluation shows that when running latency-sensitive interactive microservices, Nightcore achieves 1.36x–2.93x higher throughput and up to 69% reduction in tail latency.