XFBench: A Cross-Cloud Benchmark Suite for Evaluating FaaS Workflow Platforms

Functions-as-a-Service (FaaS) is a widely used serverless computing abstraction that helps developers build applications using event-driven, stateless functions that execute on the cloud. Commercial FaaS platforms like AWS Lambda and Azure Functions offer elastic auto-scaling and invocation-level billing to ease operations. Applications are often composed as a dataflow of FaaS functions that are orchestrated by FaaS workflow platforms like AWS Step Functions or Azure Durable
Functions. However, the proprietary nature of FaaS platforms on public clouds means that their internals are less understood. While benchmarks to characterize FaaS platforms exist, none are available for a principled evaluation of FaaS workflow platforms. Further, they are less configurable, and often limited to simple workloads and a single cloud provider. We address this by proposing XFBench, an end-to-end automated benchmarking framework for FaaS workflows, and an accompanying function, workflow and workload suite. The user provides a generic definition of the workflow and workload for benchmarking, and XFBench automatically deploys the workflows across multiple cloud platforms, generates client requests, and profiles the execution. We evaluate XFBench with realistic workflows and workloads on AWS and Azure platforms in different global regions to understand inter-function communication, function execution time, and cold start scaling and offer unique insights.