Cubed: Bounded-Memory Serverless Array Processing in Xarray

Cubed is an alternative framework for distributed processing of large arrays. Designed to respect memory constraints at all times, Cubed generalises the model of Rechunker to execute any numpy-like array operation. By using Zarr as persistent storage between steps, and Lithops as an abstraction layer, Cubed can run in a serverless fashion on a range of Cloud platforms. We will show how Cubed has been integrated with Xarray and demonstrate its performance on a tricky array workload.