Double backend design for sustainable and mockable data communication software

Computer science is a fast evolving field: software needs regular updates and maintenance, unless it becomes deprecated in a few years. Scientific software however, due to the inherent exploratory nature of research, is often developed over extended periods of time during which many new features are developed, tested and sometimes discarded for better solutions revealed by the gained experience. In order to develop software that isn't outdated before it is even used in production, careful software design and abstractions enabling easy updates and component replacements must be implemented. This is of paramount importance for "big science" experiments software that must be maintained for decades of facility operations, while the hardware infrastructure is regularly updated.
For data acquisition and processing systems, a single API double back-end design can be used: the API abstraction allows to encapsulate the "real" communication back-ends, enabling easy maintenance or replacement with on the shelf solutions, as well as the implementation of consistent "mock-up" back-ends that are indispensable for fast unit and integration testing. The mock-ups allow to simulate a complex exchange between a process and all its contacts without any contacts running, greatly reducing the cost (both run-time and required compute infrastructure) of running the tests. Finally, using simultaneously a "real" and a "mock" back-end allows to record the real back-end operations with the mock-up whenever desired ; the record can later be used to replay complex scenarios for debugging or testing.
Such a double back-end design has been implemented in a collection of libraries called phoenix, to abstract and mock-up a generic socket API. Simple mock-up and a ZeroMQ socket back-ends are provided in C++, Rust and Python, all using a compatible library to serialize messages. This basis enables the implementation of data processing pipelines that are multi-language, consistent, mockable and well encapsulated, therefore easily maintainable and extensible. The development of these tools are funded by the Horizon Europe project OSCARS with the goal of supporting diverse scientific use cases.