streamrelay: WebSocket Relay for Real-Time Streaming from Batch HPC Executors

streamrelay is a domain-agnostic Python library for streaming incremental output from batch HPC jobs to remote clients in real time. It implements a dual-channel relay pattern: a producer (running inside the HPC job) connects outbound to a relay server and forwards output token by token; a consumer (the client) connects outbound to the same channel and receives output as it is generated. Neither side accepts inbound connections, making the pattern firewall-transparent.

While demonstrated for LLM token streaming via vLLM, the library is not LLM-specific and generalizes to any HPC job that produces incremental output, such as simulations, data pipelines, or scientific workflows.

PyPI: https://pypi.org/project/streamrelay/

Source code: https://github.com/uicacer/streamrelay