Solving the One and the Many with LSF, Symphony, GPFS, and RHEL AI: A Dynamic Compute Platform for NVIDIA AI Factories

The AI factory is one infrastructure serving many workloads, yet the workload management layer remains architecturally fragmented. Training and inference differ in temporal profile, resource consumption, failure semantics, and scaling model, yet current architectures force both into a single platform. This paper proposes a multi-ontology architecture in which IBM Spectrum LSF manages batch training workloads, IBM Spectrum Symphony makes available inference services, and IBM Storage Scale (GPFS) serves as the unified coordination substrate. LSF and Symphony descend from Songnian Zhou's 1987 doctoral research on dynamic load balancing, share the same dynamic compute platform and its Load Information Manager (LIM/ELIM) infrastructure, and both implement formally definable resource ontologies. LSF's batch compute ontology provides topology-aware GPU placement and distributed training coordination proven at Oak Ridge's Summit supercomputer. Symphony's service compute ontology provides 38 per-instance ELIM inference metrics, semantic routing, GPFS-based KV cache sharing across Granite or other model tiers and sub-millisecond metric freshness. GPFS connects the two compute domains through extended attributes and Information Lifecycle Management tiering, providing so much more than its typical designation as a distributed file system. RHEL AI and its vLLM inference runtime provide the model serving layer within the Symphony compute domain, with support for over 70 model architectures across NVIDIA, AMD and Intel accelerators. Five demonstrations on commodity hardware and across multiple IBM Cloud regions validate the architecture, including multi-model vLLM inference, neuromorphic routing at 622 microseconds, cross-model KV cache transfer with 8.2x latency improvement, 47-second model handoff and an Obfuscation-as-a-Service pipeline. Within the context of NVIDIA's acquisitions of Run:ai and SchedMD, the architecture provides a vendor-independent alternative grounded in four decades of production deployment at many of the largest organizations, enterprises, and research facilities in the world.