Published December 26, 2025 | Version v2.0.0

Project VELOCITY: Overcoming the PCIe Bottleneck via Quad-Buffered Asynchronous Pipelining

Description

The VELOCITY architecture implements a Quad-Buffered Ring Pipeline that leverages the multi-channel nature of modern system RAM to eliminate "Stop-and-Go" latency during data ingestion. By implementing a 4-zone rotary buffer (Ingest $\rightarrow$ Tokenize $\rightarrow$ Pin/Feed $\rightarrow$ Writeback), the system ensures 100% saturation of the PCIe bus, effectively treating System RAM as a high-speed L4 cache for the GPU.

Phase 2: The "Infinite Model" Capability

To enable the training of models exceeding physical VRAM capacity, VELOCITY introduces a Layer-Wise Swapper ("The Teleporter") utilizing a sequential V-Cycle Architecture:

  • Downstream (Forward): Layers are streamed $0 \to N$ from System RAM to GPU. Activations are computed and offloaded back to System RAM.

  • Upstream (Backward): Layers are streamed $N \to 0$. Stored activations are retrieved, and gradients are computed and optimized in sequence.

Empirical Validation

We benchmarked the architecture on consumer hardware (NVIDIA RTX 4070 Ti, 128GB DDR5 RAM) using JAX with Pinned Memory optimization. The results confirm the effective removal of the I/O bottleneck:

  • Pipeline Saturation: The architecture sustained a synthetic data throughput of 37.5 Million tokens/second, proving that the Ring Buffer completely hides SSD and RAM latency.

  • Real-World Fine-Tuning (Simulation): In a simulation of a Llama-3 8B Decoder Layer (FP32 precision with Gradient Checkpointing), the system achieved a sustained training speed of $\approx$ 7,100 tokens/second.

  • Full 8B Backpropagation (Real Weights): We executed the full V-Cycle on the Meta Llama-3 8B model (32GB FP32 payload) on a 12GB GPU. The system successfully performed forward and backward passes for all 32 layers with an average step time of $\approx$ 101 seconds (Forward: 0.94s/layer, Backward: 1.95s/layer).

Implication: 70B Feasibility

Based on the linear scaling observed in the 8B benchmarks, we project the training time for a 70B Parameter Model (80 layers) on the same hardware to be $\approx$ 4 minutes per step. These results demonstrate that the bottleneck has been successfully shifted from I/O (Bandwidth) to Compute (FLOPs), confirming that full-parameter fine-tuning of State-of-the-Art models is feasible on consumer hardware without quantization.

Key Features

  • Ring Buffer Pipelining: Utilizes asynchronous Direct Memory Access (DMA) with Pinned Memory (Page-Locked RAM).

  • Quantization Multiplier: Integrates 4-bit (Int4) quantization logic for the transport layer to maximize bus data density.

  • Zero-Copy Loading: Eliminates redundant CPU memory moves during tensor creation.

Relation to Previous Work

This proposal extends the "Infinite Context" capabilities established in the UFCE Framework by addressing the specific constraints of model plasticity (training) rather than static retrieval.

Code Availability

The complete source code, JAX kernels, and reproduction scripts are available in the linked GitHub repository.

References:

[1] Killian, K. (2025). The UniField Coupling Equation (UFCE): A Universal Framework for Modeling Coupled Spatial-Temporal Dynamics. Zenodo. https://doi.org/10.5281/zenodo.18055873

Files

Project VELOCITY Overcoming the PCIe Bottleneck via Quad-Buffered Asynchronous Pipelining v2.pdf

Additional details

Related works

Is supplement to
Preprint: 10.5281/zenodo.18055873 (DOI)