Memory Archive: A Memory-Grounded Training Paradigm for Computer Use Agents

Memory Archive: A Memory-Grounded Training Paradigm for Computer Use Agents

This is a design specification and pre-registered research protocol, not an empirical results paper. The paradigm has not been trained end-to-end: no result in this document is empirical, every quantitative performance figure is a projection derived from principled reasoning and analogous published systems and is marked as such, and Section 5.3 pre-registers the minimum-cost experiment designed to falsify the central hypothesis, with its design, metrics, and decision rule fixed in advance.

The document addresses five structural limitations of standard Computer Use Agent (CUA) training, which pairs behavioural cloning with outcome-supervised RL: outcome sparsity, intent blindness, train-deploy format mismatch, no persistent task knowledge, and non-compositional generalisation. These are stated as design premises consistent with published failure analyses of GUI agents, not as measured results. Whether format mismatch, rather than grounding accuracy or long-horizon error compounding, is the binding constraint on CUA performance is precisely what the experimental protocol is designed to test.

The central thesis is Format Consistency. The system centres on a compiled task guide called memory.md, a structured document containing step-by-step procedural reasoning, execution commands, and visual state references. This single artifact threads through four stages of the agent lifecycle:

• Pre-Training (Format Internalization): The base model learns the grammar of GUI actuation events and step-level multimodal alignment.
• Supervised Fine-Tuning (SFT): The model is trained with cross-session memories in context, treating actuation artifacts (CommandEvent JSON) as first-class training targets alongside reasoning. A fraction of retrieved memory steps are corrupted, so the model must detect and override memory errors rather than transcribe them.
• Post-Training (Memory Adherence RL): Group Relative Policy Optimization driven by a three-component reward — Step Alignment, Visual Grounding, and Outcome Consistency — with per-step terms computed over a monotonic alignment and scored against pre-corruption originals on corrupted episodes. The Process Reward Model that contributes to Step Alignment is trained exclusively on human-annotated sessions and deployed frozen, breaking the circular dependency that would arise from critiquing VLM annotations with a critic trained on the same signal.
• Inference-Time Retrieval: A  two-stage stack (bi-encoder HNSW, then cross-encoder re-ranking) pulls relevant memories. The agent tracks execution deviation and compiles new memory.md files on task success, growing the library endogenously.

The paradigm also introduces in-training evaluation via self-generated memories, detecting overfitting, underfitting, and context-awareness without a static external benchmark. The load-bearing safeguard is executable replay verification: each candidate memory is replayed in a sandboxed OS and its terminal state checked by a programmatic task validator, so the acceptance verdict is a function of OS state rather than any learned scorer. No self-generated memory enters the training corpus without a passing replay.

The document provides full mathematical formulations, data construction specifications, algorithm details, hyperparameter guidance, failure-mode analysis, and an explicit compute-requirements model. The architectural contributions are stated as designs whose value is contingent on the pre-registered experiments, not independent of them.

Data availability: the Phase-1 corpus is publicly released at https://huggingface.co/datasets/NullVoider/Memory-Archive-Paradigm under CC BY-NC 4.0 — 101 fully annotated sessions over 36 task clusters (884 annotated steps), recorded on macOS (52 sessions) and Windows (49), each carrying the three-frame visual record per step, per-step actuation records, per-step reasoning annotations, and the compiled memory.md.