The Journal as Substrate: Unifying Deployment, Replication, Backup, and Offline Operation in Distributed Systems
Description
This paper is an analytic-theory contribution in the sense of Gregor's (2006) theory for analyzing (Type I): it identifies a fragmentation that the canonical distributed-systems literature has documented across four separate operational disciplines without recognising them as instances of a single structural condition, derives the property under which the fragmentation dissolves, and presents an instantiation as an existence proof that the alternative is realisable in production — not the design-science evaluation of an artefact (Hevner, March, Park, & Ram, 2004), which a comparative systems study would undertake separately.
For decades, deployment without downtime, cross-datacenter replication, backup, and offline operation have evolved as four parallel bodies of practice, each with its own vocabulary, apparatus, and cost model. This paper shows that this separation is contingent rather than structural. Under the substrate condition established in prior papers of this series — where the journal is code-as-data, dense, and composed of compact named operations rather than serialised states — the four disciplines reduce to a single primitive:
- Deployment becomes replay against the substrate up to the present head.
- Replication is the same replay at another site.
- Backup is the corpus held without replay.
- Offline operation is replay after a delay.
The canonical apparatuses developed for these disciplines — blue-green environments, change-data-capture pipelines, snapshot agents, and message-queue buffers — perform work that the substrate already provides by construction. Each arose independently to compensate for the absence of the substrate condition rather than to extend it. The construct introduced here names the substrate, formalises the four equivalences that follow from it, and characterises the operational regimes in which these equivalences can be observed.
Technical info (English)
Instantiation & reproduction. The four equivalences are exhibited by six laboratories (L1–L6) as existence proofs — L1 deploy=replay, L2 wire density, L3 in-proc symmetric consumer, L4 passive consumer/backup, L5 offline drain, L6 per-entry append-latency budget. They are idiomatic public-API console apps (.NET 9) built against the public runtime at commit 99e8202 (git checkout 99e8202c808c3d79ca7b1dfbdae0ffb7872c940e):
- Source code:
- Software Heritage:
swh:1:rev:99e8202c808c3d79ca7b1dfbdae0ffb7872c940e - Lab sources & harnesses:
Data package (paper05-data.zip). Bundles the six labs' source and their datasets (headlines + summary CSVs — the figures the paper reports). Lab 6's raw per-event sample log (samples.csv, ≈60 MB / ~500 000 rows) is omitted to keep the archive light and replaced by a note giving the lab source and runtime commit needed to regenerate it; L4/L5/L6 require Docker (MySQL / Azure SQL Edge). Inline file.cs:NN references in the paper resolve against the 99e8202 snapshot (per-file SWHIDs via ;path=…;lines=… on the directory SWHID swh:1:dir:c55af9b6466302ed402cb579440ceec870e449c8).
Scope note. This is a Type I analytic-theory paper: the labs are existence proofs, not a design-science evaluation — no comparative benchmarks against alternative frameworks are claimed.
Series information (English)
The Puppeteer Papers — Paper 5. A series developing a journal-as-program substrate for actor systems and its consequences. The working arc: (1) anti-porosity; (2) program–value separability; (3) reactions and the now/deferred partition; (4) preserving semantic continuity across actors; (5) substrate operations — this paper; (6) the infrastructural symptom; (7) after the substrate.
Paper 5 reframes four operational disciplines — zero-downtime deployment, cross-datacenter replication, backup, and offline operation — as instances of a single substrate primitive: replay against the journal. It presupposes the journal-as-program substrate established in Papers 1–2 and draws on the reactions and the now/deferred partition of Paper 3 for the symmetric-consumer property; the contribution is the unification of the four operations under the substrate condition, not any one operation in isolation.
Files
05-substrate-operations.md
Additional details
Software
- Repository URL
- https://github.com/alvaroNCubo/puppeteer-papers
- Programming language
- C#
- Development Status
- Active