Closing the Compiler Loop: Toward a Self-Hosting, Dependency-Free Python JIT
Authors/Creators
Description
In June 2026 the CPython Steering Council paused new work on the experimental copy-
and-patch JIT and requested a Standards-Track PEP, explicitly inviting alternative proposals
and suggesting the design describe “a JIT infrastructure that can support multiple implemen-
tation strategies” rather than a single one. We accept that invitation. Our position is that the
decisive lever for high-performance dynamic-language execution is not any single tier or trick
but co-design across every layer of the stack at once—from the source dialect, through a whole-
program-optimizing source-to-C transpiler, down to bare-metal thread context switching and a
C language extended with call-graph contracts. Building on the minipy/ShivyCX pipeline, in
which the entire runtime is written in one restricted-Python dialect and compiled as a single
whole program, we report three results that together move the system decisively toward full self-
containment. First, the runtime’s front end —a fully dynamic PEG parser—compiles straight to
native code through the same pipeline that compiles the interpreter, verified byte-identical to
CPython across a four-way differential test. Second, the transpiler itself (py2c, ∼14,000 lines)
now compiles and runs on the interpreter it produces: minipy compiles all 648 of its functions,
the module fully initializes, and it begins transpiling small programs—closing a meta-circular
loop between the compiler and its runtime. Third, we replace the external assembler: rasm,
an x86-64 assembler written in the same dialect, encodes ShivyCX’s output byte-identically
to GNU as (100/100 instruction forms) and emits ELF objects that link and run correctly
end-to-end across a program battery (9/9). With a native front end, a self-hosting transpiler,
and an in-dialect assembler, the only external tool left in the self-hosted path is the system
linker—narrowing the distance to a fully self-hosted, dependency-free JIT to a single, named
component, and directly answering the two concerns raised against the current CPython JIT:
its build-time LLVM dependency and its uncertain long-term maintainability.
Files
neopyjit3.pdf
Files
(330.7 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:50c9b6cc13ccd00b7aa9ca8432b9b60a
|
306.1 kB | Preview Download |
|
md5:b4ef4c4e2b2a1aa58b106711671027bd
|
24.6 kB | Download |
Additional details
Software
- Repository URL
- https://github.com/brentharts/ShivyC
- Programming language
- Python