1000 Tenants, One Model, One MacBook: Attacker-Tested Isolation for Multi-Tenant LLM Inference
Description
One thousand concurrent tenants. One model. One MacBook. We serve a thousand tenants through a single Qwen 2.5 3B model on an Apple M1 Pro - no dedicated instance per tenant, no hardware enclave - using a per-tenant orthonormal rotation of attention K-vectors, in probe-time and decode-time variants, composed into a three-layer serving stack (eviction-policy and depth-gated corrective layers). We do not test isolation with retrieval scores alone; we test it with a direct adversary and report the boundary it reveals. Matched-key validation scales to 1000 tenants on the production shared-prefill pipeline: a dedicated matched-only run scored 1000/1000 PASS-CLEAN with bit-exact K-vector recovery on every tenant. Wrong-key isolation is validated from T=2 to T=1000, every mis-keyed tenant producing zero content recovery and no bleed to neighbours. The cross-tenant readout over 20,000 off-diagonal pairs at T=1000 matches the 1/sqrt(d) concentration scale, and a pre-registered behavioural spot-check of 40 high-cosine pairs - including the maximum-cosine pair - found zero content recovery. Against a co-tenant adversary without model weights or known-plaintext pairs, reconstruction sits at the random baseline and membership inference is at chance: the substrate provides content-recovery isolation in this regime. Against a known-plaintext attacker, its linear-orthogonal structure is recoverable by orthogonal Procrustes once roughly d (~128) plaintext-ciphertext pairs are observed - long established in the secure-kNN literature, reproduced here on transformer attention K-vectors. The honest conclusion: a validated, scalable tenant-isolation layer against the co-tenant threat model, complementary to at-rest encryption and a trusted-execution tier that closes the known-plaintext and operator channels. We do not claim cryptographic security against structure-aware adversaries.
Files
paper1-1000tenants-v1.pdf
Files
(215.8 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:56581c572c8e879e3c496899999e4fe2
|
215.8 kB | Preview Download |
Additional details
Related works
- Is supplemented by
- Software: 10.5281/zenodo.20729953 (DOI)