leavelet
e96cfa6cbd
Replaces §9.3's object-level rank-uniform eviction with per-rank, page-level LRU GC running ON THE
WRITE THREAD (the sole slot-pool owner -> no lock, no second owner). Genuinely communication-free:
each (hash,payload) is written/read/deleted by exactly its one owner (owner = i%cp_size, and the
prefix-chained content hash makes one-hash = one-position = one-owner), so the shared LMDB's own
consistency substitutes for collectives within L3 (design §9.5/§9.6). GC touches only L3, adds zero
collectives; the spill/reload MINs that remain are all the L2 bridge.
cp_l3_store.py:
- Per-payload lazy-deletion min-heap `_gc_heap` of (last_access, seq, hash) + authoritative
`_gc_current{hash->(last_access,slot)}`. Add on write, touch via `_touch_q`, reclaim coldest.
- The write loop, each iteration: drain touches -> _gc_collect (reclaim coldest of any pool over the
0.90 start watermark down to 0.85, bounded budget) BEFORE the next write, so spill never hits a full
pool (no evict burst on the critical path). Continuous + proactive, symmetric with the spill.
- Reclaim deletes the index entry (durable) BEFORE freeing the slot, so a racing reload reads None or a
CRC-mismatched reused slot -> miss -> recompute (fail-soft); no lock / in-flight-exclusion needed.
- submit_touch (scheduler->write thread); clear() resets the GC structures.
hiradix_cache.py: wire the reload `touch` (the 3.2 gap -- hit_count/last_access never updated): at
_cp_l3_admit_reload, bump the L3 last_access of THIS rank's owned reloaded pages (ALL pages, not just
the tail, so a prefix ages uniformly). _cp_l3_insert_reloaded_node now returns the node (its
last_access seeds the touch).
Sharing handled optimally for free (page = one entry, last_access = max over touchers; prefix pages
age together; shared-hot survives) -- no object-record table, no refcount, no content-keying. 56 L3
unit tests green (new: GC reclaims coldest-to-watermark + no slot leak; touch keeps a page warm).
Cold-rebuild (scan slab headers + LMDB) is designed (§9.6) but deferred to 3.4 (crash-recovery).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.