leaveletandClaude Opus 4.8
815da6c4e5
The first 3.1 (d08ee9f8d) spilled the COLDEST resident objects and held the
protect_host eviction-pin through the entire slow disk write (released only at the
durable ack). Since the coldest objects ARE eviction's victims, a B300 cachebench
wire test wedged: evict-to-fit found everything pinned (evicted=0) ->
host_reservation_failed flood -> caching broke.
Redesign (docs_internal/cp_hicache_l3_phase3_impl_design.md §R3):
- cp_l3_store: split the single spill thread into a GATHER -> WRITE pipeline. Gather
copies each owned page off the pinned slab into a slab-independent anonymous-mmap
staging buffer (fast RAM memcpy) and acks GATHER; a separate write thread does the
O_DIRECT write + fdatasync + durable LMDB put off the staged copy and acks DURABLE.
The eviction-pin releases at the GATHER ack, so the object is evictable right after
the RAM copy; the disk write completes off the staged copy even if L2 reuses the
pages. Slab-full frees its partial slot allocation (no orphan) + acks ok=False.
- hiradix: continuous PROACTIVE spill — enqueue each just-committed object to a FIFO
deque at the replicated commit frontier (_commit_pending_backup) AND at radix splits
(the freshly-committed parent half), drained in commit order (not coldest). The deque
is capped (rank-uniform) so a disk stall can't OOM it. Spilling HOT just-committed
objects (disjoint from eviction's COLD victims) means the cold tail is already
L3-durable when evicted -> eviction just drops it, never contends for the pin.
- 3-element CP-group MIN drain [gather, durable, reload]: gather-MIN -> release_host;
durable-MIN -> l3_durable, gated by an ok-AND (a second MIN over per-op ok bits) so a
write failure on any rank means NO rank marks it durable (rank-uniform durability,
placement_digest stays green).
opus-reviewed (FIX-THEN-SHIP) + independently re-verified; review fixes folded:
split-enqueue (HIGH), deque cap for the disk-stall OOM (MED), fail-soft-on-write-failure
documented (HIGH; deliberately NOT re-enqueued — re-gather would churn under slab-full,
recompute is correctness-safe), rate-limited error logs (MED/LOW). 54 L3 unit tests
green (new: 2-phase gather-before-durable ordering, slab-full fail-soft + no slot leak).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.