leaveletandClaude Opus 4.8
9368c33b23
Two latent CP shared-KV L2 correctness fixes, landed BEFORE L3 (not in an L3 commit). Surfaced by a per-CP8-instance (CP8DP2EP16) scoping review. PREREQ-1 (CP-group scoping). The B1 commit/evict consensus collectives (writing_check ReduceOp.MIN, placement_digest MIN/MAX + its tp_world_size<=1 entry guard, drain_storage_control_queues, the evict/prefetch MINs, the flush barrier) AND cache_controller.prefetch_tp_group all derive from self.tp_group/self.tp_world_size, which was params.tp_cache_group. tp_cache_group equals the CP group ONLY at dp_size=1 (enable_dp_attention False -> tp_cpu_group, and attn_cp_size==tp_size -> _ATTN_CP==_TP) -- the sole reason B1 works today. Under CP8DP2 (DP attention, attn_tp_size=1) tp_cache_group is the size-1 attn-TP group, so every `tp_world_size>1` collective silently no-ops per rank and the placement assert self-disables -> divergent placement -> shared-slab corruption. Fix: for CP hicache (cp_size>1) scope self.tp_group to the CP cpu group (get_attention_cp_group().cpu_group -- already used for the slab-handle broadcast) + self.tp_world_size to its size. A no-op handle change at dp_size=1 (same group object); the intended fix at CP8DP2. The single init-point change propagates to every consensus collective + un-gates prefetch_tp_group + re-enables the placement assert. PREREQ-2 (flush reset). HiRadixCache.reset() cleared the radix tree + host pool but never reset CpSharedL2PageAllocator -> stale free list/ranges/committed after flush_cache (leak; the shared pool was never reclaimed). Added CpSharedL2PageAllocator.reset() (rebuild the per-slab free list all-free, drop ranges + committed, restore the freshly-built placement_digest) called from cache_controller.reset() after the ack queues are cleared. Safe: flush_cache is idle-gated (no in-flight backup/reserve). This is also L3's clear hookpoint. Validation: new test_reset_restores_freshly_constructed_all_free_state + 89/89 pool suite (torch-2.11 container) + import smoke. PREREQ-1 is a no-op at dp_size=1 (live no-regression confirmed on the next prefill restart); CP8DP2 correctness is by construction (CP-group membership verified) pending a 2-machine run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.