25f2147677
CP shared KV and HiCache now use owner-lane metadata as the authoritative capacity view for host write admission and GPU load-back planning. This removes the debug scalar capacity env and keeps CP load-back from relying on a rank-wide scalar collective when per-owner availability is already known. The load-back planner also accounts for evicting child leaves that unlock ancestor device residency, which fixes small lane deficits despite large aggregate evictable capacity. The commit also adds gated CPU timing logs for CP shared-KV MLA/index prefetch and a CUDA microbenchmark for comparing dense all-reduce with owner-packed all-gather layouts. The timing logs are intentionally behind the existing MLA prefetch log env and should not be enabled for throughput measurements. Constraint: CP shared KV owner lanes require target/draft capacity decisions to preserve page_owners rather than total-token scalars Constraint: CUDA collective benchmarks must run on target GPU hosts, not locally Rejected: Keep SGLANG_CP_HICACHE_CAPACITY_DEBUG observer env | owner-lane admission now replaces that scalar debug path Rejected: Add a silent scalar-allreduce fallback | unexpected owner-lane mismatch should fail fast or log loudly Confidence: medium Scope-risk: moderate Directive: Do not reintroduce CP capacity collectives on the scheduler hot path without proving the owner-lane metadata is insufficient Directive: Disable SGLANG_CP_SHARED_KV_LOG_MLA_PREFETCH for end-to-end performance runs; it is diagnostic and high-volume Tested: git diff --check Tested: python -m py_compile on changed runtime/test/benchmark Python files Tested: remote pytest -q test/registered/unit/mem_cache/test_cp_hicache_load_back_owner_lanes.py test/registered/unit/mem_cache/test_cp_hicache_metadata.py (81 passed, 5 warnings) Not-tested: CUDA benchmark benchmark/hicache/bench_cp_shared_kv_prefetch_collective.py Not-tested: full GLM5 E2E throughput after this commit
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.