b17976b60d
Repeated GSM8K/cache-hit traffic can drain a CP owner lane while exact allocation still succeeds. This changes L1/L2 free-room logic so the trigger accounts for the pending allocation, then evicts a minimum target chunk or the exact shortage, whichever is larger. CP load-back also performs best-effort owner-lane free-room eviction before admitting tiny cache hits that would otherwise skew one lane. The commit also adds gated bs>1 prefill timing markers around recv/bootstrap/batch/inflight polling so future hangs expose the scheduler boundary instead of only surfacing as a watchdog shutdown. The post-fix repeat GSM8K failure is recorded as an active regression to continue investigating, not as old-process noise. Constraint: Free-room policy must reduce repeated eviction and owner-lane starvation without reserving required+target pages after every allocation Rejected: Evict to required+target availability | wastes L1/L2 residency under many short cache hits Rejected: Treat free-room misses as fatal on load-back | exact capacity should remain the strict admission condition Confidence: medium Scope-risk: moderate Directive: Do not remove the gated bs>1 timing markers until repeat-GSM8K hangs have a direct failing boundary Tested: git diff --check Tested: python -m py_compile for touched Python files Tested: remote earlier test_cp_shared_kv_layout.py and test_cp_hicache_metadata.py passed 157 tests after this free-room formula Not-tested: Two full repeat GSM8K runs after this commit; latest repeat still killed prefill and requires follow-up root-cause work
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.