d696039092
CP shared-KV batching previously estimated MQA logits from full request extend/context rows, which overstated memory because CP in-seq split only computes each rank's two zigzag segments. Add CP-size aware row accounting that mirrors the fused CP MQA materialization path and take the worst local rank peak for scheduler admission. Expose SGLANG_NSA_MQA_LOGITS_CHUNK_MAX_GB as a more direct cap for one fp32 MQA logits chunk. Runtime and scheduler now both translate this GB cap into chunk rows from the actual K rows, while keeping the old row cap as a mutually-exclusive expert override. Constraint: Scheduler admission must stay CUDA-sync-free and use static budget information only. Rejected: Keep full-request q*k admission | it over-gates CP bs>1 batches because CP splits q rows per rank. Rejected: Let rows and GB caps both apply | precedence would be ambiguous during tuning. Confidence: medium Scope-risk: moderate Directive: Keep MQA logits admission tied to the fused CP MQA segment shape; do not revert to full request token counts. Tested: Local py_compile for touched runtime, scheduler, estimator, and tests. Tested: Local pytest test_cp_shared_kv_prefill_buffer_estimator.py: 8 passed. Tested: Remote g0034 cjy-glm5-new py_compile and targeted estimator/runtime tests: 13 passed. Not-tested: Full ETE high-cache-hit CP bs>1 load with SGLANG_NSA_MQA_LOGITS_CHUNK_MAX_GB. Co-authored-by: OmX <omx@oh-my-codex.dev>
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.