46be97adc0
Launching CP shared-KV prefetch from MLA prepare or the indexer can make next-layer prefix work overlap current-layer MQA/materialization instead of the attention window. Centralize the launch in the NSA backend after current-layer materialization and before attention, and leave the early indexer hook inert so the call site cannot regress silently. The accompanying notes capture the draft-as-forward-layer follow-up plan and the latest OOM diagnosis: the observed 178945-token failure matches the CP in-seq MQA logits allocation, so the follow-up fix is q-dimension chunking in _get_topk_ragged_with_cp(), not a max-prefetch-size gate. Constraint: CP shared-KV prefetch must avoid overlapping current-layer MQA/materialization and must not add silent fallback behavior. Rejected: Limit maximum prefetch size | hides the CP logits peak and can reduce cache/prefetch effectiveness. Rejected: Keep prepare/indexer launch sites | they place next-layer collectives in the wrong overlap window. Confidence: medium Scope-risk: moderate Directive: Do not reintroduce early CP prefetch launch without checking Nsight overlap and CP MQA memory peaks. Tested: Local git diff --check and py_compile for touched Python files. Tested: Remote container py_compile plus targeted pytest: 3 passed, 5 warnings. Not-tested: Full ETE under production traffic after this commit. Not-tested: CP in-seq MQA logits chunking; documented as follow-up.
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.