leaveletandClaude Opus 4.8
243f1b964c
The transfer worker iterates every non-dummy decode info for a room and calls per_layer_mgr.finish() once per info, but register_per_layer_transfer registers exactly one context per room/chunk (built for one info's dst_kv_indices). This is only sound when there is exactly one non-dummy info (required_dst_info_num == 1). With decode attn_tp < prefill attn_tp a single prefill rank holds >1 non-dummy infos; finishing once-per-info would over-pop chunk contexts and under-deliver KV to the other infos. Make the assumption explicit: register only when there is one non-dummy info, otherwise fall back to the monolithic post-forward transfer (which fans out to all infos correctly). Found by an independent first-principles audit. Adds TestRegisterGuardSingleInfo (2-info fallback, 1-info register, all-dummy fallback) exercising the real MooncakeKVManager.register_per_layer_transfer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.