f3cf95c0f1
Decode prealloc and transfer queues own the receiver lifetime once a request leaves the queue. The abort and transfer-failure paths were removing requests after streaming/releasing KV state without clearing the receiver, leaving backend-specific request tracking behind. The scheduler idle check also ignored decode retracted requests, so idle housekeeping could run while decode handoff state still existed. Constraint: This is a manual port of upstream 18989f3d48 onto a locally diverged decode queue implementation. Rejected: Direct cherry-pick | current decode queue compaction and streamer APIs differ from upstream. Confidence: high Scope-risk: narrow Directive: Any path that removes a DecodeRequest from prealloc or transfer ownership must clear its kv_receiver unless ownership is explicitly transferred. Tested: remote cjy-glm5-new pytest test/registered/unit/disaggregation/test_decode_queue_compaction.py -q: 16 passed Tested: remote cjy-glm5-new pytest test/registered/unit/disaggregation/test_decode_queue_compaction.py test/registered/unit/observability/test_scheduler_metrics_load.py -q: 17 passed Tested: git diff --check on modified files Not-tested: full disaggregated decode E2E under live traffic
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.