laoyao0822
07c9544737
Decode committed EAGLE top-k and hidden-state tensors as views into reusable metadata-buffer rows. The metadata index is freed immediately after transfer commit, while the request may wait before process_prebuilt consumes the draft state. Under concurrent cache-hit traffic a later transfer can overwrite the same row, leaving output_id copied correctly but EAGLE draft state corrupted, which matches low accept length despite successful KV/state registration. Constraint: Metadata slots are intentionally recycled right after transfer commit for throughput. Rejected: Hold metadata slots until process_prebuilt | larger lifetime change and reduces transfer capacity; cloning the small prebuilt EAGLE state is narrower. Confidence: high Scope-risk: narrow Directive: Do not store reusable metadata-buffer views on Req unless the slot lifetime is extended through all consumers. Tested: Local py_compile for decode.py and test_decode_queue_compaction.py. Tested: Remote g0034 container py_compile for decode.py and test_decode_queue_compaction.py. Tested: Remote g0034 focused clone-lifetime test: 1 passed. Tested: Remote g0034 test_decode_queue_compaction.py: 10 passed, 5 warnings. Not-tested: ETE cache-hit accept-length validation after restarting prefill/decode with this synced code.
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.