Decode queue compaction receives req_to_token rows after the prefill side has already populated cached prefix slots. Cache-hit requests therefore need the extend/suffix slice, not the leading prefix slice, when building the prebuilt transfer chunk.
Constraint: Prefill/decode disaggregation shares req_to_token rows across cached prefix and new suffix positions.
Rejected: Keep slicing from zero | cache-hit requests would copy prefix KV locs into the prebuilt suffix chunk.
Confidence: medium
Scope-risk: narrow
Directive: Do not change prepare_for_prebuilt slicing without testing cache-hit req_to_token layouts.
Tested: python -m py_compile on changed runtime files.
Not-tested: Local pytest blocked before collection by missing orjson dependency.
(cherry picked from commit 416112b617fabe71e8cff7484794af73f3e84440)
Burst decode can sit in disaggregation prealloc/transfer queues while health probes still need an immediate scheduler-alive response, and EAGLE prebuilt metadata must outlive the synthetic prebuilt step until the first real decode consume. The transfer path also needs to send final aux metadata even when there are no new KV pages in the final chunk.
This keeps decode metadata slot ownership narrow instead of cloning EAGLE tensors, adds fail-safe cleanup for prebuilt exceptions/finished prebuilt requests, fixes final empty chunk metadata transfer, and records the investigation ledger for future debugging.
Constraint: bs=1 historically worked, so decode compute/sampling behavior must not be changed without direct evidence.
Rejected: Clone EAGLE metadata tensors on transfer commit | avoids lifetime issues but adds hot-path CPU/GPU memory traffic.
Rejected: Treat prefill AbortReq logs as root cause | current evidence shows they can be downstream of decode/router aborts.
Confidence: medium
Scope-risk: moderate
Directive: Do not move EAGLE metadata slot release earlier than first real decode result processing without proving the H2D/spec_info consume point changed.
Tested: g0034 docker py_compile for touched scheduler/disagg/test files; PYTHONPATH=python python -m pytest -q test/registered/unit/disaggregation/test_decode_queue_compaction.py test/registered/unit/mem_cache/test_req_to_token_pool.py test/registered/unit/managers/test_scheduler_health_check.py -> 24 passed
Not-tested: Fresh end-to-end burst traffic after restart; decode node_rank=1 persistent log capture.
The failing runs showed EAGLE accept length collapsing when draft cache-hit suffixes used the new partial-current splice path. This keeps target partial-current reuse enabled, but returns EAGLE/NextN draft cache-hit suffixes to the previous full-materialize path with an explicit fallback warning until the draft splice path has value-level ETE proof.\n\nThe same change set also tightens the page-granular CP HiCache contract for scheduler-visible hits and makes the prefill-to-decode EAGLE handoff observable without cloning hot-path metadata. Exact non-page CP hits are floored to a page boundary for new scheduling decisions, while internal unfinished-request refresh keeps its exact accounting.\n\nConstraint: CP shared KV and HiCache operate at page granularity; exposing token-precise CP tails to scheduler-visible cache hits can force non-page partial materialization.\nConstraint: EAGLE/NextN draft has only one executable layer, so draft prefetch and draft partial-current splice need a separate correctness contract from target layers.\nRejected: Keep draft partial-current splice enabled | remote logs correlate it with avg accept length around 0.068 and median 0.\nRejected: Clone decode metadata tensors on transfer | slot ownership until process_prebuilt consumes them avoids extra hot-path copies.\nConfidence: medium\nScope-risk: moderate\nDirective: Do not re-enable draft partial-current reuse without metadata/draft-KV value checks and ETE accept-length evidence.\nTested: g0034 container py_compile for touched modules.\nTested: g0034 container PYTHONPATH=python python -m pytest -q test/registered/unit/disaggregation/test_decode_queue_compaction.py test/registered/unit/mem_cache/test_cp_hicache_metadata.py test/registered/unit/mem_cache/test_cp_shared_kv_runtime.py -> 183 passed, 5 warnings, 2 subtests passed.\nNot-tested: Fresh ETE accept-length run after this exact commit; requires user-driven traffic restart.
Transferred EAGLE metadata buffers are reusable slot views. The previous clone-based mitigation protected correctness but added copies on the transfer hot path and hid the actual lifetime contract. This change makes the transferred request own the metadata slot while it sits in the decode waiting queue, then releases it immediately after process_prebuilt has consumed top-k and hidden state into the prebuilt batch. Abort paths also release any held decode metadata slot.
Constraint: Decode disaggregation metadata buffers are reusable slot views consumed later by process_prebuilt.
Rejected: Clone transferred EAGLE tensors at commit time | correct but less efficient and masks the ownership contract.
Rejected: Release in process_batch_result_prebuilt | holds slots across forward longer than needed.
Confidence: medium
Scope-risk: moderate
Directive: Do not free successful EAGLE transfer metadata in pop_transferred unless process_prebuilt consumption is also moved earlier.
Tested: Remote py_compile for decode.py, scheduler.py, scheduler_output_processor_mixin.py, and test_decode_queue_compaction.py.
Tested: Remote focused lifecycle tests passed: 3 passed.
Tested: Remote full test_decode_queue_compaction.py passed: 11 passed, 5 warnings.
Not-tested: Fresh ETE runtime validation of EAGLE accept-length recovery after the C48 sync.
Decode committed EAGLE top-k and hidden-state tensors as views into reusable metadata-buffer rows. The metadata index is freed immediately after transfer commit, while the request may wait before process_prebuilt consumes the draft state. Under concurrent cache-hit traffic a later transfer can overwrite the same row, leaving output_id copied correctly but EAGLE draft state corrupted, which matches low accept length despite successful KV/state registration.
Constraint: Metadata slots are intentionally recycled right after transfer commit for throughput.
Rejected: Hold metadata slots until process_prebuilt | larger lifetime change and reduces transfer capacity; cloning the small prebuilt EAGLE state is narrower.
Confidence: high
Scope-risk: narrow
Directive: Do not store reusable metadata-buffer views on Req unless the slot lifetime is extended through all consumers.
Tested: Local py_compile for decode.py and test_decode_queue_compaction.py.
Tested: Remote g0034 container py_compile for decode.py and test_decode_queue_compaction.py.
Tested: Remote g0034 focused clone-lifetime test: 1 passed.
Tested: Remote g0034 test_decode_queue_compaction.py: 10 passed, 5 warnings.
Not-tested: ETE cache-hit accept-length validation after restarting prefill/decode with this synced code.
