Stabilize disaggregated decode burst handoff

Burst decode can sit in disaggregation prealloc/transfer queues while health probes still need an immediate scheduler-alive response, and EAGLE prebuilt metadata must outlive the synthetic prebuilt step until the first real decode consume. The transfer path also needs to send final aux metadata even when there are no new KV pages in the final chunk.

This keeps decode metadata slot ownership narrow instead of cloning EAGLE tensors, adds fail-safe cleanup for prebuilt exceptions/finished prebuilt requests, fixes final empty chunk metadata transfer, and records the investigation ledger for future debugging.

Constraint: bs=1 historically worked, so decode compute/sampling behavior must not be changed without direct evidence.

Rejected: Clone EAGLE metadata tensors on transfer commit | avoids lifetime issues but adds hot-path CPU/GPU memory traffic.

Rejected: Treat prefill AbortReq logs as root cause | current evidence shows they can be downstream of decode/router aborts.

Confidence: medium

Scope-risk: moderate

Directive: Do not move EAGLE metadata slot release earlier than first real decode result processing without proving the H2D/spec_info consume point changed.

Tested: g0034 docker py_compile for touched scheduler/disagg/test files; PYTHONPATH=python python -m pytest -q test/registered/unit/disaggregation/test_decode_queue_compaction.py test/registered/unit/mem_cache/test_req_to_token_pool.py test/registered/unit/managers/test_scheduler_health_check.py -> 24 passed

Not-tested: Fresh end-to-end burst traffic after restart; decode node_rank=1 persistent log capture.
This commit is contained in:
laoyao0822
2026-06-05 19:59:04 +08:00
parent 5eecc5f9c5
commit 6eea77e5e9
9 changed files with 687 additions and 28 deletions
+2 -1
View File
@@ -1482,9 +1482,10 @@ class SchedulerDisaggregationDecodeMixin:
new_batch.prepare_for_prebuilt()
try:
new_batch.process_prebuilt(self.server_args, self.future_map)
finally:
except BaseException:
for req in can_run_list:
self._free_decode_metadata_index_if_held(req)
raise
return new_batch
+9 -2
View File
@@ -946,11 +946,18 @@ class SchedulerDisaggregationPrefillMixin:
page_size,
)
if len(page_indices) == 0:
if len(page_indices) == 0 and not last_chunk:
logger.info(
f"Skip sending kv chunk for request {req.rid=} {req.bootstrap_room=} because page_indices is empty"
f"Skip sending non-final kv chunk for request {req.rid=} {req.bootstrap_room=} because page_indices is empty"
)
return
if len(page_indices) == 0:
logger.warning(
"[CP_SHARED_KV_TRANSFER][final_empty_chunk] "
"sending final aux metadata without new KV pages: "
f"{req.rid=} {req.bootstrap_room=} {start_idx=} {end_idx=} "
f"origin_len={len(req.origin_input_ids)} fill_len={len(req.fill_ids)}"
)
prefill_queue = getattr(self, "disagg_prefill_bootstrap_queue", None)
has_draft_pool = (
getattr(prefill_queue, "draft_token_to_kv_pool", None) is not None