Stabilize CP HiCache page-tail ownership under EAGLE reuse

CP shared KV and HiCache now keep page-aligned physical ownership while preserving valid-token radix semantics. Repeated tiny EAGLE exact hits free duplicate tail pages instead of leaking one allocator page, owner-lane load-back uses page-vector admission/eviction, and single-DP idle schedulers avoid entering an unnecessary MLP-sync collective.

The commit also records the current page-aligned cache contract and adds gated decode-side EAGLE accept diagnostics so future accept-length collapses can be tied to draft KV/state transfer evidence instead of more prefill cache speculation.

Constraint: CP HiCache allocator ownership is page-granular while radix matching remains valid-token based.

Constraint: New diagnostics must be gated and must not alter normal EAGLE, transfer, or cache behavior.

Rejected: Padding short requests to cp_size or 2*cp_size pages | wastes KV capacity and still hides valid-tail lifecycle bugs.

Rejected: Adding more unconditional collectives to prove CP consistency | hot-path collectives previously caused severe performance risk.

Confidence: medium

Scope-risk: broad

Directive: Do not reintroduce silent fallback for CP shared KV/HiCache paths; warning-level fallback or fail-fast is intentional.

Tested: git diff --check

Tested: local py_compile for all modified Python files

Tested: remote g0034 container py_compile for modified Python/test files

Tested: remote g0034 container PYTHONPATH=python python -m pytest -q test/registered/unit/layers/test_nsa_cp_utils.py test/registered/unit/mem_cache/test_cp_shared_kv_runtime.py test/registered/unit/mem_cache/test_cp_hicache_load_back_owner_lanes.py test/registered/unit/managers/test_scheduler_dp_attn_mixin.py => 114 passed, 5 warnings, 2 subtests passed

Not-tested: full ETE traffic rerun after this commit

Not-tested: CUDA/TAI kernel benchmark coverage for all production shapes
This commit is contained in:
laoyao0822
2026-05-30 01:20:01 +08:00
parent 21065cdfdf
commit b56a4f2e6b
16 changed files with 964 additions and 17 deletions

View File

@@ -1558,3 +1558,411 @@ Verified C22:
`73 passed, 2 subtests passed`.
- Local pytest remains blocked by missing optional deps such as `orjson`; remote
container is the authoritative verification environment for this path.
## C23: Repeated EAGLE/bigram requests can expose 65-token tiny suffixes on 64-token pages
Finding:
- A repeated request can still show `extend_lens=[65]` even when the logical
prompt is identical and page flooring is only supposed to lose at most
`page_size - 1` tokens.
- The extra token comes from EAGLE/bigram key semantics plus SGLang's existing
one-token prefix-cache holdback:
- request sequence length in the observed log: `seq_len=40385`;
- scheduler prefix match caps token keys at `input_len - 1`;
- EAGLE radix keys are bigrams, so `convert_to_bigram_key()` reduces that key
length by one more token;
- effective matchable key length becomes `40385 - 2 = 40383`;
- page-flooring to 64-token pages gives `40320`;
- forward therefore sees `40385 - 40320 = 65` extend tokens.
Implication:
- For non-EAGLE page-granular reuse, the repeated-request suffix can be up to
`1 + (page_size - 1)` tokens because the last token is held back.
- For EAGLE/bigram reuse, it can be up to `2 + (page_size - 1)` tokens. With
`page_size=64`, the maximum tiny suffix is therefore `65`, matching the
observed hang log.
- This does not mean the user prompt changed; it is an artifact of the current
cache key contract.
## C24: Tiny page-count cache-hit suffixes still enter NSA in-seq CP split
Finding:
- Latest remote log `/mnt/beegfs/cjy/sglang_cp_hicache_20260529_140440.log`
shows C22 is active: the repeated cache-hit request with
`prefix_lens=[40320]`, `extend_lens=[65]`, `page_size=64`, `cp_size=8`
skips both MLA and index async prefetchers:
`create_skip reason=extend_below_min ... min_async_extend_tokens=512`,
followed by `create_result ... has_mla=False has_index=False`.
- The same request then hangs before any `forward_probe` / `forward_layer`
line for layer 2; the next relevant lines are detokenizer heartbeat timeout.
- The earlier log `/mnt/beegfs/cjy/sglang_cp_hicache_20260529_134616.log`
showed the same 65-token repeated suffix hanging when async prefetchers were
created. Therefore async prefetch creation is not the root cause.
- `can_cp_split()` still allows NSA in-seq CP split whenever
`sum(extend_seq_lens_cpu) >= cp_size`. For the observed repeated request,
page-aligned physical coverage is only `ceil(65 / 64) = 2` pages, far below
`cp_size=8`.
- `build_page_aligned_in_seq_split_list(total_len=128, extend_len=65,
cp_size=8)` produces a degenerate split where only two page units are real and
most CP lanes get zero valid suffix tokens. This is the common condition
across both hang logs.
Correction:
- Async prefetch gating alone is insufficient. CP shared-KV page-aligned
in-seq split itself should require at least one physical suffix page per CP
rank: `ceil(extend_len / page_size) >= cp_size`.
- Below that threshold, run the request without NSA in-seq CP split rather than
generating mostly-zero CP segments. This keeps the cache page contract
intact while avoiding the fragile zero-lane distributed path.
- This must not disable target current reuse. With no async prefetcher, the
backend should still use synchronous prefix materialize + current splice for
cache-hit suffixes, and full current reuse for cache-miss/current-only
requests.
Implementation:
- `can_cp_split()` now skips CP shared-KV in-seq split when a page-aligned
cache-hit suffix has fewer padded suffix pages than CP lanes. For example
`extend_len=65,page_size=64,cp_size=8` has two physical suffix pages and no
longer creates mostly-zero CP segments.
- `can_reuse_current_extend_kv()` now treats tail-padded `out_cache_loc` as
eligible when it still contains all valid current suffix rows. The backend
slices partial-current `current_kv_cache/current_locs` down to the valid
suffix length before synchronous prefix compose, so padding rows do not become
current page-slot data.
- Current-only/full-current reuse remains unsliced in the existing path because
padded current-only CP requests already rely on the padded query/cache shape.
Verified C24:
- Remote container `/sgl-workspace/sglang-tai` on `g0034`:
- `python -m py_compile` for `utils.py`, `cp_shared_kv_runtime.py`, and
`nsa_backend.py` passed.
- Red tests first failed on the old code:
`test_can_cp_split_skips_cp_when_radix_hit_suffix_has_too_few_pages`,
`test_can_cp_split_skips_cp_when_page_units_do_not_cover_all_lanes`, and
padded-current-reuse coverage.
- After the fix, the targeted C24 tests passed: `4 passed`.
- Related unit suite passed:
`test_nsa_cp_utils.py`, `test_cp_shared_kv_runtime.py`,
`test_cp_shared_kv_layout.py` => `128 passed, 5 warnings, 2 subtests passed`.
## C25: 2026-05-29 remote "hang" snapshot is idle prefill plus L1 owner-lane eviction churn
Finding:
- Latest remote log `/mnt/beegfs/cjy/sglang_cp_hicache_20260529_143924.log`
did not show a prefill scheduler deadlock at the time of inspection:
- `/health` responded `200 OK`;
- metrics reported `num_running_reqs=0`, `num_queue_reqs=0`,
`num_prefill_prealloc_queue_reqs=0`;
- detokenizer `py-spy` stack was idle in `recv_pyobj`;
- scheduler ranks were in the normal idle receive path:
CP0 sampling at `recv_requests()` nonblocking ZMQ receive and CP1-7
sampling at `broadcast_pyobj()` from `recv_requests()`.
- The last completed user POST in that log was at `2026-05-29 14:52:08`.
After that, the log only showed periodic health/internal tiny one-token
prefill batches, so the prefill process itself had no visible user request
pending.
- The performance collapse immediately before that idle period coincided with
repeated L1/device owner-lane eviction and load-back churn, not host-L2
eviction:
- no `deterministic CP host eviction before write`,
`_evict_host_for_physical_slots`, or host-full fallback appeared in the
log after startup;
- `HiCache-load` repeatedly had low per-owner availability and evicted device
victims before CP load-back, e.g. node `873` loaded `101632` host tokens
but first evicted victims `[627,604]` for `102016` device tokens;
- generic device eviction then also evicted large backed nodes for much
smaller capacity deficits, e.g. `num_tokens=10752` evicted `79488`, and
`num_tokens=19968` evicted `91200`.
Implication:
- The current evidence points to L1/device cache thrashing under CP owner-lane
constraints once the working set is saturated. Host cache may be nearly full
externally, but this log's hot path is not host eviction; it is repeated
host-hit load-back requiring device-owner-lane free pages, followed by
coarse whole-node device eviction.
- The existing CP load-back owner-lane planner avoids random lane mismatches,
but its eviction granularity is still whole radix nodes. When leaves are
large, a small lane deficit can evict a much larger cached prefix, destroying
future L1 hits and increasing H2D load-back work.
- The generic compute-owner allocation eviction path still calls
`tree_cache.evict(EvictParams(... owner_lane_deficits=...))`, but
`owner_lane_deficits` is not consumed by `HiRadixCache.evict()`. It therefore
remains a coarse total-token eviction path and can over-evict when only a few
owner lanes are short.
Next correction target:
- Do not treat this snapshot as an async-prefetch hang. The next fix should
reduce saturated-L1 owner-lane churn:
1. add an owner-lane-aware device eviction path for compute-owner allocation,
reusing the CP load-back victim selection logic where possible;
2. cap or skip load-back when the required owner-lane eviction cost is far
larger than the host-hit benefit, unless the cache-hit prefix is large
enough to amortize the churn;
3. longer term, split/schedule radix leaves at page-aligned boundaries before
async backup/load so eviction can free page-sized or small-node units
instead of whole long-request leaves.
## C26: 2026-05-29 hang is a scheduler collective-order divergence, not only idle L1 churn
Finding:
- A later remote stack sample on `g0034` for
`/mnt/beegfs/cjy/sglang_cp_hicache_20260529_143924.log` showed a real
distributed scheduler hang:
- CP0 / CP6 were in `recv_requests()->broadcast_pyobj()`;
- CP1 / CP2 / CP5 / CP7 were in
`scheduler_dp_attn_mixin.all_gather()` from
`prepare_mlp_sync_batch_raw()->maybe_prepare_mlp_sync_batch()`;
- CP3 was in idle memory check and CP4 in the prefill event-loop boundary;
- detokenizer was idle in `recv_pyobj`.
- This means ranks disagreed on the next collective: some entered the DP/attn
MLP-sync `all_gather`, while others entered scheduler request broadcast.
That is a collective-order divergence and can deadlock even when queues and
metrics look empty.
Immediate debugging target:
- Read `event_loop_normal_disagg_prefill`,
`get_next_disagg_prefill_batch_to_run`, and
`scheduler_dp_attn_mixin.maybe_prepare_mlp_sync_batch` to identify how one CP
rank can decide it has a batch requiring MLP sync while another decides it is
idle and returns to `recv_requests`.
- The earlier owner-lane eviction churn remains a performance issue, but it is
not sufficient to explain this collective-order mismatch.
Correction C26:
- Added a single-DP idle fast path in `prepare_mlp_sync_batch_raw`: when
`local_batch is None`, `dp_size == 1`, and `SGLANG_SCHEDULER_SKIP_ALL_GATHER`
is not enabled, return `None` without building an idle batch or entering the
MLP-sync all-gather.
- Rationale: CP in-seq split runs inside one DP scheduling domain in the current
deployment. All TP/CP ranks should have identical batch presence. An idle
all-gather only exchanges zero metadata; after tiny internal health requests
it can overlap with the next request broadcast and produce the observed
split: some ranks in Gloo `broadcast_pyobj`, others in NCCL/CPU MLP-sync
`all_gather`.
- This does not remove MLP sync for real batches or for multi-DP cases where a
rank may need an idle batch because another DP worker has work.
## C27: 2026-05-29 request failure evidence is not in the prefill log
Finding:
- In the stopped run using `/mnt/beegfs/cjy/sglang_cp_hicache_20260529_152501.log`,
the prefill log does not contain the request-failure traceback:
- no `Traceback`, `RuntimeError`, scheduler-dead, health-failure, OOM, or
Python exception lines after startup;
- HTTP status counts in the prefill log are all `200`;
- the last visible prefill log line is a successful health check at
`2026-05-29 15:45:43`.
- All prefill/decode Python workers were already stopped/zombie when inspected.
Decode on `g0035`/`g0036` had no persisted stdout/stderr for this run; the
process output was attached to a TTY rather than a log file. Therefore the
concrete decode/router-side request failure cannot be reconstructed from the
available files.
- Decode workers on both decode hosts show zombie creation around
`2026-05-29 15:27:26-15:27:39`, while the prefill log continued accepting
successful `/v1/chat/completions` requests until `15:43:39` and health checks
until `15:45:43`. This strongly suggests the user-visible request failure is
downstream of decode/router availability or external process stop, not a
prefill-side HTTP 4xx/5xx captured in the HiCache log.
Implication:
- Do not infer a CP HiCache correctness failure from this prefill log alone.
The available evidence is an observability gap: decode/router failure output
was not persisted.
- For the next ETE run, decode/router stdout must be persisted with `tee` or an
equivalent log sink; otherwise a stopped decode process leaves only zombie
metadata and no Python traceback.
Next correction target:
- Continue code fixes that are already evidenced in prefill logs (owner-lane
device eviction churn and excessive tiny/internal-request CP fallback logging),
but treat this specific request failure as unrooted until decode/router logs
are captured.
C25 correction update:
- Implemented owner-lane-aware `HiRadixCache.evict()` handling when
`EvictParams.owner_lane_deficits` is set. The eviction path now plans device
victims using owner-lane contribution, so a low-priority non-contributing
leaf is no longer evicted before a higher-priority leaf that actually frees
the deficit lane.
- Reduced `_evict_for_compute_owner_lanes()` requested eviction from
`deficit_pages * page_size * cp_size` to `deficit_pages * page_size` because
the deficit vector is already counted in owner-lane pages. This removes the
previous full-CP-stripe over-eviction for a single-lane deficit.
- Remote verification on `g0034` container:
- RED before fix:
`test_owner_lane_evict_params_choose_deficit_contributing_victim` evicted
the non-contributing `[8,9,10,11]` page instead of the deficit owner page.
- GREEN after fix:
targeted test passed.
- Related suite passed:
`test_cp_hicache_load_back_owner_lanes.py` => `9 passed`.
## C28: 2026-05-29 accept length collapse means EAGLE draft rejection, not a HiCache hit-rate counter
Finding:
- Decode `accept len` is reported from `spec_num_accepted_tokens / spec_num_forward_ct`.
`update_spec_metrics()` adds `num_accepted_tokens + batch_size`, while
`_resolve_spec_overlap_token_ids()` defines `num_accepted_tokens` as
`sum(accept_lens) - len(batch.reqs)`. Therefore a logged accept length near
`1.0` means EAGLE accepted zero draft tokens and only produced the mandatory
target token per verify step.
- The current remote decode processes are running with EAGLE, but not with
`SGLANG_CP_DRAFT_SHARED_KV=1` in their environment. Prefill has
`SGLANG_CP_DRAFT_SHARED_KV=1`, CP shared KV, HiCache, current reuse, and MLA
prefetch enabled. Decode still includes draft KV buffers through its draft
worker, so this is not alone proof of missing transfer, but it is an important
deployment asymmetry to keep visible.
- Current live metrics on the new run already show low/uneven speculative
acceptance on some decode DP ranks (for example `spec_accept_length` around
`1.05` on DP4/DP9 and higher on DP8/DP13), while several ranks are still at
zero because they have not reported a recent decode window.
Implication:
- The accept-length symptom should be debugged as target/draft state divergence
or sampling/draft-quality degradation on decode, not as a direct HiCache host
capacity issue. The key invariant to verify is: after disaggregated prefill
and any cache-hit/current-reuse/load-back path, target KV, draft KV, sequence
lengths, and draft hidden state used by EAGLE are still aligned for the same
request positions.
- Next evidence should be low-frequency decode-side EAGLE acceptance diagnostics
keyed by request/cache-hit status and whether draft KV buffers were received;
per-layer prefill logs are not sufficient to prove why decode rejects drafts.
### C29. 2026-05-30 live run evidence: accept-len collapse is decode-side EAGLE rejection, not missing CP current reuse
Evidence from `/mnt/beegfs/cjy/log/sglang_cp_hicache_20260529_160555.log`, `decode0_20260529_160608.log`, and `decode1_20260529_160611.log`:
- Decode-side `accept len: 1.00` recurs across both decode nodes. Metrics at 2026-05-30 00:16 CST showed examples: DP4=1.0 on g0035 and DP9=1.0 on g0036. This means speculative draft tokens are rejected; the accepted token is only the mandatory target token.
- The low-accept events are not limited to requests with active disaggregation transfer. Log parsing showed many `accept len <= 1.001` events with `#transfer-req: 0`, so the symptom persists after transfer/preallocation, not only during the transfer window.
- Prefill CP shared KV current reuse is active in the same run. Counts: `forward_partial_current_reuse=2032`, with `used_prefetch=True=1104` and `used_prefetch=False=928`. Small-prefix cases below the async-prefetch threshold still use synchronous current compose, e.g. `reason=missing_prefetcher prefix_lens=[320] ... forward_partial_current_reuse used_prefetch=False`.
- Therefore `prefix_below_min` disables async prefix prefetcher creation only; it does not disable partial/full current reuse. It should not by itself explain accept length collapsing to 1.
- Decode startup remains asymmetric: g0035/node-rank0 command lacks `--speculative-attention-mode prefill`, while g0036/node-rank1 includes it. Both decode launch environments only expose `SGLANG_DISAGGREGATION_ALL_CP_RANKS_TRANSFER=1` and `SGLANG_DISAGGREGATION_QUEUE_SIZE=8` among CP/disagg flags.
Current hypothesis to test next: accept-len collapse comes from decode-side EAGLE draft/target state divergence or decode launch/config asymmetry, not from CP shared KV partial current reuse being disabled. Add low-frequency decode-side EAGLE diagnostics around accept-lens / draft KV availability / transferred request state before changing prefill cache logic again.
C29 correction note:
- `--speculative-attention-mode prefill` is the default in `ServerArgs`; therefore g0035 omitting the explicit flag and g0036 spelling it out is not enough to prove a runtime mismatch. Keep checking actual parsed server args if needed, but do not treat the command-line spelling difference alone as root cause.
### C30. 2026-05-30 router failures are downstream of prefill one-page cache accounting failure
Evidence from `/mnt/beegfs/cjy/log/sglang_cp_hicache_20260529_160555.log` and router errors around `2026-05-29 16:19:08`:
- Router `Prefill server failed (CRITICAL)` errors align with the prefill scheduler exiting, not with an independent router/decode routing bug.
- The direct prefill failure is the idle memory checker:
`token_to_kv_pool_allocator memory leak detected! self.max_total_num_tokens=833024, available_size=16384, evictable_size=816576, protected_size=0, session_held=0`.
- The accounting gap is exactly one page: `833024 - (16384 + 816576 + 0 + 0) = 64` tokens, matching the configured page size.
- Immediately before the failure, logs show tiny internal/prefill batches where scheduler allocation is page-granular (`#new-token: 64`) but CP shared KV metadata has valid lengths such as `seq_lens=[1]` or `seq_lens=[6]`, `real_pages=1`.
- Therefore the next root-cause target is a one-page lifecycle/accounting mismatch around tiny CP+EAGLE disagg prefill requests, especially the path `cache_unfinished_req(req)` before KV transfer and `release_kv_cache(req, tree_cache)` after transfer completion.
Implication:
- The router error is a symptom: once prefill rank workers die, decode/router cannot fetch prefill KV and reports failures.
- This is separate from the low EAGLE `accept len` symptom. Low accept length remains a decode-side draft rejection/state divergence issue; the one-page accounting failure explains prefill death and router errors.
Current hypothesis to verify before fixing:
- Disaggregated prefill inserts and locks an unfinished radix node before transfer. On transfer completion, the generic `release_kv_cache()` path calls `cache_finished_req()` again, which can re-insert/split a finished-key view. For tiny valid lengths under page granularity, this can leave exactly one physical page outside `available + evictable + protected` accounting.
### C31. Repeated tiny EAGLE requests leak one CP HiCache page when exact matched tail is not page aligned
New root-cause evidence:
- A minimal remote repro with CP HiCache, `page_size=64`, EAGLE/bigram keys, and repeated `seq_len=6` requests shows the first request is balanced after release, but every repeated exact hit loses one 64-token page:
- after first release: `avail=8128 ev=64 prot=0 sum=8192`;
- after second release: `avail=8064 ev=64 prot=0 sum=8128`;
- after third release: `avail=8000 ev=64 prot=0 sum=8064`.
- This matches the live crash where `max_total_num_tokens - (available + evictable + protected + session_held) = 64`.
- Cause: CP HiCache uses valid-token radix keys but page-granular allocator accounting. For repeated tiny EAGLE requests, the full key is an exact non-page-aligned hit, e.g. `len(keys)=5` for `seq_len=6`.
- `cache_unfinished_req()` / `cache_finished_req()` call `_free_kv_indices_range(kv_indices, cache_protected_len, new_prefix_len)` for duplicate prefixes. In CP mode `_free_kv_indices_range` ceil-aligns `start` and floor-aligns `end` unless `include_partial_tail=True`. With `start=0,end=5,page_size=64`, the duplicate range floors to empty, so the newly allocated physical page is neither freed nor represented by a new radix node. The existing cached node accounts only its own page, leaving the new duplicate page leaked.
Correction target:
- When an insertion result is an exact full-key duplicate (`new_prefix_len == len(keys)`) under CP HiCache, free the duplicate request KV range with `include_partial_tail=True`. This frees the newly allocated physical tail page while preserving the existing cached node. Keep normal partial-split/new-node behavior page-floored to avoid freeing pages retained by radix nodes.
C31 correction:
- `RadixCache.cache_finished_req()` and `cache_unfinished_req()` now free duplicate KV ranges with `include_partial_tail=True` when CP HiCache insertion reports an exact full-key duplicate (`new_prefix_len == len(keys)`).
- This is intentionally limited to exact duplicate hits. Partial/new-node insertions still use the page-floored free path so a newly inserted radix tail page is not accidentally returned to the allocator.
- Added regression coverage for repeated tiny EAGLE exact hits: three repeated `seq_len=2` requests must keep `available + evictable + protected == allocator.size` after each release.
Verified C31:
- Local compile: `python -m py_compile python/sglang/srt/mem_cache/radix_cache.py test/registered/unit/mem_cache/test_cp_hicache_load_back_owner_lanes.py` passed.
- Remote `g0034` container `/sgl-workspace/sglang-tai`:
- targeted regression passed: `test_repeated_tiny_eagle_exact_hit_frees_duplicate_tail_page`;
- related owner-lane/CP HiCache suite passed: `test_cp_hicache_load_back_owner_lanes.py => 10 passed`.
### C32. Decode accept-length collapse needs draft-transfer evidence, not more prefill cache changes
Finding:
- The latest persisted decode logs (`decode0_20260529_160608.log`,
`decode1_20260529_160611.log`) show low EAGLE acceptance after transfer has
already drained: parsing 970 decode-stat lines gives 855 lines with
`#transfer-req: 0`, average `accept len=1.342`, and 134 exact `accept len=1.00`
windows. Therefore the symptom is not limited to the prefill-to-decode
transfer polling window.
- Prefill did attach the CP draft HiCache mirror in the same run. All eight CP
ranks logged `[HiCache-draft] attached CP draft KV host pool`, and the host
budget was shared across target+draft. This rules out the simple hypothesis
that prefill had no draft host mirror.
- The logs do not prove whether the disaggregation transfer actually registered
and copied draft KV/state buffers, because `SGLANG_CP_DRAFT_SHARED_KV_DEBUG`
was not enabled; `draft_kv_buffer_count`, `draft_state_buffer_count`, and
per-room draft transfer records were therefore absent.
- Decode `accept len ~= 1` still means EAGLE draft rejection: the decode metric
adds the mandatory target token, so `1.00` is zero accepted draft tokens.
Implication:
- Do not continue changing CP partial current reuse or HiCache load-back based
solely on this accept-length symptom. The next evidence must come from
decode-side EAGLE state and disaggregation draft-transfer metadata:
transferred draft KV/state buffer counts, metadata hidden/topk presence, and
low-frequency per-request accept-lens after transfer.
- A safe next step is gated diagnostics, not an unconditional behavior change:
log startup/transfer draft-buffer availability once per rank and optionally
sample low-accept decode batches under an explicit debug env.
C32 correction:
- Added gated decode-side EAGLE acceptance diagnostics controlled by
`SGLANG_EAGLE_ACCEPT_DEBUG=1` and sampled by
`SGLANG_EAGLE_ACCEPT_DEBUG_INTERVAL` (default 128). The log is warning-level
only when a sampled batch has zero accepted draft tokens; otherwise it is
interval-sampled info. It records per-request accepted draft count, prompt
length, output length, cached token count, PD metadata tensor presence/shape,
and current `spec_info` topk/hidden shapes.
- This is diagnostic-only. It does not change EAGLE sampling, KV transfer,
cache load-back, or CP shared KV behavior.
- Synced `environ.py` and `scheduler_output_processor_mixin.py` to remote
`g0034:/mnt/beegfs/cjy/sglang-dev`; remote container py_compile passed.
C32 correction update:
- Extended `SGLANG_EAGLE_ACCEPT_DEBUG=1` to print one startup KV-manager audit per
rank on both prefill and decode. The audit includes target/draft KV buffer
counts and target/draft state buffer counts. This avoids enabling the much
noisier `SGLANG_CP_DRAFT_SHARED_KV_DEBUG` just to answer whether draft KV/state
transfer surfaces are registered.
- Synced `prefill.py` and `decode.py` to remote `g0034:/mnt/beegfs/cjy/sglang-dev`;
remote container py_compile passed for the updated files.