Port upstream IndexCache for DeepSeek-V3.2 / GLM-5 (#21405 + #27114 gate fix) to
our diverged tree, matching upstream HEAD's merged form. The NSA indexer topk is
computed every `index_topk_freq` layers; `skip_topk` layers reuse the previous
layer's topk via prev_topk_indices threaded through the model forward. Default
index_topk_freq=1 => no sharing => zero behavior change until the model config opts in.
- forward_mla.py: gate the two indexer call sites
(`if not skip_topk or (is_nextn and prev_topk_indices is None)`), else reuse
prev_topk_indices; forward_absorb_core returns (output, topk_indices) when
next_skip_topk is set (topk_indices already threaded prepare->core via inner_state).
- deepseek_v2.py: AttentionMLA.__init__ skip_topk/next_skip_topk setup
(freq/pattern/offset; is_nextn=True/True); thread prev_topk_indices through
forward/forward_prepare/op_core; DecoderLayer returns a 3-tuple (tuple-unpack
placed AFTER the CP shared-KV finally); Model.forward loop threads topk_indices.
- deepseek_nextn.py: 3-tuple decoder unpack.
- server_args.py: port the #27114 guard - raise on --enable-two-batch-overlap with
index-topk sharing (the TBO op path does not propagate topk across layers, so
shared layers would run sparse attention with no indices).
All DecoderLayer.forward callers covered (Model loop, nextn, glm4_moe_lite /
mistral_large_3_eagle inherit, TBO op-path discards via op_core). Did NOT port
#24392 (orthogonal indexer-topk capture/output infra). Import-validated on the
B300 / torch-2.11 image. Enabling it requires setting `index_topk_freq` in the
model config (Zhipu-confirmed for GLM-5.1).
Refs: WI-2026-06-07-001
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit af2a41b8d396033e378b6bdd5f6baebf2a98c301)
Port the DeepGEMM "deprecate-from-sgl-kernel, separate sgl-deep-gemm wheel"
migration (upstream PR #24268 and follow-ons) so our code runs against the
torch-2.11 dev-cu13 image, which ships DeepGEMM as the separate sgl-deep-gemm
0.1.2 wheel (import name still `deep_gemm`). Verified against upstream/main HEAD,
NOT the introducing PR — the wheel API drifted between 0.0.1 and 0.1.2.
Ports (all verified against HEAD = wheel 0.1.2):
- nsa_indexer/nsa_backend: paged-MQA context_lens must be (N_total, 1). The
0.0.1 form (batch_size, next_n) DEADLOCKS fp8_paged_mqa_logits on next_n>=2
(our EAGLE deploy uses next_n=4 on SM90/H200, which does not take the SM100-
only DG-native broadcast path). Matches HEAD's _to_2d_context_lens.
- compile_utils warmup: hasattr-guard the dropped get/set_compile_mode API;
pass m_indices positionally to m_grouped_fp8_gemm_nt_contiguous.
- fp8_utils.transform_scale_ue8m0: restore TMA-aligned stride when the DLPack
round-trip collapses a size-1 trailing dim.
- moe_runner/deep_gemm: guard the SBO masked-gemm return unpack when overlap is
inactive (#26839) — reachable via our --enable-single-batch-overlap.
- entrypoint: enable DeepGEMM PDL by default, hasattr-guarded (#23979).
- engine: require sglang-kernel >= 0.4.3 (first sgl-deep-gemm-era kernel); the
migrated (N_total,1) paths would misbehave on the old bundled DeepGEMM.
Verified-unchanged symbols left as-is: fp8_mqa_logits, fp8_gemm_nt, bf16_gemm_*,
get_mk_alignment_for_contiguous_layout, transform_sf_into_required_layout,
get/set_num_sms, the masked-gemm signature. Runtime validation pending the
torch-2.11 image (tai-kernel rebuild + harness).
Refs: WI-2026-06-07-001
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 32818784d332b5c48faf8027247cd4cd9a0f48cd)
Router needs an explicit --prometheus-port under --network=host (collides with
the metrics-enabled servers otherwise). Work item captures the full harness
bring-up: GLM-5.1 PD up e2e, 4 ports validated by smoke, #27372 abort validated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture the design trail for the per-layer/async KV transfer work: verified
problem + infra/risk map, mooncake version/CUDA-13 + async-API findings, the
upstream conn.py pick triage, the master plan, the per-layer/async design, the
g0034 env setup, and the B1a RDMA microbenchmark + GLM-5.1 PD harness scripts.
Governance work item WI-2026-06-06-001 tracked alongside.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Route supported multi-request CP shared-KV extend allocation through request-local owner plans so downstream direct-write paths can trust flattened out_cache_loc owner lanes. Unsupported page plans now fail fast instead of falling back to legacy allocation.
Include the normative govctl RFC and completed work-item trail in the tracked repository; keep transient superpowers planning outside the sglang branch.
Constraint: RFC-0001 W2 requires no legacy multi_batch fallback for supported bs>1 owner-lane allocation
Rejected: concatenating requests before owner planning | CP ownership is request-relative
Rejected: committing only docs/superpowers plan | it omits the normative RFC and work-item trail
Confidence: high
Scope-risk: moderate
Directive: Preserve flattened request order when changing CP shared-KV allocation; run govctl from the sglang repository root
Tested: remote uv test_nsa_cp_utils.py 39 OK; remote uv test_cp_shared_kv_layout.py 34 OK; remote uv test_alloc_pages_with_owners.py 10 OK; govctl check; git diff --check
Not-tested: CUDA ETE/perf paths beyond focused W1/W2 unit suites
