Commit Graph

3 Commits

Author SHA1 Message Date
1c6279eeac chore(harness): fix router prometheus port; update work item
Router needs an explicit --prometheus-port under --network=host (collides with
the metrics-enabled servers otherwise). Work item captures the full harness
bring-up: GLM-5.1 PD up e2e, 4 ports validated by smoke, #27372 abort validated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 09:51:04 +00:00
494153da1f docs(disagg): per-layer async transfer investigation, plan, and harness
Capture the design trail for the per-layer/async KV transfer work: verified
problem + infra/risk map, mooncake version/CUDA-13 + async-API findings, the
upstream conn.py pick triage, the master plan, the per-layer/async design, the
g0034 env setup, and the B1a RDMA microbenchmark + GLM-5.1 PD harness scripts.
Governance work item WI-2026-06-06-001 tracked alongside.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 09:51:04 +00:00
wxiwnd
520770da4c feat(mem-cache): enable CP shared-KV batch owner allocation
Route supported multi-request CP shared-KV extend allocation through request-local owner plans so downstream direct-write paths can trust flattened out_cache_loc owner lanes. Unsupported page plans now fail fast instead of falling back to legacy allocation.

Include the normative govctl RFC and completed work-item trail in the tracked repository; keep transient superpowers planning outside the sglang branch.

Constraint: RFC-0001 W2 requires no legacy multi_batch fallback for supported bs>1 owner-lane allocation
Rejected: concatenating requests before owner planning | CP ownership is request-relative
Rejected: committing only docs/superpowers plan | it omits the normative RFC and work-item trail
Confidence: high
Scope-risk: moderate
Directive: Preserve flattened request order when changing CP shared-KV allocation; run govctl from the sglang repository root
Tested: remote uv test_nsa_cp_utils.py 39 OK; remote uv test_cp_shared_kv_layout.py 34 OK; remote uv test_alloc_pages_with_owners.py 10 OK; govctl check; git diff --check
Not-tested: CUDA ETE/perf paths beyond focused W1/W2 unit suites
2026-06-03 03:26:08 +08:00