Commit Graph
7 Commits
Author SHA1 Message Date
wxiwnd 520770da4c feat(mem-cache): enable CP shared-KV batch owner allocation
Route supported multi-request CP shared-KV extend allocation through request-local owner plans so downstream direct-write paths can trust flattened out_cache_loc owner lanes. Unsupported page plans now fail fast instead of falling back to legacy allocation.

Include the normative govctl RFC and completed work-item trail in the tracked repository; keep transient superpowers planning outside the sglang branch.

Constraint: RFC-0001 W2 requires no legacy multi_batch fallback for supported bs>1 owner-lane allocation
Rejected: concatenating requests before owner planning | CP ownership is request-relative
Rejected: committing only docs/superpowers plan | it omits the normative RFC and work-item trail
Confidence: high
Scope-risk: moderate
Directive: Preserve flattened request order when changing CP shared-KV allocation; run govctl from the sglang repository root
Tested: remote uv test_nsa_cp_utils.py 39 OK; remote uv test_cp_shared_kv_layout.py 34 OK; remote uv test_alloc_pages_with_owners.py 10 OK; govctl check; git diff --check
Not-tested: CUDA ETE/perf paths beyond focused W1/W2 unit suites
2026-06-03 03:26:08 +08:00
wxiwnd 91facd5bb2 perf(disaggregation): fast-path common decode result processing 2026-04-19 04:23:48 +08:00
wxiwnd 14c40f4e7d perf(disaggregation): slice decode waiting queue for prebuilt batches 2026-04-19 04:23:48 +08:00
wxiwnd 6237271fc9 perf(disaggregation): compact decode queues in one pass 2026-04-19 04:23:48 +08:00
wxiwnd 9470d6d8dc perf(disaggregation): compact prefill bootstrap queue in one pass 2026-04-19 04:23:48 +08:00
wxiwnd 09acec1908 perf(disaggregation): cache static buffer metadata 2026-04-19 04:23:48 +08:00
wxiwnd 53a04a9a97 perf(disaggregation): reuse req pool freelists and alloc_extend tensors 2026-04-19 04:23:48 +08:00