laoyao0822 f290e1e35c Record CUDA IPC as CP compose main-stream choice ...

The symm-vs-IPC matrix showed CUDA IPC winning most production-shaped cases, especially bs=5/10, so the formal compose path should stay on CUDA IPC while symm remains an experimental/prefetch direction. Record the benchmark artifacts and decision to avoid re-litigating the branch choice during future rebases.\n\nConstraint: Production CP shared-KV compose must favor the measured low-risk path, not a speculative symm route.\nRejected: Make symm the main compose path | benchmark won only 3/24 cases and adds a separate synchronization contract.\nConfidence: high\nScope-risk: narrow\nDirective: Do not reintroduce symm writer-rank/barrier parameters into main nsa_backend/nsa_indexer compose without a newer benchmark that beats CUDA IPC end-to-end.\nTested: Documentation-only change; benchmark artifacts recorded from g0034 run.\nNot-tested: No service restart required.

2026-06-13 01:17:50 +08:00

..

attention_backend.md

[Doc] Add DSA/NSA attention backend to support matrix (#20326)

2026-03-11 13:40:35 -04:00

checkpoint_engine.md

[Ckpt Engine] feat: new sglang entrypoint support for update (#12216)

2025-10-30 10:39:27 +08:00

cuda_graph_for_multi_modal_encoder.md

[VLM] Add doc for ViT CUDA Graph (#16343)

2026-01-04 10:05:23 +08:00

deterministic_inference.md

[Documentation] add doc for deterministic inference (#11956)

2025-10-22 12:36:15 -05:00

dp_dpa_smg_guide.md

Enhance SMG guide with RL rollout systems benefits (#18588)

2026-02-10 20:20:45 -08:00

dp_for_multi_modal_encoder.md

[docs] fix misspellings & typos (#18276)

2026-02-05 03:35:29 +00:00

epd_disaggregation.md

[DOC] add documents for encoder global mm cache (#20636)

2026-03-15 16:44:21 -07:00

expert_parallelism.md

feat(eplb): add eplb warmup-only mode

2026-04-08 06:46:26 +08:00

forward_hooks.md

Rename: --hooks to --forward-hooks (#13994)

2025-11-26 22:26:28 +08:00

hicache_best_practices.md

[HiCache] Support heterogeneous tp for hicache storage (#18541)

2026-02-25 00:13:57 -08:00

hicache_design.md

configuration file support and nixl integration augmentation for hicache-storage-backend-extra-config (#16602)

2026-01-22 14:31:48 -08:00

hicache_storage_runtime_attach_detach.md

[Scheduler] Unify idle checks into is_fully_idle() and fix weight update test (#20296)

2026-03-10 17:50:23 -07:00

hicache.rst

[HiCache][HA 1/N] Support HiCache storage runtime attach/detach (#15892)

2026-01-26 19:33:19 -08:00

hyperparameter_tuning.md

[Docs] Rename SGLang Router to SGLang Model Gateway (#17436)

2026-01-20 12:31:10 -08:00

lora.ipynb

[CI] fix: handle missing repo in lora notebook (#19700)

2026-03-02 10:27:32 -08:00

nsa_prefill_cp_accept_len1_garbage_output_debug_ledger.md

Protect CP shared-KV cache-hit correctness under batched FP8 reuse

2026-06-07 13:26:49 +08:00

nsa_prefill_cp_all_gather.md

docs(cp): add NSA prefill CP dual all-gather explanation and Phase 1 plan

2026-04-22 04:55:15 +08:00

nsa_prefill_cp_batch_size_gt1_exploration.md

Plan batch-aware CP shared-KV execution

2026-06-03 00:37:54 +08:00

nsa_prefill_cp_bs_gt1_buffer_size_admission_plan_zh.md

Bound CP prefill batching by estimated temp memory

2026-06-11 01:33:28 +08:00

nsa_prefill_cp_compute_padding_plan_zh.md

Enable CP shared-KV compute padding without inflating cache state

2026-06-04 01:34:35 +08:00

nsa_prefill_cp_eagle_mtp_shared_kv_plan.md

Align CP shared-KV prefetch with the attention overlap window

2026-06-01 01:09:43 +08:00

nsa_prefill_cp_hicache_async_state_sync.md

Share CP HiCache host budget across target and draft KV

2026-05-27 04:28:50 +08:00

nsa_prefill_cp_hicache_bs_gt1_l2_l1_transfer_prefetch_descriptor_plan_zh.md

Reuse prepared HiCache load descriptors across CP prefill layers

2026-06-11 05:09:41 +08:00

nsa_prefill_cp_hicache_cpu_based_transfer_exploration.md

Record CPU-source limits for CP HiCache transfer

2026-06-13 00:58:47 +08:00

nsa_prefill_cp_hicache_draft_kv_strong_sync_plan.md

Keep CP HiCache draft KV invisible until joint readiness

2026-05-27 05:46:34 +08:00

nsa_prefill_cp_hicache_layer_page_first_direct_plan.md

Enable CP HiCache direct transfers to use layer-page host layout

2026-06-10 02:03:07 +08:00

nsa_prefill_cp_hicache_load_prefetch_overlap_notes.md

Make CP HiCache residency owner-lane deterministic

2026-05-28 05:54:23 +08:00

nsa_prefill_cp_hicache_no_collective_capacity_plan.md

Reduce CP HiCache capacity synchronization to owner-lane logic

2026-05-28 08:31:49 +08:00

nsa_prefill_cp_hicache_per_layer_backup_plan.md

Reduce CP HiCache capacity synchronization to owner-lane logic

2026-05-28 08:31:49 +08:00

nsa_prefill_cp_hicache_reactive_host_free_room_plan.md

Keep CP HiCache free room from collapsing under cache reuse

2026-06-07 01:23:34 +08:00

nsa_prefill_cp_index_skip_cache_communication_plan.md

Size CP HiCache host budget from compact index layers

2026-06-10 05:53:05 +08:00

nsa_prefill_cp_page_aligned_cache_contract.md

Avoid index prefetch fallback noise when disabled

2026-06-02 23:52:46 +08:00

nsa_prefill_cp_phase1_narrow_output_collection.md

Clarify the shared KV path toward compute-owned pages

2026-04-28 03:43:36 +08:00

nsa_prefill_cp_phase2_shared_kv_design.md

Clarify the shared KV path toward compute-owned pages

2026-04-28 03:43:36 +08:00

nsa_prefill_cp_phase3_current_reuse_plan.md

Reduce CP shared-KV request-boundary stalls

2026-05-05 05:18:35 +08:00

nsa_prefill_cp_phase4_page_aligned_split_plan.md

Keep CP shared-KV cache hits page-aligned without short fallback

2026-05-29 05:04:50 +08:00

nsa_prefill_cp_phase5_compute_owner_kv_layout_plan.md

Keep CP shared-KV cache hits page-aligned without short fallback

2026-05-29 05:04:50 +08:00

nsa_prefill_cp_phase6_index_materialize_reuse_plan.md

Enable index partial-current reuse without replaying prefix materialize

2026-05-31 02:31:09 +08:00

nsa_prefill_cp_phase7_materialize_triton_plan.md

Reduce CP shared-KV prepare overhead without diagnostic log noise

2026-05-12 20:19:11 +08:00

nsa_prefill_cp_phase8_mla_prefix_prefetch_plan.md

Align CP shared-KV prefetch with the attention overlap window

2026-06-01 01:09:43 +08:00

nsa_prefill_cp_phase9_two_request_overlap_plan.md

Reduce CP shared-KV prepare overhead without diagnostic log noise

2026-05-12 20:19:11 +08:00

nsa_prefill_cp_scheduler_admission_benchmark.md

Model CP scheduler admission with cache-hit pressure

2026-06-10 22:21:59 +08:00

nsa_prefill_cp_shared_kv_bs_gt1_implementation_plan_zh.md

Prevent batched CP draft from silently leaving the target path

2026-06-04 03:47:56 +08:00

nsa_prefill_cp_shared_kv_bs_gt1_implementation_plan.md

Enable batched CP shared-KV current reuse correctness

2026-06-03 07:06:21 +08:00

nsa_prefill_cp_shared_kv_bs_gt1_l1_prefetch_zero_sm_plan_zh.md

Keep CP shared-KV prefetch warnings actionable

2026-06-13 00:59:33 +08:00

nsa_prefill_cp_shared_kv_bs_gt1_parallel_workstreams_zh.md

Keep CP HiCache free room from collapsing under cache reuse

2026-06-07 01:23:34 +08:00

nsa_prefill_cp_shared_kv_bs_gt1_w3_w4_plan_zh.md

Enable batched CP shared-KV current reuse correctness

2026-06-03 07:06:21 +08:00

nsa_prefill_cp_shared_kv_ipc_collective_replacement_plan_zh.md

Record CUDA IPC as CP compose main-stream choice

2026-06-13 01:17:50 +08:00

observability.md

Update docs (#13391)

2025-11-16 19:36:33 -08:00

pd_disaggregation.md

[PD-Disagg] Support query dp rank from bootstrap server. (#19168)

2026-02-23 10:59:30 -08:00

piecewise_cuda_graph.md

Piecewise Cuda Graph Docs (#19738)

2026-03-03 11:51:17 +08:00

pipeline_parallelism.md

[Doc] Update pipeline parallelism documentation (#17414)

2026-01-20 20:10:56 +08:00

quantization.md

fix(docs): correct quantization documentation (#20301) (#20619)

2026-03-15 12:33:12 -04:00

quantized_kv_cache.md

add doc for quantized kv cache (#14348)

2025-12-04 13:01:05 -08:00

rfork.md

[1/2] Add ModelExpress coordination for remote instance weight loading - matching TP (#19920)

2026-03-18 13:38:32 -07:00

separate_reasoning.ipynb

[CI] feat: add early exit to wait_for_server when process dies (#18602)

2026-02-13 16:46:09 -08:00

server_arguments.md

feat(eplb): add eplb warmup-only mode

2026-04-08 06:46:26 +08:00

sgl_model_gateway.md

[model-gateway] Add Redis support as a history backend (#16300)

2026-01-11 01:03:00 -08:00

sglang_for_rl.md

[Feature] Implement update_weights_from_disk for SGLang-D (Diffusion … (#18306)

2026-02-18 11:24:07 -08:00

speculative_decoding.md

[Spec][Ngram] 2/N: Rename branch length to max trie depth (#21181)

2026-03-22 23:35:25 -07:00

structured_outputs_for_reasoning_models.ipynb

update pre-commit config (#18860)

2026-02-16 00:18:31 +08:00

structured_outputs.ipynb

update pre-commit config (#18860)

2026-02-16 00:18:31 +08:00

tool_parser.ipynb

[CI] feat: add early exit to wait_for_server when process dies (#18602)

2026-02-13 16:46:09 -08:00

vlm_query.ipynb

Upgrade transformers==5.3.0 (#17784)

2026-03-18 13:50:43 -07:00