chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Update docs (#13391)

Browse Source 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
This commit is contained in:
Lianmin Zheng
2025-11-16 19:36:33 -08:00
committed by GitHub
co-authored by sglang-bot
parent 2a5773440e
commit 7e626d12b7
20 changed files with 115 additions and 190 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/advanced_features/hicache.rst
+1 -1
View File
@@ -1,5 +1,5 @@
Hierarchical KV Caching (HiCache)
======================
=================================
.. toctree::
:maxdepth: 1
docs/advanced_features/observability.md
+1 -1
View File
@@ -7,7 +7,7 @@ You can query them by:
curl http://localhost:30000/metrics
```
See [Production Metrics](../references/production_metrics.md) for more details.
See [Production Metrics](../references/production_metrics.md) and [Production Request Tracing](../references/production_request_trace.md) for more details.
## Logging
docs/advanced_features/pd_multiplexing.md
-56
View File
@@ -1,56 +0,0 @@
# PD Multiplexing
## Server Arguments
| Argument | Type/Default | Description |
|-----------------------------|-------------------------|----------------------------------------------------------|
| `--enable-pdmux` | flag; default: disabled | Enable PD-Multiplexing (PD running on greenctx stream). |
| `--pdmux-config-path <path>`| string path; none | Path to the PD-Multiplexing YAML config file. |
### YAML Configuration
Example configuration for an H200 (132 SMs)
```yaml
# Number of SM groups to divide the GPU into.
# Includes two default groups:
# - Group 0: all SMs for prefill
# - Last group: all SMs for decode
# The number of manual divisions must be (sm_group_num - 2).
sm_group_num: 8
# Optional manual divisions of SMs.
# Each entry contains:
# - prefill_sm: number of SMs allocated for prefill
# - decode_sm: number of SMs allocated for decode
# - decode_bs_threshold: minimum decode batch size to select this group
#
# The sum of `prefill_sm` and `decode_sm` must equal the total number of SMs.
# If provided, the number of entries must equal (sm_group_num - 2).
manual_divisions:
- [112, 20, 1]
- [104, 28, 5]
- [96, 36, 10]
- [80, 52, 15]
- [64, 68, 20]
- [56, 76, 25]
# Divisor for default stream index calculation.
# Used when manual_divisions are not provided.
# Formula:
# stream_idx = max(
# 1,
# min(sm_group_num - 2,
# decode_bs * (sm_group_num - 2) // decode_bs_divisor
# )
# )
decode_bs_divisor: 36
# Maximum token budget for split_forward in the prefill stage.
# Determines how many layers are executed per split_forward.
# Formula:
# forward_count = max(1, split_forward_token_budget // extend_num_tokens)
split_forward_token_budget: 65536
```