diff --git a/docs/advanced_features/pd_disaggregation.md b/docs/advanced_features/pd_disaggregation.md index b40ab11b4..64cb41a9f 100644 --- a/docs/advanced_features/pd_disaggregation.md +++ b/docs/advanced_features/pd_disaggregation.md @@ -130,10 +130,12 @@ PD Disaggregation with Mooncake supports the following environment variables for To enable NVLink transport for KV cache transfers with the mooncake backend (recommended for NVL72 deployments), set the following environment variables. Note that auxiliary data transfer will still use TCP as a temporary workaround. ```bash -export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=True +export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=NVLINK export MC_FORCE_MNNVL=True ``` +The `SGLANG_MOONCAKE_CUSTOM_MEM_POOL` environment variable enables the custom memory pool. Supported values are `NVLINK` (or `True`), `BAREX`, and `INTRA_NODE_NVLINK`. + #### Prefill Server Configuration | Variable | Description | Default | |:--------:|:-----------:|:--------: diff --git a/docs/references/environment_variables.md b/docs/references/environment_variables.md index ac1b52a4b..2e0eebc19 100644 --- a/docs/references/environment_variables.md +++ b/docs/references/environment_variables.md @@ -98,6 +98,7 @@ SGLang supports various environment variables that can be used to configure its | `SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION` | Clip max new tokens estimation for memory planning | `4096` | | `SGLANG_DETOKENIZER_MAX_STATES` | Maximum states for detokenizer | Default value based on system | | `SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK` | Enable checks for memory imbalance across Tensor Parallel ranks | `true` | +| `SGLANG_MOONCAKE_CUSTOM_MEM_POOL` | Configure the custom memory pool type for Mooncake. Supports `NVLINK`, `BAREX`, `INTRA_NODE_NVLINK`. If set to `true`, it defaults to `NVLINK`. | `None` | ## Model-Specific Options