From 9ea1953331c109d55d63f86cfaa3abddffa869e2 Mon Sep 17 00:00:00 2001
From: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Date: Mon, 24 Nov 2025 11:09:41 +0800
Subject: [PATCH] [Doc] Refine fused_moe_triton configs doc (#13820)

---
 .../moe/fused_moe_triton/configs/README.md    | 47 ++++++++++++++-----
 1 file changed, 36 insertions(+), 11 deletions(-)

diff --git a/python/sglang/srt/layers/moe/fused_moe_triton/configs/README.md b/python/sglang/srt/layers/moe/fused_moe_triton/configs/README.md
index 3679e698a..2cec1f5cd 100644
--- a/python/sglang/srt/layers/moe/fused_moe_triton/configs/README.md
+++ b/python/sglang/srt/layers/moe/fused_moe_triton/configs/README.md
@@ -1,15 +1,40 @@
+# Fused MoE Triton Kernel Configurations
+
 This directory contains tuned configurations for different settings of the fused_moe kernel.
-For different settings of
-- E (number of experts)
-- N (intermediate size)
-- device_name (torch.cuda.get_device_name())
-- dtype: The data type used by the fused MoE kernel for computation. Supported types include fp8_w8a8, int8_w8a8, int8_w8a16, int4_w4a16, etc. This determines the precision and quantization scheme for both weights and activations.
-- block_shape: The block quantization shape introduced starting from DeepSeek V3/R1 models. This parameter defines the granularity for block-wise quantization, typically specified as `[block_n, block_k]` where `block_n` and `block_k` represent the block dimensions. For example, DeepSeek V3 commonly uses `[128, 128]` block shapes for efficient block-wise FP8 quantization.
 
-the JSON file contains a mapping from M (batch size) to the chosen configuration.
+## Configuration Parameters
 
-The example configurations provided are for the Mixtral model for TP2 on H100
-and TP4 on A100. Mixtral has intermediate size N = 14336, i.e. for TP2 we have
-N = 7168 and for TP4 we have N = 3584.
+Each configuration file is generated based on the following parameters:
 
-See `benchmark/kernels/fused_moe_triton/README.md` on how to generate these config files.
+- **E** (number of experts): Total number of experts in the MoE layer
+- **N** (intermediate size): The intermediate/hidden dimension size
+  - For Tensor Parallelism (TP): `N = original_intermediate_size / tp_size`
+  - Example: Mixtral has N = 14336. For TP=2, N = 7168; for TP=4, N = 3584
+- **device_name**: GPU device name from `torch.cuda.get_device_name()`
+  - Examples: `NVIDIA_H100_80GB_HBM3`, `NVIDIA_A100-SXM4-80GB`, `NVIDIA_GeForce_RTX_4090`
+- **dtype**: Data type for computation
+  - Supported types: `fp8_w8a8`, `int8_w8a8`, `int8_w8a16`, `int4_w4a16`, etc.
+  - Determines precision and quantization scheme for weights and activations
+- **block_shape**: Block quantization shape (for DeepSeek V3/R1 models)
+  - Defines granularity for block-wise quantization, specified as `[block_n, block_k]`
+  - Example: DeepSeek V3 commonly uses `[128, 128]` for efficient block-wise FP8 quantization
+- **tp_size**: Tensor Parallelism size (affects N parameter)
+- **ep_size**: Expert Parallelism size (affects E parameter when EP is enabled)
+- **per_channel_quant**: Whether per-channel quantization is used
+
+## Configuration File Format
+
+Each JSON file contains a mapping from **M** (batch size) to the optimal kernel configuration for that batch size. The configuration includes parameters like `BLOCK_M`, `BLOCK_N`, `BLOCK_K`, `GROUP_M`, number of warps, and pipeline stages.
+
+**Filename Format**:
+```
+E={E},N={N},device_name={device_name},dtype={dtype}[,block_shape={block_shape}][,per_channel_quant={bool}].json
+```
+
+## Generating Configuration Files
+
+To generate new configuration files for your specific hardware and model settings, use the tuning tools:
+
+**📖 Full Documentation**: [Tuning Triton MoE Kernels](https://github.com/sgl-project/sglang/tree/main/benchmark/kernels/fused_moe_triton)
+
+After tuning, move the generated JSON files to this directory to use them in SGLang.