From 79ddc34c1ca4bb9cbc74be313e75028dadee525f Mon Sep 17 00:00:00 2001
From: zijiexia <37504505+zijiexia@users.noreply.github.com>
Date: Mon, 19 Jan 2026 16:35:03 -0800
Subject: [PATCH] [Docs] Add new model evaluation docs (#17043)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
---
 docs/developer_guide/evaluating_new_models.md | 142 ++++++++++++++++++
 docs/index.rst                                |   1 +
 2 files changed, 143 insertions(+)
 create mode 100644 docs/developer_guide/evaluating_new_models.md

diff --git a/docs/developer_guide/evaluating_new_models.md b/docs/developer_guide/evaluating_new_models.md
new file mode 100644
index 000000000..dda43807e
--- /dev/null
+++ b/docs/developer_guide/evaluating_new_models.md
@@ -0,0 +1,142 @@
+# Evaluating New Models with SGLang
+
+This document provides commands for evaluating models' accuracy and performance. Before open-sourcing new models, we strongly suggest running these commands to verify whether the score matches your internal benchmark results.
+
+**For cross verification, please submit commands for installation, server launching, and benchmark running with all the scores and hardware requirements when open-sourcing your models.**
+
+[Reference: MiniMax M2](https://github.com/sgl-project/sglang/pull/12129)
+
+## Accuracy
+
+### LLMs
+
+SGLang provides built-in scripts to evaluate common benchmarks.
+
+**MMLU**
+
+```bash
+python -m sglang.test.run_eval \
+  --eval-name mmlu \
+  --port 30000 \
+  --num-examples 1000 \
+  --max-tokens 8192
+```
+
+**GSM8K**
+
+```bash
+python -m sglang.test.few_shot_gsm8k \
+  --host http://127.0.0.1 \
+  --port 30000 \
+  --num-questions 200 \
+  --num-shots 5
+```
+
+**HellaSwag**
+
+```bash
+python benchmark/hellaswag/bench_sglang.py \
+  --host http://127.0.0.1 \
+  --port 30000 \
+  --num-questions 200 \
+  --num-shots 20
+```
+
+**GPQA**
+
+```bash
+python -m sglang.test.run_eval \
+  --eval-name gpqa \
+  --port 30000 \
+  --num-examples 198 \
+  --max-tokens 120000 \
+  --repeat 8
+```
+> [!TIP]
+> For reasoning models, add `--thinking-mode <mode>` (e.g., `qwen3`, `deepseek-r1`, `deepseek-v3`). You may skip it if the model has forced thinking enabled.
+
+**HumanEval**
+
+```bash
+pip install human_eval
+
+python -m sglang.test.run_eval \
+  --eval-name humaneval \
+  --num-examples 10 \
+  --port 30000
+```
+
+### VLMs
+
+**MMMU**
+
+```bash
+python benchmark/mmmu/bench_sglang.py \
+  --port 30000 \
+  --concurrency 64
+```
+> [!TIP]
+> You can set max tokens by passing `--extra-request-body '{"max_tokens": 4096}'`.
+
+For models capable of processing video, we recommend extending the evaluation to include `VideoMME`, `MVBench`, and other relevant benchmarks.
+
+## Performance
+
+Performance benchmarks measure **Latency** (Time To First Token - TTFT) and **Throughput** (tokens/second).
+
+### LLMs
+
+**Latency-Sensitive Benchmark**
+
+This simulates a scenario with low concurrency (e.g., single user) to measure latency.
+
+```bash
+python -m sglang.bench_serving \
+  --backend sglang \
+  --host 0.0.0.0 \
+  --port 30000 \
+  --dataset-name random \
+  --num-prompts 10 \
+  --max-concurrency 1
+```
+
+**Throughput-Sensitive Benchmark**
+
+This simulates a high-traffic scenario to measure maximum system throughput.
+
+```bash
+python -m sglang.bench_serving \
+  --backend sglang \
+  --host 0.0.0.0 \
+  --port 30000 \
+  --dataset-name random \
+  --num-prompts 1000 \
+  --max-concurrency 100
+```
+
+**Single Batch Performance**
+
+You can also benchmark the performance of processing a single batch offline.
+
+```bash
+python -m sglang.bench_one_batch_server \
+  --model <model-path> \
+  --batch-size 8 \
+  --input-len 1024 \
+  --output-len 1024
+```
+
+You can run more granular benchmarks:
+
+- **Low Concurrency**: `--num-prompts 10 --max-concurrency 1`
+- **Medium Concurrency**: `--num-prompts 80 --max-concurrency 16`
+- **High Concurrency**: `--num-prompts 500 --max-concurrency 100`
+
+## Reporting Results
+
+For each evaluation, please report:
+
+1.  **Metric Score**: Accuracy % (LLMs and VLMs); Latency (ms) and Throughput (tok/s) (LLMs only).
+2.  **Environment settings**: GPU type/count, SGLang commit hash.
+3.  **Launch configuration**: Model path, TP size, and any special flags.
+4.  **Evaluation parameters**: Number of shots, examples, max tokens.
diff --git a/docs/index.rst b/docs/index.rst
index ebbce7fed..31749f373 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -100,6 +100,7 @@ Its core features include:
    developer_guide/development_guide_using_docker.md
    developer_guide/benchmark_and_profiling.md
    developer_guide/bench_serving.md
+   developer_guide/evaluating_new_models.md
 
 .. toctree::
    :maxdepth: 1