Update test README with CI registry documentation and 5090/H100 guidance (#17368)

2026-01-22 16:29:08 -08:00
parent 2399af5557
commit 1e8e0cca2c
1 changed files with 84 additions and 4 deletions
@@ -32,14 +32,94 @@ python3 test_choices.py
 - Ensure you added `unittest.main()` for unittest and `sys.exit(pytest.main([__file__]))` for pytest in the scripts. The CI run them via `python3 test_file.py`.
 - The CI will run some suites such as `per-commit-1-gpu`, `per-commit-2-gpu`, and `nightly-1-gpu` automatically. If you need special setup or custom test groups, you may modify the workflows in [`.github/workflows/`](https://github.com/sgl-project/sglang/tree/main/.github/workflows).

-## CI Registry Quick Peek
+## CI Registry System

-Tests in `test/registered/` declare CI metadata via lightweight markers:
+Tests in `test/registered/` use a registry-based CI system for flexible backend/schedule configuration.
+
+### Registration Functions

 ```python
-from sglang.test.ci.ci_register import register_cuda_ci
+from sglang.test.ci.ci_register import (
+    register_cuda_ci,
+    register_amd_ci,
+    register_cpu_ci,
+    register_npu_ci,
+)

-register_cuda_ci(est_time=80, suite="stage-a-test-1")
+# Per-commit test (small 1-gpu, runs on 5090)
+register_cuda_ci(est_time=80, suite="stage-b-test-small-1-gpu")
+
+# Per-commit test (large 1-gpu, runs on H100)
+register_cuda_ci(est_time=120, suite="stage-b-test-large-1-gpu")
+
+# Per-commit test (2-gpu)
+register_cuda_ci(est_time=200, suite="stage-b-test-large-2-gpu")
+
+# Nightly-only test
+register_cuda_ci(est_time=200, suite="nightly-1-gpu", nightly=True)
+
+# Multi-backend test
+register_cuda_ci(est_time=80, suite="stage-b-test-small-1-gpu")
+register_amd_ci(est_time=120, suite="stage-a-test-1")
+
+# Temporarily disabled test
+register_cuda_ci(est_time=80, suite="stage-b-test-small-1-gpu", disabled="flaky - see #12345")
+```
+
+### Choosing Between 1-GPU Suites (5090 vs H100)
+
+When adding 1-GPU tests, choose the appropriate suite based on hardware compatibility:
+
+| Suite | Runner | GPU | When to Use |
+|-------|--------|-----|-------------|
+| `stage-b-test-small-1-gpu` | `1-gpu-5090` | RTX 5090 (32GB, SM120) | 5090-compatible tests (preferred) |
+| `stage-b-test-large-1-gpu` | `1-gpu-runner` | H100 (80GB, SM90) | Large models or 5090-incompatible tests |
+
+**Use `stage-b-test-small-1-gpu` (5090) whenever possible** - this is the preferred suite for most 1-GPU tests.
+
+**Use `stage-b-test-large-1-gpu` (H100) if ANY of these apply:**
+
+1. **Architecture incompatibility (SM120/Blackwell)**:
+   - FA3 attention backend (requires SM≤90)
+   - MLA with FA3 backend
+   - FP8/MXFP4 quantization (not supported on SM120)
+   - Certain Triton kernels (shared memory limits)
+
+2. **Memory requirements**:
+   - Models >30B params or large MoE
+   - Tests requiring >32GB VRAM
+
+3. **Known 5090 failures**:
+   - Weight update/sync tests
+   - Certain spec decoding tests
+
+If a test cannot run on 5090 due to any of the above, use `stage-b-test-large-1-gpu` which runs on H100.
+
+### Available Suites
+
+**Per-Commit (CUDA)**:
+- Stage A: `stage-a-test-1` (locked), `stage-a-test-2`, `stage-a-test-cpu`
+- Stage B: `stage-b-test-small-1-gpu` (5090), `stage-b-test-large-1-gpu` (H100), `stage-b-test-large-2-gpu`
+- Stage C: `stage-c-test-large-4-gpu`, `stage-c-test-large-4-gpu-b200`, `stage-c-test-large-8-gpu-b200`
+
+**Per-Commit (AMD)**:
+- `stage-a-test-1`, `stage-b-test-small-1-gpu-amd`, `stage-b-test-large-2-gpu-amd`
+
+**Nightly**:
+- `nightly-1-gpu`, `nightly-2-gpu`, `nightly-4-gpu`, `nightly-8-gpu`, etc.
+
+### Running Tests with run_suite.py
+
+```bash
+# Run per-commit tests
+python test/run_suite.py --hw cuda --suite stage-b-test-small-1-gpu
+
+# Run nightly tests
+python test/run_suite.py --hw cuda --suite nightly-1-gpu --nightly
+
+# With auto-partitioning (for parallel CI jobs)
+python test/run_suite.py --hw cuda --suite stage-b-test-small-1-gpu \
+    --auto-partition-id 0 --auto-partition-size 4
 ```

 ## Writing Elegant Test Cases