25f2147677
CP shared KV and HiCache now use owner-lane metadata as the authoritative capacity view for host write admission and GPU load-back planning. This removes the debug scalar capacity env and keeps CP load-back from relying on a rank-wide scalar collective when per-owner availability is already known. The load-back planner also accounts for evicting child leaves that unlock ancestor device residency, which fixes small lane deficits despite large aggregate evictable capacity. The commit also adds gated CPU timing logs for CP shared-KV MLA/index prefetch and a CUDA microbenchmark for comparing dense all-reduce with owner-packed all-gather layouts. The timing logs are intentionally behind the existing MLA prefetch log env and should not be enabled for throughput measurements. Constraint: CP shared KV owner lanes require target/draft capacity decisions to preserve page_owners rather than total-token scalars Constraint: CUDA collective benchmarks must run on target GPU hosts, not locally Rejected: Keep SGLANG_CP_HICACHE_CAPACITY_DEBUG observer env | owner-lane admission now replaces that scalar debug path Rejected: Add a silent scalar-allreduce fallback | unexpected owner-lane mismatch should fail fast or log loudly Confidence: medium Scope-risk: moderate Directive: Do not reintroduce CP capacity collectives on the scheduler hot path without proving the owner-lane metadata is insufficient Directive: Disable SGLANG_CP_SHARED_KV_LOG_MLA_PREFETCH for end-to-end performance runs; it is diagnostic and high-volume Tested: git diff --check Tested: python -m py_compile on changed runtime/test/benchmark Python files Tested: remote pytest -q test/registered/unit/mem_cache/test_cp_hicache_load_back_owner_lanes.py test/registered/unit/mem_cache/test_cp_hicache_metadata.py (81 passed, 5 warnings) Not-tested: CUDA benchmark benchmark/hicache/bench_cp_shared_kv_prefetch_collective.py Not-tested: full GLM5 E2E throughput after this commit
Unit Tests
Component-level tests that do not launch a server or load model weights. Tests can use CPU or GPU — the key criterion is no server process.
Quick Start
- Find the source file under
python/sglang/srt/. - Create the corresponding test here, mirroring the source tree:
srt/mem_cache/radix_cache.py → unit/mem_cache/test_radix_cache.py srt/sampling/sampling_params.py → unit/sampling/test_sampling_params.py - Register for CI at the top of the file (after imports, before test classes):
from sglang.test.ci.ci_register import register_cpu_ci register_cpu_ci(est_time=5, suite="stage-a-test-cpu") # or: register_cuda_ci(est_time=10, suite="stage-b-test-1-gpu-small") - Run locally:
pytest test/registered/unit/ -v # all unit tests pytest test/registered/unit/mem_cache/ -v # one module - Run with coverage:
# summary pytest test/registered/unit/ --cov --cov-config=.coveragerc -v # PR incremental check (require ≥60% on changed lines) pytest test/registered/unit/ --cov --cov-config=.coveragerc --cov-report=xml diff-cover coverage.xml --compare-branch=origin/main --fail-under=60
Example
"""Unit tests for <module> — no server, no model loading."""
from sglang.test.ci.ci_register import register_cpu_ci
register_cpu_ci(est_time=5, suite="stage-a-test-cpu")
import unittest
from sglang.srt.<module> import TargetClass
from sglang.test.test_utils import CustomTestCase
class TestTargetClass(CustomTestCase):
def test_basic_behavior(self):
obj = TargetClass(...)
self.assertEqual(obj.method(), expected)
if __name__ == "__main__":
unittest.main()
Rules
- No
popen_launch_server()orEngine(...). - No model weight loading.
- Use
CustomTestCase(fromsglang.test.test_utils, adds CI retry). - Use
unittest.mockfor dependencies that are expensive to construct.