The CP shared target+draft HiCache budget was still estimating NSA target bytes per token as if every target layer had an index cache. After index skip compacts L1 and L2 index buffers to active layers, that stale estimate kept host/L2 token capacity artificially low even though the physical host index allocation was already smaller. Use the pool's index_active_layer_ids when estimating NSA HiCache bytes per token, with the existing full-layer fallback for pools that do not expose compact metadata. This keeps the shared target+draft capacity computation consistent with the actual L2 host layout. Constraint: Index skip compacts NSA index cache by active logical layers, but MLA KV remains full-layer. Rejected: Keep full-layer HiCache estimate | preserves correctness but wastes the L2 capacity recovered by compact index allocation. Confidence: high Scope-risk: narrow Directive: Keep HiCache byte estimates aligned with NSATokenToKVPool allocation metadata; otherwise target+draft token budgets will drift from real host memory use. Tested: Remote container targeted pytest for active-index HiCache estimate and shared-budget tests -> 3 passed. Tested: Remote container P3-P6 regression suite plus new capacity tests -> 203 passed, 2 subtests passed. Not-tested: Full ETE replay/GSM8K after this budget correction. Co-authored-by: OmX <omx@oh-my-codex.dev>
Unit Tests
Component-level tests that do not launch a server or load model weights. Tests can use CPU or GPU — the key criterion is no server process.
Quick Start
- Find the source file under
python/sglang/srt/. - Create the corresponding test here, mirroring the source tree:
srt/mem_cache/radix_cache.py → unit/mem_cache/test_radix_cache.py srt/sampling/sampling_params.py → unit/sampling/test_sampling_params.py - Register for CI at the top of the file (after imports, before test classes):
from sglang.test.ci.ci_register import register_cpu_ci register_cpu_ci(est_time=5, suite="stage-a-test-cpu") # or: register_cuda_ci(est_time=10, suite="stage-b-test-1-gpu-small") - Run locally:
pytest test/registered/unit/ -v # all unit tests pytest test/registered/unit/mem_cache/ -v # one module - Run with coverage:
# summary pytest test/registered/unit/ --cov --cov-config=.coveragerc -v # PR incremental check (require ≥60% on changed lines) pytest test/registered/unit/ --cov --cov-config=.coveragerc --cov-report=xml diff-cover coverage.xml --compare-branch=origin/main --fail-under=60
Example
"""Unit tests for <module> — no server, no model loading."""
from sglang.test.ci.ci_register import register_cpu_ci
register_cpu_ci(est_time=5, suite="stage-a-test-cpu")
import unittest
from sglang.srt.<module> import TargetClass
from sglang.test.test_utils import CustomTestCase
class TestTargetClass(CustomTestCase):
def test_basic_behavior(self):
obj = TargetClass(...)
self.assertEqual(obj.method(), expected)
if __name__ == "__main__":
unittest.main()
Rules
- No
popen_launch_server()orEngine(...). - No model weight loading.
- Use
CustomTestCase(fromsglang.test.test_utils, adds CI retry). - Use
unittest.mockfor dependencies that are expensive to construct.