Make CP HiCache backup admission deterministic

CP HiCache write-through under shared KV was still using rank-wide collectives to decide host reservation eviction, and per-layer backup registration could be bypassed before the final forward boundary. This moves backup registration to the final run_batch pre-forward boundary, forwards it through SessionAwareCache, exposes fallback paths as explicit warnings, and introduces deterministic owner-lane capacity planning for CP host reservation.

Constraint: CP shared-KV ranks must keep target and draft host reservations owner-lane consistent without adding hot-path collective synchronization

Constraint: Remote CUDA validation must run in the g0034 container, not locally

Rejected: Keep reserve_slots_max all_reduce as the default admission path | observed reserve collectives reaching double-digit and occasional 100ms+ latency

Rejected: Silent post-forward catch-up backup | hides when per-layer forward-overlap backup is not actually active

Confidence: medium

Scope-risk: broad

Directive: Do not reintroduce CP HiCache hot-path collectives without a measured mismatch case and explicit fallback warning

Tested: py_compile for modified Python modules and CP HiCache metadata test file in remote g0034 container

Tested: python3 -m pytest test/registered/unit/mem_cache/test_cp_hicache_metadata.py -q in remote g0034 container (75 passed, 5 warnings)

Tested: git diff --check HEAD~1..HEAD

Not-tested: Local pytest blocked by missing pybase64 in the local environment

Not-tested: Full CP HiCache + MTP E2E after the no-collective reservation change

Co-authored-by: OmX <omx@oh-my-codex.dev>

This commit is contained in:

laoyao0822

2026-05-27 23:11:23 +08:00

co-authored by

OmX

parent f355fdd39e

commit 40a8de5fd1

11 changed files with 1476 additions and 118 deletions

									
										python/sglang/srt/environ.py
									
		+1
		
												View File
												
				@@ -215,6 +215,7 @@ class Envs:

				    SGLANG_CP_SHARED_KV_MLA_PREFETCH_MIN_PREFIX_PAGES = EnvInt(-1)

				    SGLANG_CP_DRAFT_SHARED_KV = EnvBool(False)

				    SGLANG_CP_DRAFT_SHARED_KV_DEBUG = EnvBool(False)

				    SGLANG_CP_HICACHE_CAPACITY_DEBUG = EnvBool(False)

				    SGLANG_DISABLE_TAI_BIGRAM = EnvBool(False)

				    SGLANG_TEST_REQUEST_TIME_STATS = EnvBool(False)

				    SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK = EnvBool(False)