Keep CP HiCache reuse page-safe without eviction log churn

Page-aligned CP shared KV can pad out_cache_loc beyond valid current rows, so current reuse now gates MLA composition on the valid extend rows and permits draft partial-current reuse once the TAI sparse-page capability check passes. The TAI current-slot path self-tests sparse pages before use and falls back to the torch reference when the installed kernel is stale.

Eviction success and no-op diagnostics were also moved from INFO to DEBUG so owner-lane and host-admission churn does not flood production logs; true write failures remain WARNING.

Constraint: CP shared KV uses page-aligned physical reservations where valid suffix rows can be shorter than padded out_cache_loc.

Constraint: Production failure/fallback logs must remain visible, but hot successful eviction paths should not emit INFO per victim/rank.

Rejected: Keep draft partial-current reuse disabled | would preserve avoidable full materialization on draft cache-hit suffixes.

Rejected: Trust the TAI current-slot kernel unconditionally | stale kernels can corrupt sparse current-page composition.

Confidence: medium

Scope-risk: moderate

Directive: Do not reintroduce INFO logging in eviction hot paths without rate limiting and runtime evidence.

Tested: local py_compile for touched Python files

Tested: local git diff --check

Tested: remote container py_compile for touched Python files

Tested: remote PYTHONPATH=python python -m pytest -q test/registered/unit/mem_cache/test_cp_hicache_metadata.py::TestHiCacheEvictLoggingLevels::test_evict_hot_path_success_logs_are_debug_only test/registered/unit/mem_cache/test_cp_shared_kv_runtime.py -> 82 passed, 5 warnings, 2 subtests passed

Not-tested: full ETE traffic after this commit; draft partial-current accept length still needs user-driven runtime validation
This commit is contained in:
laoyao0822
2026-05-31 03:31:06 +08:00
parent 3c14b1f127
commit 0fc95b6439
7 changed files with 704 additions and 119 deletions
@@ -1,3 +1,5 @@
import inspect
import re
import sys
import types
import unittest
@@ -97,6 +99,8 @@ from sglang.srt.managers.cache_controller import (
HiCacheWriteFailure,
HiCacheWriteReservation,
)
import sglang.srt.mem_cache.common as mem_cache_common
import sglang.srt.mem_cache.hiradix_cache as hiradix_cache
from sglang.srt.mem_cache.base_prefix_cache import (
EvictParams,
InsertParams,
@@ -117,6 +121,57 @@ from sglang.test.test_utils import CustomTestCase
register_cpu_ci(est_time=2, suite="stage-a-test-cpu")
class TestHiCacheEvictLoggingLevels(CustomTestCase):
def assert_debug_marker(self, source: str, marker: str):
self.assertIn(marker, source)
self.assertRegex(
source,
r"logger\.debug\(\s*\n\s*\"" + re.escape(marker),
msg=f"{marker} should be debug-only on the success/no-op hot path",
)
self.assertNotRegex(
source,
r"logger\.info\(\s*\n\s*\"" + re.escape(marker),
msg=f"{marker} must not stay at INFO on the success/no-op hot path",
)
def test_evict_hot_path_success_logs_are_debug_only(self):
common_source = inspect.getsource(mem_cache_common)
hiradix_source = inspect.getsource(hiradix_cache)
for marker in (
"[MemCache-evict] evict_from_tree_cache:",
"[MemCache-evict] _evict_for_compute_owner_lanes: evictable_size",
"[MemCache-evict] _evict_for_compute_owner_lanes attempt=%d:",
"[MemCache-evict] _evict_for_compute_owner_lanes attempt=%d result:",
):
self.assert_debug_marker(common_source, marker)
for marker in (
"[HiCache-load] owner-lane device eviction before CP load-back: ",
"[HiCache-evict] owner-lane evict found no contributing victims: ",
"[HiCache-evict] owner-lane evict victim no longer evictable: ",
"[HiCache-evict] owner-lane evict END: requested_tokens=%d ",
"[HiCache-evict] deterministic CP host eviction before write: ",
"[HiCache-write] write_backup CP retry after deterministic host eviction: ",
"[HiCache-evict] evict START:",
"[HiCache-evict] evict END:",
"[HiCache-evict] _evict_backuped:",
"[HiCache-evict] _evict_regular:",
"[HiCache-evict] _evict_host_for_physical_slots:",
):
self.assert_debug_marker(hiradix_source, marker)
self.assertRegex(
hiradix_source,
r"logger\.warning\(\s*\n\s*\"\[HiCache-write\] write_backup CP FAILED after deterministic retry:",
)
self.assertNotRegex(
hiradix_source,
r"logger\.info\(\s*\n\s*\"\[HiCache-write\] write_backup CP FAILED after deterministic retry:",
)
class TestCpHiCacheImports(CustomTestCase):
def test_cp_hicache_public_imports_without_sgl_kernel(self):
import subprocess