Commit Graph

7 Commits

Author SHA1 Message Date
laoyao0822
d14c02b0dc Count evictable device cache when gating HiCache load-back
HiCache host hits can be skipped before load-back when the quota gate only counts immediately free KV allocator space. Under CP shared-KV pressure most reusable capacity may be represented as evictable radix-cache leaves, so the gate can incorrectly reject a host hit and leave prefill with cached-token zero despite host residency. Count device evictable cache in the quota estimate while leaving actual owner-lane allocation and eviction checks in the load path.

Constraint: CP HiCache load-back still has to respect owner-lane allocation and allocator eviction semantics.

Rejected: Force load-back regardless of quota | would bypass the scheduler pressure signal and increase OOM risk.

Rejected: Treat cache-hit zero as a transfer issue | logs showed host hits were found but skipped by quota before transfer.

Confidence: medium

Scope-risk: moderate

Directive: Do not remove evictable cache from load-back capacity accounting without checking CP HiCache host-hit behavior under device pressure.

Tested: git diff --check

Tested: remote g0034 container pytest -q test/registered/unit/managers/test_prefill_adder.py test/registered/unit/managers/test_hicache_controller_cp.py test/registered/unit/mem_cache/test_cp_hicache_metadata.py test/registered/unit/mem_cache/test_alloc_pages_with_owners.py (90 passed, 3 warnings)

Not-tested: Full ETE GLM5 CP+HiCache+EAGLE pressure run after this quota change

Co-authored-by: OmX <omx@oh-my-codex.dev>
2026-05-27 02:41:45 +08:00
36ee803aa7 test: fix Task 1 CPU unit coverage 2026-05-08 00:29:33 +08:00
95bacb8862 fix: preserve HiCache load-back allocator headroom 2026-05-08 00:18:33 +08:00
Lianmin Zheng
814202704b ci: unify PR test suite naming (#21187) 2026-03-23 00:18:45 -07:00
Shunkangz
bb737d7a82 Support Qwen3 MoE context parallel (#18233)
Co-authored-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
Co-authored-by: Jiying Dong <87510204+dongjiyingdjy@users.noreply.github.com>
2026-03-22 01:27:20 -07:00
pansicheng
97b2a89334 [RadixTree][8/N Refactor]: unify lock interface (#20330) 2026-03-16 11:49:51 +08:00
Ke Bao
c3483e8e97 [CI] Move existing unit tests into unit directory (#20631) 2026-03-15 23:25:18 +08:00