Files
sglang/python
laoyao0822 ff4717c2fb Keep L1 free-room eviction off the CP load-back hot path
CP load-back already handles exact owner-lane deficits before admitting L2->L1 reloads.  Treating free-room target misses as synchronous evict work adds avoidable latency to cache-hit requests even when the current load can fit.  The load-back path now records the free-room miss as advisory and leaves proactive replenishment to non-critical eviction paths.

Constraint: GSM8K/cache-hit runs are sensitive to CPU/control-path latency during L2->L1 reload.

Rejected: Evict to free-room on every load-back | preserves room but puts synchronous eviction on the latency-sensitive cache-hit path.

Confidence: medium

Scope-risk: moderate

Directive: Exact owner-lane deficits remain mandatory; free-room targets must not become blocking in load-back without ETE latency evidence.

Tested: python -m py_compile on changed runtime files.

Not-tested: ETE latency after this isolated change not rerun in this commit step.
(cherry picked from commit 2f42acba0d748d2d747f0531963267d86c483cde)
2026-06-08 00:17:28 +08:00
..