Keep CP HiCache owner lanes padded for valid-tail victims

Load-back owner-lane eviction can need to evict a device-resident node whose radix value is a valid tail shorter than the physical tail page. The eviction planner now pads that value only to the page boundary before deriving owner counts, matching the existing write/load capacity contract without exposing padding to radix or scheduler lengths.

Constraint: CP HiCache capacity remains page-owner based while radix node values remain valid-token based.
Constraint: Avoid collectives; owner-lane capacity must be deterministic from local metadata and logical page ids.
Rejected: Require device-resident victim values to be page-aligned | valid-tail cache nodes are now an intentional supported state.
Rejected: Pad to cp_size pages | this would waste KV and violate the page-boundary-only contract.
Confidence: medium
Scope-risk: narrow
Directive: If split-inside-tail support is added later, preserve page ownership/refcount semantics before sharing one padded physical page across radix nodes.
Tested: local py_compile for hiradix_cache.py and touched CP HiCache tests.
Tested: remote g0034 new C8 exact tests: 3 passed, 3 warnings.
Tested: remote g0034 CP HiCache impacted suites: 146 passed, 5 warnings.
Tested: remote g0034 CP shared KV C1-C5 suite: 122 passed, 5 warnings.
Not-tested: full local pytest, blocked by missing runtime dependencies such as starlette.
Not-tested: CUDA E2E runtime for this commit.
Co-authored-by: OmX <omx@oh-my-codex.dev>
This commit is contained in:
laoyao0822
2026-05-29 05:39:25 +08:00
parent 7cfc3c1324
commit 0043037f78
4 changed files with 104 additions and 7 deletions

View File

@@ -1033,14 +1033,16 @@ class HiRadixCache(RadixCache):
value = getattr(node, "value", None)
if value is None or int(value.numel()) == 0:
return tuple(0 for _ in range(cp_size))
if int(value.numel()) % self.page_size != 0:
raise RuntimeError(
"CP HiCache load-back eviction requires page-aligned device values: "
f"node_id={getattr(node, 'id', '?')} value_len={value.numel()} "
f"page_size={self.page_size}"
)
padded_value = pad_token_locs_to_page_boundary(
value,
self.page_size,
name=(
"CP HiCache load-back eviction device values "
f"node_id={getattr(node, 'id', '?')}"
),
)
first_locs = value[:: self.page_size]
first_locs = padded_value[:: self.page_size]
logical_pages = torch.div(first_locs, self.page_size, rounding_mode="floor")
owners = torch.remainder(logical_pages - 1, cp_size)
return tuple(int((owners == owner).sum().item()) for owner in range(cp_size))