Keep CP HiCache owner lanes padded for valid-tail victims
Load-back owner-lane eviction can need to evict a device-resident node whose radix value is a valid tail shorter than the physical tail page. The eviction planner now pads that value only to the page boundary before deriving owner counts, matching the existing write/load capacity contract without exposing padding to radix or scheduler lengths. Constraint: CP HiCache capacity remains page-owner based while radix node values remain valid-token based. Constraint: Avoid collectives; owner-lane capacity must be deterministic from local metadata and logical page ids. Rejected: Require device-resident victim values to be page-aligned | valid-tail cache nodes are now an intentional supported state. Rejected: Pad to cp_size pages | this would waste KV and violate the page-boundary-only contract. Confidence: medium Scope-risk: narrow Directive: If split-inside-tail support is added later, preserve page ownership/refcount semantics before sharing one padded physical page across radix nodes. Tested: local py_compile for hiradix_cache.py and touched CP HiCache tests. Tested: remote g0034 new C8 exact tests: 3 passed, 3 warnings. Tested: remote g0034 CP HiCache impacted suites: 146 passed, 5 warnings. Tested: remote g0034 CP shared KV C1-C5 suite: 122 passed, 5 warnings. Not-tested: full local pytest, blocked by missing runtime dependencies such as starlette. Not-tested: CUDA E2E runtime for this commit. Co-authored-by: OmX <omx@oh-my-codex.dev>
This commit is contained in:
@@ -1033,14 +1033,16 @@ class HiRadixCache(RadixCache):
|
||||
value = getattr(node, "value", None)
|
||||
if value is None or int(value.numel()) == 0:
|
||||
return tuple(0 for _ in range(cp_size))
|
||||
if int(value.numel()) % self.page_size != 0:
|
||||
raise RuntimeError(
|
||||
"CP HiCache load-back eviction requires page-aligned device values: "
|
||||
f"node_id={getattr(node, 'id', '?')} value_len={value.numel()} "
|
||||
f"page_size={self.page_size}"
|
||||
)
|
||||
padded_value = pad_token_locs_to_page_boundary(
|
||||
value,
|
||||
self.page_size,
|
||||
name=(
|
||||
"CP HiCache load-back eviction device values "
|
||||
f"node_id={getattr(node, 'id', '?')}"
|
||||
),
|
||||
)
|
||||
|
||||
first_locs = value[:: self.page_size]
|
||||
first_locs = padded_value[:: self.page_size]
|
||||
logical_pages = torch.div(first_locs, self.page_size, rounding_mode="floor")
|
||||
owners = torch.remainder(logical_pages - 1, cp_size)
|
||||
return tuple(int((owners == owner).sum().item()) for owner in range(cp_size))
|
||||
|
||||
Reference in New Issue
Block a user