Fix CP HiCache L3 reload duplicate-reservation crash: recheck clean-attach at reserve

_drain_l3_control_queues runs validate-candidates -> admit -> reserve in one tick.
_cp_l3_admit_reload pops the inflight key and mark_object_committed()s the l3rl
object (which stays live in the allocator) while inserting the reloaded node. A
second same-suffix candidate, validated against the pre-admit tree, then reaches
_cp_l3_reserve_reloads with its dedup guard missing (inflight key gone) and calls
allocator.reserve() on the still-live object -> ValueError: duplicate live
reservation for object 'l3rl:<hash>' payload 'target_kv' -> scheduler crash.

Re-validate _cp_l3_reload_attach_ok in _cp_l3_reserve_reloads after the piggyback
check (honoring its documented 'checked at reserve AND at admission' contract,
which the original code never enforced). If a same-tick admit inserted the suffix,
the candidate is dropped; the un-held request re-matches and hits the just-inserted
L2 node. Rank-uniform: attach_ok reads only the replicated radix and admit is
rank-uniform, so every CP rank drops the same candidate. The reserve() duplicate
raise is kept as a fail-loud backstop for true invariant violations.

Confirmed by independent root-cause + adversarial fix review (CLEAN-SHIP): the
split-retained-key path that looks residual is provably unreachable via the radix
content-path / match_prefix invariant.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-24 14:48:37 +00:00
parent fdc23f7d61
commit 8cd46c18b3

View File

@@ -1553,6 +1553,16 @@ class HiRadixCache(RadixCache):
self.ongoing_l3_reload[existing]["waiters"].append(rid) # piggyback on the in-flight reload
self.cp_l3_reload_done[rid] = False
continue
# Re-validate the clean-attach against the POST-admit tree (the contract _cp_l3_reload_attach_ok
# documents: "checked at reserve AND at admission -- the tree may change in between"). valid_cands
# was built BEFORE _cp_l3_admit_reload ran this tick; a same-tick admit of THIS suffix inserts its
# node and pops its inflight key while leaving the l3rl object committed-and-live in the allocator.
# Without this recheck the dedup guard above misses (inflight key gone) and reserve() below would
# raise "duplicate live reservation" on the still-live object. If the suffix is now present, drop
# the candidate; the (un-held) request re-matches and hits the just-inserted L2 node. Rank-uniform
# (replicated radix + rank-uniform admit -> every rank skips the same candidate).
if not self._cp_l3_reload_attach_ok(lhn, skey):
continue
if len(self.ongoing_l3_reload) >= self.cp_l3_reload_max_inflight:
continue # cap reached (len is replicated) -> recompute
ranges = self._cp_l3_reserve_reload_object(reload_key, store.payloads, c, allocator)