fix: call _update_leaf_status in inc/dec_node_lock_ref to prevent phantom evictable nodes
inc_node_lock_ref/dec_node_lock_ref adjusted evictable_size_ but did not call _update_leaf_status, so nodes could become "phantoms" — counted in evictable_size_ but missing from evictable_leaves. Under load with write-behind, this caused eviction to return 0 despite large evictable_size_, leading to OOM: Available tokens: 778624 (evictable_size=706880) evict_result=(num_tokens_evicted=0) The race: write-behind locks node X via inc_node_lock_ref (X stays in evictable_leaves). A request path touches X via inc_lock_ref, which calls _update_leaf_status and removes X from evictable_leaves. Request finishes, dec_lock_ref keeps X out (lock_ref still >0). Write ack calls dec_node_lock_ref dropping lock_ref to 0, but never calls _update_leaf_status — X is permanently lost. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -665,6 +665,7 @@ class RadixCache(BasePrefixCache):
|
||||
self.evictable_size_ -= len(node.key)
|
||||
self.protected_size_ += len(node.key)
|
||||
node.lock_ref += 1
|
||||
self._update_leaf_status(node)
|
||||
|
||||
def dec_node_lock_ref(self, node: TreeNode):
|
||||
"""Decrement lock_ref on a single node without walking to root.
|
||||
@@ -679,6 +680,7 @@ class RadixCache(BasePrefixCache):
|
||||
self.evictable_size_ += len(node.key)
|
||||
self.protected_size_ -= len(node.key)
|
||||
node.lock_ref -= 1
|
||||
self._update_leaf_status(node)
|
||||
|
||||
def evictable_size(self):
|
||||
return self.evictable_size_
|
||||
|
||||
Reference in New Issue
Block a user