Keep CP HiCache valid tails page-owned during extension

CP shared KV HiCache now treats non-page-aligned valid-tail nodes as page-owned when a later request extends beyond them. The prefix probe, match path, insert path, and prepared backup start now agree on flooring the reusable prefix to the previous physical page boundary, so prepared backup metadata cannot start mid-page or fail to attach at insertion.

Duplicate frees under CP HiCache now go through a page-safe free helper. Insert and unfinished duplicate ranges free only fully unprotected pages; no-insert completion still releases the right tail owned only by the finishing request.

Constraint: CP HiCache allocator frees whole physical pages even when called with token-granular locs.

Rejected: Partial-page sharing/refcounting | too complex for the current page-as-minimum-unit contract.

Rejected: Fix only prepare_write_backup_for_req | match_prefix and insert would still expose exact valid-tail hits and desynchronize prepared backup length.

Confidence: medium

Scope-risk: moderate

Directive: Do not expose non-page-aligned CP valid-tail hits to extending requests unless partial-page ownership is explicitly implemented end-to-end.

Tested: remote g0034 py_compile for touched files

Tested: remote g0034 test_cp_hicache_metadata.py 97 passed

Tested: remote g0034 test_cp_shared_kv_runtime.py 73 passed

Not-tested: test_cp_shared_kv_layout.py aborts during installed sgl_kernel architecture-specific op loading before assertions
This commit is contained in:
laoyao0822
2026-05-29 21:49:17 +08:00
parent 2a9dfcca6f
commit 40cf691c78
4 changed files with 414 additions and 10 deletions
@@ -1151,6 +1151,44 @@ Tests needed:
- Add a CUDA/remote regression or source-level guard proving CUDA hot path cannot
call `pad_token_locs_to_page_boundary()` with a non-page-start span.
Additional finding while implementing C16:
- Flooring only `prepare_write_backup_for_req()` is insufficient. The same
exact-valid-tail extension must be treated consistently by radix
`match_prefix()` and final `insert()`.
- Example with `page_size=64`: an existing 100-token valid-tail node and a new
120-token request must expose/reuse only the first 64 tokens. If
`match_prefix()` exposes 100 or `insert()` later reuses 100, the prepared
backup from 64..119 either starts mid-page in host accounting or becomes
unattached/mismatched at insert time.
- Therefore C16 is a three-site contract: prefix probe, scheduler-visible
prefix match, and insertion duplicate accounting all floor exact valid-tail
extensions to the previous physical page boundary.
Implemented C16:
- Added a shared exact-valid-tail extension floor helper in `HiRadixCache`.
- `_probe_existing_radix_prefix_len_no_split()` now floors an exact
non-page-aligned valid-tail hit when the request extends beyond that tail.
- `match_prefix()` now exposes only the floored page-boundary prefix in the same
situation and splits the radix node at that page boundary.
- `insert()` applies the same floor, stops after the page-boundary split, and
inserts the recomputed suffix from that boundary so prepared CP backup length
remains attachable.
- `prepare_write_backup_for_req()` still guards stale scheduler
`cache_protected_len` by flooring the raw backup start after first checking
that there is an actual suffix to backup.
Completed C16 tests:
```text
remote g0034 container:
test_prepare_write_backup_for_req_floors_mid_page_prefix_hit
test_cp_prepare_probe_floors_exact_valid_tail_when_request_extends
test_cp_match_prefix_floors_exact_valid_tail_when_request_extends
test_cp_insert_extends_from_page_boundary_after_exact_valid_tail
```
## C17: Duplicate/no-insert frees are still token-granular under CP pages
Finding:
@@ -1194,6 +1232,36 @@ Tests needed:
- `cache_unfinished_req()` duplicate-free path with a non-page-aligned protected
prefix.
Implemented C17:
- Added page-floor/ceil helpers for CP free ranges.
- Routed `cache_finished_req()` duplicate frees, finished no-insert frees, and
`cache_unfinished_req()` duplicate frees through a CP page-safe range helper.
- Under CP HiCache, duplicate free ranges now skip partial left/right pages and
only pass full unprotected pages to the page allocator. The no-insert
finished path skips the partial left protected-prefix page but still includes
the finished request's right tail page because no radix node retains it. This
prevents a token-granular free from releasing a physical page that still
contains protected prefix or radix-owned valid-tail KV without leaking
no-insert tail pages.
Completed C17 tests:
```text
remote g0034 container:
test_cache_finished_req_cp_insert_duplicate_free_skips_partial_page
test_cache_finished_req_cp_no_insert_frees_only_full_unprotected_pages
test_cache_unfinished_req_cp_duplicate_free_skips_partial_page
```
Not-tested C16/C17:
- `test_cp_shared_kv_runtime.py` passed in the remote container.
- `test_cp_shared_kv_layout.py` was attempted in the same remote container, but
collection aborted inside installed `sgl_kernel` architecture-specific op
loading before any assertion ran. The CP HiCache metadata/regression suite
passed in the same container.
## C18: `cache_protected_len` now mixes valid-token and physical-page semantics
Finding: