Keep CP HiCache valid tails page-owned during extension
CP shared KV HiCache now treats non-page-aligned valid-tail nodes as page-owned when a later request extends beyond them. The prefix probe, match path, insert path, and prepared backup start now agree on flooring the reusable prefix to the previous physical page boundary, so prepared backup metadata cannot start mid-page or fail to attach at insertion. Duplicate frees under CP HiCache now go through a page-safe free helper. Insert and unfinished duplicate ranges free only fully unprotected pages; no-insert completion still releases the right tail owned only by the finishing request. Constraint: CP HiCache allocator frees whole physical pages even when called with token-granular locs. Rejected: Partial-page sharing/refcounting | too complex for the current page-as-minimum-unit contract. Rejected: Fix only prepare_write_backup_for_req | match_prefix and insert would still expose exact valid-tail hits and desynchronize prepared backup length. Confidence: medium Scope-risk: moderate Directive: Do not expose non-page-aligned CP valid-tail hits to extending requests unless partial-page ownership is explicitly implemented end-to-end. Tested: remote g0034 py_compile for touched files Tested: remote g0034 test_cp_hicache_metadata.py 97 passed Tested: remote g0034 test_cp_shared_kv_runtime.py 73 passed Not-tested: test_cp_shared_kv_layout.py aborts during installed sgl_kernel architecture-specific op loading before assertions
This commit is contained in:
@@ -1151,6 +1151,44 @@ Tests needed:
|
||||
- Add a CUDA/remote regression or source-level guard proving CUDA hot path cannot
|
||||
call `pad_token_locs_to_page_boundary()` with a non-page-start span.
|
||||
|
||||
Additional finding while implementing C16:
|
||||
|
||||
- Flooring only `prepare_write_backup_for_req()` is insufficient. The same
|
||||
exact-valid-tail extension must be treated consistently by radix
|
||||
`match_prefix()` and final `insert()`.
|
||||
- Example with `page_size=64`: an existing 100-token valid-tail node and a new
|
||||
120-token request must expose/reuse only the first 64 tokens. If
|
||||
`match_prefix()` exposes 100 or `insert()` later reuses 100, the prepared
|
||||
backup from 64..119 either starts mid-page in host accounting or becomes
|
||||
unattached/mismatched at insert time.
|
||||
- Therefore C16 is a three-site contract: prefix probe, scheduler-visible
|
||||
prefix match, and insertion duplicate accounting all floor exact valid-tail
|
||||
extensions to the previous physical page boundary.
|
||||
|
||||
Implemented C16:
|
||||
|
||||
- Added a shared exact-valid-tail extension floor helper in `HiRadixCache`.
|
||||
- `_probe_existing_radix_prefix_len_no_split()` now floors an exact
|
||||
non-page-aligned valid-tail hit when the request extends beyond that tail.
|
||||
- `match_prefix()` now exposes only the floored page-boundary prefix in the same
|
||||
situation and splits the radix node at that page boundary.
|
||||
- `insert()` applies the same floor, stops after the page-boundary split, and
|
||||
inserts the recomputed suffix from that boundary so prepared CP backup length
|
||||
remains attachable.
|
||||
- `prepare_write_backup_for_req()` still guards stale scheduler
|
||||
`cache_protected_len` by flooring the raw backup start after first checking
|
||||
that there is an actual suffix to backup.
|
||||
|
||||
Completed C16 tests:
|
||||
|
||||
```text
|
||||
remote g0034 container:
|
||||
test_prepare_write_backup_for_req_floors_mid_page_prefix_hit
|
||||
test_cp_prepare_probe_floors_exact_valid_tail_when_request_extends
|
||||
test_cp_match_prefix_floors_exact_valid_tail_when_request_extends
|
||||
test_cp_insert_extends_from_page_boundary_after_exact_valid_tail
|
||||
```
|
||||
|
||||
## C17: Duplicate/no-insert frees are still token-granular under CP pages
|
||||
|
||||
Finding:
|
||||
@@ -1194,6 +1232,36 @@ Tests needed:
|
||||
- `cache_unfinished_req()` duplicate-free path with a non-page-aligned protected
|
||||
prefix.
|
||||
|
||||
Implemented C17:
|
||||
|
||||
- Added page-floor/ceil helpers for CP free ranges.
|
||||
- Routed `cache_finished_req()` duplicate frees, finished no-insert frees, and
|
||||
`cache_unfinished_req()` duplicate frees through a CP page-safe range helper.
|
||||
- Under CP HiCache, duplicate free ranges now skip partial left/right pages and
|
||||
only pass full unprotected pages to the page allocator. The no-insert
|
||||
finished path skips the partial left protected-prefix page but still includes
|
||||
the finished request's right tail page because no radix node retains it. This
|
||||
prevents a token-granular free from releasing a physical page that still
|
||||
contains protected prefix or radix-owned valid-tail KV without leaking
|
||||
no-insert tail pages.
|
||||
|
||||
Completed C17 tests:
|
||||
|
||||
```text
|
||||
remote g0034 container:
|
||||
test_cache_finished_req_cp_insert_duplicate_free_skips_partial_page
|
||||
test_cache_finished_req_cp_no_insert_frees_only_full_unprotected_pages
|
||||
test_cache_unfinished_req_cp_duplicate_free_skips_partial_page
|
||||
```
|
||||
|
||||
Not-tested C16/C17:
|
||||
|
||||
- `test_cp_shared_kv_runtime.py` passed in the remote container.
|
||||
- `test_cp_shared_kv_layout.py` was attempted in the same remote container, but
|
||||
collection aborted inside installed `sgl_kernel` architecture-specific op
|
||||
loading before any assertion ran. The CP HiCache metadata/regression suite
|
||||
passed in the same container.
|
||||
|
||||
## C18: `cache_protected_len` now mixes valid-token and physical-page semantics
|
||||
|
||||
Finding:
|
||||
|
||||
Reference in New Issue
Block a user