Files
sglang/python
leavelet 2ea3728a43 L3 3.0: host-slab accessor (layer_page_first <-> contiguous blob)
CpSharedL2SlabAccessor (HIGH-1): memcpy-gather a page's N_layers strided slab slices into a
contiguous buffer (spill) + scatter a contiguous blob back into the strided slices (reload) —
the cheap RAM bridge that lets host stay layer_page_first (inference perf) while disk is
page-contiguous. Generic CpL3SlabLayout(n_layers, page_num, slice_bytes) with verified-from-source
factories: for_mla (layer_num,page_num,page_size,kv_cache_dim,itemsize -> 2.74 MiB/page) and
for_index (n_active_layers,indexer_page_num,indexer_page_stride -> 0.17/0.63 MiB/page). gather_into/
scatter_from take an offset so the spill writes straight after the 64B slot header (zero extra copy).
Pure (mmap+ints), no torch. 8/8 tests. Canonical layer 0..N-1 order shared by spill+reload.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 21:31:12 +00:00
..
2026-06-10 05:54:43 +08:00