Files
sglang/python
leavelet 3d65944a22 Remove stale layer_transfer_counter prefetch guards (port from e293d4a39 fix #3)
CpSharedKVMlaPrefetcher.create and CpSharedKVIndexPrefetcher.create
returned None whenever ``token_to_kv_pool.layer_transfer_counter`` was
set, which permanently disables NSA indexer prefetch whenever HiCache
is active — even when no H2D transfer is in progress.

The guard is unnecessary: buffer getters synchronize via wait_until(),
and the prefetcher's stream calls wait_stream(current_stream) before
materialization. Removing it restores intended prefetch parallelism
under HiCache + NSA.

Only fix #3 of e293d4a39 is portable to this branch — fixes #1 (SHM
timeout) and #2 (SHM exception safety) target shm_allreduce.py and the
batched-CP coordination path in hiradix_cache.py, neither of which
exists here.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 00:40:39 +08:00
..