chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Avoid index prefetch fallback noise when disabled

Browse Source 
The missing index prefetcher state is expected when MLA/index prefetch is disabled. Logging it as an index-prefetch fallback made the normal synchronous materialize path look like a failed fast path and created noisy reports.

This keeps consume-miss warnings for enabled prefetchers, but suppresses missing-prefetcher warnings when the prefetch feature gate is off.

Constraint: Index prefetch is controlled by the same MLA prefetch enable gate in the current pipeline.
Rejected: Suppress all index prefetch fallback warnings | consume misses still indicate an enabled prefetch pipeline failed to serve a requested buffer.
Confidence: high
Scope-risk: narrow
Directive: Missing prefetcher is not a fallback when SGLANG_CP_SHARED_KV_ENABLE_MLA_PREFETCH=0; do not re-add warning noise without a separate enable gate.
Tested: Local py_compile for nsa_indexer.py and test_cp_shared_kv_runtime.py.
Tested: Remote g0034 targeted regression set passed 4 tests: disabled sync materialize no warning, consume miss warning, first-layer no-warning, enabled create-skip warning.
Not-tested: Full CUDA ETE run with logs.
This commit is contained in:
laoyao0822
2026-06-02 23:52:22 +08:00
parent 401de0f8ce
commit 937f89ef89
3 changed files with 101 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

49
python/sglang/srt/layers/attention/nsa/nsa_indexer.py
View File

@@ -17,6 +17,7 @@ from sglang.srt.layers.attention.nsa import index_buf_accessor
from sglang.srt.layers.attention.nsa.cp_shared_kv_runtime import (
cp_shared_kv_debug_enabled,
cp_shared_kv_debug_log,
cp_shared_kv_mla_prefetch_enabled,
cp_shared_kv_mla_prefetch_log,
cp_shared_kv_mla_prefetch_log_enabled,
cp_shared_kv_mla_prefetch_should_log_layer,
@@ -131,6 +132,14 @@ def _log_cp_shared_kv_index_prefetch_fallback(
)
def _should_log_missing_index_prefetcher() -> bool:
# When MLA/index prefetch is disabled, a missing index prefetcher is the
# expected sync-materialize path, not a fast-path fallback. Still log consume
# misses when a prefetcher exists because those indicate an enabled prefetch
# pipeline failed to provide the requested layer/current buffer.
return cp_shared_kv_mla_prefetch_enabled()
class BaseIndexerMetadata(ABC):
@abstractmethod
def get_seqlens_int32(self) -> torch.Tensor:
@@ -384,17 +393,18 @@ class Indexer(MultiPlatformOp):
tuple(logical_page_table.shape),
)
else:
_log_cp_shared_kv_index_prefetch_fallback(
"current_missing_prefetcher",
"index prefetcher is unavailable; falling back to sync "
"partial-current index compose. layer=%s cp_rank=%s "
"prefix_lens=%s extend_lens=%s logical_page_table_shape=%s",
layer_id,
layout.cp_rank,
prefix_lens,
extend_lens,
tuple(logical_page_table.shape),
)
if _should_log_missing_index_prefetcher():
_log_cp_shared_kv_index_prefetch_fallback(
"current_missing_prefetcher",
"index prefetcher is unavailable; falling back to sync "
"partial-current index compose. layer=%s cp_rank=%s "
"prefix_lens=%s extend_lens=%s logical_page_table_shape=%s",
layer_id,
layout.cp_rank,
prefix_lens,
extend_lens,
tuple(logical_page_table.shape),
)
slot_remap = get_or_build_shared_paged_buffer_slot_remap(
forward_batch,
page_buffer=index_buffer,
@@ -463,14 +473,15 @@ class Indexer(MultiPlatformOp):
tuple(logical_page_table.shape),
)
else:
_log_cp_shared_kv_index_prefetch_fallback(
"missing_prefetcher",
"index prefetcher is unavailable; falling back to sync paged "
"materialize. layer=%s cp_rank=%s logical_page_table_shape=%s",
layer_id,
layout.cp_rank,
tuple(logical_page_table.shape),
)
if _should_log_missing_index_prefetcher():
_log_cp_shared_kv_index_prefetch_fallback(
"missing_prefetcher",
"index prefetcher is unavailable; falling back to sync paged "
"materialize. layer=%s cp_rank=%s logical_page_table_shape=%s",
layer_id,
layout.cp_rank,
tuple(logical_page_table.shape),
)
if cp_shared_kv_debug_enabled():
cp_shared_kv_debug_log(