chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

chore(work): record stub-shadowing fix + nsa_pool_host env-failure findings

Browse Source 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
孙远航
2026-06-07 10:16:01 +00:00
parent b1cbacffae
commit a1d5652c7a
1 changed files with 5 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
gov/work/2026-06-06-mooncake-per-layer-kv-transfer-to-cut-ttft-under-high-cache-hit.toml
View File

@@ -149,6 +149,11 @@ date = "2026-06-07"
scope = "cp_per_layer_transfer"
content = "[session recap, re-added after a reset --hard during git cleanup] (1) Root-caused + fixed chunked per-layer KV corruption: register_per_layer_transfer hardcoded chunk_page_start=0; must be chunk_key//page_size (absolute page offset) to match monolithic send() and the worker's dst_kv_indices[absolute positions] indexing. Verified 3 ways (opus audit, deployed-module invariant, 24 unit tests incl 2 regressions) + E2E (flag-on chunked coldchunk sha=75793204 == flag-off monolithic; pre-fix was garbage f5ee6d71). (2) Added guard: per-layer path only when exactly one non-dummy decode info (else monolithic fallback); 3 guard tests. (3) Debugged verify req[1]/req[3] run-to-run nondeterminism -> PRE-EXISTING cache-precision issue (user-confirmed), per-layer EXONERATED (cold deterministic); flat-distribution mechanism (top1-top2 gap predicts flips: Q0/Q2 gap~2.7 stable, Q1/Q3 gap~0.9 flip). Sweep V1 prefill-only restart gave 503 -> needs full restart (deferred). (4) Git: stripped docs_internal from all branch commits (filter-branch, 31->24, pruned 7 pure-docs), gitignored docs_internal+tai-kernel, rebased 24 cleaned commits onto upstream f75ffff8d 'Protect CP shared-KV cache-hit correctness under batched FP8 reuse' (clean, no conflicts; merge verified). commit.gpgsign=false set repo-local."
[[content.journal]]
date = "2026-06-07"
scope = "cp_shared_kv"
content = "Post-rebase test validation on f75ffff8d. Root-caused + FIXED a test-isolation bug in f75ffff8d's test_cp_shared_kv_runtime.py: it installs CPU-CI sgl_kernel stubs at import (sys.modules.setdefault + torch.library FRAGMENT). On GPU, if collected before real sgl_kernel loads, the empty stub shadows it process-wide -> later real-kernel test (fast_topk_transform_ragged_fused) calls a None lambda -> TypeError; sys.modules cleanup makes it WORSE (segfault from FRAGMENT double-registration). Proven via probe (sgl_kernel.__file__=None, fast_topk=<lambda>). Fix: import real sgl_kernel first (setdefault keeps real, FRAGMENT hits already-registered, setattr fills only missing); CPU-CI unaffected. Verified GPU: cp_shared_kv_runtime alone 120 pass; combined cp+topk 125 pass. Commit b1cbacffa. SEPARATE pre-existing: test_nsa_pool_host_unit.py 3 fail IN ISOLATION (env-specific, NOT mine): one is f75ffff8d's own fail-fast rejecting CUDA src_indices; two are cudaErrorHostMemoryAlreadyRegistered (host-pinned-mem conflict from running tests on g0034 while the live prefill server pins hicache memory). per-layer suite still 24/24. Branch rebased on f75ffff8d, docs_internal stripped+gitignored; user handles push."
[[content.acceptance_criteria]]
text = "govctl check passes"
status = "pending"