From 899828fe22f72659dc97ff5d2c0db3a353e70f61 Mon Sep 17 00:00:00 2001 From: leavelet Date: Sun, 7 Jun 2026 09:53:45 +0000 Subject: [PATCH] chore(work): record per-layer chunked fix, guard, cache-precision debug, rebase Co-Authored-By: Claude Opus 4.8 (1M context) --- ...r-layer-kv-transfer-to-cut-ttft-under-high-cache-hit.toml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/gov/work/2026-06-06-mooncake-per-layer-kv-transfer-to-cut-ttft-under-high-cache-hit.toml b/gov/work/2026-06-06-mooncake-per-layer-kv-transfer-to-cut-ttft-under-high-cache-hit.toml index d743f4f07..a54bfbffa 100644 --- a/gov/work/2026-06-06-mooncake-per-layer-kv-transfer-to-cut-ttft-under-high-cache-hit.toml +++ b/gov/work/2026-06-06-mooncake-per-layer-kv-transfer-to-cut-ttft-under-high-cache-hit.toml @@ -144,6 +144,11 @@ date = "2026-06-07" scope = "harness" content = "#27372 ABORT TEST PASSED e2e: 8 reqs, 4 aborted mid-transfer (before TTFT~800ms = the corruption window), 4 completed, unexpected_err=0; post-abort clean batch 3/3 healthy; prefill+decode both 200 after. No crash/corruption -> decode->prefill ABORT protocol validated under real concurrent transfer. LOAD 8/8 ok, ttft p50=1484/p99=1671ms. B0 (timing markers, parse_b0.py): forward run_batch p50=476ms (p99=167s = cold-shape DeepGEMM JIT outlier, not real); result+send kickoff p50=1.47ms; transfer poll-to-done p50=1.48ms/p99=58ms. KEY B0 INSIGHT: the scheduler poll-to-transfer-done is TINY (1.48ms) -> transfer is ALREADY overlapped at BATCH granularity (scheduler doesn't block on it). So lever A's win is NOT scheduler time - it is the PER-REQUEST TTFT (the transfer duration the decode waits for before first token). Proper B0 needs request-level transfer duration (req_time_stats #24416 transfer_total_mb/speed) at varying cache-hit, not the scheduler event-loop markers. NEXT B0: enable per-request time-stats logging or read the transfer metric." +[[content.journal]] +date = "2026-06-07" +scope = "cp_per_layer_transfer" +content = "[session recap, re-added after a reset --hard during git cleanup] (1) Root-caused + fixed chunked per-layer KV corruption: register_per_layer_transfer hardcoded chunk_page_start=0; must be chunk_key//page_size (absolute page offset) to match monolithic send() and the worker's dst_kv_indices[absolute positions] indexing. Verified 3 ways (opus audit, deployed-module invariant, 24 unit tests incl 2 regressions) + E2E (flag-on chunked coldchunk sha=75793204 == flag-off monolithic; pre-fix was garbage f5ee6d71). (2) Added guard: per-layer path only when exactly one non-dummy decode info (else monolithic fallback); 3 guard tests. (3) Debugged verify req[1]/req[3] run-to-run nondeterminism -> PRE-EXISTING cache-precision issue (user-confirmed), per-layer EXONERATED (cold deterministic); flat-distribution mechanism (top1-top2 gap predicts flips: Q0/Q2 gap~2.7 stable, Q1/Q3 gap~0.9 flip). Sweep V1 prefill-only restart gave 503 -> needs full restart (deferred). (4) Git: stripped docs_internal from all branch commits (filter-branch, 31->24, pruned 7 pure-docs), gitignored docs_internal+tai-kernel, rebased 24 cleaned commits onto upstream f75ffff8d 'Protect CP shared-KV cache-hit correctness under batched FP8 reuse' (clean, no conflicts; merge verified). commit.gpgsign=false set repo-local." + [[content.acceptance_criteria]] text = "govctl check passes" status = "pending"