chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Commit Graph

  • 0f9b445131 Keep compute-padding source offsets out of valid rerange output laoyao0822 2026-06-07 23:53:29 +08:00
  • c9a39ccdd2 Preserve decode suffix KV locations after cache hits laoyao0822 2026-06-07 23:53:15 +08:00
  • a1d5652c7a chore(work): record stub-shadowing fix + nsa_pool_host env-failure findings leavelet 2026-06-07 10:16:01 +00:00
  • b1cbacffae test(cp_shared_kv): load real sgl_kernel first to stop CPU-CI stub leaking leavelet 2026-06-07 10:15:35 +00:00
  • 899828fe22 chore(work): record per-layer chunked fix, guard, cache-precision debug, rebase leavelet 2026-06-07 09:53:45 +00:00
  • 24bafabea3 chore: gitignore docs_internal and tai-kernel (internal harness/notes/kernels) leavelet 2026-06-07 09:52:39 +00:00
  • 243f1b964c fix(cp_per_layer_transfer): gate per-layer path on a single non-dummy decode info leavelet 2026-06-07 08:59:02 +00:00
  • e9354c41bc fix(cp_per_layer_transfer): use absolute chunk_page_start for chunked KV dst mapping leavelet 2026-06-07 08:53:10 +00:00
  • beffd3a82f feat(disagg): unify per-layer transfer over chunked prefill (lever A) leavelet 2026-06-07 07:33:46 +00:00
  • e12afe8ced perf(disagg): coarse-grained per-layer transfer + skip re-registration (lever A) leavelet 2026-06-07 05:31:46 +00:00
  • 5fdf439c7e fix(disagg): dedup per-layer enqueues so high-cache-hit can't hang (lever A) leavelet 2026-06-07 04:43:39 +00:00
  • 5791684864 feat(disagg): broaden lever A to full sequence incl cached prefix (high cache-hit) leavelet 2026-06-07 04:30:33 +00:00
  • 76dc60a1d0 test(disagg): finish() latency breakdown instrumentation (lever A A5 root-cause) leavelet 2026-06-07 03:54:43 +00:00
  • 64f767bb42 fix(disagg): cover notifier-missed layers via post-forward fallback (lever A) leavelet 2026-06-07 02:02:46 +00:00
  • 246dbddac0 fix(disagg): per-layer completion can't hang + worker CUDA device + diagnostics leavelet 2026-06-07 01:44:00 +00:00
  • 501faa8f5b fix(disagg): tensor-safe prefix check + graceful per-layer register (lever A) leavelet 2026-06-07 01:15:23 +00:00
  • ae18e3adc8 feat(disagg): wire per-layer overlapped transfer end-to-end (lever A, A3-step3) leavelet 2026-06-07 01:01:40 +00:00
  • aa6acc9485 feat(disagg/mooncake): build_per_layer_context assembly (lever A, A3-step2) leavelet 2026-06-07 00:49:23 +00:00
  • 6bd5d5760c feat(disagg): per-layer block-address computation (lever A, A3-step2 core) leavelet 2026-06-07 00:41:28 +00:00
  • 74e22a0721 feat(disagg): register per-layer transfer manager notifier (lever A, A3-step1) leavelet 2026-06-07 00:39:29 +00:00
  • 7ac8067db1 feat(disagg): per-layer transfer manager (lever A, A2 orchestration) leavelet 2026-06-07 00:35:19 +00:00
  • e0d47bfc41 feat(disagg): per-layer transfer context for overlap (lever A, A2 core) leavelet 2026-06-07 00:31:58 +00:00
  • a09aec30a3 feat(disagg/mooncake): async per-layer KV transfer primitive (lever A foundation) leavelet 2026-06-07 00:09:38 +00:00
  • 1c6279eeac chore(harness): fix router prometheus port; update work item leavelet 2026-06-07 00:02:11 +00:00
  • 9a91c73e09 feat(disagg/mooncake): add non-blocking async transfer wrapper bindings leavelet 2026-06-06 23:20:19 +00:00
  • 494153da1f docs(disagg): per-layer async transfer investigation, plan, and harness leavelet 2026-06-06 23:15:03 +00:00
  • 9a40e42384 fix(disagg/mooncake): notify prefill on decode-side abort leavelet 2026-06-06 22:45:36 +00:00
  • c6c88c2617 fix(disagg): account KV transfer metrics by actually-sent bytes leavelet 2026-06-06 22:24:49 +00:00
  • 2524fe4c9d fix(disagg/mooncake): mark abort Failed and skip aborted chunks in worker leavelet 2026-06-06 22:03:52 +00:00
  • 0cd5c38a9e fix(disagg/mooncake): clip CP/NSA state indices to the shorter side leavelet 2026-06-06 21:59:43 +00:00
  • f75ffff8d9 Protect CP shared-KV cache-hit correctness under batched FP8 reuse laoyao0822 2026-06-07 13:26:49 +08:00
  • b17976b60d Keep CP HiCache free room from collapsing under cache reuse laoyao0822 2026-06-07 01:23:15 +08:00
  • 68defa3923 Expose CP shared-KV batch timing evidence laoyao0822 2026-06-06 14:28:26 +08:00
  • 81bfd4bfd3 Reduce CP shared-KV batch overhead without reverting bs1 planning laoyao0822 2026-06-06 02:03:16 +08:00
  • 1b99de7459 Expose gated evidence for CP shared-KV bs>1 debugging laoyao0822 2026-06-05 23:48:08 +08:00
  • 6eea77e5e9 Stabilize disaggregated decode burst handoff laoyao0822 2026-06-05 19:59:04 +08:00
  • 5eecc5f9c5 Preserve CP ragged topk coordinates under batch planning laoyao0822 2026-06-05 07:45:29 +08:00
  • c8510938b9 Repair CP current-slot composition instead of disabling reuse laoyao0822 2026-06-05 06:25:06 +08:00
  • e200091638 Expose CP shared-KV transfer page-count mismatches before decode laoyao0822 2026-06-04 23:14:22 +08:00
  • f50e2b1e00 Prevent stale CP shared-KV contracts from corrupting prefill laoyao0822 2026-06-04 20:22:29 +08:00
  • 3d6007246b Stabilize CP shared-KV batch padding semantics laoyao0822 2026-06-04 07:25:11 +08:00
  • 02af370e87 Batch CP HiCache host admission before write reservation laoyao0822 2026-06-04 04:32:40 +08:00
  • 108e7d866d Gate CP shared-KV prefill batching behind explicit limits laoyao0822 2026-06-04 04:17:32 +08:00
  • d7723aca07 Prevent batched CP draft from silently leaving the target path laoyao0822 2026-06-04 03:47:56 +08:00
  • 3e3f1b776b Enable CP shared-KV compute padding without inflating cache state laoyao0822 2026-06-04 01:34:35 +08:00
  • b3913046b6 Batch CP HiCache backup submits across requests laoyao0822 2026-06-03 10:23:30 +08:00
  • 19dcd6c4dc Batch CP shared-KV index work for bs>1 fast paths laoyao0822 2026-06-03 08:45:32 +08:00
  • d36f62a3cd Merge CP shared-KV batch owner allocation laoyao0822 2026-06-03 07:22:04 +08:00
  • 50d0008705 Enable batched CP shared-KV current reuse correctness laoyao0822 2026-06-03 07:06:21 +08:00
  • 520770da4c feat(mem-cache): enable CP shared-KV batch owner allocation wxiwnd 2026-06-03 03:26:08 +08:00
  • a7472c415f Preserve request boundaries in CP shared-KV index top-k laoyao0822 2026-06-03 02:36:31 +08:00
  • f8b4f1915e Make CP shared-KV direct writes enforce batch-local ownership laoyao0822 2026-06-03 02:13:57 +08:00
  • e4cf8d18b4 Preserve CP narrow output while planning real batches laoyao0822 2026-06-03 01:21:43 +08:00
  • 0158e28689 Plan batch-aware CP shared-KV execution laoyao0822 2026-06-03 00:37:54 +08:00
  • 937f89ef89 Avoid index prefetch fallback noise when disabled laoyao0822 2026-06-02 23:52:22 +08:00
  • 401de0f8ce Reduce CP HiCache L2 allocator scan cost laoyao0822 2026-06-02 23:35:12 +08:00
  • 7c8fa2f71c Remove full-cache scans from CP owner-lane allocation laoyao0822 2026-06-02 08:41:00 +08:00
  • ce3a20d11b Stabilize CP HiCache residency under L1/L2 pressure laoyao0822 2026-06-02 07:58:02 +08:00
  • 8be4a3a8b5 Surface CP shared-KV fast-path misses laoyao0822 2026-06-02 06:47:40 +08:00
  • 6a25c312c7 Cache completed chunked prefill pages in CP HiCache laoyao0822 2026-06-02 06:22:17 +08:00
  • 5bd68768d9 Prevent unattached CP HiCache write acks from crashing chunked prefill laoyao0822 2026-06-02 05:17:42 +08:00
  • c2d25ff591 Prevent CP HiCache pending-backup splits from killing chunked prefill laoyao0822 2026-06-02 04:51:39 +08:00
  • d1627d1da3 Preserve CP HiCache page-aligned transfer findings laoyao0822 2026-06-02 00:16:03 +08:00
  • 6ef4face89 Preserve FP8 CP shared-KV page contracts laoyao0822 2026-06-01 03:33:44 +08:00
  • 46be97adc0 Align CP shared-KV prefetch with the attention overlap window laoyao0822 2026-06-01 01:09:43 +08:00
  • 4342de0463 Avoid redundant CP collectives on sync shared-KV materialize laoyao0822 2026-05-31 23:59:23 +08:00
  • b4f5c3bc0e Record CP shared KV IPC materialize evidence before runtime wiring laoyao0822 2026-05-31 18:28:59 +08:00
  • a149289554 Ground CP shared KV collective choices in production-shaped evidence laoyao0822 2026-05-31 17:04:00 +08:00
  • 71872bb851 Expose empty chat messages behind conversion failures laoyao0822 2026-05-31 04:06:50 +08:00
  • 0fc95b6439 Keep CP HiCache reuse page-safe without eviction log churn laoyao0822 2026-05-31 03:31:06 +08:00
  • 3c14b1f127 Enable index partial-current reuse without replaying prefix materialize laoyao0822 2026-05-31 02:31:09 +08:00
  • 251a48fb0a Stabilize CP HiCache page-first direct transfers on CUDA 13 laoyao0822 2026-05-31 01:26:07 +08:00
  • b328baec7c Stabilize EAGLE draft cache hits under CP HiCache laoyao0822 2026-05-30 22:31:43 +08:00
  • 10296a5fef Prevent stale CP HiCache tails from overlapping new page owners laoyao0822 2026-05-30 05:51:53 +08:00
  • e9c341afe8 Hold decode EAGLE metadata until prebuilt consumption laoyao0822 2026-05-30 04:51:50 +08:00
  • 07c9544737 Preserve transferred EAGLE state past metadata-slot reuse laoyao0822 2026-05-30 03:36:42 +08:00
  • b7364d23f9 Keep draft cache-hit KV on current-suffix compose laoyao0822 2026-05-30 03:04:34 +08:00
  • 5f343c65ca Keep EAGLE diagnostics from breaking disagg startup laoyao0822 2026-05-30 01:37:05 +08:00
  • b56a4f2e6b Stabilize CP HiCache page-tail ownership under EAGLE reuse laoyao0822 2026-05-30 01:20:01 +08:00
  • 21065cdfdf Keep tiny CP cache-hit suffixes off async prefetch laoyao0822 2026-05-29 22:04:01 +08:00
  • 40cf691c78 Keep CP HiCache valid tails page-owned during extension laoyao0822 2026-05-29 21:49:17 +08:00
  • 2a9dfcca6f Keep current reuse independent of async CP prefetch laoyao0822 2026-05-29 19:34:27 +08:00
  • 2b1524bd8c Apply CP HiCache page-floor policy before backup and insert laoyao0822 2026-05-29 07:10:53 +08:00
  • 8001c4ae8e Floor CP HiCache tail splits to page boundaries laoyao0822 2026-05-29 06:57:16 +08:00
  • c9f790cde9 Fail fast when CP MLA partial-current compose is unavailable laoyao0822 2026-05-29 06:46:39 +08:00
  • 4f9bb7ce30 Record CP shared-KV C11-C14 closure evidence laoyao0822 2026-05-29 06:26:49 +08:00
  • dd69c5970c Standardize CP shared-KV runtime fallback markers laoyao0822 2026-05-29 06:02:13 +08:00
  • e09c8256d7 Expose CP shared-KV allocation fallback as a warning laoyao0822 2026-05-29 05:59:02 +08:00
  • 38532a1fc9 Prevent incompatible CP shared-KV transfer mapping laoyao0822 2026-05-29 05:53:37 +08:00
  • 7e06eaebbf Prove draft HiCache mirrors target valid-tail pages laoyao0822 2026-05-29 05:43:49 +08:00
  • 0043037f78 Keep CP HiCache owner lanes padded for valid-tail victims laoyao0822 2026-05-29 05:39:25 +08:00
  • 7cfc3c1324 Preserve CP HiCache valid tails while padding physical pages laoyao0822 2026-05-29 05:28:53 +08:00
  • c551623ca8 Keep CP shared-KV cache hits page-aligned without short fallback laoyao0822 2026-05-29 05:04:50 +08:00
  • 1e1807f2bc reset prefetch default env laoyao0822 2026-05-29 02:27:05 +08:00
  • c3fc3ff752 Stabilize CP shared-KV prefetch around draft cache hits laoyao0822 2026-05-29 00:33:41 +08:00
  • 26c792939d Gate tiny CP shared-KV prefetch by token threshold laoyao0822 2026-05-28 08:57:45 +08:00
  • 25f2147677 Reduce CP HiCache capacity synchronization to owner-lane logic laoyao0822 2026-05-28 08:31:49 +08:00
  • ff33446787 Make CP HiCache residency owner-lane deterministic laoyao0822 2026-05-28 05:54:23 +08:00
  • 2c94b8de23 Ground HiCache backup overlap policy in measured tails laoyao0822 2026-05-28 03:43:05 +08:00
  • 67d52346de Route page-first direct HiCache loads through TAI laoyao0822 2026-05-28 02:37:56 +08:00