chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Commit Graph

  • 40a8de5fd1 Make CP HiCache backup admission deterministic laoyao0822 2026-05-27 22:39:05 +08:00
  • f355fdd39e Overlap CP HiCache backup without exposing partial host state laoyao0822 2026-05-27 09:50:47 +08:00
  • 03529319a1 Keep CP HiCache host visibility behind per-layer target and draft backup laoyao0822 2026-05-27 07:43:10 +08:00
  • 367dff06f3 Keep CP HiCache draft KV invisible until joint readiness laoyao0822 2026-05-27 05:46:34 +08:00
  • 71c4f66968 Enforce draft KV strong-sync for CP HiCache hits laoyao0822 2026-05-27 05:23:31 +08:00
  • 8571fe0cd9 Share CP HiCache host budget across target and draft KV laoyao0822 2026-05-27 04:28:50 +08:00
  • d14c02b0dc Count evictable device cache when gating HiCache load-back laoyao0822 2026-05-27 02:41:45 +08:00
  • e5982dcceb fix: call _update_leaf_status in inc/dec_node_lock_ref to prevent phantom evictable nodes leavelet 2026-05-23 10:20:54 +00:00
  • 3d65944a22 Remove stale layer_transfer_counter prefetch guards (port from e293d4a39 fix #3) leavelet 2026-05-27 00:40:39 +08:00
  • 3186382be1 Drop env-gated CP HiCache page-index validator leavelet 2026-05-27 00:15:35 +08:00
  • 1d630def95 Fix CP HiCache load_cp owner-pattern mismatch (cache-hit corruption) leavelet 2026-05-25 01:11:07 +08:00
  • f2834b3403 Preserve draft NSA state during CP disaggregated transfer laoyao0822 2026-05-24 21:46:08 +08:00
  • fa80c6278e Preserve draft KV across CP HiCache hits laoyao0822 2026-05-24 00:45:30 +08:00
  • 315eaaff56 Preserve draft KV across CP HiCache hits laoyao0822 2026-05-24 00:45:30 +08:00
  • d655fad040 Dispatch EAGLE radix bigram builder to tai-kernel leavelet 2026-05-24 03:21:52 +08:00
  • ec7e9fbc57 Avoid full-prompt embedding in CP MTP prefill laoyao0822 2026-05-23 15:48:36 +08:00
  • bacad1d498 fix: update leaf status for hicache leavelet 2026-05-22 14:09:48 +00:00
  • bd6e28f8ce fix: embed before CP split in nextn to prevent TP all_reduce shape mismatch leavelet 2026-05-22 11:21:09 +00:00
  • 8ee249f953 fix: re-export log_cp_draft_shared_kv_debug from cp_shared_kv_runtime leavelet 2026-05-14 17:44:08 +00:00
  • 99b669f8b9 Reduce prefill EAGLE memory pressure under CP shared KV laoyao0822 2026-05-13 22:29:18 +08:00
  • 3fc7a5c18c Move CP shared-KV prefetch reduce into attention overlap laoyao0822 2026-05-08 17:39:24 +08:00
  • aa27a444f6 Reuse TAI range prepare for CP MQA current path laoyao0822 2026-05-08 17:23:38 +08:00
  • f20ef7ace4 Avoid redundant CP KV rebuild on shared-KV MLA path laoyao0822 2026-05-07 06:37:50 +08:00
  • 96bf7a2594 Reduce CP shared-KV prepare overhead without diagnostic log noise laoyao0822 2026-05-07 03:43:26 +08:00
  • c5c30a3f50 Reuse CP shared KV remaps across layer materialization laoyao0822 2026-05-12 20:02:22 +08:00
  • 099bcfb41e fix: copy spec_info in ScheduleBatch.copy() to preserve Eagle hidden states in overlap prefill ThomasX 2026-05-11 04:44:25 +08:00
  • 3f1e5c5896 fix: initialize cp_shared_kv_layout for draft model runners ThomasX 2026-05-11 02:51:22 +08:00
  • 29a395fe95 feat: add structured debug logging for CP HiCache write/evict/load paths Add logger.info statements with [CacheCtrl-write], [MemCache-evict], [MemCache-alloc], [HiCache-write], [HiCache-evict], [HiCache-load] tags across cache_controller.py, common.py, and hiradix_cache.py to improve observability of cache operations. ThomasX 2026-05-11 00:00:04 +08:00
  • 3a5928ef53 feat: add page-aligned index validation for CP HiCache direct transfers ThomasX 2026-05-10 02:02:54 +08:00
  • f38937723e fix: address CP HiCache review regressions ThomasX 2026-05-08 04:11:25 +08:00
  • 8844c303e2 fix: synchronize CP HiCache host eviction semantics ThomasX 2026-05-08 03:20:47 +08:00
  • eecc8e21ec docs: defer CP HiCache performance follow-up ThomasX 2026-05-08 03:07:18 +08:00
  • 9617d52639 fix: keep HiCache storage config runtime import ThomasX 2026-05-08 03:03:10 +08:00
  • 3e9152194b test: cover CP HiCache host integration ThomasX 2026-05-08 02:59:27 +08:00
  • 39f40c02b4 fix: handle empty HiCache match prefixes ThomasX 2026-05-08 02:47:25 +08:00
  • 829f3ebceb feat: restore CP HiCache host hits as logical locs ThomasX 2026-05-08 02:34:16 +08:00
  • f10a8983c3 fix: keep CP HiCache load host indices on CPU ThomasX 2026-05-08 02:26:46 +08:00
  • 94df7a6909 feat: map CP HiCache loads to owned physical slots ThomasX 2026-05-08 02:16:04 +08:00
  • 08d518c2ed fix: preserve CP host siblings during stale cleanup ThomasX 2026-05-08 02:06:04 +08:00
  • 2df832dec0 fix: unblock CP host eviction past stale leaves ThomasX 2026-05-08 01:59:14 +08:00
  • f1a7e605e8 feat: split and evict CP HiCache metadata ThomasX 2026-05-08 01:50:07 +08:00
  • bebbbfd0e8 fix: treat CP HiCache metadata as backup state ThomasX 2026-05-08 01:41:53 +08:00
  • 08a95a35a6 fix: implement CP HiCache write retry eviction ThomasX 2026-05-08 01:35:55 +08:00
  • 068aa2f910 feat: track CP HiCache write-back state ThomasX 2026-05-08 01:29:39 +08:00
  • a8816f18dd test: clean CP HiCache controller imports ThomasX 2026-05-08 01:23:12 +08:00
  • 301420f976 feat: map CP HiCache writes to owned physical slots ThomasX 2026-05-08 01:14:58 +08:00
  • eb0ec4299d test: cover CP HiCache metadata validation edges ThomasX 2026-05-08 01:09:32 +08:00
  • 1ede8bb999 test: harden CP HiCache metadata invariants ThomasX 2026-05-08 01:06:58 +08:00
  • 5fbe43381f feat: add CP HiCache node metadata ThomasX 2026-05-08 01:00:09 +08:00
  • da4994ec26 fix: fail fast for CP HiCache storage args ThomasX 2026-05-08 00:55:35 +08:00
  • 54f8cab7e7 test: enforce early CP HiCache storage rejection ThomasX 2026-05-08 00:50:52 +08:00
  • ffcbf904e4 fix: reject CP shared KV with HiCache storage ThomasX 2026-05-08 00:42:08 +08:00
  • 36ee803aa7 test: fix Task 1 CPU unit coverage ThomasX 2026-05-08 00:29:33 +08:00
  • 95bacb8862 fix: preserve HiCache load-back allocator headroom ThomasX 2026-05-08 00:18:33 +08:00
  • 1f074f434e docs: add CP HiCache host plan ThomasX 2026-05-08 00:12:39 +08:00
  • 43ad2fe52d Reduce CP shared KV overhead without changing ownership semantics laoyao0822 2026-05-06 05:27:43 +08:00
  • 5e5ac5e2e7 Route CP shared MLA store through TAI fused kernels without runtime spam laoyao0822 2026-05-06 00:54:47 +08:00
  • 49eaf9ffde Reduce CP shared-KV request-boundary stalls laoyao0822 2026-05-05 05:18:35 +08:00
  • a638d71d53 Avoid PAGED topk metadata scans after MQA laoyao0822 2026-05-03 23:08:17 +08:00
  • 9eb9d82b51 Remove CP indexer syncs around MQA logits laoyao0822 2026-05-03 05:11:36 +08:00
  • 9fec89ba09 Keep Phase8 prefetch on the deferred-consume path laoyao0822 2026-05-03 03:47:31 +08:00
  • bc23a81884 Overlap CP shared KV prefix materialization for cached MLA prefill laoyao0822 2026-05-03 03:09:59 +08:00
  • 5769b63082 Reduce CP shared KV materialize and direct-write overhead laoyao0822 2026-05-02 07:07:28 +08:00
  • 2317952a01 Document MLA prefix prefetch before Phase 8 implementation laoyao0822 2026-05-02 07:05:41 +08:00
  • 91fa31bcac Enable compute-owner KV layout by page-aligning NSA CP split laoyao0822 2026-05-01 00:54:16 +08:00
  • 47bd2fdf1f Reduce shared KV materialize synchronization laoyao0822 2026-04-28 04:09:19 +08:00
  • d31535589b Clarify the shared KV path toward compute-owned pages laoyao0822 2026-04-28 03:43:13 +08:00
  • d015e3fb01 Reduce CP shared KV current-chunk materialization laoyao0822 2026-04-26 23:29:34 +08:00
  • 5af232e9de Define the current-reuse boundary for shared KV Phase 3 laoyao0822 2026-04-26 05:57:33 +08:00
  • f8fca72635 Expand prefill CP KV capacity by sharding persistent NSA KV laoyao0822 2026-04-26 04:11:17 +08:00
  • 76c830dda8 docs(cp): plan NSA prefill CP shared KV phase2 laoyao0822 2026-04-24 16:31:03 +08:00
  • 7cc4e2dee8 docs(cp): design NSA prefill CP shared KV phase2 laoyao0822 2026-04-24 06:06:31 +08:00
  • e7a8c4b52f docs(cp): add NSA prefill CP dual all-gather explanation and Phase 1 plan laoyao0822 2026-04-22 04:55:15 +08:00
  • bc9a9f128b perf(cp): add narrow output collection for NSA prefill CP laoyao0822 2026-04-22 04:55:06 +08:00
  • 7403f14511 feat(kernel): add quanted silu fusion kernel laoyao0822 2026-04-13 22:02:24 +08:00
  • 91facd5bb2 perf(disaggregation): fast-path common decode result processing wxiwnd 2026-04-09 00:25:35 +08:00
  • 14c40f4e7d perf(disaggregation): slice decode waiting queue for prebuilt batches wxiwnd 2026-04-08 23:09:03 +08:00
  • 6237271fc9 perf(disaggregation): compact decode queues in one pass wxiwnd 2026-04-08 22:37:40 +08:00
  • 9470d6d8dc perf(disaggregation): compact prefill bootstrap queue in one pass wxiwnd 2026-04-08 21:59:16 +08:00
  • 09acec1908 perf(disaggregation): cache static buffer metadata wxiwnd 2026-04-08 20:58:19 +08:00
  • 53a04a9a97 perf(disaggregation): reuse req pool freelists and alloc_extend tensors wxiwnd 2026-04-08 20:12:47 +08:00
  • a371eacd86 fix(deepgemm): add nsa init in deepgemm init laoyao0822 2026-04-05 00:13:52 +08:00
  • d9b6e90b35 fix(dp_ctrl): fix imbalanced DP rank assignment laoyao0822 2026-03-26 00:12:46 +08:00
  • cc11cac77c fix(b_overlap): single batch overlap dim not match laoyao0822 2026-03-25 23:27:03 +08:00
  • 6efa9b4350 fix(disaggregation): check_status not found case crackdown laoyao0822 2026-03-25 21:21:39 +08:00
  • ef607c35c9 fix(spec_v2): support EAGLE with spec_v2 laoyao0822 2026-03-25 21:17:09 +08:00
  • 101100e25b feat(eplb): add eplb warmup-only mode laoyao0822 2026-03-25 21:21:34 +08:00
  • d5ec233834 feat(mtp): add tbo mtp support laoyao0822 2026-03-25 21:20:47 +08:00
  • 1c04d6078b feat(token_limit): limit input token < 131072 laoyao0822 2026-03-25 21:12:30 +08:00
  • 7c524ccacc feat(tool_call): changed return type of tool_call laoyao0822 2026-03-25 21:12:01 +08:00
  • 9b5ecc8c1f feat(drain): graceful shutdown and admin drain laoyao0822 2026-03-25 21:11:50 +08:00
  • 150d4db41d feat(scheduler): support scheduler recv skip in dp laoyao0822 2026-03-23 22:07:50 +08:00
  • 663459d2ef feat(deepgemm): deepgemm jit and init in model_runner leavelet 2026-03-23 22:06:55 +08:00
  • 2ae0237d3e fix(deepep): tuning and enhance deepep on single node leavelet 2026-03-23 22:05:02 +08:00
  • e72880b467 feat(eplb): add eplb warmup leavelet 2026-03-25 20:25:49 +08:00
  • 2d288ba8c9 [Bugfix] fix npu get kv_item_lens in PD separation when use ASCEND_US… (#15852) XDaoHong 2026-03-23 15:56:47 +08:00
  • 59cb9a9da6 [Spec][Ngram] 3/N: Fix synchronization issues in Ngram.cpp (#21186) kpham-sgl 2026-03-23 00:37:07 -07:00
  • 814202704b ci: unify PR test suite naming (#21187) Lianmin Zheng 2026-03-23 00:18:45 -07:00
  • 3d312643b9 [BUGFIX] Fix CP residual size mismatch crash when tp_size == attn_cp_size (#21170) yudian0504 2026-03-23 15:12:58 +08:00
  • 7757a9ddd0 ci: remove IS_BLACKWELL env var; auto-detect Blackwell (#21118) Lianmin Zheng 2026-03-22 23:44:48 -07:00