chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Commit Graph

  • 50d1c2fc5a CP HiCache trace: compose gather-output + span-owner + descriptor + MoE-stage hashes leavelet 2026-06-17 04:03:47 +00:00
  • a447ae8317 CP HiCache trace: NSA indexer-K round-trip hash + forward-side attn/MoE hashes leavelet 2026-06-17 03:05:30 +00:00
  • e3cfba8441 CP HiCache trace: key KV-value hashes by host-slot fingerprint + zero-KV flag leavelet 2026-06-17 02:29:00 +00:00
  • 5ed20f493a CP HiCache trace: add KV-value round-trip fingerprint (backup vs reload) leavelet 2026-06-17 02:04:05 +00:00
  • 38ef664b4c Add env-gated CP HiCache KV round-trip corruption tracing leavelet 2026-06-17 01:14:31 +00:00
  • 34763d92b1 Cache CP shared-KV flattened request row ids per forward leavelet 2026-06-13 13:19:46 +00:00
  • 2c7353d061 Fix spec_v2 CP draft extend hidden-state shape mismatch leavelet 2026-06-13 09:06:00 +00:00
  • 1d168d061f Preserve pending CP HiCache backups through final insert laoyao0822 2026-06-13 03:44:23 +08:00
  • ceb5345410 Account decode handoff queues in DP load snapshots laoyao0822 2026-06-13 02:35:10 +08:00
  • 69ca7045ea Preserve chunked request affinity state laoyao0822 2026-06-13 02:34:55 +08:00
  • ee843a946b Keep chunked CP prefills solo during bs>1 admission laoyao0822 2026-06-13 01:28:35 +08:00
  • f290e1e35c Record CUDA IPC as CP compose main-stream choice laoyao0822 2026-06-13 01:17:50 +08:00
  • 254d667853 Keep CP batch slot descriptors on IPC-compatible runtime laoyao0822 2026-06-13 01:15:33 +08:00
  • e0c388076e Keep CP compose on CUDA IPC after syh rebasing laoyao0822 2026-06-13 01:02:56 +08:00
  • 4d5c7f32d6 Keep CP shared-KV prefetch warnings actionable laoyao0822 2026-06-12 08:11:51 +08:00
  • da8bd7f813 Record CPU-source limits for CP HiCache transfer laoyao0822 2026-06-12 08:11:14 +08:00
  • e6ab699f8f Record CE-first direction for CP shared-KV L1 prefetch laoyao0822 2026-06-12 06:02:13 +08:00
  • cd4412a4b8 Reuse IPC descriptors across CP shared-KV layers laoyao0822 2026-06-12 05:03:53 +08:00
  • d7eb90dff2 Record compact IPC staging capacity contract laoyao0822 2026-06-12 04:15:55 +08:00
  • bafb55044b Keep CP current IPC staging proportional to touched pages laoyao0822 2026-06-12 03:49:10 +08:00
  • 142e7a5a64 Keep CP shared-KV fast paths off dense current collectives laoyao0822 2026-06-12 03:07:47 +08:00
  • 2387787ebc Fail fast when CP compose would hide dense fallback laoyao0822 2026-06-12 23:57:40 +08:00
  • 9d65bdba95 Measure mqa-logits chunk pipelining: not worth it (S2b closed) leavelet 2026-06-12 08:27:19 +00:00
  • cf46defddd Harden the affinity wiring per opus review (F-1, F-2) leavelet 2026-06-12 08:07:36 +00:00
  • 6a1e862f48 Group cache-hit prefills into dense batches (SGLANG_CP_PREFILL_AFFINITY_GROUP) leavelet 2026-06-12 07:44:39 +00:00
  • 54c056af83 Let tail chunks co-batch with cache-hit requests (SGLANG_CP_PREFILL_MIX_CHUNKED) leavelet 2026-06-12 06:38:47 +00:00
  • f9ce28ee5c Add mqa-logits chunking cost micro-benchmark (专题 S2a) leavelet 2026-06-12 06:16:41 +00:00
  • a75164ef48 Demote the P/D source-fingerprint mismatch to a warning leavelet 2026-06-12 05:15:11 +00:00
  • 58f2350738 Give prefill inflight transfers a liveness bound leavelet 2026-06-12 04:55:35 +00:00
  • d66758d929 Run the symm staging fill on byte views (no fp8 index_copy kernel) leavelet 2026-06-12 04:02:19 +00:00
  • 9c6cf278b0 Seed the IPC agreement before prefetch hit/miss can diverge leavelet 2026-06-11 22:56:31 +00:00
  • 1923fcd67e Wire the symm current exchange into the CP shared-KV prefetchers leavelet 2026-06-11 22:39:47 +00:00
  • d63fbd4d79 Fold the symm compose into one barrier-gated mega gather leavelet 2026-06-11 22:09:29 +00:00
  • e8e5edb230 Shrink the symm compose region to a compact current-page staging leavelet 2026-06-11 21:50:34 +00:00
  • f74f0a557f Keep CP pending-split probes collective-free laoyao0822 2026-06-11 20:27:06 +08:00
  • a24111a5f4 Cut per-layer CPU on the prefill launch path: validators, plans, spans leavelet 2026-06-11 21:01:00 +00:00
  • b356773d2f Harden symm compose per review: prefetch gate, round parity, hot-path CPU leavelet 2026-06-11 20:26:27 +00:00
  • c55e406176 Add symm-heap current-page exchange for CP shared-KV compose (Step B) leavelet 2026-06-11 20:13:04 +00:00
  • f47b739e30 Restructure bs>1 CP shared-KV compose to one gather + one collective leavelet 2026-06-11 19:16:28 +00:00
  • ecf7998c80 Route undrained-ack reservation refusal around the capacity retry leavelet 2026-06-11 10:41:39 +00:00
  • 47cb9b9d11 Make the duplicate-reservation ack gate O(1) for fresh node ids leavelet 2026-06-11 10:34:11 +00:00
  • 698a3e2431 Enforce one in-flight CP write ack per node at the producer leavelet 2026-06-11 10:22:26 +00:00
  • dfa168abe9 Prevent duplicate CP HiCache write acks from crashing split drain laoyao0822 2026-06-11 17:46:22 +08:00
  • 75d7d8772e Unify over-length errors into the PayloadTooLargeError 413 format leavelet 2026-06-11 08:01:02 +00:00
  • d01601c171 Demote per-chunk CP per-layer transfer register log to debug leavelet 2026-06-11 06:25:37 +00:00
  • 9c8e3e99cb Align OpenAI serving behavior with Para deployments laoyao0822 2026-06-11 05:59:08 +08:00
  • 8cfdb0466e Reduce CP per-layer transfer success-log noise laoyao0822 2026-06-11 05:49:05 +08:00
  • 7284a469a2 Reuse prepared HiCache load descriptors across CP prefill layers laoyao0822 2026-06-11 05:09:41 +08:00
  • adf357b02c Default CP bs>1 extend admission to chunk budget laoyao0822 2026-06-11 03:51:41 +08:00
  • d696039092 Align MQA logits admission with CP split runtime shape laoyao0822 2026-06-11 03:51:20 +08:00
  • 250fab291d Account for MQA logits in CP batch admission laoyao0822 2026-06-11 03:25:22 +08:00
  • ddc1233955 Bound CP MQA logits buffers with row chunking laoyao0822 2026-06-11 03:15:52 +08:00
  • e0ea8a485c Revert NSA MQA logits chunking while performance is unresolved laoyao0822 2026-06-11 03:04:03 +08:00
  • 4e49751406 Bound NSA MQA logits peak memory laoyao0822 2026-06-11 03:03:01 +08:00
  • cc908dd556 Reuse DeepGEMM JIT cache across restarts laoyao0822 2026-06-11 02:55:04 +08:00
  • 3a43727216 Bound CP prefill batching by estimated temp memory laoyao0822 2026-06-11 01:33:28 +08:00
  • 9a9893e571 Model CP scheduler admission with cache-hit pressure laoyao0822 2026-06-10 22:21:59 +08:00
  • 4f65d7a176 Bound CP prefill batches by cached-token pressure laoyao0822 2026-06-10 22:21:46 +08:00
  • 342c552ab3 Decouple CP prefill batching from generic token cap laoyao0822 2026-06-10 21:36:22 +08:00
  • afdd1d0992 Demote per-request load-back log to debug leavelet 2026-06-10 10:22:35 +00:00
  • 3eef989739 Harden IndexCache for first e2e: validation, fail-fast, prefetch gating leavelet 2026-06-10 06:17:52 +00:00
  • 5bec69f40a Drop redundant np.isin from CP-rank KV page filter leavelet 2026-06-10 05:15:16 +00:00
  • b742371fe3 Skip get_load() snapshot when no DP load-balancer consumes it leavelet 2026-06-10 05:12:40 +00:00
  • e6864fbea7 Demote HiCache/CacheCtrl per-op logs from info to debug leavelet 2026-06-10 05:10:45 +00:00
  • d221212565 Batch EAGLE spec output D2H copies in disagg prefill results leavelet 2026-06-10 05:07:51 +00:00
  • 87a22b17ce Fuse SamplingBatchInfo tensor construction into one pass leavelet 2026-06-10 05:04:39 +00:00
  • 0d065a8ab0 Speed up Python bigram key conversion with C-level zip leavelet 2026-06-10 04:30:59 +00:00
  • 8979c81a22 Make RadixKey slicing zero-copy to fix quadratic match_prefix leavelet 2026-06-10 04:28:03 +00:00
  • ffff715f00 Plan bs>1 CP shared-KV L1 prefetch safely laoyao0822 2026-06-10 07:04:39 +08:00
  • f78c414b64 Skip unused FP8 NSA K-cache dequant work laoyao0822 2026-06-10 07:00:43 +08:00
  • c8c736e75c new pyprojectoml laoyao0822 2026-06-10 05:54:43 +08:00
  • af20334f4c Size CP HiCache host budget from compact index layers laoyao0822 2026-06-10 05:53:05 +08:00
  • 1ebde44e59 Compact skipped NSA index-cache state safely laoyao0822 2026-06-10 05:37:06 +08:00
  • d21952b903 Reduce inactive NSA index-cache transfer safely laoyao0822 2026-06-10 04:28:26 +08:00
  • 6229c7da60 Plan index-skip cache and transfer reduction laoyao0822 2026-06-10 04:28:26 +08:00
  • a32050d1d8 Expose NSA index sharing as launch-time overrides laoyao0822 2026-06-10 03:24:33 +08:00
  • 24da983ff5 Enable CP HiCache direct transfers to use layer-page host layout laoyao0822 2026-06-10 02:03:07 +08:00
  • 5e22279670 Carry CP-local hidden contract into EAGLE v2 prefill draft extend laoyao0822 2026-06-09 22:42:54 +08:00
  • df2a3696cd Gate CP split validation to eligible extend forwards laoyao0822 2026-06-09 19:54:59 +08:00
  • 81eb138a26 Avoid CP HiCache owner-lane planning stalls during load-back laoyao0822 2026-06-09 06:55:15 +08:00
  • 50fde834ae Keep CP compute padding out of sparse MoE laoyao0822 2026-06-09 01:48:24 +08:00
  • 4d2fdd14ac Keep kernel guard aligned with deployment env laoyao0822 2026-06-09 00:45:54 +08:00
  • eabe0bbfad Preserve CP slot-remap reference paths after SYH pick laoyao0822 2026-06-09 00:19:15 +08:00
  • b35495a28d chore(work): journal bs>1 contamination fix + B300 P/D validation leavelet 2026-06-08 13:22:34 +00:00
  • 165b6a01f1 fix(nsa): correct bs>1 CP shared-KV KV per-request page_inverse + CP/EAGLE compute-padding leavelet 2026-06-08 13:22:34 +00:00
  • 81a0191331 perf(deepgemm): CP-aware dense warmup M-grid for NSA prefill CP leavelet 2026-06-08 13:22:34 +00:00
  • 134b88b9b4 fix(server_args): keep NSA prefill/decode backends in the same family for fp8 leavelet 2026-06-08 13:22:34 +00:00
  • 98fdae3375 fix(config): restore GlmMoeDsa index_topk_freq dropped by HF config load leavelet 2026-06-07 19:32:26 +00:00
  • ce3b5bc874 feat(nsa): port IndexCache (indexer topk sharing across layers) leavelet 2026-06-07 19:26:50 +00:00
  • 4cf8885a74 perf(nsa): cache MQA-logits budget + drop indexer rope clones leavelet 2026-06-07 18:59:28 +00:00
  • db5ab48b19 feat(deepgemm): migrate to sgl-deep-gemm 0.1.2 wheel API leavelet 2026-06-07 17:33:32 +00:00
  • 430ed85083 Record CP cache-hit GSM8K findings and cleanup state laoyao0822 2026-06-08 20:57:30 +08:00
  • a32c639147 Remove temporary CP cache-hit trace instrumentation laoyao0822 2026-06-08 20:57:08 +08:00
  • 3698b0c22c Lock bs>1 CP compute-padding invariants laoyao0822 2026-06-08 20:56:48 +08:00
  • b58513cba4 Preserve request-slot identity for CP cache-hit reuse laoyao0822 2026-06-08 20:55:58 +08:00
  • fb5ccaff26 Preserve CP shared-KV cache-hit diagnostics laoyao0822 2026-06-08 03:39:50 +08:00
  • 2aa0b7313e Fix RAGGED CP cache-hit current KV reuse laoyao0822 2026-06-08 03:39:34 +08:00
  • b324407def Trace CP HiCache cache-hit state for GSM8K regressions laoyao0822 2026-06-07 23:56:37 +08:00
  • ff4717c2fb Keep L1 free-room eviction off the CP load-back hot path laoyao0822 2026-06-07 23:56:04 +08:00
  • 8f23b06427 Lock fp8 CP cache-hit roundtrips across batch requests laoyao0822 2026-06-07 23:53:42 +08:00