 Liangsheng YinandLianmin Zheng
|
836873b99f
|
Fix memory leak when aborting decode request in PD-Disagg (#9817)
Co-authored-by: Lianmin Zheng <15100009+merrymercy@users.noreply.github.com>
|
2025-08-30 14:36:03 +08:00 |
|
|
Liangsheng Yinandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
a23c30205d
|
Raise error when topk>1 and page>1 for paged attention backends. (#9784)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-29 12:47:34 +08:00 |
|
|
Liangsheng Yin
|
d0934a5192
|
gpt-oss blog reproduction document (#9728)
|
2025-08-28 10:15:08 +08:00 |
|
|
Liangsheng Yinandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
0ff7241995
|
Improve bench_one_batch_server script (#9608)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-26 10:38:37 +08:00 |
|
|
Liangsheng Yin
|
9b5f0f64f5
|
Fix tiny misalign with previous truncation setting in tokenizer_manager (#9430)
|
2025-08-21 14:05:35 +08:00 |
|
|
Liangsheng Yin
|
eb19ccadae
|
[bug] fix errors related to context length in SD (#9388)
|
2025-08-21 10:32:34 +08:00 |
|
|
Liangsheng Yinandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
08ebdf79d0
|
Fix the --allow-auto-truncate argument in tokenizer manager. (#9391)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-20 16:56:47 +08:00 |
|
|
Liangsheng Yin
|
4d98e48649
|
Revert "[Misc] feat: Deepgemm update for sgl-kernel (#8790)" to fix kernel CI (#9260)
|
2025-08-17 22:59:50 +08:00 |
|
|
Liangsheng Yin
|
0c8594e67d
|
Optional extension for green context (#9231)
|
2025-08-15 21:33:52 +08:00 |
|
|
Liangsheng Yin
|
445f9dca6e
|
Runtime check CUDA driver version to avoid unresolved green context symbols (#9021)
|
2025-08-12 09:26:10 -07:00 |
|
|
Liangsheng Yin
|
f9afa7dceb
|
Fix docs for clip max new tokens (#9082)
|
2025-08-11 13:15:21 -07:00 |
|
|
Liangsheng Yin
|
7a27e798ca
|
[CI] Do not trigger pd-disaggregation CI in draft PR (#8737)
|
2025-08-04 05:12:20 +08:00 |
|
|
Liangsheng Yin
|
9f47d686e5
|
Fix fused MoE when routed_scaling_factor is None (#8709)
|
2025-08-03 12:42:01 +08:00 |
|
|
Liangsheng Yin
|
603f5ce020
|
[Bug] fix green context's incompatibility with cuda < 12.4 (#8701)
|
2025-08-02 15:23:11 -07:00 |
|
|
Liangsheng Yin
|
f9f0138f80
|
Revert "[1/2] sgl-kernel: Fuse routed scaling factor into select_experts" (#8706)
|
2025-08-02 20:14:30 +08:00 |
|
|
Liangsheng Yin
|
e7dc163f57
|
add SVG logo (#8603)
|
2025-07-31 15:56:26 +08:00 |
|
|
Liangsheng Yin
|
76139bfba0
|
update mooncake in dockerfile (#7480)
|
2025-06-24 02:29:30 +08:00 |
|
|
Liangsheng Yin
|
25549433e8
|
Fix prefill OOM due to wrong token calculation when page > 1 (#7397)
|
2025-06-24 02:12:29 +08:00 |
|
|
Liangsheng Yin
|
05c9bc8956
|
[minor] simplify the TokenToKVPoolAllocator (#7414)
|
2025-06-22 12:37:18 +08:00 |
|
|
Liangsheng Yin
|
5ea5d22170
|
Fix CPU offloading for MLA memory pool (#7409)
|
2025-06-22 02:39:05 +08:00 |
|
|
Liangsheng Yin
|
c494386728
|
minor fix (#7245)
|
2025-06-16 23:30:26 +08:00 |
|
|
Liangsheng Yin
|
930746d93c
|
Improve log status (#7115)
|
2025-06-12 14:38:24 +08:00 |
|
|
Liangsheng Yinandfzyzcjy
|
78689d3393
|
PD Rust LB (PO2) (#6437)
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
|
2025-05-29 20:50:10 +08:00 |
|
|
Liangsheng Yin
|
17d080b7ae
|
Remove Cargo.lock, add it into .gitignore (#6438)
|
2025-05-20 12:01:32 +08:00 |
|
|
Liangsheng Yin
|
a3e4e9bf9e
|
Better PD initialization (#5751)
|
2025-05-07 01:12:57 +08:00 |
|
|
Liangsheng Yin
|
6d4d3bc81d
|
Fix not "import os" (#6057)
|
2025-05-06 22:06:41 +08:00 |
|
|
Liangsheng Yin
|
40d9b8acce
|
Improve overlap scheduling (#5788)
|
2025-04-28 11:19:16 +08:00 |
|
|
Liangsheng Yin
|
beb65c7433
|
[PD]Reduce kv transfer threads (#5791)
|
2025-04-27 23:03:30 +08:00 |
|
|
Liangsheng Yin
|
c55550cbf0
|
[PD] Better logs (#5715)
|
2025-04-25 17:25:45 +08:00 |
|
|
Liangsheng Yin
|
e69a219074
|
Enhance GPU memory settings (#5604)
|
2025-04-21 15:15:00 -07:00 |
|
|
Liangsheng Yin
|
33b16ad178
|
Distinguish bootstrap key only in decode server (#5422)
|
2025-04-15 20:59:28 +08:00 |
|
|
Liangsheng Yin
|
44afde82d7
|
Fix PD disaggregation bugs (#5326)
|
2025-04-14 19:27:30 +08:00 |
|
|
Liangsheng Yin
|
f765579046
|
Fix typo: infight -> inflight (#5357)
|
2025-04-14 01:25:30 +08:00 |
|
|
Liangsheng Yin
|
8616357a97
|
Fix deepseek awq v3 (#3450)
|
2025-02-12 22:09:52 +08:00 |
|
|
Liangsheng Yin
|
e7ebecf82e
|
Fix cache hit rate when chunked prefill (#2555)
|
2024-12-26 03:14:28 -08:00 |
|
|
Liangsheng Yin
|
5f12f0e7af
|
Fix chunked prefill when ignore eos (#2290)
|
2024-12-01 00:37:53 -08:00 |
|
|
Liangsheng YinandLianmin Zheng
|
b9fd178f1b
|
Fix retraction + overlap (#1860)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2024-10-31 18:27:42 -07:00 |
|
|
Liangsheng Yin
|
07bf2e846a
|
Allow consecutive ports when launching multiple sglang servers. (#1802)
|
2024-10-26 06:43:24 +00:00 |
|
|
Liangsheng Yin
|
a628dd8e31
|
Set ZMQ buffer size heuristic (#1801)
|
2024-10-25 23:15:56 -07:00 |
|
|
Liangsheng Yin
|
1e8903414a
|
Fix possible ZMQ hanging (#1800)
|
2024-10-25 23:07:07 -07:00 |
|
|
Liangsheng Yin
|
a2f5e7555f
|
Fix memory leak when doing chunked prefill (#1787)
|
2024-10-25 08:01:17 -07:00 |
|
|
Liangsheng Yin
|
3f5ac88d02
|
Fix out of memory message. (#1771)
|
2024-10-23 15:20:39 -07:00 |
|
|
Liangsheng Yin
|
5e1558f1f2
|
Update max_req_len and max_req_input_len (#1748)
|
2024-10-21 16:12:04 -07:00 |
|
|
Liangsheng Yin
|
94cde10920
|
Llama3.2 vision model support (#1551)
|
2024-10-21 15:01:21 -07:00 |
|
|
Liangsheng Yin
|
efb099cdee
|
Fix prefill oom (#1743)
|
2024-10-21 03:54:35 -07:00 |
|
|
Liangsheng Yin
|
9e0dac1ad7
|
Fix regex and logprob conflicts when chunked prefilling (#1703)
|
2024-10-17 18:33:21 -07:00 |
|
|
Liangsheng Yin
|
b6b4094621
|
Fix filter_batch function call (#1681)
|
2024-10-15 22:59:26 -07:00 |
|
|
Liangsheng Yin
|
5d0ba4038f
|
Refine the add request reasons to avoid corner cases. (#1574)
|
2024-10-04 18:00:18 -07:00 |
|
|
Liangsheng Yin
|
100f5b8bc9
|
Simplify flashinfer dispatch (#1552)
|
2024-10-01 00:28:42 -07:00 |
|
|
Liangsheng Yin
|
619bb6ddda
|
Dispatch flashinfer wrappers (#1550)
|
2024-09-30 23:12:36 -07:00 |
|
|
Liangsheng Yin
|
b88ea90d4a
|
Fix bugs of logprobs_nums (#1548)
|
2024-09-30 17:09:54 -07:00 |
|
|
Liangsheng Yin
|
99ec439da4
|
Organize Attention Backends (#1547)
|
2024-09-30 15:54:18 -07:00 |
|
|
Liangsheng Yin
|
55b974f96f
|
Process image in parallel (#1539)
|
2024-09-29 18:52:43 -07:00 |
|
|
Liangsheng Yin
|
fd9ad817ec
|
Organize image inputs (#1531)
|
2024-09-29 06:28:55 +00:00 |
|
|
Liangsheng Yin
|
4353acb469
|
minor: fix config (#1524)
|
2024-09-27 01:49:16 -07:00 |
|
|
Liangsheng Yin
|
7c162fa9c5
|
Fix schedule bug (#1451)
|
2024-09-17 22:59:32 -07:00 |
|
|
Liangsheng Yin
|
36078fb247
|
fix schedule bug (#1450)
|
2024-09-17 16:33:53 -07:00 |
|
|
Liangsheng Yin
|
70b6802982
|
Optimize conflicts between CUDA graph and vocab mask tensors (#1392)
|
2024-09-13 20:27:53 -07:00 |
|
|
Liangsheng Yin
|
4462137777
|
Add no commit to main rule (#1393)
|
2024-09-12 05:40:45 +08:00 |
|
|
Liangsheng Yin
|
144bc70fcc
|
Organize flashinfer indices update (#1378)
|
2024-09-10 17:38:59 -07:00 |
|
|
Liangsheng Yin
|
fbb4754cb8
|
Fix vocab mask update bug (#1376)
|
2024-09-10 13:10:36 -07:00 |
|
|
Liangsheng Yin
|
69b3bb9ae1
|
Unify forward mode (#1360)
|
2024-09-09 13:49:29 -07:00 |
|
|
Liangsheng Yin
|
05bea6883c
|
Fix some online scheduling delay (#1345)
|
2024-09-07 20:46:27 -07:00 |
|
|
Liangsheng Yin
|
ab4a83b259
|
Optimize schedule (#1339)
|
2024-09-05 14:30:26 -07:00 |
|
|
Liangsheng Yin
|
a5a134f39f
|
Fix bugs in sampler with CUDA graph / torch.compile (#1306)
|
2024-09-02 23:18:48 +00:00 |
|
|
Liangsheng Yin
|
47f20da223
|
Fix regex mask (#1296)
|
2024-09-01 21:50:58 -07:00 |
|
|
Liangsheng Yin
|
381dd57bd6
|
Sampler cudagraph (#1253)
|
2024-08-28 18:58:52 -07:00 |
|
|
Liangsheng Yin
|
1ece2cda3d
|
Fix bench latency benchmark (#1225)
|
2024-08-28 00:37:32 -07:00 |
|
|
Liangsheng Yin
|
c61a1b6f97
|
Torch compile CI throughput test (#1223)
|
2024-08-26 13:52:58 -07:00 |
|
|
Liangsheng YinandYineng Zhang
|
75ce37f401
|
Move sampler into CUDA graph (#1201)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2024-08-26 07:02:50 -07:00 |
|
|
Liangsheng Yin
|
632d506d0b
|
minor: improve CI and dependencies (#1212)
|
2024-08-26 04:26:31 +00:00 |
|
|
Liangsheng Yin
|
364d3d72a7
|
Fix broken penalty (#1184)
|
2024-08-22 08:16:35 +00:00 |
|
|
Liangsheng Yin
|
83e23c69b3
|
Improve code style of sampler (#1168)
|
2024-08-21 16:48:24 -07:00 |
|
|
Liangsheng Yin
|
5d0d40d0eb
|
Fix CI accuracy && time out limit (#1133)
|
2024-08-16 21:41:11 -07:00 |
|
|
Liangsheng Yin
|
f624f6a6cc
|
Fix port conflicts between local CI and runner CI. (#1131)
|
2024-08-16 15:12:38 -07:00 |
|
|
Liangsheng Yin
|
3694f8f996
|
Mixed style of chunked prefill (#1013)
|
2024-08-16 09:13:00 +00:00 |
|
|
Liangsheng Yin
|
73cf6834f2
|
Support stop_token_ids in sglang API (#1092)
|
2024-08-15 00:31:39 +00:00 |
|
|
Liangsheng Yinandzhyncs
|
a34dd86a7d
|
Use dtype to control generate (#1082)
Co-authored-by: zhyncs <me@zhyncs.com>
|
2024-08-14 15:58:07 +00:00 |
|
|
Liangsheng Yin
|
e205527cb1
|
Fix jump forward final state circular path bug. (#1084)
|
2024-08-13 21:14:05 -07:00 |
|
|
Liangsheng Yin
|
fb7421db0d
|
minor: some potential bugs (#1044)
|
2024-08-12 05:35:44 +00:00 |
|
|
Liangsheng Yin
|
7de6034534
|
Fix the prefix indices (#1037)
|
2024-08-11 17:57:02 -07:00 |
|
|
Liangsheng Yin
|
7b6a5332ca
|
Fix triton args init (#1034)
|
2024-08-11 12:11:26 -07:00 |
|
|
Liangsheng Yin
|
fcc0f5ed99
|
Fix wrong assert (#1028)
|
2024-08-11 09:22:16 +00:00 |
|
|
Liangsheng Yin
|
43fbb6d919
|
Fix input_ids && rename to fill_ids (#1021)
|
2024-08-10 16:24:12 -07:00 |
|
|
Liangsheng Yin
|
62757db6f0
|
Reduce the overhead when cache is disabled (#1010)
|
2024-08-09 16:36:57 -07:00 |
|
|
Liangsheng Yin
|
73fa2d49d5
|
Some warnings to crash when CI (#1009)
|
2024-08-09 15:16:23 -07:00 |
|
|
Liangsheng Yin
|
1ac304eeb4
|
Adjust InputeMetadata and ScheduleBatch (#981)
|
2024-08-08 01:11:22 -07:00 |
|
|
Liangsheng Yin
|
6ed4e3b8fb
|
Fix chunked prefill (#984)
|
2024-08-07 22:28:42 -07:00 |
|
|
Liangsheng Yin
|
2b8257f325
|
Adjust max prefix len (#980)
|
2024-08-08 00:41:26 +00:00 |
|
|
Liangsheng Yin
|
7623091d97
|
RadixCache method adjust (#977)
|
2024-08-07 15:52:24 -07:00 |
|
|
Liangsheng Yin
|
f724f1f1e9
|
PrefillAdder abstraction (#968)
|
2024-08-07 13:47:28 -07:00 |
|
|
Liangsheng Yin
|
4d929107ae
|
Run purge-cache only in sgl-project (#976)
|
2024-08-07 13:16:36 -07:00 |
|
|
Liangsheng Yin
|
fbe0c818c2
|
Purge self-runner's pip cache weekly (#975)
|
2024-08-07 12:43:12 -07:00 |
|
|
Liangsheng Yin
|
a01ddd9605
|
misc: fix the req_to_token member change (#967)
|
2024-08-07 01:52:10 -07:00 |
|
|
Liangsheng Yin
|
7fa54a1ab3
|
Make req_pool_indices on CPU (#960)
|
2024-08-07 01:41:25 -07:00 |
|
|
Liangsheng Yin
|
87e8c090e9
|
Organize code (rename, movement) (#953)
|
2024-08-06 20:50:32 -07:00 |
|
|
Liangsheng Yin
|
ad56e68495
|
Fix stuck in get_new_prefill_batch (#948)
|
2024-08-06 01:05:58 -07:00 |
|
|
Liangsheng Yin
|
bb66cc4c52
|
Fix CI && python3.8 compatible (#920)
|
2024-08-04 16:02:05 -07:00 |
|
|
Liangsheng Yin
|
947402c829
|
Reorder CI unit tests. (#908)
|
2024-08-03 16:18:50 -07:00 |
|
|
Liangsheng Yin
|
12ce3befb6
|
Update runner docs (#879)
|
2024-08-01 17:37:47 -07:00 |
|