chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
5,112 Commits 1 Branch 0 Tags
f3b6760213edaefc9564f2535309e07be4611f49
Commit Graph

476 Commits

Author SHA1 Message Date
Lianmin Zheng
f3b6760213 [Auto Sync] Update server_args.py (20250906) (#10117) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-09-06 16:59:36 -07:00
DevashishLal-CB
13705dae06 [Fix] Add speculative_draft_model_revision to server_args (#5255) 
Signed-off-by: Devashish Lal <devashish@rivosinc.com>
 2025-09-05 19:45:46 +08:00
fzyzcjy
df97b31f37 Tiny support setting numa nodes for different ranks (#10006) 2025-09-05 19:01:27 +08:00
Liangsheng Yin
6e95f5e5bd Simplify Router arguments passing and build it in docker image (#9964) 2025-09-05 12:13:55 +08:00
Yingchun Lai
b32ab0705e metrics: support customer buckets for prompt/generation_tokens_histogram (#9634) 2025-09-04 22:22:08 +08:00
Hubert Lu
2c562fd2d0 Fix Llama 4 with MXFP4 dynamic quant on MI35x (#9993) 2025-09-04 00:48:58 -07:00
Yineng Zhang
2c7ca33abb Revert "[Fix] DeepSeek EP accuracy issue on B200 GPUs (#9946)" (#9955) 2025-09-02 23:49:56 -07:00
Al-Ekram Elahee Hridoy
6243c36702 [Fix] DeepSeek EP accuracy issue on B200 GPUs (#9946) 2025-09-02 19:31:15 -07:00
Lianmin Zheng
60e37f8028 Move parsers under a single folder (#9912) 2025-09-02 18:25:04 -07:00
ybyang
5f77e1292d Support Multi Process Tokenizer Manager(#6555) (#8964) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-01 01:00:13 -07:00
hzh0425
c2a26e725c feature(eplb): add min-rebalancing-utilization-threshold for eplb (#8345) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-08-30 11:24:29 +08:00
Liangsheng Yin
a23c30205d Raise error when topk>1 and page>1 for paged attention backends. (#9784) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-29 12:47:34 +08:00
Zhiqiang Xie
001f51940a [HiCache] change the default policy to write through (#9772) 2025-08-28 18:28:39 -07:00
Lianmin Zheng
fce7ae33f8 [Sync] Update server_args.py (20250828) (#9745) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-08-28 10:33:00 -07:00
Lianmin Zheng
fd71b11b1d move is_sm90_supported/is_sm100_supported to python/sglang/srt/utils.py (#9679) 2025-08-27 03:34:29 -07:00
hzh0425
c04c17edfa refactor(hicache): Introduce generic HiCacheStorageConfig for improved configuration management (#9555) 
Co-authored-by: Teng Ma <805522925@qq.com>
 2025-08-26 17:55:20 -07:00
Jonas
a0a77d937b Fix Harmony reasoning parser for and auto-separation for gpt-oss models (#9190) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang20 <zhaochenyang20@gmail.com>
Co-authored-by: minleminzui <2969413251@qq.com>
Co-authored-by: maocheng23 <maocheng@berkeley.edu>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-25 15:26:26 -07:00
Yineng Zhang
ebd9dbe71b fix: revert #8593 (#9581) 2025-08-25 01:29:06 -07:00
fzyzcjy
2600fc0d47 Overlapped weight offload (#8034) 2025-08-23 02:06:46 -07:00
fzyzcjy
0374304a2c Add enable_flashinfer_mxfp4_bf16_moe for higher precision and slower moe backend (#9004) 2025-08-23 15:38:40 +08:00
Chanh Nguyen
127d4b0d5e Support GC Freezing to improve latency & throughput (#9241) 
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2025-08-23 13:43:09 +08:00
Hubert Lu
f445a1d9a3 [AMD] Fix Llama 4 FP8 accuracy issues on MI300X (#7699) 2025-08-22 13:13:45 -07:00
Xuchun Shang
cebf45994b [bugfix] Make --enable-hierarchical-cache and --disable-radix-cache mutually exclusive (#9452) 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
 2025-08-22 17:49:52 +08:00
Qiaolin Yu
9ec314c6ac Support speculative decoding in the trtllm_mha attention backend (#9331) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2025-08-21 23:53:35 -07:00
Pavani Majety
3cc3d9b950 Add Support for Page Size greater than 1 for Flashinfer MLA Backend (#8593) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-08-21 18:15:06 -07:00
Yineng Zhang
0f9318f7d0 feat: update auto_choose_speculative_params (#9470) 2025-08-21 17:12:12 -07:00
pranavm-nvidia
64574ef8c0 Enables speculative decoding for the trtllm_mla attention backend (#9238) 2025-08-21 01:18:21 -07:00
Nicolas Castet
c10b8e6a0f Support DP attention with GPT-OSS (#9359) 2025-08-20 16:36:31 -07:00
Lifu Huang
b0980af89f Support pinning adapter via server args. (#9249) 2025-08-20 16:25:01 -07:00
fzyzcjy
42c8704560 Add PDL support for quant kernel and rope kernel (#9106) 2025-08-20 01:56:29 -07:00
Lianmin Zheng
1ec9769753 [Docs] Update contribution guide (#9383) 2025-08-19 23:37:45 -07:00
Keyang Ru
886454e8e7 [MISC] use dynamic choices for tool-call-parser argument (#9316) 2025-08-18 13:02:10 -07:00
kk
1c1f8a118e Combine fp4.py and mxfp4.py into one file and support dynamic mxfp4 quantization in mxfp4.py (#9049) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-08-16 19:01:54 -07:00
Trevor Morris
eff4eb3fdd Add fp4 quantize before all-gather for Flashinfer cutlass MoE DP (max throughput) (#7667) 2025-08-15 22:08:11 -07:00
Cheng Wan
e3e75a786a Fix the deprecation warning for enable_flashinfer_mxfp4_moe (#9214) 2025-08-14 23:59:35 -07:00
Cheng Wan
295895120d [6/N] MoE Refactor: Cleanup MoE-related configs (#8849) 2025-08-14 21:14:53 -07:00
Yineng Zhang
27985c27aa feat: update model config (#9202) 2025-08-14 15:15:27 -07:00
Chengxing Xie
c1c7dc4534 feat: Add model version tracking with API endpoints and response metadata (#8795) 2025-08-14 12:13:46 -07:00
Lianmin Zheng
9e426466af Clean up allocators (#9134) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-13 13:56:04 -07:00
Ke Bao
0ff6d1fce1 Support FA3 backend for gpt-oss (#9028) 2025-08-13 10:41:50 -07:00
huangtingwei
0edda32001 Support page first layout zero copy for mooncake store (#8651) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-08-12 15:59:26 -07:00
jacky.cheng
25caa7a8a9 [AMD] Support Wave attention backend with AMD GPU optimizations (#8660) 
Signed-off-by: Stanley Winata <stanley.winata@amd.com>
Signed-off-by: Harsh Menon <harsh@nod-labs.com>
Signed-off-by: nithinsubbiah <nithinsubbiah@gmail.com>
Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
Signed-off-by: xintin <gaurav.verma@amd.com>
Co-authored-by: Harsh Menon <harsh@nod-labs.com>
Co-authored-by: Stanley Winata <stanley.winata@amd.com>
Co-authored-by: Stanley Winata <68087699+raikonenfnu@users.noreply.github.com>
Co-authored-by: Stanley Winata <stanley@nod-labs.com>
Co-authored-by: Ivan Butygin <ivan.butygin@gmail.com>
Co-authored-by: nithinsubbiah <nithinsubbiah@gmail.com>
Co-authored-by: Nithin Meganathan <18070964+nithinsubbiah@users.noreply.github.com>
Co-authored-by: Ivan Butygin <ibutygin@amd.com>
 2025-08-12 13:49:11 -07:00
Chang Su
a218490136 (gpt-oss, oai, chat): Remove Harmony Integration and Implement Native GPT-OSS Tool Call Support (#9043) 2025-08-11 18:59:18 -07:00
Faraz
f508cd3cb7 TRTLLM-MLA FP8 path (#8638) 
Signed-off-by: Faraz Khoubsirat <58580514+farazkh80@users.noreply.github.com>
 2025-08-11 14:02:13 -07:00
Xiaoyu Zhang
44e86480e8 fuse allreduce and residual_rmsnorm (#8731) 2025-08-11 13:50:53 -07:00
633WHU
3bffe11279 Fix chunked prefill size validation for disabled state (#8973) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-11 11:05:29 -07:00
Baizhou Zhang
75e6a7cde1 Support radix cache for Lora feature (#7216) 2025-08-11 10:14:11 -07:00
Lianmin Zheng
2449a0afe2 Refactor the docs (#9031) 2025-08-10 19:49:45 -07:00
Lianmin Zheng
706bd69cc5 Clean up server_args.py to have a dedicated function for model specific adjustments (#8983) 2025-08-08 19:56:50 -07:00
Lianmin Zheng
a947154286 Revert "Support Multi Process Tokenizer Manager" (#8960) 2025-08-08 02:28:27 -07:00