Yuwei An
|
4ac8e09df0
|
Piecewise CUDA Graph Support & Torch Compile Backend (#10062)
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
|
2025-10-12 11:55:57 +08:00 |
|
 
|
20a6c0a63d
|
Beta spec-overlap for EAGLE (#11398)
Co-authored-by: Lianmin Zheng <15100009+merrymercy@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2025-10-12 11:02:22 +08:00 |
|
 Lorenzo LuandChang Su
|
b5dcfd4154
|
Add option to disable any_whitespace for xgrammar and llguidance backends. (#8919)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-10-11 22:24:58 +08:00 |
|
ykcombat
|
f5754d1256
|
[Documentation][Configuration] Server args and documentation of PD-Multiplexing. (#11427)
|
2025-10-11 21:36:07 +08:00 |
|
Zhiyu
|
129d299278
|
Enable native ModelOpt quantization support (2/3) (#9991)
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
|
2025-10-11 07:48:14 +00:00 |
|
Lianmin Zheng
|
9b8ebb2798
|
move more files under srt/utils (#11285)
|
2025-10-09 16:46:15 -07:00 |
|
Trevor Morris
|
a4b424c632
|
[DeepSeek-V3.2] Include indexer kv cache when estimating kv cache size (#11309)
|
2025-10-08 23:59:46 -07:00 |
|
Sundara Raman Ramachandran
|
53bd00d975
|
[Generative Score API] Multi-Item scoring with custom attention mask. (#10979)
|
2025-10-08 18:47:32 -07:00 |
|
 Lifu HuangandHieu Pham
|
edefab0c64
|
[2/2] Support MHA prefill with FlashAttention 4. (#10937)
Co-authored-by: Hieu Pham <hyhieu@gmail.com>
|
2025-10-08 00:54:20 -07:00 |
|
Cheng Wan
|
3c06b673af
|
[8/N] MoE Refactor: deprecate EPMoE (#11211)
|
2025-10-07 21:51:41 -07:00 |
|
Chang Su
|
7ba3de0e92
|
[oai serving chat] Add argument --sampling-defaults and fix ChatCompletionRequest defaults (#11304)
|
2025-10-08 00:36:05 +00:00 |
|
Xinyuan Tong
|
e3c7f09146
|
Update tool parser and related documentation (#11223)
|
2025-10-07 11:03:40 -07:00 |
|
Lianmin Zheng
|
708f4ff490
|
Rename max_micro_batch_size -> pp_max_micro_batch_size (#11279)
|
2025-10-06 15:50:56 -07:00 |
|
Zhiyu
|
155cbb51f0
|
Enable native ModelOpt quantization support (1/3) (#7149)
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
|
2025-10-06 13:24:15 -07:00 |
|
+2        
|
efbc687c28
|
Support DeepSeek V3.2 Exp (#11061)
Co-authored-by: Stefan He <11166516+hebiao064@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <95566987+hnyls2002@users.noreply.github.com>
Co-authored-by: Baizhou Zhang <56809903+fridge003@users.noreply.github.com>
Co-authored-by: DarkSharpness <76582120+darksharpness@users.noreply.github.com>
Co-authored-by: ZhengdQin <46387172+zhengdqin@users.noreply.github.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-10-06 00:24:15 -07:00 |
|
 Yuan Luoandluoyuan.luo
|
590f2da052
|
[Feat] Support Torch Symm Mem AllReduce (#10571)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
|
2025-10-05 13:55:19 -07:00 |
|
fzyzcjy
|
fdc4e1e570
|
Tiny move files to utils folder (#11166)
|
2025-10-03 22:40:06 +08:00 |
|
 Matt Nappoandmolocule
|
8c57490210
|
[Feature] Option to save model weights to CPU when memory saver mode is enabled (#10873)
Co-authored-by: molocule <34072934+molocule@users.noreply.github.com>
|
2025-10-03 16:48:19 +08:00 |
|
b8zhong
|
a2faf8940c
|
[1/n] Enable DCA CUDA graph capture (#9537)
|
2025-10-03 11:30:00 +08:00 |
|
Lianmin Zheng
|
6a261aaca5
|
Minor fixes for server_args, parallel_state, and test_deterministic.py (#11159)
|
2025-10-02 12:12:49 -07:00 |
|
b8zhong
|
948278f173
|
fix cpp JIT compilation issue of ngram speculative decoding (#10837)
|
2025-10-02 21:05:01 +08:00 |
|
fzyzcjy
|
5e786cca3a
|
Support single batch overlap (#10422)
|
2025-10-02 18:04:36 +08:00 |
|
Liangsheng Yin
|
25e7dbe8af
|
Fix ngram spec with page size > 1 (#11135)
|
2025-10-02 12:34:23 +08:00 |
|
 Shisong Maandmashisong
|
86cb4db058
|
[Feature] Add EIC as sglang HiCache Storage backend (#10271)
Co-authored-by: mashisong <mashisong@bytedance.com>
|
2025-10-01 08:43:34 -05:00 |
|
Trevor Morris
|
a6cc86df9d
|
Fix DSR1 accuracy for flashinfer_trtllm MoE with FP8 quantization (#11081)
|
2025-09-30 10:33:12 -07:00 |
|
Zaili Wang
|
229d2b95f1
|
[CPU] Adding Memory Capacity Acquisition Functionality (#11102)
|
2025-09-30 08:41:20 -07:00 |
|
narutolhy
|
d17986f8c6
|
Enable optional FP32 compute for LM Head (#10729)
Thanks to MiniMax Team and Chenyang Zhao's support.
|
2025-09-29 20:45:17 -07:00 |
|
 li-kesenandYi Zhang
|
2bc61dd194
|
Remove hybrid_linear_attn attention backend and refactor attention registry (#10816)
Co-authored-by: Yi Zhang <1109276519@qq.com>
|
2025-09-30 10:16:16 +08:00 |
|
Lianmin Zheng
|
dda34c2f93
|
Fix mem fraction static for nightly tests (#11076)
|
2025-09-29 12:57:41 -07:00 |
|
 Lianmin Zhengandsglang-bot
|
a17e70f5cc
|
Use more general heuristics to set the default value of --mem-fraction-static (#10975)
Co-authored-by: sglang-bot <sglangbot@gmail.com>
|
2025-09-29 10:11:03 -07:00 |
|
 Zhihao Zhanganda4zhangfei
|
24f7cb1ece
|
[speculative decoding] rename lookahead to ngram (#11010)
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
|
2025-09-28 21:06:59 -07:00 |
|
huangtingwei
|
e05555fad8
|
[HiCacheStorage] mooncake store support page_first_direct layout (#10591)
|
2025-09-28 20:45:48 -07:00 |
|
Mick
|
2e7633982c
|
fix: show failed models in nightly ci (#10986)
|
2025-09-28 12:38:29 -07:00 |
|
![gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)   
|
62e2e99db6
|
fix: make inference deterministic for large TP (#10930)
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-09-27 02:46:45 -07:00 |
|
 hzh0425andTeng Ma
|
c8a5d12abe
|
[HiCache]: Support dynamic loading backends for hicache (#10551)
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
|
2025-09-26 18:34:11 -07:00 |
|
 
|
05a3526654
|
Restruct gpu_memory_settings in a unify function and relax max_cuda_graph_bs (#10372)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: sglang-bot <sglangbot@gmail.com>
|
2025-09-26 15:10:49 -07:00 |
|
 hzh0425andShangming Cai
|
7ec5b4e89c
|
[PD-HiCache]: Support Async Offloading KVCache In Decode Side (#10192)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2025-09-25 23:20:49 -07:00 |
|
![gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
f68dd998b9
|
Rename customer label -> custom label (#10899)
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-09-25 16:19:53 -07:00 |
|
yi wang
|
fce170480a
|
integrate AIBrix KVcache (#10376)
|
2025-09-25 14:47:09 +08:00 |
|
 
|
d7b20dd65d
|
chore: Initial support for input config files (#10534)
Co-authored-by: root <root@umbriel-b200-017.ipp4a1.colossus.nvidia.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-09-24 14:45:52 -07:00 |
|
Lianmin Zheng
|
ea338676b5
|
Clean up server args (#10770)
|
2025-09-23 00:22:32 -07:00 |
|
 ronnie_zhengandwangqihui01
|
095093ee5a
|
[Ascend] optimize Qwen-vl on Ascend (#10556)
Co-authored-by: wangqihui01 <wangqh10@163.com>
|
2025-09-22 17:18:16 -07:00 |
|
 Baizhou ZhangandYineng Zhang
|
aa1c5cf5bd
|
Add warnings and remove dependency for deterministic inference (#10724)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-09-22 10:56:02 -07:00 |
|
 
|
e2ac7888b8
|
[2/2] Support deterministic inference for temperature > 0 (#10678)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
|
2025-09-21 19:36:08 -07:00 |
|
Ethan (Yusheng) Su
|
134b4f7ec2
|
Support deterministic inference with triton backend (#10694)
|
2025-09-22 09:20:40 +08:00 |
|
Lifu Huang
|
08ecd0aa2a
|
[3/4] Speed up CSGMV backend perf by 10% through dynamic chunking + kernel optimization (#10592)
|
2025-09-20 22:47:48 -07:00 |
|
fzyzcjy
|
720c1c8ca3
|
Super tiny fix extra logs (#10697)
|
2025-09-20 21:30:54 -07:00 |
|
![gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Yineng Zhangandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
d403c143e3
|
feat: update server args (#10696)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-09-20 18:52:19 -07:00 |
|
Stefan He
|
cba0d8c309
|
[Feature] Support deterministic inference with FA3 backend (#10651)
|
2025-09-20 17:50:21 -07:00 |
|
 
|
8ecef73f12
|
[1/2] Support deterministic inference with flashinfer attention backend (#10645)
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
|
2025-09-19 23:34:29 -07:00 |
|