chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
7,966 Commits 1 Branch 0 Tags
726fe3e72e181262ffb65f621cdd855c63393fe9
Commit Graph

44 Commits

Author SHA1 Message Date
Yuan Luo
3912ee4991 [VLM] feat: support chunked vit attention (#14907) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-15 12:11:02 +08:00
Tianyu Guo
9acb21ae27 feat: support EPD disaggregation (#12263) 
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: Nicholas <45984215+liusy58@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2025-12-14 22:30:08 +08:00
Nicholas
f57d4fe78e [feat] use cachebuffer to store mm feature to speedup hash (#14386) 2025-12-08 10:35:20 +08:00
Lianmin Zheng
ca52ed425f Clean up imports and move files (#14317) 2025-12-02 16:31:54 -08:00
kun-llfl
106df4eac5 Fix mrope_positions size when req is retracted (#13700) 
Signed-off-by: Kun(llfl) <i@imux.top>
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-02 11:38:20 +08:00
Byron Hsu
edbeaf3b88 [MM][style] rename inputs_embeds to input_embeds for consistency (#14240) 2025-12-01 11:36:51 -08:00
Byron Hsu
0825d7f4c6 [piecewise] Refactor VLM to support input embed buffer and remove external embedder hack (#14155) 2025-11-30 21:43:09 -08:00
Yuan Luo
475962a139 [VLM] Support Piecewise CUDA Graph for InternVL (#13640) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-21 13:21:51 +08:00
Yuan Luo
af6bcadcf7 [VLM] Support Piecewise CUDA Graph for Qwen2.5-VL (#13055) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Yuhao Yang <yhyang201@gmail.com>
 2025-11-20 10:23:44 +08:00
Yuan Luo
706502ff6c [VLM] Support PP for Qwen2.5-VL (#13075) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2025-11-12 23:18:44 +08:00
Yuhao Yang
1240ac13b8 vlm: fix tiny multimodal cache bug (#12984) 2025-11-10 21:30:19 +08:00
Mick
f5b3ccd9a5 feat: basic support for server-level multimodal cache (#10775) 2025-11-10 00:27:50 +08:00
kousakawang
7efd8b3d1f [FEAT] Shared mem pool based cuda ipc for multi-modal data transport (#11917) 
Co-authored-by: kousakawang <wanghanpei@bytedance.com>
Co-authored-by: Yuan Luo <4908075+yuan-luo@users.noreply.github.com>
 2025-11-02 16:46:37 +08:00
Mick
86b04d25b3 model: qwen3-omni (thinker-only) (#10911) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-10-16 13:20:38 -07:00
Liangsheng Yin
516738b096 Depreate global_server_args_dict (#11528) 2025-10-13 19:34:43 +08:00
Cheng Wan
1bdd010291 Revert "Deprecate global_server_args_dict" (#11520) 2025-10-12 17:40:40 -07:00
Liangsheng Yin
1083e7e3df Deprecate global_server_args_dict (#11331) 2025-10-13 01:20:47 +08:00
Zheng Li
4f564b9e83 model: support qwen3-vl series (#10323) 
Co-authored-by: ocss884 <ocss.lin@gmail.com>
Co-authored-by: cao1zhg <653506626@qq.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: 瑀澈 <yuche.lz@alibaba-inc.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-23 10:15:52 -07:00
Lzhang-hub
37d83c6e6d Qwen2.5-VL eagle3 infer (#8801) 2025-09-07 20:44:34 -07:00
VDV1985
ba861293cf [feat]Ascend NPU Gemma-3-12b and Gemma-3-27b support (#8909) 2025-08-31 00:25:07 -07:00
Binyao Jiang
66d6be0874 Bug fix: use correct mm_items in embed_mm_inputs (#8893) 2025-08-16 19:55:56 -07:00
Binyao Jiang
7b81f956eb Fix qwen2 audio not working bug (#8600) 2025-08-09 00:42:29 -07:00
Zheng Wengang
2d120f8b18 [Feature][Multimodal] Implement LRU cache for multimodal embeddings (#8292) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-06 23:21:40 -07:00
Mick
0bcc195f4e fix: minor fix TransportProxyTensor under tp (#8382) 2025-07-27 00:38:49 -07:00
Mick
3212c2ad3f vlm: optimize tensor transport (#6003) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-07-26 17:41:01 +08:00
Mick
534756749a chore: improvements on mm_utils (#7737) 2025-07-26 17:38:56 +08:00
Xinyuan Tong
8430bfe3e9 [Refactor] simplify multimodal data processing (#8107) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-07-20 21:43:09 -07:00
Lifu Huang
d969504d9a Fix flaky CI: test_vlm_models (#8006) 2025-07-14 14:56:41 -07:00
Mick
b5e3d6031c vlm: support video as an input modality (#5888) 2025-07-09 23:48:35 -07:00
Xinyuan Tong
4bab50a6b5 Fix llama4 vision (#7840) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-07-08 14:00:03 -07:00
Xinyuan Tong
3a911b854d Refactor mm processors and Enable mixed modality processing (#7629) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-06-30 23:14:48 -07:00
Lianmin Zheng
22352d47a9 Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632) 
Co-authored-by: Kan Wu <wukanustc@gmail.com>
 2025-06-29 23:16:19 -07:00
Lianmin Zheng
ce3a3e8783 Move multimodal processors into a separate folder (#7581) 2025-06-27 11:58:24 -07:00
Xinyuan Tong
681fdc264b Refactor vlm embedding routine to use precomputed feature (#6543) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-05-24 18:39:21 -07:00
Chang Su
4685fbb888 [VLM] Support chunk prefill for VLM (#6355) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-05-22 20:32:41 -07:00
Mick
01dd39bac1 refactor: minor refactors regarding multimodal processing (#6187) 2025-05-17 22:53:20 -07:00
Yury Sulsky
f19a9204cd Support precomputed multimodal features for Qwen-VL and Gemma3 models. (#6136) 
Co-authored-by: Yury Sulsky <ysulsky@tesla.com>
 2025-05-16 12:26:15 -07:00
applesaucethebun
2ce8793519 Add typo checker in pre-commit (#6179) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-11 12:55:00 +08:00
Mick
c998d04b46 vlm: enable radix cache for qwen-vl models (#5349) 
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-04-23 20:35:05 -07:00
Lianmin Zheng
177320a582 Clean up imports (#5467) 2025-04-16 15:26:49 -07:00
Mick
fbebcb7aa4 model: support mllama4 (#5144) 2025-04-09 09:28:44 -07:00
Mick
5cb552b1d4 refactor: multimodal data (#4754) 2025-03-31 09:57:51 -07:00
Mick
1e86457c90 model: Minicpmo (#3023) 2025-03-24 20:08:40 -07:00
Mick
11577cedb7 refactor: bug fixes and refactor for vlm (#4661) 2025-03-22 22:48:49 -07:00