fzyzcjy
|
b6cf3532b5
|
Tiny refactor ModelConfig.from_server_args (#5219)
|
2025-05-08 01:02:43 -07:00 |
|
xm:D
|
3409aaab32
|
Support InternVL3 (#5350)
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
|
2025-05-01 22:38:59 -07:00 |
|
Ying Sheng
|
11383cec3c
|
[PP] Add pipeline parallelism (#5724)
|
2025-04-30 18:18:07 -07:00 |
|
Ke Bao
|
dd408ee481
|
Auto set draft model path for MTP (#5793)
|
2025-04-29 16:25:40 -07:00 |
|
Liangsheng Yin
|
40d9b8acce
|
Improve overlap scheduling (#5788)
|
2025-04-28 11:19:16 +08:00 |
|
fzyzcjy
|
5fc4b6004e
|
Add sanity check for max_running_requests (#5016)
|
2025-04-20 17:56:49 -07:00 |
|
Cheng Wan
|
038bc5d521
|
Support --enable-llama4-multimodal (#5254)
|
2025-04-11 01:24:14 -07:00 |
|
Lianmin Zheng
|
74e0ac1dbd
|
Clean up import vllm in quantization/__init__.py (#4834)
|
2025-03-28 10:34:10 -07:00 |
|
XinyuanTong
|
42a45df043
|
[Fix] self.worker assignment in TpModelWorker and refactor references (#4788)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-03-27 20:28:38 -07:00 |
|
Wei Wu
|
91ba98fe50
|
[Fix] Resolve GPU Memory Leak in update_weights_from_tensor (#4446)
|
2025-03-17 08:54:30 +00:00 |
|
Ying Sheng
|
d3d4d76758
|
[Eagle] Refactor eagle speculative decoding (#3986)
Co-authored-by: Ke Bao <ISPObaoke@163.com>
|
2025-03-05 08:06:07 -08:00 |
|
Lianmin Zheng
|
ac2387279e
|
Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
|
2025-03-03 00:12:04 -08:00 |
|
fzyzcjy
|
e3e0bc50a9
|
[Feature] SPMD for SGLang + Verl (#3852)
|
2025-02-28 09:53:10 -08:00 |
|
giorgiopiatti-dfinity
|
8b6a4486ec
|
fix missing revision arg when loading tokenizer (#2982)
|
2025-01-19 11:36:07 -08:00 |
|
Lianmin Zheng
|
8b6ce52e92
|
Support multi-node DP attention (#2925)
Co-authored-by: dhou-xai <dhou@x.ai>
|
2025-01-16 11:15:00 -08:00 |
|
Lianmin Zheng
|
ad20b7957e
|
Eagle speculative decoding part 3: small modifications to the general scheduler (#2709)
Co-authored-by: kavioyu <kavioyu@tencent.com>
|
2025-01-02 02:09:08 -08:00 |
|
fzyzcjy
|
9183c23eca
|
Speed up update_weights_from_tensor (#2695)
|
2025-01-02 02:05:19 -08:00 |
|
Lianmin Zheng
|
b0524c3789
|
Eagle speculative decoding part 2: Fix cuda graph + DP attention hanging (#2684)
Co-authored-by: yukavio <kavioyu@gmail.com>
|
2024-12-31 02:25:05 -08:00 |
|
fzyzcjy
|
fd28640dc5
|
Add update_weights_from_tensor (#2631)
|
2024-12-28 13:30:27 -08:00 |
|
Yineng Zhang
|
85e1a6f3aa
|
Update model_loader deps and qqq quantization deps (#2220) (#2318)
Co-authored-by: HandH1998 <1335248067@qq.com>
|
2024-12-02 23:22:13 +08:00 |
|
Chayenne
|
983bfcf386
|
Online weight updates from torch.distributed (#2279)
|
2024-12-01 23:23:18 -08:00 |
|
Chayenne
|
7d1485d376
|
Add get weights by parameter name for llama (#2266)
|
2024-11-29 23:36:38 -08:00 |
|
Chayenne
|
7d5d1d3d29
|
udate weights from disk (#2265)
|
2024-11-30 01:17:00 +00:00 |
|
Xuehai Pan
|
62a4a339eb
|
docs: fix module docstrings and copyright headers (#2077)
|
2024-11-22 22:16:53 +08:00 |
|
Lianmin Zheng
|
b7a065eae3
|
Use cuda event wait and synchronization instead of busy waiting (#2089)
|
2024-11-19 00:21:46 -08:00 |
|
Lianmin Zheng
|
b110453802
|
Simplify logits penalizer (#2086)
|
2024-11-18 17:48:28 -08:00 |
|
Ke Bao
|
62832bb272
|
Support cuda graph for DP attention (#2061)
|
2024-11-17 16:29:20 -08:00 |
|
Ke Bao
|
976bc302e5
|
Support DP MLA (#1970)
|
2024-11-16 09:01:43 +00:00 |
|
Lianmin Zheng
|
0abbf289a8
|
Unify the model type checking (#1905)
|
2024-11-03 12:25:39 -08:00 |
|
Liangsheng Yin
|
5e1558f1f2
|
Update max_req_len and max_req_input_len (#1748)
|
2024-10-21 16:12:04 -07:00 |
|
Lianmin Zheng
|
b48edff67f
|
Split the overlapped version of TpModelWorkerClient into a separate file (#1726)
|
2024-10-20 00:29:29 -07:00 |
|
Lianmin Zheng
|
59cbf47626
|
Unify the memory pool api and tp worker API (#1724)
|
2024-10-19 23:19:26 -07:00 |
|
Lianmin Zheng
|
12cad0feae
|
Simplify the interface of tp_worker (#1718)
|
2024-10-19 17:39:38 -07:00 |
|
Lianmin Zheng
|
769bf11c05
|
Fix the race condition in overlap mode (#1712)
|
2024-10-19 06:50:56 -07:00 |
|
Lianmin Zheng
|
dbec2f1847
|
Launch a thread to overlap CPU and GPU (#1687)
|
2024-10-16 11:20:17 -07:00 |
|
Lianmin Zheng
|
7ee6c259ff
|
Simplify the event loop and expose --num-continuous-decode-steps as an argument (#1652)
|
2024-10-12 21:35:30 -07:00 |
|
Lianmin Zheng
|
63ba2f8d7b
|
Clean up batch data structures: Introducing ModelWorkerBatch (#1544)
|
2024-09-30 06:41:49 -07:00 |
|
Lianmin Zheng
|
36d5acfca5
|
Rename InputMetadata -> ForwardBatch (#1543)
|
2024-09-30 02:41:11 -07:00 |
|
Lianmin Zheng
|
3f0fe08d37
|
Let ModelRunner take InputMetadata as input, instead of ScheduleBatch (#1541)
|
2024-09-29 20:28:45 -07:00 |
|
Lianmin Zheng
|
f86c1e611f
|
Move scheduler code from tp_worker.py to scheduler.py (#1538)
|
2024-09-29 17:42:45 -07:00 |
|
Lianmin Zheng
|
048685430d
|
Improve process creation (#1534)
|
2024-09-29 02:36:12 -07:00 |
|
Liangsheng Yin
|
fd9ad817ec
|
Organize image inputs (#1531)
|
2024-09-29 06:28:55 +00:00 |
|
Ying Sheng
|
9aa6553d2a
|
[Feature] Support reward model LxzGordon/URM-LLaMa-3.1-8B (#1525)
|
2024-09-27 23:32:11 -07:00 |
|
Lianmin Zheng
|
2854a5ea9f
|
Fix the overhead due to penalizer in bench_latency (#1496)
|
2024-09-23 07:38:14 -07:00 |
|
Liangsheng Yin
|
36078fb247
|
fix schedule bug (#1450)
|
2024-09-17 16:33:53 -07:00 |
|
Lianmin Zheng
|
27b557aea7
|
Clean up model loader (#1440)
|
2024-09-16 18:16:27 -07:00 |
|
zifeitong
|
93dffd699b
|
Add constrained_json_whitespace_pattern to ServerArgs (#1438)
|
2024-09-16 13:29:18 -07:00 |
|
Lianmin Zheng
|
9ba1f09760
|
[Fix] Fix logprob and normalized_logprob (#1428)
|
2024-09-15 06:36:06 -07:00 |
|
Liangsheng Yin
|
70b6802982
|
Optimize conflicts between CUDA graph and vocab mask tensors (#1392)
|
2024-09-13 20:27:53 -07:00 |
|
Lianmin Zheng
|
b912de11b0
|
Make stop reason a dict instead of str (#1407)
|
2024-09-12 20:47:31 -07:00 |
|