 Ke Bao
|
976bc302e5
|
Support DP MLA (#1970)
|
2024-11-16 09:01:43 +00:00 |
|
|
Lianmin Zheng
|
2f2e07439c
|
Fix weight update for data parallelism (#2050)
|
2024-11-16 00:30:39 -08:00 |
|
|
HAI
|
e5c6715003
|
Fix core (MI300X) with --enable-overlap (#2048)
|
2024-11-15 21:24:42 -08:00 |
|
|
Lianmin Zheng
|
2558d6a675
|
Fix the default arguments of bench_offline_throughput.py & simplify detokenizer manager (#2042)
|
2024-11-15 05:02:44 -08:00 |
|
|
Lianmin Zheng
|
54479d6f30
|
Fix grammar backend for tensor parallelism (#2020)
|
2024-11-13 01:49:45 -08:00 |
|
|
Lianmin Zheng
|
ba069a24d3
|
Fix grammar backend (#2018)
|
2024-11-12 21:17:38 -08:00 |
|
|
DarkSharpnessandLianmin Zheng
|
125b1199c5
|
support parallel grammar preprocessing (#1996)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2024-11-12 08:45:28 -08:00 |
|
|
Lianmin Zheng
|
78c1d6445f
|
Fix finish reason (#2013)
|
2024-11-11 23:24:41 -08:00 |
|
|
Lianmin Zheng
|
befc6beb86
|
Fix a typo in io_struct.py (#2008)
|
2024-11-11 16:34:10 -08:00 |
|
|
yizhang2077
|
a8aad9357d
|
qwen2vl fix bug for #1971 #1897 (#1984)
|
2024-11-10 08:10:45 -08:00 |
|
|
Lianmin ZhengandMohit Reddy
|
1929c06762
|
Simplify prometheus metrics (#1981)
Co-authored-by: Mohit Reddy <mohitreddy1996@users.noreply.github.com>
|
2024-11-10 04:39:32 -08:00 |
|
|
Lianmin Zheng
|
9c939a3d8b
|
Clean up metrics code (#1972)
|
2024-11-09 15:43:20 -08:00 |
|
|
Enrique Shockwave
|
f11eb90fe4
|
Initialize model_worker_batch variable (#1973)
|
2024-11-09 11:28:02 -08:00 |
|
|
Lianmin Zheng
|
a509552087
|
[minor] Improve code style and compatibility (#1961)
|
2024-11-08 02:19:41 -08:00 |
|
|
Chayenne
|
c77c1e05ba
|
fix black in pre-commit (#1940)
|
2024-11-08 07:42:47 +08:00 |
|
|
Xuehai Pan
|
a5e0defb5a
|
minor: Add basic editorconfig and pre-commit hooks to enforce style for whitespaces (#1926)
|
2024-11-06 13:46:04 +00:00 |
|
|
|
a146d9990e
|
support prometheus metrics (#1853)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2024-11-05 20:42:53 -08:00 |
|
|
Lianmin ZhengandKyle Corbitt
|
2ce32db6fb
|
Let reward model take text inputs instead of message lists (#1907)
Co-authored-by: Kyle Corbitt <kyle@corbt.com>
|
2024-11-03 13:27:12 -08:00 |
|
|
Lianmin Zheng
|
0abbf289a8
|
Unify the model type checking (#1905)
|
2024-11-03 12:25:39 -08:00 |
|
|
Lianmin Zheng
|
c17c578108
|
Simplify tokenizer manager (#1904)
|
2024-11-03 08:38:26 -08:00 |
|
|
Lianmin Zheng
|
838dcda162
|
Simplify tokenizer manager (#1899)
|
2024-11-03 03:52:38 -08:00 |
|
|
Lianmin Zheng
|
efbc116a0f
|
Do not use longest prefix matching when #queue-req is large (#1896)
|
2024-11-03 01:45:20 -07:00 |
|
|
Liangsheng YinandLianmin Zheng
|
b9fd178f1b
|
Fix retraction + overlap (#1860)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2024-10-31 18:27:42 -07:00 |
|
|
Lianmin ZhengandLiangsheng Yin
|
a2e0424abf
|
Fix memory leak for chunked prefill 2 (#1858)
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
|
2024-10-31 14:51:51 -07:00 |
|
|
Byron Hsu
|
438526a814
|
Refactor tokenizer manager (#1846)
|
2024-10-30 21:32:18 -07:00 |
|
|
Lianmin Zheng
|
f7102fbd2b
|
Fix mixed chunked prefill (#1850)
|
2024-10-30 21:20:41 -07:00 |
|
|
Byron Hsu
|
a7a0a6886b
|
Make decode log interval configurable (#1847)
|
2024-10-30 19:59:20 -07:00 |
|
|
Ying Sheng
|
4e2af03cfa
|
[Production] Drain requests before exit when receive SIGTERM (#1838)
|
2024-10-30 10:22:56 -07:00 |
|
|
Byron Hsu
|
680cad2023
|
fix get_memory_pool_size deadlock for DP (#1830)
|
2024-10-28 23:07:14 -07:00 |
|
|
Byron Hsu
|
0a24eb850a
|
Fix update_weights deadlock for DP (#1825)
|
2024-10-28 12:02:23 -07:00 |
|
|
Lianmin Zheng
|
86fc0d79d0
|
Add a watch dog thread (#1816)
|
2024-10-27 02:00:50 -07:00 |
|
|
Lianmin Zheng
|
86e0dde555
|
Improve the user control of new_token_ratio (#1811)
|
2024-10-26 16:39:41 -07:00 |
|
|
Lianmin Zheng
|
2b80978859
|
Provide an argument to set the maximum batch size for cuda graph (#1809)
|
2024-10-26 15:09:33 -07:00 |
|
|
Byron Hsu
|
c26507484f
|
fix int conversion for SGLANG_CPU_COUNT (#1803)
|
2024-10-26 00:09:44 -07:00 |
|
|
Liangsheng Yin
|
1e8903414a
|
Fix possible ZMQ hanging (#1800)
|
2024-10-25 23:07:07 -07:00 |
|
|
Lianmin Zheng
|
fb99aaa527
|
[Fix] Fix --skip-tokenizer-init (#1798)
|
2024-10-25 18:51:59 -07:00 |
|
|
DarkSharpness
|
b77a02cdfd
|
[Performance] Support both xgrammar and outlines for constrained decoding (#1752)
|
2024-10-25 21:47:02 +00:00 |
|
|
Lianmin Zheng
|
e646c5901e
|
Fix logprob in the overlapped mode (#1795)
|
2024-10-25 11:06:57 -07:00 |
|
|
Lianmin Zheng
|
c555ce2ca2
|
Revert "Fix memory leak when doing chunked prefill" (#1797)
|
2024-10-25 10:24:44 -07:00 |
|
|
Liangsheng Yin
|
a2f5e7555f
|
Fix memory leak when doing chunked prefill (#1787)
|
2024-10-25 08:01:17 -07:00 |
|
|
Lianmin Zheng
|
8f8f96a621
|
Fix the perf regression due to additional_stop_token_ids (#1773)
|
2024-10-23 16:45:21 -07:00 |
|
|
Liangsheng Yin
|
3f5ac88d02
|
Fix out of memory message. (#1771)
|
2024-10-23 15:20:39 -07:00 |
|
|
Lianmin Zheng
|
9af7b88e3c
|
[Fix] Fix abort in dp (#1767)
|
2024-10-23 10:46:29 -07:00 |
|
|
Ying ShengandByron Hsu
|
2fce449b1c
|
[API] add get memory pool size (#1760)
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2024-10-23 07:02:29 +00:00 |
|
|
Lianmin Zheng
|
ad4125d1a9
|
Fuse more ops & Simplify token mapping (#1758)
|
2024-10-22 23:20:43 -07:00 |
|
|
Byron Hsu
|
17536e7e3d
|
Fix edge case for truncated (#1747)
|
2024-10-23 00:00:25 -04:00 |
|
|
Liangsheng Yin
|
5e1558f1f2
|
Update max_req_len and max_req_input_len (#1748)
|
2024-10-21 16:12:04 -07:00 |
|
|
Liangsheng Yin
|
94cde10920
|
Llama3.2 vision model support (#1551)
|
2024-10-21 15:01:21 -07:00 |
|
|
Lianmin Zheng
|
7ce3606891
|
Faster overlap mode scheduler (#1738)
|
2024-10-21 04:30:52 -07:00 |
|
|
Liangsheng Yin
|
efb099cdee
|
Fix prefill oom (#1743)
|
2024-10-21 03:54:35 -07:00 |
|