 Lifu Huang
|
254f62d879
|
Support spec decoding when LoRA is applied to target model (#12903)
|
2025-11-16 13:20:23 -08:00 |
|
|
Lifu Huang
|
ce832d7034
|
Add env var to control custom Triton kernel cache and set CSGMV as default backend. (#12176)
|
2025-10-27 17:49:32 -07:00 |
|
|
Lifu Huang
|
d9b31011aa
|
Skip TestLlama4LoRA in CI (#12098)
|
2025-10-25 12:16:36 -07:00 |
|
|
Lifu Huang
|
b0d20cdec7
|
Set csgmv as default lora backend. (#11488)
|
2025-10-15 23:53:24 -05:00 |
|
|
Lifu Huang
|
780fbf2f38
|
[Fix] Fix accuracy bug in CSGMV kernel caching key. (#11579)
|
2025-10-14 20:25:56 -07:00 |
|
|
Lifu Huang
|
92473e2e34
|
Support LoRA in bench_serving oai interface (#11318)
|
2025-10-08 01:28:58 -07:00 |
|
|
Lifu HuangandHieu Pham
|
edefab0c64
|
[2/2] Support MHA prefill with FlashAttention 4. (#10937)
Co-authored-by: Hieu Pham <hyhieu@gmail.com>
|
2025-10-08 00:54:20 -07:00 |
|
|
Lifu Huang
|
748f86f3de
|
[Bug] Fix incorrect assertion in FA4 and add UT. (#11182)
|
2025-10-06 14:58:39 -07:00 |
|
|
Lifu Huang
|
e98d9346c7
|
[1/2] Support FA4 for MHA Prefill in sgl-kernel (#10940)
|
2025-09-28 19:59:14 -07:00 |
|
|
Lifu Huang
|
9241f4fd20
|
Move cached kernel to srt.utils (#10776)
|
2025-09-22 23:00:36 -07:00 |
|
|
Lifu Huang
|
2101d93b4f
|
Fix CI TestChunkedSGMV (#10737)
|
2025-09-22 16:09:58 +08:00 |
|
|
Lifu Huang
|
635ccda673
|
[4/4] Introduce CachedKernel to reduce CSGMV kernel launch overheads by 60% (#10709)
|
2025-09-21 22:26:42 -07:00 |
|
|
Lifu Huang
|
08ecd0aa2a
|
[3/4] Speed up CSGMV backend perf by 10% through dynamic chunking + kernel optimization (#10592)
|
2025-09-20 22:47:48 -07:00 |
|
|
Lifu Huang
|
3f41b48c40
|
[2/2] Introduce Chunked-SGMV kernels and corresponding LoRA backend for improved performance (#10286)
|
2025-09-15 16:04:03 -07:00 |
|
|
Lifu Huang
|
941002945b
|
[1/2] Refactor LoRA to support backend-specific batch preprocessing. (#10251)
|
2025-09-10 09:58:37 -07:00 |
|
|
Lifu Huang
|
e903f695c8
|
Fix potential flakiness in test_lora_qwen3 (#10250)
|
2025-09-10 08:04:39 +00:00 |
|
|
Lifu Huang
|
1fbfdebe6b
|
[chore] fix dead links in doc (#9913)
|
2025-09-02 00:28:26 -07:00 |
|
|
Lifu HuangandXinyuan Tong
|
d4bce29721
|
Fix incorrect logic in chat template handling. (#9336)
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2025-08-20 16:25:36 -07:00 |
|
|
Lifu Huang
|
b0980af89f
|
Support pinning adapter via server args. (#9249)
|
2025-08-20 16:25:01 -07:00 |
|
|
Lifu Huang
|
4b74c3fcca
|
[chore] Clean up redundant lora_weight_names concept to simplify code (#9131)
|
2025-08-17 12:36:58 -07:00 |
|
|
Lifu Huang
|
29a610b4d9
|
Fix broken CI TestRequestLengthValidation (#9095)
|
2025-08-11 22:59:56 -07:00 |
|
|
Lifu Huang
|
5ded39cab2
|
Fix race condition in async lora unload (#9084)
|
2025-08-11 22:59:29 -07:00 |
|
|
Lifu Huang
|
e322a94d1f
|
Reduce CI duration of test_lora_update. (#9024)
|
2025-08-10 15:34:04 -07:00 |
|
|
Lifu Huang
|
f8a173bb50
|
Improve LoRA Perf by Deprecating FlashInfer and Eliminating Redundant Tensor Ops (#8940)
|
2025-08-10 01:04:45 -07:00 |
|
|
Lifu Huang
|
6e2151183b
|
Fix incorrect default get_hidden_dim logic (#8987)
|
2025-08-09 00:25:38 -07:00 |
|
|
Lifu Huang
|
6210e2c4f0
|
Support GPU pinning for LoRA (#8697)
|
2025-08-06 19:39:45 -07:00 |
|
|
Lifu Huang
|
7cb20754fa
|
[Fix] Fix several issues preventing gemma3n LoRA support. (#8776)
|
2025-08-04 17:11:46 -07:00 |
|
|
Lifu Huang
|
8675bdf246
|
Support limiting max loaded loras in CPU. (#8650)
|
2025-08-03 00:02:23 -07:00 |
|
|
Lifu Huang
|
46e9d1c7c1
|
Increase tolerance to address CI failures (#8643)
|
2025-08-01 02:32:10 -07:00 |
|
|
Lifu Huang
|
67e53b16f5
|
Bump transfomers to 4.54.1 to fix Gemma cache issue. (#8541)
|
2025-07-30 19:50:54 -07:00 |
|
|
Lifu HuangandStefan He
|
fb16fbaf52
|
Fix incorrect KV cache allocation for MTP models. (#8482)
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
|
2025-07-28 22:54:50 -07:00 |
|
|
Lifu Huang
|
df90645525
|
Support overlapped lora updates (#8213)
|
2025-07-27 13:00:44 -07:00 |
|
|
Lifu Huang
|
761546315c
|
Remove slot usage in code to be backward-compatible with python 3.9 (#8396)
|
2025-07-26 21:24:22 -07:00 |
|
|
Lifu Huang
|
5c705b1dce
|
Add perf tests for LoRA (#8314)
|
2025-07-26 14:55:22 -07:00 |
|
|
Lifu Huang
|
8abd3e77fe
|
Introduce Stable LoRA ID System for Overlapped Updates and Prefix Caching (#8261)
|
2025-07-23 00:32:16 -07:00 |
|
|
Lifu Huang
|
4e3defe5a7
|
Support start up LoRA server without initial adapters (#8019)
|
2025-07-19 15:38:09 -07:00 |
|
|
Lifu Huang
|
3de617a75b
|
Fix LoRA buffer contamination during adapter eviction (#8103)
|
2025-07-19 13:14:08 -07:00 |
|
|
Lifu Huang
|
d969504d9a
|
Fix flaky CI: test_vlm_models (#8006)
|
2025-07-14 14:56:41 -07:00 |
|
|
Lifu Huang
|
e2ed9d049a
|
Refactor dynamic LoRA update to fix incorrect handling of variant weight shapes (#7844)
|
2025-07-13 18:36:01 -07:00 |
|
|
Lifu Huang
|
2b0e1d1ce0
|
[Minor] Fix sporadic CI timeout caused by underestimated tests. (#7850)
|
2025-07-08 01:01:49 -07:00 |
|
|
Lifu Huang
|
ea4bf12286
|
Fix division-by-zero bug in LoRA triton kernels. (#7785)
|
2025-07-06 00:45:29 -07:00 |
|
|
Lifu Huang
|
01f9873048
|
Fix CI test OOM issue. (#7799)
|
2025-07-05 15:11:02 -07:00 |
|
|
Lifu Huang
|
1a08358aed
|
Improve error handling for requests with unloaded LoRA path(s) (#7642)
|
2025-07-01 20:05:34 -07:00 |
|
|
Lifu Huang
|
49538d111b
|
Support dynamic LoRA loading / unloading in engine/server API (#7446)
|
2025-06-27 21:00:27 -07:00 |
|
|
Lifu Huang
|
2373faa317
|
Fix flakiness in LoRA batch test. (#7552)
|
2025-06-27 19:51:43 -07:00 |
|
|
Lifu Huang
|
d7374d7467
|
Fix broken CI TestVILAServer (#7610)
|
2025-06-27 15:01:03 -07:00 |
|
|
Lifu Huang
|
1998ce4046
|
Refactor LoRAManager and LoRAMemoryPool state management logic for dynamic LoRA loading support (#7412)
|
2025-06-21 16:09:19 -07:00 |
|
|
Lifu Huang
|
e07d064729
|
Support LoRA in MMMU benchmark script. (#7218)
|
2025-06-15 21:17:57 -07:00 |
|
|
Lifu Huang
|
98538822d5
|
Add Phi-4-mm to supported VLM supported model list. (#7178)
|
2025-06-13 23:17:40 -07:00 |
|
|
Lifu Huang
|
021f76e4f4
|
[Perf] Refactor LoRAManager to eliminate stream syncs and redundant computations (#6994)
|
2025-06-11 16:18:57 -07:00 |
|
|
Lifu Huang
|
b1e5a33ae3
|
Eliminate stream sync to speed up LoRA batch init (#6960)
|
2025-06-09 00:22:45 -07:00 |
|
|
Lifu Huang
|
4474eaf552
|
Support LoRA in TestOpenAIVisionServer and fix fused kv_proj loading bug. (#6861)
|
2025-06-04 22:08:30 -07:00 |
|
|
Lifu Huang
|
0a9bfc20ab
|
[Minor] Always append newline after image token when parsing chat message (#6797)
|
2025-05-31 20:50:33 -07:00 |
|
|
Lifu Huang
|
094fbdacd5
|
Fix incorrect LoRA weight loading for fused gate_up_proj (#6734)
|
2025-05-31 13:41:44 -07:00 |
|
|
Lifu Huang
|
477a101cbd
|
Refactor LoRA handling to support adapter tensors in fused format (#6585)
|
2025-05-26 21:51:54 -07:00 |
|
|
Lifu Huang
|
79a39ac0cc
|
follow-up: move Idefics2 to a shared location to eliminate unexpected dependency. (#6603)
|
2025-05-26 19:23:59 -07:00 |
|
|
Lifu Huang
|
0d503090aa
|
Supported precomputed feature for Kimi VL (#6599)
|
2025-05-26 01:24:13 -07:00 |
|
|
Lifu Huang
|
022012aae8
|
Support Phi-4 Multi-Modal (text + vision only) (#6494)
|
2025-05-24 21:43:38 -07:00 |
|
|
Lifu Huang
|
3cf1473a09
|
Use monotonic clock for interval measurement (#6211)
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-05-17 16:49:18 -07:00 |
|
|
Lifu Huang
|
3e350a931e
|
[Bug] Fix accidental logger override caused by internVL. (#6282)
|
2025-05-13 23:29:25 -07:00 |
|
|
Lifu Huang
|
6e2da51561
|
Replace time.time() to time.perf_counter() for benchmarking. (#6178)
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-05-11 14:32:49 -07:00 |
|
|
Lifu Huang
|
1232f7e8b7
|
Update dev container config to support live code sync and improve docker setup guide (#6018)
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-05-04 22:33:46 -07:00 |
|
|
Lifu Huang
|
2b63798c7d
|
[Minor] Fix duplicate method definitions in conversation.py (#6012)
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-05-04 13:02:53 -07:00 |
|
|
Lifu Huang
|
1acca3a2c6
|
FA3 speed up: skip len operation and get batch size directly from forward batch (#5969)
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-05-02 00:26:12 -07:00 |
|