 Lianmin Zheng
|
c9745ee082
|
Fix pandas dependency in CI (#3818)
|
2025-02-24 05:56:57 -08:00 |
|
|
laixinandHandH1998
|
1a6e97577a
|
Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730)
Co-authored-by: HandH1998 <1335248067@qq.com>
|
2025-02-24 05:43:35 -08:00 |
|
|
Baizhou Zhang
|
b110084654
|
Refactor flashinfer logic for deepseek v3 and fix accuracy bug (#3785)
|
2025-02-24 04:07:25 -08:00 |
|
|
Lianmin Zheng
|
27a46317b6
|
Fix dependency (#3813)
|
2025-02-24 03:50:58 -08:00 |
|
|
Lianmin Zheng
|
c979580817
|
Update readme (#3809)
|
2025-02-24 00:31:08 -08:00 |
|
|
|
6c7a152c5a
|
Hierarchical Caching for SGLang (#2693)
Co-authored-by: Wenxuan Tan <wenxuan.tan@wisc.edu>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-02-23 21:56:30 -08:00 |
|
|
Baizhou Zhangandzhaochenyang20
|
4d2a88bdff
|
[Docs]Add instruction for manually stopping nsys profiler (#3795)
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
|
2025-02-23 13:21:48 -08:00 |
|
|
fzyzcjy
|
45360b2fa9
|
Improve: Rename TokenizerManager to StdOrchestrator (#3116)
|
2025-02-23 00:30:58 -08:00 |
|
|
fzyzcjy
|
3f41b18455
|
Improve: Extract generation_manager from tokenizer_manager (#3115)
|
2025-02-22 23:25:45 -08:00 |
|
|
Mick
|
45205d88a0
|
bench: Add MMMU benchmark for vLM (#3562)
|
2025-02-22 08:10:59 -08:00 |
|
|
fzyzcjy
|
9087694006
|
Improve: Use TypeBasedDispatcher in DetokenizerManager (#3117)
|
2025-02-21 19:50:46 -08:00 |
|
|
fzyzcjy
|
a3339d8cac
|
Bug: Fix weight loader error when LM head weights are tied (#3766)
|
2025-02-21 17:53:12 -08:00 |
|
|
Chayenne
|
14d90617b0
|
Bug: fix lm head weights in Qwen models (#3777)
|
2025-02-21 16:49:31 -08:00 |
|
|
fzyzcjy
|
d37f95511d
|
Improve: Tiny fix Olmo2 (#3348)
|
2025-02-21 16:09:35 -08:00 |
|
|
Zhiyu
|
c66b2c9cf1
|
Add support for nvidia modelopt fp8 kv cache (#3223)
|
2025-02-22 07:04:58 +08:00 |
|
|
simveit
|
20b765a26e
|
Model: Support Qwen 72B RM model. (#3772)
|
2025-02-21 14:38:21 -08:00 |
|
|
Chayenne
|
e310722266
|
Docs: Update offline_engine_api and add links (#3773)
|
2025-02-21 14:15:52 -08:00 |
|
|
Shi Shuai
|
e074e76b31
|
docs: Add offline engine launch example and documentation (#3771)
|
2025-02-21 11:25:52 -08:00 |
|
|
simveit
|
4592afc27d
|
Docs: Fix layout to docs (#3733)
|
2025-02-21 11:24:13 -08:00 |
|
|
Shenggui Li
|
9af0e21ef5
|
[bug] fixed batch api for DeepSeek V3/R1 (#3754)
|
2025-02-21 10:28:16 -08:00 |
|
|
Shi Shuai
|
c7c79b16cd
|
[Fix] OpenAI API adapter tokenizer encoding (#3432)
|
2025-02-21 09:24:15 -08:00 |
|
|
Shakhizat Nurgaliyev
|
d8d75d256a
|
Change description of nvidia jetson docs (#3761)
|
2025-02-21 20:44:22 +08:00 |
|
|
Andrew Smith
|
1df6eabd5d
|
feat: Add SageMaker support (#3740)
|
2025-02-21 19:31:09 +08:00 |
|
|
zixuanzhang226
|
0c227ee373
|
feat: update grouped_topk to support softmax and sigmoid (#3680)
|
2025-02-21 16:30:15 +08:00 |
|
|
HAI
|
5c54ef0352
|
AMD/ROCm: update AITER repo to ROCm/aiter (#3747)
|
2025-02-21 00:18:08 -08:00 |
|
|
Shenggui Li
|
c6a4852136
|
[docs] added torch.compile cache to dpsk manual (#3737)
|
2025-02-21 00:11:40 -08:00 |
|
|
Trayvon Panandyqtianust
|
4f678c87d8
|
docs: fix 404 link (#3588)
Co-authored-by: yqtianust <yqtian@ust.hk>
|
2025-02-20 15:15:57 -08:00 |
|
|
|
e79f7420be
|
[Fix] Fix bugs and refactor codes in lora for better scalability. (#3652)
Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
|
2025-02-20 11:51:57 -08:00 |
|
|
Baizhou Zhang
|
ac05310098
|
[Docs] Modify ep related server args and remove cublas part of deepseek (#3732)
|
2025-02-21 03:37:56 +08:00 |
|
|
chenxiaobing
|
d5d80ab477
|
[Bugfix] Fix scores mask for moe topk (#3705)
|
2025-02-21 02:17:23 +08:00 |
|
|
Ke Bao
|
ddcf9fe3be
|
Optimize triton attention custom mask (#3731)
|
2025-02-21 00:54:41 +08:00 |
|
|
HAI
|
6252ade985
|
revert BLOCK and num_warps on HIP (#3722)
|
2025-02-20 23:30:18 +08:00 |
|
|
yizhang2077
|
1eb8eade2b
|
add control for cutlass fp8 blockwise gemm (#3727)
|
2025-02-20 16:10:35 +08:00 |
|
|
Chayenne
|
3c7bfd7eab
|
Docs: Fix layout with sub-section (#3710)
|
2025-02-19 15:44:30 -08:00 |
|
|
simveit
|
bb121214c2
|
Variance measure for reasoning benchmark (#3677)
|
2025-02-20 03:49:49 +08:00 |
|
|
|
55de40f782
|
[Docs]: Fix Multi-User Port Allocation Conflicts (#3601)
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
Co-authored-by: simveit <simp.veitner@gmail.com>
|
2025-02-19 11:15:44 -08:00 |
|
|
Cheng Wan
|
6b0aeb58fd
|
[moe] optim: reduce memory consumption in fused_moe (#3692)
|
2025-02-20 02:25:05 +08:00 |
|
|
Peter Pan
|
bb3e526823
|
[k8s] remove unnecessary hostIPC for security concern (#3700)
|
2025-02-20 02:11:21 +08:00 |
|
|
Zhanghao Wu
|
f93e915817
|
[Docs] Add SkyPilot DeepSeek example (#3706)
|
2025-02-20 02:10:23 +08:00 |
|
|
Xiaoyu Zhang
|
55a7ec388f
|
use warp shuffle style reduce and flashinfer vectorize (#3628)
|
2025-02-19 20:53:51 +08:00 |
|
|
Yineng Zhang
|
fe0673f1cc
|
set NCCL_IB_GID_INDEX=3 for multi node NVIDIA InfiniBand if needed (#3698)
|
2025-02-19 20:50:22 +08:00 |
|
|
Mick
|
99c1b9d2ee
|
fix: apply cache size limit of attention mask for VisionAttention (#3657)
|
2025-02-19 20:16:48 +08:00 |
|
|
|
634a3561ac
|
AMD Prefill optimize (#3665)
Co-authored-by: AMD-dteng <dteng@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
|
2025-02-18 09:35:58 -08:00 |
|
|
Mick
|
424848d26f
|
fix: remove dependency on latest transformers impl (#3635)
|
2025-02-19 01:14:11 +08:00 |
|
|
Ke Bao
|
e5ce395a6c
|
Fix draft decode max batch size (#3676)
|
2025-02-18 23:03:26 +08:00 |
|
|
Yineng Zhang
|
f983213a1f
|
update pr-test (#3663)
|
2025-02-18 17:23:43 +08:00 |
|
|
Baizhou Zhang
|
67fc595bb8
|
[Feature] Apply Cublas Grouped Gemm kernel (#3629)
|
2025-02-18 15:18:31 +08:00 |
|
|
Yineng Zhang
|
07ab4d4a2d
|
fix #3654
|
2025-02-18 15:16:16 +08:00 |
|
|
saienduri
|
522e18eaeb
|
Update amd docker image. (#3654)
|
2025-02-17 20:12:55 -08:00 |
|
|
ybyang
|
c51dc2cc8d
|
Docs: Deploy multi-node inference (LWS method) using sglang in a K8s cluster (#3624)
|
2025-02-17 18:14:20 -08:00 |
|