 JieXin Liang
|
69af3ec35f
|
[doc] add note for get_num_kv_splits in triton_backend (#6444)
|
2025-05-19 21:40:21 -07:00 |
|
|
JieXin LiangandStefan He
|
1f30c05d4a
|
[fix] fix fa3 forward_decode with spec_decode (#6395)
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
|
2025-05-18 12:50:15 -07:00 |
|
|
JieXin Liang
|
9a405274e2
|
[misc] remove redundant platform codes (#6298)
|
2025-05-15 00:51:30 -07:00 |
|
|
JieXin Liang
|
17299f088a
|
[misc] deep_gemm fallback to NVRTC when NVCC not found (#6252)
|
2025-05-13 01:41:35 -07:00 |
|
|
JieXin Liang
|
e9bebafb19
|
[fix] remove mixtral from is_fa3_default_architecture (#6191)
|
2025-05-11 00:15:54 -07:00 |
|
|
JieXin Liang
|
c178abdabc
|
[fix] fix determine_n_share_experts_fusion (#6118)
|
2025-05-10 01:19:09 -07:00 |
|
|
JieXin Liang
|
5e02330137
|
[perf] dsv3 bmm fallback to bf16 (#5662)
|
2025-05-08 11:43:39 -07:00 |
|
|
JieXin Liang
|
f1ff736d68
|
[fix] fix pyproject.toml dependencies (#6119)
|
2025-05-08 02:14:36 -07:00 |
|
|
JieXin Liang
|
b70957fcf8
|
[refactor] slightly tidy fp8 module (#5993)
|
2025-05-07 17:28:24 -07:00 |
|
|
JieXin Liang
|
8ebde73f7d
|
[perf] H100 DeepSeek-V3 fused moe tuned config (#5998)
|
2025-05-03 14:02:26 -07:00 |
|
|
JieXin Liang
|
3cff963335
|
[fix] kimi-vl test in test_vision_openai_server.py (#5910)
|
2025-04-29 23:59:10 -07:00 |
|
|
JieXin Liangandalcanerian
|
e4b6133b78
|
[fix] relax mem_fraction_static for h200 (#5893)
Co-authored-by: alcanerian <alcanerian@gmail.com>
|
2025-04-29 17:01:12 -07:00 |
|
|
JieXin Liang
|
f55933e1cc
|
[misc] more decode step log for batch_one_batch (#5565)
|
2025-04-26 19:50:28 -07:00 |
|
|
JieXin Liang
|
3c4dc38a9a
|
[fix] fix bench_one_batch_server (#5607)
|
2025-04-26 18:49:45 -07:00 |
|
|
JieXin Liang
|
eebfdb9459
|
[fix] fix potential bumpy throughtput with deepgemm (#5722)
|
2025-04-26 18:12:48 -07:00 |
|
|
JieXin Liangandalcanerian
|
917324862e
|
[fix] reduce dp capture bs (#5634)
Co-authored-by: alcanerian <alcanerian@gmail.com>
|
2025-04-22 11:08:45 -07:00 |
|
|
JieXin Liangandispobock
|
4418f599a5
|
Fix FA3 DeepSeek prefill performance regression (#5624)
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2025-04-22 01:41:41 -07:00 |
|
|
JieXin Liang
|
506be6b892
|
[fix] fix compile_deep_gemm missing kv_b_proj (#5620)
|
2025-04-22 00:06:36 -07:00 |
|
|
JieXin Liang
|
2343d8df7d
|
[fix] force use deepgemm in compile_deep_gemm (#5618)
|
2025-04-21 21:36:02 -07:00 |
|
|
JieXin Liang
|
c2942907d5
|
[feature] enable pre compile jit deep_gemm (#5580)
|
2025-04-21 16:52:53 -07:00 |
|
|
JieXin Liang
|
97cb762bb6
|
[misc] remove is_cuda_available (#5319)
|
2025-04-20 18:16:51 -07:00 |
|
|
JieXin Liang
|
bca832c7c6
|
[Fix] fix outlines and xgrammar (#4947)
|
2025-04-20 13:31:25 -07:00 |
|
|
JieXin Liang
|
99456bcacb
|
[perf] introduce deep gemm group_gemm_masked as bmm (#5432)
|
2025-04-20 00:38:27 -07:00 |
|
|
JieXin Liang
|
f88f7e1943
|
[misc] fix ci flaky case (#5352)
|
2025-04-15 01:37:16 -07:00 |
|
|
JieXin Liang
|
bdde237562
|
[perf] experimental enhance fp8 per-tensor quant (#5370)
|
2025-04-14 12:35:43 -07:00 |
|
|
JieXin Liang
|
a995a773a0
|
[fix] remove cuda_device_count_stateless (#5060)
|
2025-04-04 00:18:26 -07:00 |
|
|
JieXin Liang
|
51ac297ace
|
[feat] interface for platforms abstraction (#4928)
|
2025-03-31 00:04:21 -07:00 |
|
|
JieXin Liang
|
9e93ef3f8e
|
[fix] fix illegal mem access and clean up triton attention backend (#4571)
|
2025-03-20 02:01:52 -07:00 |
|
|
JieXin Liang
|
4942074174
|
[fix] fix initialization of _ENABLE_TORCH_INFERENCE_MODE (#4549)
|
2025-03-19 09:57:59 -07:00 |
|
|
JieXin Liang
|
c0e9a36c5f
|
Optimize Triton decoding kernel for dynamic workload (#4553)
|
2025-03-18 21:25:38 -07:00 |
|
|
JieXin Liang
|
0212d2e288
|
[Fix] use torch.inference_mode() instead of torch.no_grad() (#4372)
|
2025-03-16 22:54:16 -07:00 |
|
|
JieXin Liang
|
1a3fa75f2f
|
[Fix] use torch.cat instead of torch.concat to prevent entering the Autograd backends. (#4466)
|
2025-03-16 00:02:47 -07:00 |
|
|
JieXin Liang
|
0540fef7a1
|
[Fix] fix _yarn_linear_ramp_mask with device parameter (#4337)
|
2025-03-12 02:28:19 -07:00 |
|