chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
10,448 Commits 1 Branch 0 Tags
0a43229be13227e017a472f4b52afaf54a7a1643
Commit Graph
6887 Commits
Author SHA1 Message Date
Yuan Luoandluoyuan.luo 7da590d4d0 [Qwen3.5] Support Qwen3.5 Pipeline Parallelism (#19670) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-07 23:34:08 +08:00
YeChang Guorootronnie_zheng
 		13bdc7bf4a [Feature][NPU]: add runtime support for AutoRound quantized models (#16699) 
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2026-03-07 18:03:55 +03:00
Артем Савкинandronnie_zheng 5297b02c88 [Diffusion] [NPU] Wan2.2-T2V-A14B-Diffusers modelslim quantization support (#17996) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2026-03-07 17:26:44 +03:00
xingsy97 f8d4eb7022 [Docs] Add docstrings to JIT kernel include headers (#19770) 2026-03-07 20:48:00 +08:00
Ratish P ef6540b439 [diffusion]: add width/height passthrough for OpenAI image API (#19970) 2026-03-07 20:43:46 +08:00
David Wang 19c51fe2fa fix(rope): restore K writeback in fused rope + kv store kernel (#19636) 2026-03-07 20:41:35 +08:00
Fan YinandBaizhou Zhang 43d6a32045 [sgl-kernel] rebase FlashMLA 0217 (#18902) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-03-07 00:30:52 -08:00
danielafrimi f8bbf56de7 Refactor NemotronHConfig to canonical layers_block_type and add MTP block-type support (#19950) 
Signed-off-by: dafrimi <dafrimi@nvidia.com>
 2026-03-06 23:22:03 -08:00
Lancer b91fb8393e [diffusion] fix: fix multi-prompt generation and support multiple prompts in cli (#19960) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-07 13:01:59 +08:00
Eitan Turok 31e93e4486 [diffusion] fix: fix TeaCache silently fails with --enable-teacache (#19964) 2026-03-07 13:00:11 +08:00
Qiaolin Yu 925185f9ec Fix flashinfer backend with pcg (#20061) 2026-03-06 20:01:43 -08:00
Feng Su 8a411a9a2a [Tracing] Remove the deprecated tracing code from mini_lb (#19409) 2026-03-07 11:19:23 +08:00
Mohammad Miadh Angkad f88acf8780 [JIT Kernel] Reland NVFP4 kernels to JIT (#20012) 2026-03-07 10:31:08 +08:00
Yilong Zhao 6ffc74efd7 [Metrics] Add overlap bubble timing, full KV usage gauge, and prefill cuda graph tracking (#19982) 2026-03-06 17:41:27 -08:00
shubham singhalandsys-lpot-val a0d085c16d Adding correct path for module not found error while collecting test (#19778) 
Co-authored-by: sys-lpot-val <sys_lpot_val@intel.com>
 2026-03-06 16:26:16 -08:00
R0CKSTAR e818f8219a Fix none-comparison (E711) warnings (#19745) 
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
 2026-03-06 16:15:21 -08:00
R0CKSTAR 0c4f98ed4e [diffusion] hardware: add set_musa_arch on MUSA (misc, 15/N) (#19381) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-03-06 16:14:41 -08:00
MARATRIX 069d4c577b Fix Kimi K2.5 PP layer range exposure for PD disaggregation (#19959) 
Signed-off-by: yafeng.li <yafeng.li@mthreads.com>
 2026-03-06 16:14:02 -08:00
Liangsheng YinandClaude Opus 4.6 ddcecdea49 [Core] Unify max_num_reqs dp_size division for pool sizing (#20063) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-06 16:12:59 -08:00
Kangyan-ZhouandClaude Opus 4.6 7a12255b6e fix: set first_token_time before computing decode_throughput for single-batch completions (#19984) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 16:11:41 -08:00
Aurick Qiao 5c8e28698c Add cleanup for _ATTN_TP in parallel_state.py (#19978) 2026-03-06 15:43:31 -08:00
Shu Wang 61de303f0a Fix fallback to default tactic (flashinfer autotuner) with trtllm_fp4_block_scale_moe (#19189) 2026-03-06 15:15:04 -08:00
Kangyan-ZhouandClaude Opus 4.6 e89069ee64 Fallback to torch.cuda.mem_get_info() when nvidia-smi is unavailable (#18957) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 15:00:08 -08:00
Liangsheng Yin 604db4471d [Core] Clarify memory variable naming in model runner (#20060) 2026-03-06 14:00:46 -08:00
Liangsheng Yin 7a6cf0e9ba [Core] Extract _calculate_mamba_ratio and _init_pools from init_memory_pool (#20058) 2026-03-06 13:37:22 -08:00
Mohammad Miadh Angkad 759700c808 Fix SM120 triton_kernels MXFP4 block_k for GPT-OSS (#20040) 2026-03-06 10:53:08 -08:00
R0CKSTAR de1a0afcbc [MUSA][10/N] Add GGUF support (#18357) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-03-06 10:50:35 -08:00
JohnHerry e8f2b80340 [diffusion] improve: improve code readability of DenoisingStage (#20003) 2026-03-06 23:23:44 +08:00
xingsy97 54634b9a40 [Kernel] Dispatch exp/sin/cos through dtype_trait (#19798) 2026-03-06 22:57:52 +08:00
Johnsonms 2d266c73ea Migrate renorm kernels from sgl-kernel to FlashInfer JIT (#18854) 2026-03-06 22:53:28 +08:00
Xiaoyu ZhangandMick 6d22c9f369 [Diffusion] Move hf kernels diffusion cuda kernels skills to SGLD (#20001) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-03-06 22:16:06 +08:00
Yuan Luo f7de9375ac [GDN][Qwen3-Next][Qwen3.5] Fuse fused_gdn_gating and fused_recurrent_gated_delta_rule_update in verify_target (#19775) 2026-03-06 21:42:44 +08:00
Prozac614anddaiweitao e3b581ce6b [diffusion] fix: remove num_frames in wan2_1_t2v_1_3b_lora_1gpu test (#20009) 
Co-authored-by: daiweitao <dwti614707404@163.com>
 2026-03-06 21:36:43 +08:00
Kangyan-ZhouandClaude Opus 4.6 25e678d933 [diffusion] endpoint: add /server_info and /model_info endpoints for gateway discovery (#20020) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 21:36:13 +08:00
inkcherry 84aaa69795 [AMD] Use bfloat16 for correction_bias in AITER FP8 path to avoid runtime dtype conversion for dsv3 (#19843) 2026-03-06 00:57:12 -08:00
Clint 27053aa5ed Fix MLA decode path returning unwritten (padded) rows (#19902) 2026-03-06 00:54:29 -08:00
xdtbyndandsglang-npu-bot 0252ca8255 [Bugfix] Fix the bug blocking the startup of Llama-3.2-11b-Vision-Instruct (#19638) 
Co-authored-by: sglang-npu-bot <sglangnpu@163.com>
 2026-03-06 16:21:50 +08:00
Zheng Wengang da27d9bff6 [Bug-Fix][EPD]: skip log waiting-image-req for zmq_to_tokenzer/mooncake (#19555) 2026-03-06 14:39:22 +08:00
MookandRatish P be9a9e4819 refactor(multimodal/test): centralize model names and shared utilities in test_utils (#19354) 
Co-authored-by: Ratish P <114130421+Ratish1@users.noreply.github.com>
 2026-03-05 20:09:42 -08:00
Baizhou Zhang 51e5dc845a Revert "[Kernel Slimming] Migrate NVFP4 kernels to JIT" (#20005) 2026-03-05 19:40:00 -08:00
sushil Dubey 6e5a2de354 [diffusion] fix: fix reading multiple prompts from prompt file (#19075) 
Signed-off-by: Sushil Dubey <sushil.dubey@intel.com>
 2026-03-06 11:23:31 +08:00
Simo Lin 9502369488 fix(grpc): add server-side keepalive options to prevent GOAWAY (#19986) 
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
 2026-03-05 18:56:35 -08:00
liupeng374 5471e4a492 [NPU][Feature] eliminate dsv3 redundant rotary embed calculation (#19842) 2026-03-06 09:02:14 +08:00
chenxu214 b912d7ae19 [OPT]Skip the first delayer to maximize the BS of the decoding. (#19836) 2026-03-06 08:53:19 +08:00
shadowxz109 261be85ecc Support mrope_position_delta cache 2026-03-06 08:50:53 +08:00
Xinyuan Tong 9ebffef1ef [FIX] NSA backend page_table overflow in speculative decoding target_verify (#19016) 2026-03-05 16:04:58 -08:00
Ajay Anubolu 13af7cbb02 fix: use consistent time denominator for throughput metrics in bench_one_batch_server (#19223) 2026-03-05 15:58:17 -08:00
Chang Su dd2bbe6d62 fix(grpc): use context.abort() with proper status codes instead of in-band errors (#19972) 
Signed-off-by: Chang Su <chang.s.su@oracle.com>
 2026-03-05 14:53:18 -08:00
Qiaolin Yu 46dced64ea Adjust padding size to improve triton_kernels moe performance (#19174) 2026-03-05 14:50:40 -08:00
kpham-sglandhnyls2002 346a4131cf [Spec] Refactor NaN/OOB checks to async maybe_detect_* with env-var control (#19899) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-03-05 13:51:05 -08:00