chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
9,885 Commits 1 Branch 0 Tags
cc63c99f112f781605017db018fced686cdc94de
Commit Graph
33 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yi Zhong ed22720c07 [JIT kernel] hd=512,1024 in JIT QK norm (cta based) (#17515) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-02-16 16:07:24 +08:00
Yi Zhong dc1309fc7e Avoid kimi linear stream sync (#16186) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-02-12 09:27:22 +08:00
Yi Zhong bf89cc3803 [ModelOPT] Support Qwen 3 Next Coder NVFP4 (#18224) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-02-08 22:29:07 +00:00
Yi Zhong ca36d88fa6 [ModelOpt] Fix broken Qwen3-235B-A22B-Instruct-2507-NVFP4 launch (#18189) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-02-08 14:35:28 +00:00
Yi Zhong c2ab3713e9 [Performance] Optimize Mllama LayerNorm -> Upd (#9725) 2026-01-31 16:02:57 -08:00
Yi Zhong 8acd4d7d7e Make flashMLA work on: Cu13, B300 (#17600) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-01-28 00:12:47 +08:00
Yi ZhongandBrayden Zhong 08fcda2f63 add the fa4 mm backend and varlen func (#13539) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-01-23 23:12:06 +08:00
Yi Zhong 458fe5a337 [docs] Show user the fastAPI docs available (#17510) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-01-21 14:26:25 +00:00
Yi Zhong ec9b48ea96 Add olmo3 in supported docs (#13672) 
Signed-off-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
 2026-01-16 12:18:16 -05:00
Yi Zhong 7dde3438e2 Show how to use cu13 image with B300 (#17170) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-01-15 15:25:20 -05:00
Yi Zhong d1110e1c3e docs only add kimi k2 thinking and kimi linear (#15789) 2026-01-15 12:09:52 -05:00
Yi Zhong 84ab32a2a0 Reduce some small cpu overhead in stream fetch (#16587) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-01-09 21:53:06 +00:00
Yi Zhongandb8zhong cda35611d4 [perf] Add two stream norm for Olmo3 speedup 5% (#13681) 
Signed-off-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
Co-authored-by: b8zhong <b8zhong@uwaterloo.ca>
 2026-01-08 17:01:35 +00:00
vincentzed ddb3970eb4 Tiny env cleanup in deepgemm (#15706) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2025-12-24 00:58:46 -08:00
vincentzed ac320a6f04 Move some quant args to its own section in environ variables doc (#15722) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2025-12-23 20:11:08 -08:00
Vincent ZhongandBrayden Zhong 2ac5b98395 fix: fix rmsnorm -> layernorm in qwen3 omni (#11791) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-06 12:12:57 +08:00
Vincent Zhong 6bc3062894 Fix launch of Olmo3 (#13666) 
Signed-off-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
 2025-11-20 12:04:30 -08:00
Vincent Zhong 0d116b9a0b Clean up deprecated tile_tokens_dim for next flashinfer (#13341) 
Signed-off-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
 2025-11-16 00:50:15 +08:00
Vincent Zhong b4d2da106e [docs] upd docker files names everywhere (#12133) 2025-10-25 18:20:23 -07:00
Vincent Zhong e8640ee9be [smol] [perf] Inverse perm improvement (#11482) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2025-10-21 19:18:10 +08:00
Vincent Zhong a220536f40 [ perf ] Replace json-> orjson in hot path (#11221) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2025-10-12 20:30:58 +08:00
Vincent Zhong be740acdb0 [smol] [perf] Qwen3-VL in place op. (#11481) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2025-10-12 20:25:30 +08:00
Vincent Zhong 36a6b8dbfc Update v1/responses to be more OpenAI-compatible. (#9624) 2025-10-05 18:47:46 +00:00
Vincent Zhong 71959545df Fix gemma 3 launch with transformers: the error: AttributeError: 'TransformersForCausalLM' object has no attribute 'tp_size' (#9614) 2025-09-29 06:18:29 +00:00
Vincent Zhong 0b14159fc4 Add reasoning examples for GPT-OSS in Markdown examples (#9626) 2025-09-15 11:27:40 +08:00
Vincent Zhongandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 1489cd6c02 [docs / oneliner] update mmmu docs instruction (#9768) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-15 11:26:39 +08:00
Vincent Zhong 9a0d0b754d [Performance] Improve Qwen RMSNorm by replacing with native RMSNorm op (#9709) 2025-08-31 17:20:50 +08:00
Vincent Zhong 327f7b7c87 fix(grok): remove duplicate replicate_lm_head configuration (#9549) 2025-08-23 19:49:24 -07:00
Vincent Zhongandb8zhong 45a31a82e4 docs: Update documentation to reflect xgrammar as default grammar backend (#6601) 
Co-authored-by: b8zhong <b8zhong@uwaterloo.ca>
 2025-05-27 13:29:13 +08:00
Vincent ZhongandBrayden Zhong e9ef39d2e9 docs: Update the MD files (#6373) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-17 09:23:16 -07:00
vzed 95c231e50d Tool Call: Add chat_template_kwargs documentation (#5679) 2025-05-04 13:12:40 -07:00
vzed 094891c01a fix: Use is not None instead of != None for None checks. (#5687) 2025-04-26 19:26:57 -07:00
vzed df2cf583ce we fix the non existent access of decrypted_config_file (#5685) 2025-04-26 18:32:37 -07:00