chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Commit Graph

  • 673c11ba73 [Minor] Temporarily skipping deepep large mtp test (#14586) Baizhou Zhang 2025-12-07 13:59:16 -08:00
  • 3b47973af8 [CI] Tiny speed up VLM CI (#14517) b8zhong 2025-12-07 13:30:41 -08:00
  • f6423b626c Rename TensorRT Model Optimizer to Model Optimizer (#14455) Zhiyu 2025-12-07 13:18:20 -08:00
  • 84efe54bc4 Fix FP8 KV Triton type issue and add regression test (#14553) Hudson Xing 2025-12-07 10:51:46 -08:00
  • 948b6acee8 [BugFix] fix prefixcache performance and accuracy on ascend (#13573) khalilzhk 2025-12-08 02:16:20 +08:00
  • f124539a01 [NPU]LoRA: Adding Torch Native backend (#14132) Vladimir Serov 2025-12-07 21:16:07 +03:00
  • c8683ae305 [diffusion] cli: profiling utilities support (#14185) AichenF 2025-12-08 00:59:45 +08:00
  • 125e17efd5 Add small model test for spec v2 + dp + trtllm_mla (#14576) Liangsheng Yin 2025-12-07 23:55:00 +08:00
  • 88c459c6c8 Tiny remove wrong import from python.sglang (#14577) Liangsheng Yin 2025-12-07 22:07:49 +08:00
  • ae6a6630e4 Add Expert Parallelism (EP) support for kimi-k2-thinking (#13725) Xiaoyu Zhang 2025-12-07 20:28:57 +08:00
  • 26d95008b6 [apply][2/2] Fused qk_norm_rope for Qwen3-MoE (#13998) Yuan Luo 2025-12-07 20:25:18 +08:00
  • f2b5dcc976 Add unit-test-backend-8-gpu-b200 to rerun-stage command (#14569) Alison Shao 2025-12-07 03:09:07 -08:00
  • 9abcab3ffa [DLLM] feat: Add threshold based parallel decoding support (#14412) Tiwei Bie 2025-12-07 18:25:33 +08:00
  • e5135b73f4 Add CUDA kernel size analysis tool for sgl-kernel optimization (#14544) Xiaoyu Zhang 2025-12-07 15:29:41 +08:00
  • 41d61faa99 [FLA] Add explicit kernel arguments to kda.py for Kimi Linear support (#14561) Alison Shao 2025-12-06 22:45:41 -08:00
  • 3c7886ec4c Fix attention backend logic for Qwen3-Next on SM100 (#14560) Chen1022 2025-12-07 14:03:34 +08:00
  • 0e4d8790d7 feat(ci): add framework target to release-docker workflows (#14559) ishandhanani 2025-12-06 21:25:24 -08:00
  • 6d5d76ad97 remove unecessary dual stream token threshold from the rest of models (qwen moe, kimi linear, etc.) (#14337) b8zhong 2025-12-06 19:57:26 -08:00
  • 91c9c14c28 DOC update nemo-skills in docs (#14555) George Armstrong 2025-12-06 19:03:08 -08:00
  • 32a32cf7d2 Enhance prefill PP node robustness (#14494) Rain H 2025-12-07 10:00:54 +08:00
  • be4a3ec376 support piecewise cuda graph for Olmo models (#14476) Minglei Zhu 2025-12-06 17:57:44 -08:00
  • ff6e3ea934 [docs] Add missing word in argument description (#14205) almaslof 2025-12-07 03:56:54 +02:00
  • dd91d38e6a [Doc] Add short explanation on page size (#14557) b8zhong 2025-12-06 17:26:30 -08:00
  • 5f6f550af8 Update DeepSeek V3 docs to use B200 (#14447) Lee Nau 2025-12-06 17:22:11 -08:00
  • 5edbe351cb Update CI_PERMISSIONS.json (#14552) harrisonlimh 2025-12-06 14:01:32 -08:00
  • d2b42477c7 chore: bump sgl-kernel version to 0.3.18.post3 (#14518) sglang-bot 2025-12-06 13:15:16 -08:00
  • 9dfa01a435 [Misc]Register and refactor some environs for dpsk-fp4 and DeepEp (#14538) Baizhou Zhang 2025-12-06 12:29:16 -08:00
  • e592ee6545 [Qwen3-next] remove heuristics and add radix cache kl test (#14520) Hanming Lu 2025-12-06 12:11:40 -08:00
  • cee93a6f26 [Bug fix] Add /model_info endpoint to mini_lb (#14535) Alison Shao 2025-12-06 11:34:03 -08:00
  • bc388471d2 [1/n] Fix hanging during DeepGemm Warmup (#14493) Baizhou Zhang 2025-12-06 10:44:02 -08:00
  • 3e40c63674 fix "GrammarMatcher has terminated after accepting the stop token, but is trying to find the next token mask" when both reasoning and spec are enabled (#14464) gongwei-130 2025-12-06 06:15:22 -08:00
  • 80122e4f4c [diffusion] lora: fix LoRA dtype handling and weight attribute access for z-image model (#14543) WenhaoZhang 2025-12-06 22:14:44 +08:00
  • e12c6b320f [model-gateway][tracing]: implement request tracing using OpenTelemetry with trace context propagation (HTTP) (#13897) Feng Su 2025-12-06 21:59:04 +08:00
  • 6d41791823 [diffusion] perf: add QKV fusion optimization for Flux models (#14505) Xiaoyu Zhang 2025-12-06 20:44:16 +08:00
  • 35a9a07370 [diffusion] refactor: simplify sampling params' override logic (#14539) Mick 2025-12-06 20:23:49 +08:00
  • ea177372bd support mtp with deepseek r1 nvfp4 model (#13115) Rain Jiang 2025-12-06 00:45:54 -08:00
  • 42fcf5438f Revert "tiny remove deprecated endpoint call" (#14533) Baizhou Zhang 2025-12-05 23:48:54 -08:00
  • d257bf87b9 [CPU] add mamba fla kernels for Qwen3-next (#12324) blzheng 2025-12-06 14:16:23 +08:00
  • 7b0c7ad163 Revise DP Multi-Modal Encoder Document (#14290) Yuhao Yang 2025-12-06 12:56:53 +08:00
  • d30d6b368f [bug] fix notebook to include new keys from model_info (#14528) Simo Lin 2025-12-05 20:44:03 -08:00
  • d881f31488 [diffusion] chore: temporarily upgrade diffusers to make Z-image compatible with Cache-DiT (#14530) Mick 2025-12-06 12:39:37 +08:00
  • 2ac5b98395 fix: fix rmsnorm -> layernorm in qwen3 omni (#11791) Vincent Zhong 2025-12-05 23:12:57 -05:00
  • a0dde90af6 [CODEOWNER] update codeowner for qwen3-next related (#14522) Hanming Lu 2025-12-05 16:15:10 -08:00
  • b988c18eae Fix safetensors validation to catch corruption after download (#14465) Alison Shao 2025-12-05 16:04:00 -08:00
  • e41664ba1a [Docs] Add /rerun-stage command to contribution guide (#14521) Alison Shao 2025-12-05 15:46:47 -08:00
  • 3d1b591aa1 Tiny use trtllm_mha as default when possible (#14291) fzyzcjy 2025-12-06 06:26:03 +08:00
  • e11f795f63 chore: bump sgl-kernel version to 0.3.18.post3 (#14427) sglang-bot 2025-12-05 14:09:02 -08:00
  • 959a17466d [model-gateway] fix code owner for wasm (#14516) Simo Lin 2025-12-05 13:48:25 -08:00
  • b72f0268c7 [model-gateway] fix left over sgl-router names in wasm (#14514) Simo Lin 2025-12-05 13:16:58 -08:00
  • 09376fd738 [model-gateway] fix logs in smg workflow (#14513) Simo Lin 2025-12-05 12:50:50 -08:00
  • aed835e32d [model-gateway] fix left over sgl-router names to sgl-model-gateway (#14512) Simo Lin 2025-12-05 12:41:48 -08:00
  • 49dfa1d891 [model-gateway] change sgl-router to sgl-model-gateway (#14312) Simo Lin 2025-12-05 12:04:48 -08:00
  • 1ea6b740a7 [model-gateway] Make Tokenizer Builder Aware of Env Vars Like HF_ENDPOINT (#14405) Wenyi Xu 2025-12-06 02:42:06 +08:00
  • e73173b0d7 Fix removing worker will make it healthy forever in prometheus metrics (#14420) fzyzcjy 2025-12-06 02:40:42 +08:00
  • 16e8463a90 Add Mistral Large 3 basic test to PR CI (#14460) Alison Shao 2025-12-05 10:38:52 -08:00
  • cf9a774cf8 [model-gateway] fix server info comment (#14508) Simo Lin 2025-12-05 10:09:58 -08:00
  • ec7b2c16d9 tiny remove deprecated endpoint call (#13607) b8zhong 2025-12-05 09:54:49 -08:00
  • 1569fc7f49 [model-gateway] reorganized conversation handler (#14507) Simo Lin 2025-12-05 09:31:27 -08:00
  • 5a46fb153d [model-gateway] Add WASM support for middleware (#12471) Tony Lu 2025-12-06 01:29:07 +08:00
  • 38daa29466 Add fused FP8 KV cache write kernel for TRTLLM MHA backend (#14093) Hudson Xing 2025-12-05 08:53:55 -08:00
  • 66984a8b3d [diffusion] feat: support cache-dit integration (#14234) blahblah 2025-12-06 00:52:22 +08:00
  • 889b46ea50 [Spec] Mamba2 support in target models (#13434) roikoren755 2025-12-05 18:50:46 +02:00
  • 05284378d6 [model-gateway] move conversation to first class routing (#14506) Simo Lin 2025-12-05 08:44:13 -08:00
  • a89045603b [diffusion] chore: set allowing overriding protected fields of sampling params as default behavior (#14471) Mick 2025-12-06 00:22:42 +08:00
  • 662809874c Add Mistral Large 3 to nightly CI tests (#14459) Alison Shao 2025-12-05 07:16:27 -08:00
  • 205f041e96 Add Mistral Large 3 Eagle Support (#14466) elvischenv 2025-12-05 23:11:41 +08:00
  • 7235a7fbe9 [misc] add model arch and type to server info and use it for harmony (#14456) Simo Lin 2025-12-05 06:51:00 -08:00
  • 8fce9e7b2a support GLM-V vision model dp (#14097) Yuxuan Zhang 2025-12-05 21:03:54 +08:00
  • 5347732219 [diffusion] fix: Fix profiler trace missing Python stack in diffusion pipeline (#14499) Xiaoyu Zhang 2025-12-05 20:12:35 +08:00
  • 2ce121a1c3 Enable RadixCache for Mamba2 models (#13584) roikoren755 2025-12-05 12:23:58 +02:00
  • 35ba6fe19e [diffusion] fix: fix CLIP text encoder attention mask not used (#14364) WenhaoZhang 2025-12-05 16:30:10 +08:00
  • 498ea41ca6 dockerfile: add runtime stage + ubuntu 24.04 (#13861) ishandhanani 2025-12-05 00:28:36 -08:00
  • 7c744d137d [diffusion] cli: add argument --adjust-frames and --override-protected-fields (#13996) GMI Xiao Jin 2025-12-04 23:32:06 -08:00
  • 46b05ef58f [diffusion] fix: fix bug about pin memory when offloading (#14472) zyksir 2025-12-04 23:26:30 -08:00
  • beec8eed6a [diffusion] chore: further improve model searching logic (#14484) Mick 2025-12-05 15:04:55 +08:00
  • b76e303e6a clean up gemlite usage (#14444) Minglei Zhu 2025-12-04 21:52:56 -08:00
  • 80a575e4e8 Add YAMY1234 to CI Permission (#14475) Baizhou Zhang 2025-12-04 21:25:49 -08:00
  • 4c5074eb78 Add AMD stage support to /rerun-stage command and fix related bugs (#14463) Alison Shao 2025-12-04 21:13:52 -08:00
  • 41429a8c10 [ez] Fix typing (#14473) Yinghai Lu 2025-12-04 20:23:13 -08:00
  • 532037df78 Add CI permissions for user 'yushengsu-thu' (#14468) Alison Shao 2025-12-04 19:01:50 -08:00
  • fa0ca97694 [diffusion] improve: further optimize model load (#13836) zyksir 2025-12-04 18:45:20 -08:00
  • b5d3998508 Rename secrets.WHL_TOKEN -> secrets.GH_PAT_FOR_WHL_RELEASE (#14421) sglang-bot 2025-12-04 18:24:54 -08:00
  • 2ecee7571c [Bug] fix not desired disable fused share experts caused by rocm logic (#14432) Junrong Lin 2025-12-05 09:55:07 +08:00
  • 6d37e70883 ministral3 (#14251) Xinyuan Tong 2025-12-04 22:31:26 +00:00
  • c1006fd8a1 [model-gateway] grpc to leverage event type (#14450) Simo Lin 2025-12-04 14:30:19 -08:00
  • 29c6c2ea9d [model-gateway] add mistral 3 image processor (#14445) Simo Lin 2025-12-04 13:16:04 -08:00
  • eb85fa6daf [model-gateway] move all responses api event from oai to proto (#14446) Simo Lin 2025-12-04 13:11:03 -08:00
  • 0e6441b434 fix: Correct environment variable syntax in docker-compose configuration (#8287) Kay Yan 2025-12-05 05:03:39 +08:00
  • 88d1bab537 add doc for quantized kv cache (#14348) b8zhong 2025-12-04 13:01:05 -08:00
  • 922756aaa1 [FIX] trtllm-moe-fp4-renorm for Qwen series models (#14350) Sam 2025-12-05 04:52:21 +08:00
  • d8faf2f361 [model-gateway] move oai header util to router header util (#14441) Simo Lin 2025-12-04 11:02:50 -08:00
  • 7dfcc78155 [DeepseekV3.2][NSA][Indexer] Fix PAGED top-k transform for NSA indexer chunked execution on H200 (#14325) YAMY 2025-12-04 10:25:03 -08:00
  • 7f3308bc68 [model-gateway] extract conversation out of oai router (#14440) Simo Lin 2025-12-04 10:16:14 -08:00
  • fdc2ef58db [model-gateway] add llama4 vision image processor (#14438) Simo Lin 2025-12-04 09:52:36 -08:00
  • 1808df48fe [NPU]add nightly-test-npu (#14143) Cherry_ming 2025-12-05 00:43:35 +08:00
  • b01fc161eb [model-gateway] introduce request ctx for oai router (#14434) Simo Lin 2025-12-04 08:31:44 -08:00
  • 788628b56f [diffusion] feat: Add Configurable Generator Device and Seed Support via API (#14366) WenhaoZhang 2025-12-05 00:25:09 +08:00
  • 11d33c0e8f [model-gateway] add phi4 vision image processor (#14430) Simo Lin 2025-12-04 07:40:33 -08:00
  • 441420e149 Add mooncake transfer_engine_bench into maunal test (#14429) Liangsheng Yin 2025-12-04 22:35:00 +08:00
  • 29a2d4b59f Add 'NPU' to the runtime exception message in get_device (#14225) Raul Torres 2025-12-04 14:34:31 +00:00