chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Commit Graph

  • efa473348b [Spec Decoding] Support MTP for dsv3.2 (#11652) Paiiii 2025-10-19 23:44:22 +08:00
  • d658f0497e [overlap-spec] fix stop condition and trimming (#11819) Liangsheng Yin 2025-10-19 22:00:20 +08:00
  • 57e25de756 Revert "Fix: Dynamic RoPE Cache Expansion to Prevent Position-ID Out-of-Bounds in EAGLE + Long-Sequence Workloads" (#11827) Liangsheng Yin 2025-10-19 19:44:06 +08:00
  • 12eb02e982 Change bf16 to fp8 for some gemms in attention for DeepSeek ckpt v2 (#11805) fzyzcjy 2025-10-19 16:15:13 +08:00
  • 002d037359 Avoid generation gets hanging when user specifies multiple event loops (#5162) fzyzcjy 2025-10-19 16:12:49 +08:00
  • a27825ae01 Support not officially supported high sgl-kernel version with low srt version (#11786) fzyzcjy 2025-10-19 16:11:59 +08:00
  • ce399e154c Make single-batch overlap compatible with NextN (#11804) fzyzcjy 2025-10-19 16:10:44 +08:00
  • ea6275dfbc Tiny add hints when users send requests to wrong place (#11808) fzyzcjy 2025-10-19 16:10:20 +08:00
  • eb7318f1c2 support tokenized batch request (#11091) narutolhy 2025-10-19 00:05:02 -07:00
  • 6058fb520c Update CODEOWNERS for layer quantization path (#11818) Lianmin Zheng 2025-10-18 21:17:17 -07:00
  • 80407b0493 Fix: Dynamic RoPE Cache Expansion to Prevent Position-ID Out-of-Bounds in EAGLE + Long-Sequence Workloads (#10788) YAMY 2025-10-18 20:37:43 -07:00
  • b288f4f440 Improve send_sone script (#11817) Liangsheng Yin 2025-10-19 11:28:16 +08:00
  • 6d6ea5af0c fix: do not wrap invalid grammar objects during constrained generation (#11328) tazjin 2025-10-19 05:54:33 +03:00
  • 1dacedd2db make sure logit bias is applied during eagle spec decoding verification (#11555) Marin 2025-10-19 04:53:33 +02:00
  • b5e14b2b78 [1/2][feature] support openai like classification api (#11618) ybyang 2025-10-19 10:32:48 +08:00
  • d513ee93ef [2/2] [feature] support openai like classification api in router (#11670) ybyang 2025-10-19 10:31:08 +08:00
  • a7ae61ed77 [router] Add Configurable L0 and L1 Tokenizer Caching (#11688) Simo Lin 2025-10-18 18:33:53 -07:00
  • fda0cb2a30 Fix Dockerfile not installing correct version of DeepEP for arm build (#11773) kyleliang-nv 2025-10-18 15:06:05 -07:00
  • ebda73dc72 Use cutlass fp4 gemm by default (#11813) Qiaolin Yu 2025-10-18 14:10:15 -07:00
  • f4f8a1b4d8 ci: update lmms-eval to speed up multimodal CI (#11000) b8zhong 2025-10-18 11:51:19 -07:00
  • c44e985dc2 feat(example/fastapi): support --startup-timeout using Qwen3-Next-80B-A3B-Instruct as example (#11710) Kindyaa 2025-10-19 02:50:34 +08:00
  • f9a7d9b3dc support server arg override KV cache to bf16 to avoid slow cases (#11749) b8zhong 2025-10-18 11:49:48 -07:00
  • a93f10a722 [overlap-spec] support page size > 1 (#11772) Liangsheng Yin 2025-10-19 02:09:13 +08:00
  • 585e1223f0 [HiCache] feat: add more eviction policy (#11506) Teng Ma 2025-10-18 23:49:45 +08:00
  • a7043c6f0d Bump torch_memory_saver to avoid installing pre-release versions (#11797) fzyzcjy 2025-10-18 16:20:42 +08:00
  • 67e34c56d7 Fix install instructions and pyproject.tomls (#11781) Lianmin Zheng 2025-10-18 01:08:01 -07:00
  • 1d726528f7 Eager Compiler for Torch Compile (#11803) Yuwei An 2025-10-18 00:18:52 -07:00
  • f4488e9dd9 set default attention backend for deterministic inference (#11801) Minglei Zhu 2025-10-18 00:01:24 -07:00
  • e68a2b5b2f [RL] use cpu group to prepare_mlp_sync_batch_raw when the server is offloaded (#10152) Zilin Zhu 2025-10-18 14:29:35 +08:00
  • 31b9f19e54 [RL] support weight update with DP attention (#11669) Zilin Zhu 2025-10-18 14:26:19 +08:00
  • 547003bdd0 fix command line usage of profiling (#11793) Qiaolin Yu 2025-10-17 21:54:36 -07:00
  • f7ab955455 fix(glm45): disable reduce scatter (#11665) Jimmy 2025-10-18 12:19:20 +08:00
  • dbbd4e1891 Try add back no-commit-to-branch (#11799) fzyzcjy 2025-10-18 12:05:12 +08:00
  • ca240eefb4 [router][grpc] Support parallel queue puts in grpc_request_manager and remove mutex for grpc_client (#11798) Chang Su 2025-10-17 20:49:43 -07:00
  • 6c7c92eb02 Enable lint on main (#11794) fzyzcjy 2025-10-18 10:08:50 +08:00
  • 5b214b50b6 [Refactor] move deep_gemm_wrapper out of quantization (#11784) Cheng Wan 2025-10-17 18:57:54 -07:00
  • 13219e1e48 completely remove mixed mode deterministic test as prefix mode could cover it (#11783) Minglei Zhu 2025-10-17 17:46:03 -07:00
  • 33e9bbec35 Make single-batch overlap compatible with offloading (#11614) fzyzcjy 2025-10-18 08:45:54 +08:00
  • dcb8f090ad Super tiny fix CI (#11788) fzyzcjy 2025-10-18 08:41:58 +08:00
  • 9eefe2c0b7 Set CUDA_VISIBLE_DEVICES to achieve one GPU per process (#9170) Lianmin Zheng 2025-10-17 17:30:06 -07:00
  • 69fe3c9726 Manually flip deepep_mode for cuda_graph (#11666) Zilin Zhu 2025-10-18 08:05:48 +08:00
  • 8af8491298 Support casting bf16 NextN moe to fp8 (#11613) fzyzcjy 2025-10-18 08:02:15 +08:00
  • 505329cab0 Support shared experts overlap in cutlass moe (#11611) fzyzcjy 2025-10-18 07:59:40 +08:00
  • 8a382fd399 Super tiny fix missing input throughput (#11607) fzyzcjy 2025-10-18 07:58:48 +08:00
  • 627974405d [Lint] Add python/sglang to ruff F401 checks and remove unused imports in files (#11685) Chang Su 2025-10-17 16:49:46 -07:00
  • 2614adf9ca [Fix] Skip visual layers when applying LoRA to Qwen2VL modules (#11519) Antonin Vidon 2025-10-17 18:39:57 -04:00
  • fdd7c69d65 [Auto Sync] Update common.py (20251017) (#11782) Lianmin Zheng 2025-10-17 15:03:42 -07:00
  • b9a54e0968 [minor] sync code on python/sglang/test/test_deterministic.py and improve ci tests (#11777) Lianmin Zheng 2025-10-17 14:25:22 -07:00
  • 20b8d2306c Cleaning indexer for DeepSeek V3.2 (#11682) Baizhou Zhang 2025-10-17 15:47:21 -05:00
  • d1984e218c [router][grpc] Remove timeout for connections and remove max_tokens deprecation warning log (#11775) Chang Su 2025-10-17 12:36:36 -07:00
  • b79f75fd53 [Auto Sync] Update scheduler.py (20251017) (#11738) Yineng Zhang 2025-10-17 12:36:07 -07:00
  • 8fcc69e7c4 Turn on shm_allreduce and shm_allgather for fp16 (#10725) Chunyuan WU 2025-10-18 03:35:20 +08:00
  • f440baa136 [Feature] Reuse flashinfer workspace for PD-Multiplexing. (#11540) ykcombat 2025-10-18 02:35:06 +08:00
  • 2bc3fcd420 [doc] update router document (#11767) Keyang Ru 2025-10-17 10:26:54 -07:00
  • a5978a20f0 [router] fix grpc client time out to 1h (#11768) Simo Lin 2025-10-17 10:26:12 -07:00
  • e483c1eae5 [router] Fix UTF-8 Boundary Panic in Stop Sequence Decoder (#11766) Simo Lin 2025-10-17 10:21:00 -07:00
  • da681f35d3 Revert "Set csgmv as default lora backend. (#11488)" (#11735) Yineng Zhang 2025-10-17 10:01:36 -07:00
  • 9b0f725b1d add tuned fuse moe kernel for qwen3 235b fp8 on h200 (#11730) pdasgup 2025-10-17 09:55:09 -07:00
  • cde5a6e30f Abstraction for spec worker and code cleanup (#11643) Liangsheng Yin 2025-10-17 23:31:36 +08:00
  • 3e4c7da2f5 ci: reduce and refactor vlm ut and combine test files (#11062) Mick 2025-10-17 23:24:50 +08:00
  • d88ac9bc9a [overlap-spec] Make plan stream an option (#11724) Liangsheng Yin 2025-10-17 15:48:57 +08:00
  • ce11dd82dc [CI] Try fix broken event loop init (#11746) Liangsheng Yin 2025-10-17 13:30:17 +08:00
  • 9e87b60f37 [router][CI] Clean up deprecated fields in pr-test-pd-router.yml (#11739) Chang Su 2025-10-16 19:01:00 -07:00
  • 7780230a15 Revert "[router] fix get_models endpoint for openai router (#11687)" (#11740) Keyang Ru 2025-10-16 18:36:53 -07:00
  • dc01313da1 [router] Add rustfmt and set group imports by default (#11732) Chang Su 2025-10-16 17:33:29 -07:00
  • 7a7f99beb7 [router] add spec.rs to enables tests under spec folder (#11734) Keyang Ru 2025-10-16 16:07:26 -07:00
  • fd389df96e Reduce the image processing latency in VLM (#11541) StonyPort 2025-10-17 06:00:03 +08:00
  • b0d1d717e1 Revert "make radix cache deterministic" (#11728) Baizhou Zhang 2025-10-16 16:36:15 -05:00
  • c7962868c1 [router] Fix tool_choice normalization in ChatCompletionRequest and fix ut (#11731) Chang Su 2025-10-16 14:20:13 -07:00
  • 4f24ab1718 [router][grpc] add dissag info to warm up in grpc server (#11727) Simo Lin 2025-10-16 14:19:55 -07:00
  • 64affab495 [router] fix p and d worker filtering and bootstrap port handling (#11729) Simo Lin 2025-10-16 14:19:39 -07:00
  • 4c9bcb9d56 [Router] Refactor protocol definitions: split spec.rs into modular files (#11677) Keyang Ru 2025-10-16 13:44:44 -07:00
  • 86b04d25b3 model: qwen3-omni (thinker-only) (#10911) Mick 2025-10-17 04:20:38 +08:00
  • 85ebeecf06 chore: bump SGLang version to 0.5.3.post3 (#11693) sglang-bot 2025-10-16 13:14:55 -07:00
  • 0dd6cf16ba [ci]use H20 to run disaggregation test (#11543) Hank Han 2025-10-17 02:42:42 +08:00
  • 0975ba99bc [router] fix get_models endpoint for openai router (#11687) Keyang Ru 2025-10-16 09:00:08 -07:00
  • 1de3924b18 [CI] Add GLM4MoE model test (#11706) Shangming Cai 2025-10-16 16:25:58 +08:00
  • 3cceaa381a [Bugfix] Fix Qwen3/DSV3/DSV3.2 model support (#11510) Even Zhou 2025-10-16 15:14:09 +08:00
  • b0d20cdec7 Set csgmv as default lora backend. (#11488) Lifu Huang 2025-10-15 21:53:24 -07:00
  • cbac499750 Split test_intel_amx_attention_backend.py to pass CI of timeout (#11370) YanbingJiang 2025-10-16 10:22:32 +08:00
  • 476c67d7fc Fix missing a2a backend init of GLM4.5 MoE Block (#11692) Shangming Cai 2025-10-16 10:13:08 +08:00
  • 3289da5b41 [sgl-kernel] support hadamard (#11663) Fan Yin 2025-10-16 10:00:44 +08:00
  • 868403f642 [PD] Add PD support for hybrid model (Qwen3-Next, DeepSeek V3.2 Exp) (#10912) Shangming Cai 2025-10-16 09:59:14 +08:00
  • 97d857c096 [Mamba] Increase default mamba_full_memory_ratio to 0.9 (#11679) Hanming Lu 2025-10-15 18:56:43 -07:00
  • 52a54a26b2 docs: Add Contributor Covenant Code of Conduct (#11689) Yineng Zhang 2025-10-15 18:50:26 -07:00
  • cd7e1bd591 Sync code and test CI; rename some env vars (#11686) Lianmin Zheng 2025-10-15 18:37:03 -07:00
  • 729b7edf72 enable rmsnorm on XPU (#10248) Huaiyu, Zheng 2025-10-16 08:54:18 +08:00
  • 4c03dbaaef [CI][XPU]enable sglang CI on Intel XPU (#9493) DiweiSun 2025-10-16 08:13:19 +08:00
  • baf277a9bf chore: bump SGLang version to 0.5.3.post2 (#11680) sglang-bot 2025-10-15 16:49:14 -07:00
  • f5d30dae89 [router] Refactor StopSequenceDecoder to Use Sequence for Incremental Decoding (#11676) Simo Lin 2025-10-15 16:31:03 -07:00
  • 2479b89405 [router][grpc] Simplify model_id determination (#11684) Chang Su 2025-10-15 15:56:58 -07:00
  • 5464457251 [sgl-kernel] Optimize gguf test (#11667) Fan Yin 2025-10-16 06:45:53 +08:00
  • 6c01844f45 [sgl-kernel][3/N]Support Expert Specialization Grouped GEMM (#11674) Qi Yuhang 2025-10-16 04:39:31 +08:00
  • f226d3da2a Fix missing json imports in serving_responses.py (#11681) Chang Su 2025-10-15 13:01:55 -07:00
  • d2478cd4ff [router] Fix response api related spec (#11621) Keyang Ru 2025-10-15 09:59:38 -07:00
  • 30ea4c462b [tool call] Fix prev_tool_call_arr management in base_format_detector.py (#11367) Chang Su 2025-10-15 09:51:51 -07:00
  • 6d0364681c Fix 1-step draft model forward (#11653) Shangming Cai 2025-10-15 19:11:33 +08:00
  • 8221f9ae8b Tiny cleanup some eagle unused codes (#11660) Liangsheng Yin 2025-10-15 17:24:08 +08:00
  • ab9187a20b docs: update sglang installation guide (#11659) Yineng Zhang 2025-10-15 00:35:48 -07:00
  • 6b143d62a2 Clean up some Qwen3-Next and deterministic code (#11585) Stefan He 2025-10-15 00:19:37 -07:00