chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Commit Graph

  • fadf18fdd5 docs: update installation (#7366) Yineng Zhang 2025-06-19 12:00:19 -07:00
  • f88e70853e [Bugfix][PD] Set conclude state before clear when failure happens (#7362) Shangming Cai 2025-06-20 02:26:53 +08:00
  • 4f838c09cd [PD] Transfer hidden states for mtp when disaggregation (#7242) Atream 2025-06-20 02:22:47 +08:00
  • d20a073bc3 Put _normalize_rid before other normalization in io_struct (#7363) Chang Su 2025-06-19 11:21:25 -07:00
  • 47367b768d [Refactor] Clean up radix cache related API (#7303) DarkSharpness 2025-06-19 09:58:48 -07:00
  • 650127a173 Reintroduce tiny fix sampler error when prob is not contiguous (#7354) fzyzcjy 2025-06-20 00:39:34 +08:00
  • 3774f07825 Multi-Stage Awake: Support Resume and Pause KV Cache and Weights separately (#7099) Stefan He 2025-06-19 00:56:37 -07:00
  • 9179ea1595 add seed in CPU UTs to avoid flaky failure (#7333) Chunyuan WU 2025-06-19 10:12:14 +08:00
  • 20a503c7d1 fix: resolve blackwell deepep image issue (#7331) Yineng Zhang 2025-06-18 17:04:02 -07:00
  • ffd1a26e09 Add more refactored openai test & in CI (#7284) Jinn 2025-06-18 15:52:55 -05:00
  • 09ae5b20f3 Merge PDLB (Prefill-Decode Load Balancer) into SGLang Router (#7096) Simo Lin 2025-06-18 11:28:15 -07:00
  • 712bf9ec9b [pd] optimize dockerfile for pd disaggregation (#7319) ybyang 2025-06-19 02:26:41 +08:00
  • 9c6a0656a3 Fix profiler error when there are idle passes (#7003) fzyzcjy 2025-06-19 01:55:01 +08:00
  • 2ae809c5c1 Fix mini_lb for PD with long output: limit chunk size of decode response (#7301) ch-tiger1 2025-06-19 01:46:46 +08:00
  • 1de4db9bef update invalid link in doc (#7297) linzhuo 2025-06-18 16:37:36 +08:00
  • 31fccf5a4f chore: change logs fromINFO to DEBUG for dp and add force quit for tokenizer manager (#7251) ishandhanani 2025-06-18 01:36:43 -07:00
  • b783c1cb82 Fix hicache benchmark script bug - some sampled input_request is [] (#7300) Binyao Jiang 2025-06-17 23:47:11 -07:00
  • 094c116f7d Update python API of activation, topk, norm and rope and remove vllm dependency (#6614) YanbingJiang 2025-06-18 13:11:50 +08:00
  • e56685ac1b Upstreaming hicache bug fixes (#7267) Zhiqiang Xie 2025-06-17 17:44:57 -07:00
  • c26d7349d3 [PD] Add custom memory pool option to support Mooncake PD with NVLink (#7264) shangmingc 2025-06-18 08:21:37 +08:00
  • ceaa85c9e6 [PD] Support get local ip from NIC for PD disaggregation (#7237) shangmingc 2025-06-18 08:19:26 +08:00
  • 0650e5176f fix: only enable flash_attn test on sm80 sm90 (#7289) Yineng Zhang 2025-06-17 16:56:41 -07:00
  • fc554105f6 ci: Fix test_ebnf_generate_all_optional_function_params (#7288) Chang Su 2025-06-17 16:39:42 -07:00
  • 4f204db57c fix: resolve b200 dsv3 mtp issue (#7286) Yineng Zhang 2025-06-17 16:22:46 -07:00
  • 3eb4a800e8 Fix AWQ Dequant and Weight Loading of deepseek v2 (#6842) AniZpZ 2025-06-18 04:45:10 +08:00
  • e726131523 bugfix(tool call ebnf): Fix EBNF generation for optional function parameters (#7283) Chang Su 2025-06-17 13:36:07 -07:00
  • 8c16da334e Fix Deepseek R1 0528 FP4 tensor name mismatch issue during weights loading. (#7164) Charles Chen 2025-06-17 11:26:23 -07:00
  • a39d928782 support qwen2 running on ascend npu device (#7022) Yijie Zhu 2025-06-18 02:24:10 +08:00
  • 10d60cd41b feat: mtp support dp-attention (#6081) u4lr451 2025-06-17 15:33:28 +08:00
  • 8a10c4c3d9 update ci node for xeon (#7265) DiweiSun 2025-06-17 14:44:08 +08:00
  • 405780bcf0 [amd] Opt dsv3 moe (#7160) kk 2025-06-17 13:26:51 +08:00
  • 1dffee31ac OAI Server Skeleton & Core Utility Endpoints (#7179) yhyang201 2025-06-17 11:45:55 +08:00
  • 70c471a868 [Refactor] OAI Server components (#7167) Xinyuan Tong 2025-06-16 20:45:20 -07:00
  • 1a9c2c9214 Fix AMD speculative decoding (#7252) Lianmin Zheng 2025-06-16 17:01:33 -07:00
  • 873ae12cee support custom weight loader for model runner (#7122) KavioYu 2025-06-17 07:28:15 +08:00
  • c64290dcb5 Use seq_len_fill_value in the cuda graph runners (#7233) Lianmin Zheng 2025-06-16 15:57:07 -07:00
  • 8e2363dc15 fix amd EP MoE FP8 issue (#7125) Alex Sun 2025-06-17 06:29:45 +08:00
  • f9dc9dd28b chore: bump v0.4.7.post1 (#7248) Yineng Zhang 2025-06-16 15:20:29 -07:00
  • 62a7aa2efc Update CI flakes. (#7244) Sai Enduri 2025-06-16 15:19:32 -07:00
  • 5ca07eed90 [fix] fix DeepGEMM blackwell input quant & ut & fix style and log (#7247) JieXin Liang 2025-06-17 02:45:54 +08:00
  • e30ef368ab Feat/support rerank (#6058) woodx 2025-06-17 01:50:01 +08:00
  • 91a066ec6a Tiny remove comments about DeepEP on H20 (#7234) fzyzcjy 2025-06-17 00:13:57 +08:00
  • c494386728 minor fix (#7245) Liangsheng Yin 2025-06-16 23:30:26 +08:00
  • 53a525bf33 [Eagle] Fix kernel call after updating speculative sampling kernels (#7231) Lianmin Zheng 2025-06-16 07:25:59 -07:00
  • 7ddf8e83d2 [EAGLE] Fix draft kv cache layout for fa3 and topk > 1 (#7239) Lianmin Zheng 2025-06-16 05:47:51 -07:00
  • 8321f8e45e Release sgl-kernel 0.1.9 (#7232) Lianmin Zheng 2025-06-16 03:37:40 -07:00
  • cfceb83d05 Fix sampling for speculative decoding & simplify kernels (#7207) Lianmin Zheng 2025-06-16 03:28:30 -07:00
  • b1286a116a [EAGLE] Refactor code for page size > 1 & more simplifications (#7213) Lianmin Zheng 2025-06-16 03:04:29 -07:00
  • 21615cc3fe Minor style and doc fix (#7228) Lianmin Zheng 2025-06-16 01:03:13 -07:00
  • 0ae1e9a755 refine fused_moe benchmark (#7221) Xiaoyu Zhang 2025-06-16 12:21:32 +08:00
  • e07d064729 Support LoRA in MMMU benchmark script. (#7218) Lifu Huang 2025-06-15 21:17:57 -07:00
  • 3c2274fbee Implement gather before attn (#6378) Cheng Wan 2025-06-15 21:08:56 -07:00
  • d2679f5109 Fix ChunkCache object has no attribute 'disable' (#7217) Baizhou Zhang 2025-06-15 20:55:15 -07:00
  • 96be97bfff Minor PD style fix (#7215) Byron Hsu 2025-06-15 16:12:12 -07:00
  • 88f9c347b2 [PD] use int32 for kv indices & get num_reserved_decode_tokens from server_args (#7214) Byron Hsu 2025-06-15 11:51:03 -07:00
  • fff10809bf Revert "[EAGLE] Refactor code for page size > 1 & more simplifications" (#7210) Lianmin Zheng 2025-06-15 02:48:00 -07:00
  • 5f1ab32717 [EAGLE] Refactor code for page size > 1 & more simplifications (#7163) Lianmin Zheng 2025-06-14 23:16:23 -07:00
  • 7df7c679b6 feat: use zstd for docker (#7205) Yineng Zhang 2025-06-14 23:13:29 -07:00
  • 38af4f68a9 Fix grammar abort & Minor style fixes (#7204) Lianmin Zheng 2025-06-14 22:49:41 -07:00
  • a6305c7d50 Lianmin/simplify memory pool (#7202) Lianmin Zheng 2025-06-14 22:25:37 -07:00
  • a023856b12 Move host memory pools into a separate file (#7200) Lianmin Zheng 2025-06-14 21:31:42 -07:00
  • db0cc57e75 [PD] Support decode retract and update decode.py (#7196) Byron Hsu 2025-06-14 19:48:05 -07:00
  • 349bb2c92a Fix error when disabling new DeepGEMM (#7198) fzyzcjy 2025-06-15 10:24:54 +08:00
  • 0b8939bc2c Fix NCCL 2.27.3 not in docker image (#7195) fzyzcjy 2025-06-15 09:36:11 +08:00
  • ed89837cf4 chore: upgrade sgl-kernel v0.1.8.post2 (#7186) JieXin Liang 2025-06-15 09:26:18 +08:00
  • 55561e2553 [fix] fix determine_num_fused_shared_experts (#7180) JieXin Liang 2025-06-15 08:41:22 +08:00
  • 4473320380 chore: bump v0.1.8.post2 (#7189) Yineng Zhang 2025-06-14 17:01:48 -07:00
  • 0bd67ba2bd Fix a minor bug related to DeepGEMM upgrade (#7191) Zhijian Liu 2025-06-14 16:54:40 -07:00
  • 7d316991b2 [PD] Update prefill.py (#7190) Byron Hsu 2025-06-14 15:59:54 -07:00
  • ab1a4fa5cb [fix] fix cutlass_mla_backend with cuda_graph and add sm_scale for sgl-kernel cutlass_mla (#7184) JieXin Liang 2025-06-15 03:45:41 +08:00
  • ed54bf9d19 [fix] fix dsv3 weight loader tqdm and simplify shared experts fusion (#7181) JieXin Liang 2025-06-15 02:56:29 +08:00
  • b57d87c297 Fix shared experts fusion + weight requant (#7177) fzyzcjy 2025-06-14 17:35:18 +08:00
  • 98538822d5 Add Phi-4-mm to supported VLM supported model list. (#7178) Lifu Huang 2025-06-13 23:17:40 -07:00
  • f47a1b1d0f Increase timeout in test/srt/test_disaggregation.py (#7175) Lianmin Zheng 2025-06-13 23:12:14 -07:00
  • 93cec4335f Support new DeepGEMM (#7172) fzyzcjy 2025-06-14 14:00:17 +08:00
  • ba589b88fc Improve test cases for eagle infer (#7173) Lianmin Zheng 2025-06-13 22:25:13 -07:00
  • 50876abc47 Add test for refactored openai server (#7161) Jinn 2025-06-13 22:42:57 -05:00
  • b4c41f7276 Refactor DeepGEMM integration (#7150) fzyzcjy 2025-06-14 11:41:03 +08:00
  • 8b8f2e7463 Support new DeepGEMM input format in silu_and_mul_masked_post_quant_fwd (#7153) fzyzcjy 2025-06-14 11:40:24 +08:00
  • 0fc3d992bb Split the eagle test into two files (#7170) Lianmin Zheng 2025-06-13 20:14:26 -07:00
  • be2d985df8 Minor style change of triton backend (#7165) Lianmin Zheng 2025-06-13 16:01:23 -07:00
  • 5b1afa7814 Re-quantize DeepSeek model weights to support DeepGEMM new input format (#7156) fzyzcjy 2025-06-14 06:57:45 +08:00
  • c49c1d9226 Remove 200us slow concat kernel (part 2: srt) (#7020) fzyzcjy 2025-06-14 06:19:31 +08:00
  • 0f1dfa1efe Tiny add sanity checks for DeepGEMM inputs (#7157) fzyzcjy 2025-06-14 05:36:27 +08:00
  • e3ec6bf4b6 Minor speed up block_quant_dequant (#6814) fzyzcjy 2025-06-14 05:32:46 +08:00
  • b04df75acd Fix DeepEP error in some environments (#7154) fzyzcjy 2025-06-14 05:27:25 +08:00
  • bec3e48402 Support new DeepGEMM format in per token group quant (part 2: srt) (#7155) fzyzcjy 2025-06-14 05:25:40 +08:00
  • 8ab7d93c2e chore: bump v0.1.8.post1 (#7152) Yineng Zhang 2025-06-13 03:14:26 -07:00
  • 5c66c4424f Support new DeepGEMM format in per token group quant (#7146) fzyzcjy 2025-06-13 17:00:22 +08:00
  • aa46ed34d2 Remove 200us slow concat kernel (part 1: kernel) (#7145) fzyzcjy 2025-06-13 16:58:29 +08:00
  • 2f4ec752bc filter by num_hidden_layers (#7056) pansicheng 2025-06-13 15:53:09 +08:00
  • da47621ccc Minor speedup topk postprocessing (#7058) fzyzcjy 2025-06-13 15:50:18 +08:00
  • 22a6b9fc05 Remove unnecessary metadata_expand.max_seq_len_k operations in fa3 to… (#7140) Binyao Jiang 2025-06-12 23:25:52 -07:00
  • b02df20a8d Minor add utility to read expert distribution recorder output (#7134) fzyzcjy 2025-06-13 07:50:05 +08:00
  • bd7cfbd2f8 [Fix] Reduce busy polling when scheduler is idle (#6026) Povilas Kanapickas 2025-06-13 00:58:22 +03:00
  • 4b9971e401 Add gfx950 support for sgl-kernel. (#7092) sogalin 2025-06-13 02:07:48 +08:00
  • dcc79d325b Update CODEOWNERS (#7126) Lianmin Zheng 2025-06-12 02:12:49 -07:00
  • 7046e0fab7 feat: update blackwell setup (#7119) Yineng Zhang 2025-06-12 01:54:40 -07:00
  • 930746d93c Improve log status (#7115) Liangsheng Yin 2025-06-12 14:38:24 +08:00
  • 84727a5139 [sgl-kernel] Add cuda kernel for moe_ep_silu_and_mul (#6919) Yuan Luo 2025-06-12 11:43:08 +08:00