chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Commit Graph

  • cc858953a0 Fix recv_requests (#2405) Lianmin Zheng 2024-12-08 04:08:04 -08:00
  • 6128f7cff5 fix: specify dtype with begin_forward aka plan (#2404) Yineng Zhang 2024-12-08 20:07:30 +08:00
  • a2486eb58f Fix a bug with logprob streaming + chunked prefill (#2403) Lianmin Zheng 2024-12-08 03:55:27 -08:00
  • 61dec545b0 Remove unused vars in the triton backend (#2401) Ke Bao 2024-12-08 19:37:03 +08:00
  • 96db0f666d Update killall_sglang.sh (#2397) Lianmin Zheng 2024-12-08 01:56:26 -08:00
  • 7dc66fcb40 Optimize Triton decoding kernel for long context (#2394) Ke Bao 2024-12-08 17:17:37 +08:00
  • 1f09e84b9a nit: Remove busy waiting on scheduler (#2382) SangBin Cho 2024-12-08 01:06:15 -08:00
  • 63dfab1bea Fix shape error that occurred when loading lora weight of gemma2 model. (#2330) Sangchun Ha (Patrick) 2024-12-08 18:04:08 +09:00
  • ef995dae1e [router] Health check on worker before adding to the router (#2392) Byron Hsu 2024-12-07 15:39:54 -08:00
  • 75ae968959 minor: update killall script (#2391) Yineng Zhang 2024-12-08 04:21:00 +08:00
  • 95f93f493a Fp8 MoE optimizations on AMD (#2388) HAI 2024-12-07 05:18:26 -08:00
  • aaac33fd8d fix: update xgrammar v0.1.6 (#2390) Yineng Zhang 2024-12-07 21:09:16 +08:00
  • d332aa3b0c fix: resolve fp8 moe issue (#2387) Yineng Zhang 2024-12-07 19:28:53 +08:00
  • c36736c841 [router] Add remove worker api (#2380) Byron Hsu 2024-12-06 17:16:03 -08:00
  • 1bf9e34745 [router] add remove tenant method in the radix tree (#2379) Byron Hsu 2024-12-06 11:53:15 -08:00
  • 499c85f131 [Router] remove duplicate char count (#2378) Byron Hsu 2024-12-06 11:26:07 -08:00
  • e5f227c0ee Release v0.4.0.post1 (#2375) Lianmin Zheng 2024-12-06 06:08:19 -08:00
  • 0e7409adb6 Fix the overlap for xgrammar (#2377) Lianmin Zheng 2024-12-06 05:49:29 -08:00
  • 3cde5eb629 docs: Improve instructions for supporting new models (#2363) vchzls 2024-12-06 20:27:17 +08:00
  • f5b2a3aa67 Use proc.join instead of busy waiting (#2374) Lianmin Zheng 2024-12-06 02:01:23 -08:00
  • f68175967c docs: update adoption (Meituan) (#2373) Yineng Zhang 2024-12-06 17:59:26 +08:00
  • 67b657945a [router] support /add_worker api (#2369) Byron Hsu 2024-12-06 01:17:04 -08:00
  • 37ee906f61 Add more support for intel Gaudi accelerators (#2357) Qun Yang 2024-12-06 17:16:33 +08:00
  • 34b364e073 optimize cuda graph max_bs_settings on low-end gpus (#2360) Xiaoyu Zhang 2024-12-06 17:13:04 +08:00
  • 84d96b3ae5 Move FP8 to SGLang (#2370) Yineng Zhang 2024-12-06 15:42:10 +08:00
  • 3d32e4a32c Resubmit MoE-EP (#2371) xiaobochen 2024-12-06 15:05:21 +08:00
  • 64fceab8af [router] use 2-gpu-runner (#2368) Byron Hsu 2024-12-05 17:46:21 -08:00
  • 71e2a27753 Fix the cuda graph capture range for small #max-running-requests (#2359) Lianmin Zheng 2024-12-05 13:42:47 -08:00
  • 4a63c181f1 Fix AWQ with enable MLA (#2364) Ke Bao 2024-12-06 00:46:48 +08:00
  • 2b0fc5941d [Minor] Code style improvements (#2355) Lianmin Zheng 2024-12-04 19:02:08 -08:00
  • 9cc733b38c move apply_torchao_config_ to model_runner (#2342) Jerry Zhang 2024-12-04 17:26:42 -08:00
  • d693ec0427 Make torch TP composable with torch.compile (#2352) Ke Wen 2024-12-04 17:26:00 -08:00
  • 18ea841f40 Add Docs For SGLang Native Router (#2308) Chayenne 2024-12-04 15:41:22 -08:00
  • 786be44da5 Fix Docs CI When Compile Error (#2323) Chayenne 2024-12-04 11:19:46 -08:00
  • 2db4469808 minor: limit the range of vllm versions (#2350) Yineng Zhang 2024-12-05 02:00:34 +08:00
  • ed45e509df Check gpu availability at server args creation (#2340) Ata Fatahi 2024-12-04 09:53:02 -08:00
  • ec52464dde MLA prefill w/o weight absorption (#2349) Ke Bao 2024-12-05 01:50:28 +08:00
  • eb0c1f5373 docs: add SGLang v0.4 blog (#2341) Yineng Zhang 2024-12-05 01:24:51 +08:00
  • b2986d7aa5 Adding SGLang FP8 Utils (#2348) HAI 2024-12-04 03:01:33 -08:00
  • f8b0326934 chore: bump v0.4.0 (#2338) Yineng Zhang 2024-12-04 03:55:41 +08:00
  • 0495796517 [router] Copy license when publishing & bump version (#2339) Byron Hsu 2024-12-03 10:27:43 -08:00
  • 1228f7ca69 Fix gptq for moe layers (#2300) Lianmin Zheng 2024-12-03 07:12:33 -08:00
  • fda628d8f2 fix: resolve cmake url for Dockerfile.dev (#2335) Yineng Zhang 2024-12-03 21:22:19 +08:00
  • 07ec07ad1f Improve torch compile for fused moe (#2327) Lianmin Zheng 2024-12-03 01:58:25 -08:00
  • 83b340e371 Add missing license for router wheel (#2324) Ata Fatahi 2024-12-03 00:06:25 -08:00
  • 0639bf15d1 ROCm Container: set SGLANG_SET_CPU_AFFINITY=1 (#2328) HAI 2024-12-02 23:20:33 -08:00
  • aa47f64223 Revert "[feat] Enable chunked prefill for llava-onevision" (#2329) Ying Sheng 2024-12-02 23:11:13 -08:00
  • 3ddb1c4679 [Minor] Fix logger and style (#2325) Lianmin Zheng 2024-12-02 20:45:53 -08:00
  • 480e38a733 [feat] Enable chunked prefill for llava-onevision (#2281) Ying Sheng 2024-12-02 20:19:02 -08:00
  • 69e2d4fb66 Relax to include more AMD GPUs (#2319) HAI 2024-12-02 19:05:58 -08:00
  • 85e1a6f3aa Update model_loader deps and qqq quantization deps (#2220) (#2318) Yineng Zhang 2024-12-02 23:22:13 +08:00
  • 33deca81b5 Add more fused moe benchmark utilities (#2314) Lianmin Zheng 2024-12-02 04:26:55 -08:00
  • 18108abe5d [Minor] Fix code style (#2311) Lianmin Zheng 2024-12-02 02:27:36 -08:00
  • c54bda300a Use rocminfo instead of rocm-smi for more OS/WSL support (#2310) HAI 2024-12-02 00:15:45 -08:00
  • 3c79ad35ca [Fix] Fix the padded hash value for image tokens (#2309) Lianmin Zheng 2024-12-01 23:36:28 -08:00
  • 983bfcf386 Online weight updates from torch.distributed (#2279) Chayenne 2024-12-01 23:23:18 -08:00
  • 28bc60dcab misc: update build setup (#2306) Yineng Zhang 2024-12-02 02:03:49 +08:00
  • 7301a39b13 fix: resolve CodeQL cpp issue (#2305) Yineng Zhang 2024-12-01 23:55:19 +08:00
  • 47eb139f81 feat: use warp reduce as a simple example (#2304) Yineng Zhang 2024-12-01 22:43:50 +08:00
  • 5c18a03733 Fix logprob for completions (#2301) Lianmin Zheng 2024-12-01 05:17:05 -08:00
  • 5c91a315d7 feat: support sgl-kernel pypi (#2302) Yineng Zhang 2024-12-01 20:11:21 +08:00
  • 3dbd73d319 minor: rm unused _grouped_size_compiled_for_decode_kernels (#2299) Yineng Zhang 2024-12-01 19:24:12 +08:00
  • e9a6203dee feat: skip good first issue (#2298) Yineng Zhang 2024-12-01 19:18:57 +08:00
  • 62c516ac45 Add a simple torch native attention backend (#2241) Qun Yang 2024-12-01 19:01:25 +08:00
  • fc78640e00 minor: support flashinfer nightly (#2295) Yineng Zhang 2024-12-01 18:55:26 +08:00
  • 906d795f15 Feat: upgrade outlines & support compatibility with the old version (#2292) gobraves 2024-12-01 18:07:27 +08:00
  • 118b6af35e feat: add should_use_tensor_core (#2179) Yineng Zhang 2024-12-01 18:01:16 +08:00
  • 9449a95431 [CI] Balance CI tests (#2293) Lianmin Zheng 2024-12-01 01:47:30 -08:00
  • 5f12f0e7af Fix chunked prefill when ignore eos (#2290) Liangsheng Yin 2024-12-01 00:37:53 -08:00
  • d5b95cbb53 adapt vllm distributed module to sglang (#2244) yizhang2077 2024-12-01 15:54:52 +08:00
  • 0303ca918f [CI] Fix missing files in run_suite.py (#2288) Lianmin Zheng 2024-11-30 23:53:34 -08:00
  • 00181098dd feat: add Dockerfile for development (#2289) Yineng Zhang 2024-12-01 15:27:52 +08:00
  • 4936be8acc Revert "Revert "[FEAT] Support GGUF format"" (#2287) Lianmin Zheng 2024-11-30 22:14:48 -08:00
  • 1bfa511b95 [CI] Fix ci tests (#2284) Lianmin Zheng 2024-11-30 21:12:03 -08:00
  • f5b5f2bff9 Revert "[Fix] fix assertion error for chunked prefill when disabling cache" (#2286) Lianmin Zheng 2024-11-30 19:03:42 -08:00
  • 7e4c6dd8da Revert "[FEAT] Support GGUF format" (#2285) Lianmin Zheng 2024-11-30 19:03:26 -08:00
  • d622851dc9 [Fix] fix assertion error for chunked prefill when disabling cache (#2282) Rui Wang 2024-12-01 09:53:43 +08:00
  • 883c955489 [FEAT] Support GGUF format (#2215) Yang Zheng 2024-11-30 16:44:48 +08:00
  • 0d6a49bd7d [CI] Kill zombie processes (#2280) Lianmin Zheng 2024-11-30 00:24:30 -08:00
  • ccaf1f997c [CI] Print summary on github actions (#2274) Lianmin Zheng 2024-11-29 23:48:54 -08:00
  • 7d1485d376 Add get weights by parameter name for llama (#2266) Chayenne 2024-11-29 23:36:38 -08:00
  • 7d5d1d3d29 udate weights from disk (#2265) Chayenne 2024-11-29 17:17:00 -08:00
  • b53d6cbda3 Add new contributors so they can trigger CI automatically (#2269) Lianmin Zheng 2024-11-29 16:37:52 -08:00
  • 01017d4c20 Support LoRA in Completion API (#2243) bjmsong 2024-11-30 08:13:38 +08:00
  • 94e167ea5a Fix the default chunked prefill size (#2268) Lianmin Zheng 2024-11-29 16:03:32 -08:00
  • 262e370f78 [benchmark] Add fused_moe_triton benchmark and tuning tools (#2225) Xiaoyu Zhang 2024-11-30 05:36:45 +08:00
  • 419a57e771 minor: add sgl-kernel dir (#2261) Yineng Zhang 2024-11-30 02:27:35 +08:00
  • fae4e5e99a chore: bump v0.3.6.post3 (#2259) Yineng Zhang 2024-11-30 01:41:16 +08:00
  • afe1e46586 [Minor] fix the style for multimodal models (#2257) Lianmin Zheng 2024-11-29 04:24:20 -08:00
  • f50a6cf443 Fix hash collision for multi modal models (#2256) Lianmin Zheng 2024-11-29 03:15:58 -08:00
  • fe97a2d40f Simplify tokenizer manager (#2254) Lianmin Zheng 2024-11-29 02:18:51 -08:00
  • 8b48496aaf Revert "Revert "Add simple CPU offloading support"" (#2253) Ying Sheng 2024-11-28 23:58:54 -08:00
  • 4057ea82c9 Revert "Add simple CPU offloading support" (#2252) Ying Sheng 2024-11-28 23:36:55 -08:00
  • 4f2ee48ed1 Update backend.md (#2251) Lianmin Zheng 2024-11-28 23:18:07 -08:00
  • 71ff2728a1 Update backend.md (#2250) Lianmin Zheng 2024-11-28 23:14:36 -08:00
  • b7038fec9b [fix] Fix prefix caching for multi-image/video (#2239) Ying Sheng 2024-11-28 12:08:13 -08:00
  • 65fdb28929 fix missing launch server import (#2242) Enrique Shockwave 2024-11-28 13:24:47 +00:00
  • b2ccf36d4d Fix memory leak during abort (#2238) Lianmin Zheng 2024-11-28 02:22:15 -08:00
  • d4fc1a70e3 Crash the server correctly during error (#2231) Lianmin Zheng 2024-11-28 00:22:39 -08:00
  • db674e3d24 Add OLMo2 model. (#2233) Jani Monoses 2024-11-28 10:15:20 +02:00