chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Commit Graph

  • d9b3b01883 enable marlin kernels (#286) Enrique Shockwave 2024-03-13 02:10:12 +00:00
  • 745ea007ac Fix Incorrect CURL Request Example in README (#287) Arsalan 2024-03-12 22:09:38 -04:00
  • ad1dd74673 Fix flashinfer >= 0.0.3 compat (#282) Qubitium 2024-03-12 21:45:58 +08:00
  • eb4308c4c9 adding the triton docker build minimal example (#242) Arsalan 2024-03-12 03:16:06 -04:00
  • b2eb080501 Fix Runtime missing some ServerArgs options (#281) Qubitium 2024-03-11 22:32:15 +08:00
  • 4aa5dd2c5f Update version to v0.1.13 (#280) Lianmin Zheng 2024-03-11 05:49:27 -07:00
  • 13662fd533 Fix RuntimeEndpoint (#279) Lianmin Zheng 2024-03-11 05:24:24 -07:00
  • d5ae2ebaa2 Add Support for API Key Authentication (#230) Alessio Dalla Piazza 2024-03-11 13:16:10 +01:00
  • 1b35547927 Organize server_args (#277) Liangsheng Yin 2024-03-11 20:06:52 +08:00
  • faba293a0d Improve gemma and documentations (#278) Lianmin Zheng 2024-03-11 04:43:39 -07:00
  • 89885b31ef Gemma Support (#256) Liangsheng Yin 2024-03-11 12:14:27 +08:00
  • 64fe311593 replace skip_embed with input_embeds (#222) Geary.Z 2024-03-11 10:04:52 +08:00
  • a7ace9c88d Fix qwen config (#261) Liangsheng Yin 2024-03-11 09:54:18 +08:00
  • a833de05d3 Add logo (#275) Lianmin Zheng 2024-03-10 18:51:47 -07:00
  • 30d67b2bca Add set_var to interpreter.py (#263) Lin Tianchuan 2024-03-07 23:20:11 +08:00
  • b0b722ee8e Refactor ChatTemplate for Enhanced Clarity and Efficiency (#201) Xinwei Xiong 2024-03-03 17:52:36 +08:00
  • 01b07ea3ac Add SSL Cert Functionality (#224) Srinivas Billa 2024-03-03 09:41:41 +00:00
  • dfb13ac455 Fix addr reuse in check_port (#253) Liangsheng Yin 2024-03-03 17:09:16 +08:00
  • ec90b9c054 Upload agent_calls.jsonl download link (#226) Liangsheng Yin 2024-02-24 19:03:46 +08:00
  • 9759d927cf fix chatml template (#195) Enrique Shockwave 2024-02-24 08:34:22 +00:00
  • 8d0a7fae3b Fix interpreter.py get_var(var_name) in text iter when stream is not enabled (#198) Zhang Wenbin 2024-02-24 16:27:34 +08:00
  • c4e9ebe3a4 Fix stop str merging (#225) Liangsheng Yin 2024-02-24 16:05:21 +08:00
  • 3c2c5869ad Support outlines > 0.0.31 (#219) Cody Yu 2024-02-23 23:06:17 -08:00
  • 4cb9aaedf3 Fix logprobs with logprob_start_len (#193) Cody Yu 2024-02-22 10:33:03 -08:00
  • 9de9a46815 Added the ability to Modify the Context Length (#210) psych0v0yager 2024-02-20 18:22:56 -06:00
  • ce3b261053 Update README.md (#207) Ikko Eltociear Ashimine 2024-02-20 02:09:03 +09:00
  • 91e036334f Adjust outlines version. (#200) Liangsheng Yin 2024-02-17 13:40:39 +08:00
  • 2a74748b2f Pin outlines version (#196) Cody Yu 2024-02-16 13:01:40 -08:00
  • 63ba630bbb Refactor decoding logprob and add completion_tokens_wo_jump_forward (#189) Cody Yu 2024-02-15 10:54:20 -08:00
  • 6493256b7d improve print Lianmin Zheng 2024-02-12 12:43:48 +00:00
  • 06008bc295 Fix server launch for jupyter notebook (#186) Lianmin Zheng 2024-02-12 04:43:14 -08:00
  • bb824da41a Add Together and AzureOpenAI examples (#184) Lianmin Zheng 2024-02-12 01:06:38 -08:00
  • 931213245c correct reference dtype openai.py (#181) Yaya Sy 2024-02-11 22:26:20 +01:00
  • c97fdae4aa correct a mistake on the README.md (#182) Yaya Sy 2024-02-11 22:25:57 +01:00
  • 624b21e742 Update version to 0.1.12 (#178) Lianmin Zheng 2024-02-11 06:43:45 -08:00
  • c51020cf0c Fix the chat template for llava-v1.6-34b & format code (#177) Lianmin Zheng 2024-02-11 05:50:13 -08:00
  • 50afed4eaa Support extra field regex in OpenAI API (#172) Cody Yu 2024-02-10 17:21:33 -08:00
  • 4d303c4fa3 Fix token usage with jump forward (#174) Cody Yu 2024-02-09 20:06:15 -08:00
  • 37b42297f8 import outlines (#168) Liangsheng Yin 2024-02-09 10:13:02 +08:00
  • cba5027332 Fix BaseCache metric (#170) Cody Yu 2024-02-08 17:23:09 -08:00
  • a6aa46dd3f minor Ying Sheng 2024-02-08 04:35:25 +00:00
  • 405f26b00b Add Auth Token to RuntimeEndpoint (#162) Srinivas Billa 2024-02-08 04:07:31 +00:00
  • b1a3a454ee add --disable-disk-cache (#160) Liangsheng Yin 2024-02-08 00:50:12 +08:00
  • 79e6b84bec Update README.md Ying Sheng 2024-02-06 23:14:59 -08:00
  • 26c3494152 [Submodule] Change FlashInfer to import (#156) Cody Yu 2024-02-06 19:28:29 -08:00
  • cb8e1982f8 Update README.md Ying Sheng 2024-02-06 18:44:37 -08:00
  • 23f05005fd Format code & move functions (#155) Lianmin Zheng 2024-02-06 13:27:46 -08:00
  • a7334aeea1 Support decode token logprobs (#130) Cody Yu 2024-02-06 12:24:55 -08:00
  • ee1df26a77 Update README.md Lianmin Zheng 2024-02-06 11:35:42 -08:00
  • 3ae78a09b3 Add gptq quantization model support (#141) Arcmoon 2024-02-07 03:35:04 +08:00
  • ccbe1e67d8 Temporary fix OpenAI API for Pydantic v1/v2 (#153) Cody Yu 2024-02-06 11:34:15 -08:00
  • e2bf732bc3 add openai error handler with retry and logger (#148) LiviaSun 2024-02-06 12:38:41 +08:00
  • 322421fae3 Add warmup to SRT server (#146) Cody Yu 2024-02-05 14:21:16 -08:00
  • 8ff870bf3e improve docs Lianmin Zheng 2024-02-05 11:22:06 +00:00
  • 26f0bedc8f jump-forward rename (#144) Liangsheng Yin 2024-02-05 16:50:37 +08:00
  • 82fa69b3cc fix undfined variable (#142) Yaya Sy 2024-02-04 23:27:52 +01:00
  • 8fb7459e08 update json decoding docs Lianmin Zheng 2024-02-03 17:42:01 -08:00
  • bb3a3b6675 Support Faster JSON decoding for llava (#137) Liangsheng Yin 2024-02-03 23:32:05 +08:00
  • 45d6592d40 Fix no-cache mode (#136) Ying Sheng 2024-02-03 04:59:06 -08:00
  • f6bfe3aaff Release 0.1.11 (#134) Ying Sheng 2024-02-03 02:50:13 -08:00
  • e095b16236 Add max_prefill_num_token into server arguments (#133) Ying Sheng 2024-02-03 02:35:54 -08:00
  • 67be11c790 fix bug of race condition in copy() Ying Sheng 2024-02-03 01:38:00 -08:00
  • cd8c3ccd95 Fix is_multimodal_model judge (#132) Liangsheng Yin 2024-02-03 11:48:01 +08:00
  • 9c121f2a45 minor fix: result dump format hnyls2002 2024-02-02 09:58:24 +00:00
  • 03e04b2331 update docs for Yi-VL Lianmin Zheng 2024-02-01 22:44:05 +00:00
  • 864425300f Yi-VL Model (#112) Christopher Chou 2024-02-01 08:33:22 -08:00
  • 79cb018e4b Add city doc benchmark mode (#129) Liangsheng Yin 2024-02-01 13:38:47 +08:00
  • c7af9f7393 Fix a bug in llava-hd Lianmin Zheng 2024-01-31 18:52:15 +00:00
  • 876db8dc7a Update sampling_params.md Lianmin Zheng 2024-01-31 10:18:43 -08:00
  • ad82bac6f5 Fix model loading & format code (#125) Lianmin Zheng 2024-01-30 23:49:52 -08:00
  • 71b54eea7d Add cache metrics (#119) Cody Yu 2024-01-30 22:13:14 -08:00
  • 74b3bfaaf8 format code Lianmin Zheng 2024-01-30 16:36:10 +00:00
  • 4a634cf646 [Feature] Allow specifying all ports to use in advance (#116) Jay Zhou 2024-01-30 08:34:51 -08:00
  • a49dc52bfa release v0.1.10 Lianmin Zheng 2024-01-30 15:37:43 +00:00
  • 873d0e8537 Ignore detokenization error Lianmin Zheng 2024-01-30 14:52:06 +00:00
  • 1d0fbe8e43 [Feature] Adds basic support for image content in OpenAI chat routes (#113) Keith Stevens 2024-01-30 23:12:33 +09:00
  • 97aa9b3284 Improve docs & Add JSON decode example (#121) Lianmin Zheng 2024-01-30 05:45:27 -08:00
  • 0617528632 Update quick start examples (#120) Lianmin Zheng 2024-01-30 04:29:32 -08:00
  • 4ea92f8307 Format code (#118) Lianmin Zheng 2024-01-29 17:08:12 -08:00
  • 6b0af2853c Add qwen2 (#114) Junyang Lin 2024-01-30 09:06:02 +08:00
  • 6f560c761b Improve the control of streaming and improve the first token latency in streaming (#117) Lianmin Zheng 2024-01-29 17:05:42 -08:00
  • cd6872334e Fix Mistral model loading (#108) Cody Yu 2024-01-26 09:38:43 -08:00
  • 81561f8e2d Flush Cache API (#103) Liangsheng Yin 2024-01-26 13:32:59 +08:00
  • 3a581e9949 Dynamic model class loading (#101) Cody Yu 2024-01-25 15:29:07 -08:00
  • 0147f940dd fix batch error for llava-hd (#98) shiyi.c_98 2024-01-25 07:56:25 -08:00
  • 23950056f0 support speculative execution for openai API (#48) parasol-aser 2024-01-25 03:57:06 -06:00
  • 93414c8238 Add a link to HF paper page Lianmin Zheng 2024-01-24 22:25:33 -08:00
  • ed7c7eca0e Update README.md Lianmin Zheng 2024-01-24 16:52:21 -08:00
  • 0c457bae8f Handle grayscale images in expand2square (#97) isaac-vidas 2024-01-24 19:23:11 -05:00
  • d3fc86a43e Improve Chinese character streaming when the last char is half Chinese word. (#95) Haotian Liu 2024-01-24 14:23:27 -06:00
  • 01ee0fbc05 fast regex decode Liangsheng Yin 2024-01-25 01:16:25 +08:00
  • 711d343530 add a batch llava example Lianmin Zheng 2024-01-24 11:44:07 +00:00
  • 6dceab4d17 bump version to 0.1.9 Lianmin Zheng 2024-01-24 11:37:25 +00:00
  • c70b3cfa9e Bump the version to v0.1.8 (#93) Lianmin Zheng 2024-01-24 03:33:34 -08:00
  • 489796c7ea minor performance fix Ying Sheng 2024-01-24 10:45:44 +00:00
  • fa7a696d04 Fix max_new_tokens for limited memory Lianmin Zheng 2024-01-24 10:44:32 +00:00
  • bef0b35902 Fix llava & Fix multiprocessing Lianmin Zheng 2024-01-24 10:35:31 +00:00
  • c6576e820c Llava-hd Support (#92) shiyi.c_98 2024-01-24 01:51:21 -08:00
  • 99258181c6 set start method to spawn Lianmin Zheng 2024-01-24 08:55:38 +00:00
  • 3de54a1b55 Add health endpoint to SGLang runtime server (#90) isaac-vidas 2024-01-23 22:00:28 -05:00