chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Commit Graph

  • c0ae70c8ed Improve logging & fix litellm dependency. (#512) Lianmin Zheng 2024-06-07 12:51:40 -07:00
  • 87260b7bfd Litellm Backend (#502) 胡译文 2024-06-08 03:24:28 +08:00
  • 651a23ee7c remove redundant pad_input_ids function (#500) Amos You 2024-06-07 12:23:29 -07:00
  • bf3e271fe0 Update vllm to v0.4.3 (#511) Lianmin Zheng 2024-06-07 12:11:31 -07:00
  • 3bc01ac137 [Minor] improve code style Lianmin Zheng 2024-06-03 18:11:34 -07:00
  • 9f009261f2 Improve docs Lianmin Zheng 2024-06-01 17:46:08 -05:00
  • 159cc741e4 Make the server random by default (#493) Lianmin Zheng 2024-05-31 23:33:34 -07:00
  • 7d1ebc2d71 update the script: examples/usage/llava_video/srt_example_llava_v.sh (#491) Yuanhan Zhang 2024-06-01 14:31:56 +08:00
  • 83525a1df2 Revert "Make the server random by default" (#492) Ying Sheng 2024-05-31 12:00:21 -07:00
  • 80a33ce8b0 Do not set the default value of global random seed (#488) Lianmin Zheng 2024-05-29 18:41:18 -04:00
  • 1a57e41679 do not launch workers in parallel Lianmin Zheng 2024-05-27 23:00:16 -07:00
  • adc974268a Update docs (#486) Lianmin Zheng 2024-05-27 22:46:04 -07:00
  • 0463f7fb52 Support data parallelism (static) (#480) Ying Sheng 2024-05-27 21:24:10 -07:00
  • 565d727409 improve logging & fix vllm version Lianmin Zheng 2024-05-27 14:32:05 -07:00
  • 09de730dee Improve benchmark scripts & add more models (#484) Lianmin Zheng 2024-05-27 14:13:26 -07:00
  • 55c1643627 Improve benchmark scripts & rename some scripts (#477) Lianmin Zheng 2024-05-26 12:51:45 -07:00
  • 2b605ab1d7 [Feat/Fix] Refactoring Llava models into single file (#475) Li Bo 2024-05-27 03:29:51 +08:00
  • 947bda73fe Add benchmark scripts (#476) Ying Sheng 2024-05-26 12:09:03 -07:00
  • f06e90c2cf Optimize retract (#440) Liangsheng Yin 2024-05-26 00:07:26 +08:00
  • 2cea6146d8 Improve logging & add logit cap (#471) Lianmin Zheng 2024-05-24 03:48:53 -07:00
  • 44c998fcb5 Add the instruction link to the LLaVA-NeXT-Video at README (#463) Yuanhan Zhang 2024-05-24 18:38:20 +08:00
  • 3167d8dabc fix test bug in srt_llava_next_test.py (#470) bing 2024-05-24 18:38:01 +08:00
  • 0fafc5606b port fp8 mixtral (#460) Lianmin Zheng 2024-05-21 11:46:35 -07:00
  • 19d2135cb8 Use model loader from vllm (#459) Lianmin Zheng 2024-05-21 09:13:37 -07:00
  • ced77c6626 Rename api_num_spec_tokens -> num_api_spec_tokens (#458) Lianmin Zheng 2024-05-20 18:44:23 -07:00
  • 8dbdc018a3 Abort disconnected requests (#457) Lianmin Zheng 2024-05-20 18:41:21 -07:00
  • 3e684be7a3 Fix openai speculative execution (#456) Ying Sheng 2024-05-20 17:01:13 -07:00
  • ec380dfd30 openai chat speculative execution (#250) LiviaSun 2024-05-18 22:23:53 -07:00
  • 5b647543c1 Fix the broken --disable-radix-cache (#451) Liangsheng Yin 2024-05-19 13:00:12 +08:00
  • 8210ec60f4 Improve error handling & abort disconnected requests (#449) Lianmin Zheng 2024-05-17 05:49:31 -07:00
  • 5be9eb8a8c Add PUT for generate api (#448) Ying Sheng 2024-05-17 02:35:15 -07:00
  • c05956e534 Simplify port allocation (#447) Lianmin Zheng 2024-05-16 18:07:30 -07:00
  • d75dc20fae Add finish_reason to OpenAI API (#446) Matthias Gerstgrasser 2024-05-16 14:55:05 -07:00
  • 690d162d97 Format code (#441) Liangsheng Yin 2024-05-14 22:40:46 +08:00
  • 664287b2a7 [Feat] Add llava qwen, llava mistral (#419) Kaichen Zhang - NTU 2024-05-14 13:17:50 +08:00
  • e0ae5d42ec Update version to 0.1.16 (#438) Lianmin Zheng 2024-05-13 17:29:17 -07:00
  • 32de16ce2f Fix streaming (#437) Lianmin Zheng 2024-05-13 17:26:18 -07:00
  • 0992d85f92 support llava video (#426) Yuanhan Zhang 2024-05-14 07:57:00 +08:00
  • 5dc55a5f02 Handle truncation errors (#436) Lianmin Zheng 2024-05-13 15:56:00 -07:00
  • 4231a42fa8 Fix import of global_config Lianmin Zheng 2024-05-13 12:11:55 -07:00
  • 455c9ccc4a Update readme (#434) Lianmin Zheng 2024-05-13 00:17:02 -07:00
  • 39191c8515 Cache optimizations (#418) Liangsheng Yin 2024-05-13 12:47:13 +08:00
  • 562b8857d8 Improve error handling (#433) Lianmin Zheng 2024-05-12 20:49:04 -07:00
  • 04c0b21488 Allow input_ids in the input of the /generate endpoint (#363) Shannon Shen 2024-05-12 12:29:00 -10:00
  • 6e09cf6a15 Misc fixes (#432) Lianmin Zheng 2024-05-12 15:05:40 -07:00
  • 72bb344388 Update version to 0.1.15 (#431) Lianmin Zheng 2024-05-12 14:22:33 -07:00
  • 2d580e7a89 Fix flashinfer (#430) Lianmin Zheng 2024-05-12 08:18:53 -07:00
  • 3fc97f6709 Move openai api server into a separate file (#429) Lianmin Zheng 2024-05-12 06:41:32 -07:00
  • abc548c707 Minor fix for the import path (#428) Lianmin Zheng 2024-05-12 05:10:35 -07:00
  • aee4f523cf Fix logit processor bugs (#427) Lianmin Zheng 2024-05-12 04:54:07 -07:00
  • 7023f413c6 Clean up (#422) Lianmin Zheng 2024-05-11 20:55:00 -07:00
  • 09deb20dee Optimize the memory usage of logits processor (#420) Lianmin Zheng 2024-05-11 16:56:42 -07:00
  • 33b242df30 Compat with latest VLLM 0.4.2 main + fork.number rename + Flashinfer 0.0.4 (#380) Qubitium 2024-05-12 07:37:49 +08:00
  • a511a2d089 restrict vllm version Lianmin Zheng 2024-05-09 15:49:29 -07:00
  • 6ec65f4555 Make public APIs more standard. (#416) Liangsheng Yin 2024-05-09 15:39:22 +08:00
  • e2c31fca5c Include finish reason in meta info response (#415) Enrique Shockwave 2024-05-09 08:14:01 +01:00
  • d5de20a3ee Fix sync() when fork(1) (#412) Liangsheng Yin 2024-05-08 15:15:18 +08:00
  • 4a1c6ae2ce Add Cohere Command R chat template (#411) YoungJoong Noah Kim 2024-05-07 16:18:15 +09:00
  • 14522e6a26 Organize Benchmark (#381) Liangsheng Yin 2024-05-05 16:14:17 +08:00
  • 183df47282 SamplingParams add "spaces_between_special_tokens" argument (#392) ZhouXingg 2024-05-01 07:17:12 +08:00
  • 5c5aba5900 Adding RAG tracing & eval cookbook using Parea (#390) Joschka Braun 2024-04-30 19:13:28 -04:00
  • ba67101f99 Fix chatml template (#406) Lianmin Zheng 2024-04-30 15:53:39 -07:00
  • 95c4e0dfac Format Benchmark Code (#399) Liangsheng Yin 2024-04-28 21:06:22 +08:00
  • 19818b9c2f Minor: style improvement of radix_cache and memory_pool (#395) Liangsheng Yin 2024-04-26 01:01:36 +08:00
  • 9216b10678 Improve performance when running with full parallel (#394) Liangsheng Yin 2024-04-25 17:29:07 +08:00
  • da19434c2f Benchmark Updates (#382) Liangsheng Yin 2024-04-24 02:23:01 +08:00
  • 150d7020ed Revert removing the unused imports (#385) Liangsheng Yin 2024-04-23 22:36:33 +08:00
  • 9acc6e3504 add .isort.cfg (#378) Liangsheng Yin 2024-04-22 22:38:09 +08:00
  • cf9d8efdd3 llama3 instruct template (#372) Enrique Shockwave 2024-04-21 17:40:12 +01:00
  • 1bf1cf1953 Reduce overhead when fork(1) (#375) Liangsheng Yin 2024-04-21 17:25:14 +08:00
  • e822e5900b Optimize radix tree matching (#364) Ke Bao 2024-04-18 00:47:37 +08:00
  • ca4f1ab89c Update model support in readme (#370) Ying Sheng 2024-04-17 00:16:32 -07:00
  • 2b6d999191 Fix issue #367 – System message not supported for Anthropic (anthropic.BadRequestError) (#368) Fronx 2024-04-16 20:18:24 +02:00
  • 65501a9cf1 Fix commandr import; format code Lianmin Zheng 2024-04-16 18:10:12 +00:00
  • db611066ad support command-r (#369) ZhouXingg 2024-04-17 01:36:51 +08:00
  • c93293c57e Update README.md (#358) Ikko Eltociear Ashimine 2024-04-10 00:39:30 +09:00
  • 62b3812b69 Time cost utils (#355) Liangsheng Yin 2024-04-09 23:27:31 +08:00
  • 550a4f78f3 Fix typos in infer_batch.py (#354) Tom Dörr 2024-04-09 09:10:05 +02:00
  • ff99c38a07 Add timeout to get_meta_info (#346) SimoneRaponi 2024-04-03 16:22:06 +02:00
  • c9de3e169c Eliminate 2 gpu ops during sampling when logit_bias is zero (#338) Qubitium 2024-04-03 13:56:06 +08:00
  • ed27a6b992 Revert "Eliminate 2 gpu ops during sampling when logit_bias is zero" (#345) Liangsheng Yin 2024-04-03 12:45:01 +08:00
  • 463c6632a8 Eliminate 2 gpu ops during sampling when logit_bias is zero (#343) Liangsheng Yin 2024-04-02 19:14:55 +08:00
  • b0890631a0 fix gemma import error Ying Sheng 2024-04-01 07:35:58 +00:00
  • cb389c91bc Fix llava parallelism/fork bug (#315) Junlong Li 2024-03-29 10:24:54 +08:00
  • eddaa2b599 Add support for new autogptq quant_config.checkpoint_format (#332) Qubitium 2024-03-29 10:24:16 +08:00
  • 2af565b3bb [model] DBRX-instruct support (#337) Liangsheng Yin 2024-03-29 01:05:19 +08:00
  • 3842eba5fa Logprobs Refractor (#331) Liangsheng Yin 2024-03-28 14:34:49 +08:00
  • 24e59f5350 model_runner simplify (#329) Liangsheng Yin 2024-03-24 19:48:37 +08:00
  • 7523541962 model_rpc style improvement (#293) Liangsheng Yin 2024-03-24 15:41:24 +08:00
  • 64ee9c030e Openrouter usage example (#327) Jani Monoses 2024-03-23 19:16:24 +02:00
  • 30d17840fc Update dependencies (#326) Jani Monoses 2024-03-23 19:15:58 +02:00
  • ce216c80dc Cleanup codebase: removed unnecessary code/logic (#298) Qubitium 2024-03-24 01:15:16 +08:00
  • 51104cd405 Update version to v0.1.14 (#324) Lianmin Zheng 2024-03-22 13:42:22 -07:00
  • e2b2f0a213 Support oai in benchmark/mmlu (#323) Lianmin Zheng 2024-03-22 13:37:57 -07:00
  • b57abe1663 Add StableLM model. (#301) Jani Monoses 2024-03-22 22:24:08 +02:00
  • e57f079275 Use Anthropic messages API (#304) Jani Monoses 2024-03-22 22:23:31 +02:00
  • 08df63a6f8 [Fix/Potential Bugs] Can not correctly import models in python/sglang/srt/models (#311) Li Bo 2024-03-23 03:19:58 +08:00
  • 77835756a7 Fix outlines-0.0.35 incompatibility (#291) ZhouGongZaiShi 2024-03-23 03:19:11 +08:00
  • ed31579971 Fix marlin model loading compat with autogptq (#290) Liurl 2024-03-13 13:15:43 +08:00
  • 92e2d74fd0 Fix env (docker) compat due to __file__ usage (#288) Qubitium 2024-03-13 13:02:48 +08:00