chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Commit Graph

  • f14569f64a ci: remove workflow path trigger (#1096) Yineng Zhang 2024-08-14 18:36:24 +08:00
  • 8f790ac100 Fix a bug in cuda graph runner (#1094) Lianmin Zheng 2024-08-14 03:25:38 -07:00
  • 616b59f384 [Feature] modify Runtime to support skip_tokenizer_init (#1088) rainred 2024-08-14 15:28:04 +08:00
  • c8423ca311 ci: update timeout and retry (#1086) Yineng Zhang 2024-08-14 15:27:35 +08:00
  • e205527cb1 Fix jump forward final state circular path bug. (#1084) Liangsheng Yin 2024-08-13 21:14:05 -07:00
  • 0909bb0d2f [Feat] Add window attention for gemma-2 (#1056) Ying Sheng 2024-08-13 17:01:26 -07:00
  • ad3e4f1619 Update the mixtral to use the better FusedMoE layer (#1081) Lianmin Zheng 2024-08-13 15:44:25 -07:00
  • 312e849255 Example file for docker compose and k8s (#1006) Lucien 2024-08-14 06:07:57 +08:00
  • 95f5fbf1a7 Fix create_abort_task, GenerateReqInput does not have rids. (#1079) rainred 2024-08-13 20:47:22 +08:00
  • cebd78d83e ci: add accuracy timeout (#1078) Yineng Zhang 2024-08-13 20:12:58 +08:00
  • 0076f11541 fix: use devel for Triton's compiler requirements (#1074) Yineng Zhang 2024-08-13 19:08:43 +08:00
  • f7fb68d292 ci: add moe test (#1053) Yineng Zhang 2024-08-13 16:43:23 +08:00
  • 396a13e6ad ci: add cancel pr workflow (#1070) Yineng Zhang 2024-08-13 16:16:50 +08:00
  • 65915f9f3e fix: temporary solution for DeepSeek V2 H100 layout conversion issue (#1060) Yineng Zhang 2024-08-13 13:48:54 +08:00
  • 162f3ccb01 Fix layernorm input shape (#1066) Ke Bao 2024-08-13 13:48:07 +08:00
  • 65e89baea9 fix: not use the default port (#1068) Yineng Zhang 2024-08-13 13:12:56 +08:00
  • 6a38efa834 feat: replace all rmsnorm and silu (#1057) Yineng Zhang 2024-08-13 00:15:59 +08:00
  • b0ad0c1bc8 chore: bump v0.2.12 (#1048) Yineng Zhang 2024-08-12 18:59:38 +08:00
  • c877292cc1 Re-organize CI tests (#1052) Lianmin Zheng 2024-08-12 03:39:01 -07:00
  • 0c1c72a0b4 Fix accuracy test (#1051) Lianmin Zheng 2024-08-12 02:48:40 -07:00
  • 41598e0d8e Add longer accuracy test on CI (#1049) Lianmin Zheng 2024-08-12 02:21:38 -07:00
  • 89f23a5178 docs: update setup github runner (#1050) Yineng Zhang 2024-08-12 16:11:38 +08:00
  • cb99ba4fc6 feat: update Dockerfile (#1033) Yineng Zhang 2024-08-12 14:24:06 +08:00
  • 32f6144323 fix: Fix returned prefill logits and add output str test (#1046) Ying Sheng 2024-08-11 23:13:45 -07:00
  • fb1f28cbbb Clean up the comments and names under python/sglang/srt/layers (#1047) Lianmin Zheng 2024-08-11 22:54:37 -07:00
  • fb7421db0d minor: some potential bugs (#1044) Liangsheng Yin 2024-08-11 22:35:44 -07:00
  • 14b6493087 Delete the useless test/srt/test_throughput.py (#1045) Lianmin Zheng 2024-08-11 21:31:52 -07:00
  • 8207637029 Improve end-to-end throughput test and its coverage (#1039) Lianmin Zheng 2024-08-11 18:27:33 -07:00
  • 7de6034534 Fix the prefix indices (#1037) Liangsheng Yin 2024-08-11 17:57:02 -07:00
  • d84c5e70f7 Test the case when max_new_tokens is very large (#1038) Lianmin Zheng 2024-08-11 16:41:03 -07:00
  • d785412077 Fix the case when max_new_tokens is too large (#1025) Lianmin Zheng 2024-08-11 15:20:18 -07:00
  • 7b6a5332ca Fix triton args init (#1034) Liangsheng Yin 2024-08-11 12:11:26 -07:00
  • 4080e82244 Fix the case where r.prefix_indices is None (#1031) Lianmin Zheng 2024-08-11 04:53:51 -07:00
  • c245b78973 hotfix: add CustomOp abstraction (#1027) Yineng Zhang 2024-08-11 17:45:59 +08:00
  • 9dae407812 Improve type annotation (#1029) Lianmin Zheng 2024-08-11 02:44:59 -07:00
  • fcc0f5ed99 Fix wrong assert (#1028) Liangsheng Yin 2024-08-11 02:22:16 -07:00
  • a97df79124 Clean up readme and arguments of chunked prefill (#1022) Lianmin Zheng 2024-08-11 01:18:52 -07:00
  • 33d61356b8 misc: update issue template (#1024) Yineng Zhang 2024-08-11 15:34:30 +08:00
  • 94752ac811 feat: use FlashInfer rmsnorm and silu (#907) Yineng Zhang 2024-08-11 12:57:13 +08:00
  • 43fbb6d919 Fix input_ids && rename to fill_ids (#1021) Liangsheng Yin 2024-08-10 16:24:12 -07:00
  • 54fb1c80c0 Clean up unit tests (#1020) Lianmin Zheng 2024-08-10 15:09:03 -07:00
  • b68c4c073b fix: force max new tokens to be 1 for embedding request (#1019) Ying Sheng 2024-08-10 13:46:42 -07:00
  • e712837d38 misc: update test config (#990) Yineng Zhang 2024-08-11 02:20:30 +08:00
  • 7599badeaf Support embedding input as a list (#1014) Ying Sheng 2024-08-10 08:39:05 -07:00
  • 62757db6f0 Reduce the overhead when cache is disabled (#1010) Liangsheng Yin 2024-08-09 16:36:57 -07:00
  • 73fa2d49d5 Some warnings to crash when CI (#1009) Liangsheng Yin 2024-08-09 15:16:23 -07:00
  • 61728884d7 Fix benchmark latency (#1007) Mingyi 2024-08-09 13:18:58 -07:00
  • 9cf0a5bada Add skip_tokenizer_init args. (#959) gryffindor-rr 2024-08-10 03:14:13 +08:00
  • b16e856f11 Add openai embedding API (#997) Ying Sheng 2024-08-09 11:19:18 -07:00
  • 05c50a82b8 Minor bugfix on benchmark serving (#1005) Roger Wang 2024-08-09 09:53:50 -07:00
  • b568df5d03 fix: resolve correctness_test issue (#1002) Yineng Zhang 2024-08-09 21:21:42 +08:00
  • 10bca45bc6 bugfix: penalizers to be merged before reqs (#1001) Juwan Yoo 2024-08-09 04:46:24 -07:00
  • b91a4cb1b1 support models from www.modelscope.cn (#994) liuyhwangyh 2024-08-09 17:52:14 +08:00
  • 95a28019ba test: negative value testing for frequency, presence penalizers (#995) Juwan Yoo 2024-08-08 23:30:50 -07:00
  • e040a2450b Add e5-mistral embedding model - step 3/3 (#988) Ying Sheng 2024-08-08 16:31:19 -07:00
  • 9f662501a3 Move torch.compile configs into cuda_graph_runner.py (#993) Ying Sheng 2024-08-08 13:20:30 -07:00
  • ab7875941b feat: frequency, min_new_tokens, presence, and repetition penalties (#973) Juwan Yoo 2024-08-08 04:21:08 -07:00
  • 228cf47547 Create contributor_guide.md (#992) Ying Sheng 2024-08-08 03:58:47 -07:00
  • 3a79613c28 support more optioin about usage in stream mode (#985) yichuan~ 2024-08-08 17:41:57 +08:00
  • 1ac304eeb4 Adjust InputeMetadata and ScheduleBatch (#981) Liangsheng Yin 2024-08-08 01:11:22 -07:00
  • 20a4f927dc Add io struct for embedding models [unreachable code] - step 2/3 (#987) Ying Sheng 2024-08-08 00:52:31 -07:00
  • 0de7c2d09e Add e5-mistral modules [unreachable code] - step 1/3 (#983) Ying Sheng 2024-08-08 00:04:15 -07:00
  • 6ed4e3b8fb Fix chunked prefill (#984) Liangsheng Yin 2024-08-07 22:28:42 -07:00
  • 00023d622a [minor] Update type annotation in tokenizer_manager.py (#982) Ying Sheng 2024-08-07 18:48:45 -07:00
  • c62d560c03 #590 Increase default , track changes in examples and documentation (#971) foszto 2024-08-08 02:54:46 +02:00
  • 2b8257f325 Adjust max prefix len (#980) Liangsheng Yin 2024-08-07 17:41:26 -07:00
  • 7623091d97 RadixCache method adjust (#977) Liangsheng Yin 2024-08-07 15:52:24 -07:00
  • f724f1f1e9 PrefillAdder abstraction (#968) Liangsheng Yin 2024-08-07 13:47:28 -07:00
  • 6db27f7b3b misc: correct the int data type for token ids and indices (#969) Zhiqiang Xie 2024-08-07 13:40:07 -07:00
  • 4d929107ae Run purge-cache only in sgl-project (#976) Liangsheng Yin 2024-08-07 13:16:36 -07:00
  • fbe0c818c2 Purge self-runner's pip cache weekly (#975) Liangsheng Yin 2024-08-07 12:43:12 -07:00
  • dc9d06d886 chore: bump v0.2.11 (#970) Yineng Zhang 2024-08-07 20:47:53 +08:00
  • c31f084c71 chore: update vllm to 0.5.4 (#966) Yineng Zhang 2024-08-07 19:15:41 +08:00
  • a01ddd9605 misc: fix the req_to_token member change (#967) Liangsheng Yin 2024-08-07 01:52:10 -07:00
  • 7fa54a1ab3 Make req_pool_indices on CPU (#960) Liangsheng Yin 2024-08-07 01:41:25 -07:00
  • 05abd1261c misc: add compute capability in check_env (#965) Yineng Zhang 2024-08-07 16:39:36 +08:00
  • 5f6fa04a3f misc: simplify test (#964) yichuan~ 2024-08-07 16:23:27 +08:00
  • 58a0970853 misc: update issue template (#963) Yineng Zhang 2024-08-07 15:41:21 +08:00
  • ff68ae857a Show more error messages for warmup errors (#932) Ying Sheng 2024-08-06 23:57:06 -07:00
  • 795eab6dda Add support for Batch API test (#936) yichuan~ 2024-08-07 14:52:10 +08:00
  • 41bb1ab10d fix nsys cannot profile cuda kernel (#957) Meng, Peng 2024-08-07 11:51:21 +08:00
  • 87e8c090e9 Organize code (rename, movement) (#953) Liangsheng Yin 2024-08-06 20:50:32 -07:00
  • ad56e68495 Fix stuck in get_new_prefill_batch (#948) Liangsheng Yin 2024-08-06 01:05:58 -07:00
  • ffb15744b5 Support multiple args options (#941) yichuan~ 2024-08-06 02:12:53 +08:00
  • a9c833d580 Fix union operator (#940) Ke Bao 2024-08-06 00:46:34 +08:00
  • 94e0115186 Feat: add alternative choices selection methods (#835) Aidan Cooper 2024-08-05 11:27:49 +01:00
  • b216a545b3 Remove leftover auth_token (#934) Aidan Cooper 2024-08-05 11:25:48 +01:00
  • fde8340550 docs: update README (#935) Yineng Zhang 2024-08-05 18:06:06 +08:00
  • fd7926e46e Fix prompt len in parallel sampling (#928) yichuan~ 2024-08-05 15:56:08 +08:00
  • 399cad91f3 Update README.md (#927) Ying Sheng 2024-08-04 23:01:35 -07:00
  • 0a4f5f9bea Test regex in vision api (#926) Ying Sheng 2024-08-04 22:52:41 -07:00
  • 3bc99e6fe4 Test openai vision api (#925) Ying Sheng 2024-08-04 20:51:55 -07:00
  • ebf69964cd latency test enhancement - final part (#921) min-xu-et 2024-08-04 18:15:23 -07:00
  • 141e8c71a3 Bump version to 0.2.10 (#923) Ying Sheng 2024-08-04 16:52:51 -07:00
  • d53dcf9c98 Support more OpenAI API test (#916) yichuan~ 2024-08-05 07:43:09 +08:00
  • bb66cc4c52 Fix CI && python3.8 compatible (#920) Liangsheng Yin 2024-08-04 16:02:05 -07:00
  • 975adb802b Update hyperparameter_tuning.md (#918) Ying Sheng 2024-08-04 13:51:52 -07:00
  • 0d4f3a9fcd Make API Key OpenAI-compatible (#917) Ying Sheng 2024-08-04 13:35:44 -07:00
  • afd411d09f enhance latency test - part 2 (#915) min-xu-et 2024-08-04 12:27:25 -07:00
  • e1eae1fd15 Support MLA for DeepSeek-V2 with Triton - step 1 (#905) Ke Bao 2024-08-05 01:40:33 +08:00