chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

[Embeddings Performance Testing] Add performance test for embedding models (#12359)

Browse Source
This commit is contained in:
Vedant V Jhaveri
2025-11-17 15:35:18 -08:00
committed by GitHub
parent ea89a3a0c5
commit aac07bf7fd
3 changed files with 248 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
.github/workflows/pr-test.yml vendored
View File

@@ -706,6 +706,18 @@ jobs:
cd test/srt
python3 -m unittest test_bench_serving.TestBenchServing.test_score_api_batch_scaling
- name: Benchmark Embeddings online latency and throughput
timeout-minutes: 10
run: |
cd test/srt
python3 -m unittest test_bench_serving.TestBenchServing.test_embeddings_api_latency_throughput
- name: Benchmark Embeddings online latency and throughput (batch size scaling)
timeout-minutes: 10
run: |
cd test/srt
python3 -m unittest test_bench_serving.TestBenchServing.test_embeddings_api_batch_scaling
performance-test-2-gpu:
needs: [check-changes, unit-test-backend-2-gpu, sgl-kernel-build-wheels]
if: always() && !failure() && !cancelled() &&