Commit Graph

21 Commits

Author SHA1 Message Date
hlu1
0e86de7c0b Remove deepseek-r1 from THINKING_MODE_CHOICES in run_eval.py (#17178) 2026-01-15 16:53:06 -08:00
hlu1
aeb480c11f Add top-p to run_eval.py (#16844) 2026-01-10 17:10:37 +08:00
Hudson Xing
f4ab2ec5be Add unified metrics collection framework (v1) (#16064)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-03 16:30:38 -08:00
mlmz
1f1f05a85e vlm: refactor engine vlm params and support processor output as input (#14091)
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: zhaochenyang20 <zhaochenyang20@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: BenYao21 <cyao22@asu.edu>
Co-authored-by: minleminzui <minleminzui@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
2025-12-20 18:31:24 +08:00
ゆり
3e4d431a44 [Feature] Add AIME25 dataset support for SGLang simple_eval (#14990)
Co-authored-by: zkexorability <zkexorability@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2025-12-15 21:59:40 -08:00
Baizhou Zhang
ab3ffd1c8e Add nightly accuracy test for DeepSeek V3.2 (#14935) 2025-12-13 12:11:16 -08:00
Binyao Jiang
312df1d6c0 Fix TestGLM41VPPAccuracy test flakiness (#14848) 2025-12-10 16:59:58 -08:00
Binyao Jiang
0aa65f94f1 [Fix] Improve longbench prompt and other logics (#11474) 2025-10-12 15:04:28 -07:00
Al-Ekram Elahee Hridoy
533e58a15d Feature/longbench v2 evaluation utils (#10949) 2025-10-07 14:17:31 +08:00
Liangsheng Yin
04b86b3c5c [hot-fix] Fix CI break which caused by adding thinking_mode in eval (#11192) 2025-10-03 18:29:27 +08:00
hlu1
d6777a706d Add --thinking-mode to run_eval (#11189)
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
2025-10-03 16:49:39 +08:00
Liangsheng Yin
9710f718fb [Eval] Add --repeat in run_eval (#11101) 2025-09-30 23:35:54 +08:00
Mick
777eb53897 ci: refactor nightly test (#10495) 2025-09-26 15:24:30 -07:00
fzyzcjy
442534aa44 Add CI for gpt-oss model on hopper (#8851) 2025-08-09 00:34:23 -07:00
Lifu Huang
6e2da51561 Replace time.time() to time.perf_counter() for benchmarking. (#6178)
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
2025-05-11 14:32:49 -07:00
Lianmin Zheng
ad4125d1a9 Fuse more ops & Simplify token mapping (#1758) 2024-10-22 23:20:43 -07:00
Lianmin Zheng
0c1c72a0b4 Fix accuracy test (#1051) 2024-08-12 19:48:40 +10:00
Lianmin Zheng
41598e0d8e Add longer accuracy test on CI (#1049) 2024-08-12 09:21:38 +00:00
Ying Sheng
995af5a54b Improve the structure of CI (#911) 2024-08-03 23:09:21 -07:00
Ying Sheng
e90e3a50d4 Add benchmark: HumanEval (#889) 2024-08-02 00:46:41 -07:00
Ying Sheng
ae7ee01a8e Add accuracy test to CI: MMLU (#882) 2024-08-01 21:20:17 -07:00