Co-authored-by: zhangkaihong.zkh <zhangkaihong.zkh@antgroup.com> Co-authored-by: 有禾 <zhangdonghao.zdh@antgroup.com> Co-authored-by: yudian0504 <138860534+yudian0504@users.noreply.github.com> Co-authored-by: 悠扬 <youyang.zmy@antgroup.com> Co-authored-by: xinxingyang <xinxing.yangxx@antgroup.com> Co-authored-by: zmy460290 <zmy460290@antgroup.com>
Code Structure
eval: The evaluation utilities.lang: The frontend language.multimodal_gen: Inference framework for accelerated image/video generation.srt: The backend engine for running local models. (SRT = SGLang Runtime).test: The test utilities.api.py: The public APIs.bench_offline_throughput.py: Benchmark the performance in the offline mode.bench_one_batch.py: Benchmark the latency of running a single static batch without a server.bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.bench_serving.py: Benchmark online serving with dynamic requests.check_env.py: Check the environment variables and dependencies.global_config.py: The global configs and constants.launch_server.py: The entry point for launching a local server.profiler.py: The profiling entry point to send profile requests.utils.py: Common utilities.version.py: Version info.