chenchenghao/sglang

Fork 0

Files

T

Xiaoyu Zhang e42fa009d4 [Diffusion] diffusion profile and opt skills (#19540 )

2026-03-02 15:06:29 +08:00

28 KiB

Raw Blame History

name, description

name	description
diffusion-benchmark-and-profile	End-to-end benchmark and per-layer kernel profiling guide for SGLang Diffusion models. Use when measuring SGLang Diffusion generation performance, running latency benchmarks across Qwen-Image, FLUX, Z-Image-Turbo, and Wan2.2 models, profiling DiT layer kernel breakdown with torch.profiler or nsys+gputrc2graph.py, investigating performance bottlenecks, or tracking performance regressions. Always verify output correctness before and after any optimization.

name

description

diffusion-benchmark-and-profile

End-to-end benchmark and per-layer kernel profiling guide for SGLang Diffusion models. Use when measuring SGLang Diffusion generation performance, running latency benchmarks across Qwen-Image, FLUX, Z-Image-Turbo, and Wan2.2 models, profiling DiT layer kernel breakdown with torch.profiler or nsys+gputrc2graph.py, investigating performance bottlenecks, or tracking performance regressions. Always verify output correctness before and after any optimization.

SGLang Diffusion Benchmark and Profile Guide

Overview This skill covers how to run end-to-end benchmarks for SGLang Diffusion (sglang generate) across a standard set of models, profile per-layer kernel execution inside the DiT, and use those results to continuously improve performance. These metrics collectively reflect the overall performance of the current SGLang Diffusion release.

Correctness First: Any performance optimization must be validated for correctness before being considered complete. Faster but incorrect output is not an improvement. Always compare generated images/videos against a reference baseline before and after any change.

Primary Metric: Denoise Latency The most important latency signal is the denoising loop latency — the total time spent running the DiT forward pass across all inference steps. This is the dominant cost in every diffusion model and the main target for optimization. End-to-end latency (including VAE decode and text encoding) is also recorded as a secondary metric, but denoising latency is the key indicator of DiT model performance.

Prerequisites

Tool Dependency Check

Before running any benchmark or profiling command, verify that all required tools are available. Run the following check script:

#!/usr/bin/env bash
set -euo pipefail
PASS=0; FAIL=0

check() {
  local label=$1; shift
  if "$@" &>/dev/null; then
    echo "  [OK]  $label"
    ((PASS++)) || true
  else
    echo "  [MISS] $label"
    ((FAIL++)) || true
  fi
}

echo "=== SGLang Diffusion Benchmark Prerequisites ==="

# Core runtime
check "sglang CLI"             python3 -c "import sglang"
check "torch"                  python3 -c "import torch; assert torch.cuda.is_available()"
check "CUDA available"         python3 -c "import torch; torch.zeros(1).cuda()"

# Profiling (torch.profiler is built into torch — no extra install needed)
check "torch.profiler"         python3 -c "import torch.profiler"

# nsys (optional, for Level 2 profiling)
check "nsys in PATH"           which nsys

# gputrc2graph.py dependencies
# The tool lives in the sglang repo at examples/profiler/nsys_profile_tools/gputrc2graph.py
# Set SGLANG_REPO to your local sglang repo root, e.g.:
#   export SGLANG_REPO=/workspace/sglang
SGLANG_REPO="${SGLANG_REPO:-$(python3 -c "import sglang, os; print(os.path.abspath(os.path.join(os.path.dirname(sglang.__file__), '../../..')))" 2>/dev/null || echo "")}"
GPUTRC="${SGLANG_REPO}/examples/profiler/nsys_profile_tools/gputrc2graph.py"
check "gputrc2graph.py exists (set SGLANG_REPO if missing)" test -f "$GPUTRC"
check "pandas"                 python3 -c "import pandas"
check "regex"                  python3 -c "import regex"
check "plotly (optional)"      python3 -c "import plotly"

echo ""
echo "Result: $PASS passed, $FAIL missing"
if [ "$FAIL" -gt 0 ]; then
  echo ""
  echo "Install missing dependencies:"
  echo "  pip install pandas regex plotly   # for gputrc2graph.py"
  echo "  # nsys: install NVIDIA Nsight Systems from https://developer.nvidia.com/nsight-systems"
fi

Minimum required for benchmarking: sglang, torch with CUDA. Additional for Level 1 profiling: torch.profiler (bundled with torch, always available). Additional for Level 2 profiling: nsys on PATH, gputrc2graph.py present, pandas, regex. plotly is only needed to generate the HTML chart; result.csv is generated regardless.

Download Required Input Images

Some models (image editing / image-guided video generation) require input images:

mkdir -p /workspace/gen_benchmark/figs
cd /workspace/gen_benchmark/figs

wget -O cat.png https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png
wget -O astronaut.jpg https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg

Standard Benchmark Model Suite

All commands include --warmup (pre-warm torch.compile) and --enable-torch-compile to reflect real production deployment performance. Timing is measured after warmup completes.