[diffusion] doc: add multimodal-gen profiling doc (#15069)

Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

python/sglang/multimodal_gen/docs/profiling.md

@@ -0,0 +1,111 @@
+# Profiling Multimodal Generation
+
+This guide covers profiling techniques for multimodal generation pipelines in SGLang.
+
+## PyTorch Profiler
+
+PyTorch Profiler provides detailed kernel execution time, call stack, and GPU utilization metrics.
+
+### Denoising Stage Profiling
+
+Profile the denoising stage with sampled timesteps (default: 5 steps after 1 warmup step):
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image \
+  --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+  --seed 0 \
+  --profile
+```
+
+**Parameters:**
+- `--profile`: Enable profiling for the denoising stage
+- `--num-profiled-timesteps N`: Number of timesteps to profile after warmup (default: 5)
+  - Smaller values reduce trace file size
+  - Example: `--num-profiled-timesteps 10` profiles 10 steps after 1 warmup step
+
+### Full Pipeline Profiling
+
+Profile all pipeline stages (text encoding, denoising, VAE decoding, etc.):
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image \
+  --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+  --seed 0 \
+  --profile \
+  --profile-all-stages
+```
+
+**Parameters:**
+- `--profile-all-stages`: Used with `--profile`, profile all pipeline stages instead of just denoising
+
+### Output Location
+
+Trace files are saved to the `./logs/` directory by default. The file name format depends on the profiling mode:
+
+- **Denoising Stage Profiling**: `{request_id}-{num_steps}_steps-global-rank{rank}.trace.json.gz`
+- **Full Pipeline Profiling**: `{request_id}-full_stages-global-rank{rank}.trace.json.gz`
+
+Example: `mocked_fake_id_for_offline_generate-5_steps-global-rank0.trace.json.gz`
+
+### View Traces
+
+Load and visualize trace files at:
+- https://ui.perfetto.dev/ (recommended)
+- chrome://tracing (Chrome only)
+
+For large trace files, reduce `--num-profiled-timesteps` or avoid using `--profile-all-stages`.
+
+## Nsight Systems
+
+Nsight Systems provides low-level CUDA profiling with kernel details, register usage, and memory access patterns.
+
+### Installation
+
+See the [SGLang profiling guide](https://github.com/sgl-project/sglang/blob/main/docs/developer_guide/benchmark_and_profiling.md#profile-with-nsight) for installation instructions.
+
+### Basic Profiling
+
+Profile the entire pipeline execution:
+
+```bash
+nsys profile \
+  --trace-fork-before-exec=true \
+  --cuda-graph-trace=node \
+  --force-overwrite=true \
+  -o QwenImage \
+  sglang generate \
+    --model-path Qwen/Qwen-Image \
+    --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+    --seed 0
+```
+
+### Targeted Stage Profiling
+
+Use `--delay` and `--duration` to capture specific stages and reduce file size:
+
+```bash
+nsys profile \
+  --trace-fork-before-exec=true \
+  --cuda-graph-trace=node \
+  --force-overwrite=true \
+  --delay 10 \
+  --duration 30 \
+  -o QwenImage_denoising \
+  sglang generate \
+    --model-path Qwen/Qwen-Image \
+    --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+    --seed 0
+```
+
+**Parameters:**
+- `--delay N`: Wait N seconds before starting capture (skip initialization overhead)
+- `--duration N`: Capture for N seconds (focus on specific stages)
+- `--force-overwrite`: Overwrite existing output files
+
+## Notes
+
+- **Reduce trace size**: Use `--num-profiled-timesteps` with smaller values or `--delay`/`--duration` with Nsight Systems
+- **Stage-specific analysis**: Use `--profile` alone for denoising stage, add `--profile-all-stages` for full pipeline
+- **Multiple runs**: Profile with different prompts and resolutions to identify bottlenecks across workloads