chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[diffusion] multi-platform: support diffusion on amd and fix encoder loading on MI325 (#13760)

Browse Source 
Co-authored-by: Sabre Shao <sabre.shao@amd.com>
Co-authored-by: Yusheng (Ethan) Su <yushengsu.thu@gmail.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
Co-authored-by: xsun <sunxiao04@gmail.com>
This commit is contained in:
Yuzhen Zhou
2025-12-19 15:38:46 +08:00
committed by GitHub
co-authored by Sabre Shao Yusheng Su Hubert Lu xsun
parent f2d64e6782
commit 4bf06635fc
34 changed files with 823 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/multimodal_gen/README.md
+6 -2
View File
@@ -12,7 +12,11 @@ SGLang Diffusion has the following features:
- Broad model support: Wan series, FastWan series, Hunyuan, Qwen-Image, Qwen-Image-Edit, Flux
- Fast inference speed: enpowered by highly optimized kernel from sgl-kernel and efficient scheduler loop
- Ease of use: OpenAI-compatible api, CLI, and python sdk support
- Diverse hardware support: H100, H200, A100, B200, 4090
- Multi-platform support: NVIDIA GPUs (H100, H200, A100, B200, 4090) and AMD GPUs (MI300X, MI325X)
### AMD/ROCm Support
SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the [ROCm installation guide](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) for setup instructions.
## Getting Started
@@ -20,7 +24,7 @@ SGLang Diffusion has the following features:
uv pip install 'sglang[diffusion]' --prerelease=allow
```
For more installation methods (e.g. pypi, uv, docker), check [install.md](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install.md).
For more installation methods (e.g. pypi, uv, docker), check [install.md](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install.md). ROCm/AMD users should follow the [ROCm quickstart](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) that includes the additional kernel builds and attention backend settings we validated on MI300X.
## Inference