From 7de650c83c4d63b9107184bd5cf36303d89e8d28 Mon Sep 17 00:00:00 2001 From: R0CKSTAR Date: Wed, 4 Feb 2026 04:44:57 +0800 Subject: [PATCH] [diffusion] hardware: support diffusion models on MTGPU (doc, 6/N) (#17346) Signed-off-by: Xiaodong Ye --- python/sglang/multimodal_gen/README.md | 9 +++++-- python/sglang/multimodal_gen/docs/install.md | 4 +++- .../multimodal_gen/docs/install_musa.md | 24 +++++++++++++++++++ 3 files changed, 34 insertions(+), 3 deletions(-) create mode 100644 python/sglang/multimodal_gen/docs/install_musa.md diff --git a/python/sglang/multimodal_gen/README.md b/python/sglang/multimodal_gen/README.md index 76d925543..daef8764e 100644 --- a/python/sglang/multimodal_gen/README.md +++ b/python/sglang/multimodal_gen/README.md @@ -18,14 +18,19 @@ SGLang Diffusion has the following features: SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the [ROCm installation guide](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) for setup instructions. +### Moore Threads/MUSA Support + +SGLang Diffusion supports Moore Threads GPUs (MTGPU) through the MUSA software stack. On MUSA platforms, we use the Torch SDPA backend for attention. See the [MUSA installation guide](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_musa.md) for setup instructions. + ## Getting Started ```bash uv pip install 'sglang[diffusion]' --prerelease=allow ``` -For more installation methods (e.g. pypi, uv, docker), check [install.md](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install.md). ROCm/AMD users should follow the [ROCm quickstart](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) that includes the additional kernel builds and attention backend settings we validated on MI300X. - +For more installation methods (e.g. pypi, uv, docker), check [install.md](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install.md). +* ROCm/AMD users should follow the [ROCm quickstart](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) that includes the additional kernel builds and attention backend settings we validated on MI300X. +* MUSA/Moore Threads users should follow the [MUSA quickstart](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_musa.md) that includes the attention backend settings we validated on MTT S5000. ## Inference diff --git a/python/sglang/multimodal_gen/docs/install.md b/python/sglang/multimodal_gen/docs/install.md index 49966516f..c77e77c2d 100644 --- a/python/sglang/multimodal_gen/docs/install.md +++ b/python/sglang/multimodal_gen/docs/install.md @@ -2,7 +2,9 @@ You can install sglang-diffusion using one of the methods below. -This page primarily applies to common NVIDIA GPU platforms. For AMD Instinct/ROCm environments see the dedicated [ROCm quickstart](install_rocm.md), which lists the exact steps (including kernel builds) we used to validate sgl-diffusion on MI300X. +This page primarily applies to common NVIDIA GPU platforms. +* For AMD Instinct/ROCm environments see the dedicated [ROCm quickstart](install_rocm.md), which lists the exact steps (including kernel builds) we used to validate sgl-diffusion on MI300X. +* For Moore Threads GPU (MTGPU) with the MUSA software stack, see the [MUSA quickstart](install_musa.md), which lists the exact steps we used to validate sgl-diffusion on MTT S5000. ## Method 1: With pip or uv diff --git a/python/sglang/multimodal_gen/docs/install_musa.md b/python/sglang/multimodal_gen/docs/install_musa.md new file mode 100644 index 000000000..b7474c3c2 --- /dev/null +++ b/python/sglang/multimodal_gen/docs/install_musa.md @@ -0,0 +1,24 @@ +# MUSA Quickstart for SGLang-Diffusion + +This page covers installation and usage of SGLang-Diffusion on Moore Threads GPU (MTGPU) with the MUSA software stack. + +## Install from Source + +```bash +# Clone the repository +git clone https://github.com/sgl-project/sglang.git +cd sglang + +# Install the Python packages +pip install --upgrade pip +rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml +pip install -e "python[all_musa]" +``` + +## Quick Test + +```bash +sglang generate --model-path black-forest-labs/FLUX.1-dev \ + --prompt "A logo With Bold Large text: SGL Diffusion" \ + --save-output +```