[diffusion][llm] macOS support (#19549)
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com> Co-authored-by: Mick <mickjagger19@icloud.com>
This commit is contained in:
@@ -1,3 +1,9 @@
|
||||
## Apple MPS
|
||||
|
||||
| Environment Variable | Default | Description |
|
||||
|----------------------|---------|--------------------------------------------------------------|
|
||||
| `SGLANG_USE_MLX` | not set | Set to `1` to enable MLX fused Metal kernels for norm ops on MPS |
|
||||
|
||||
## Caching Acceleration
|
||||
|
||||
These variables configure caching acceleration for Diffusion Transformer (DiT) models.
|
||||
|
||||
@@ -7,7 +7,11 @@ SGLang Diffusion is an inference framework for accelerated image and video gener
|
||||
- **Broad Model Support**: Wan series, FastWan series, Hunyuan, Qwen-Image, Qwen-Image-Edit, Flux, Z-Image, GLM-Image, and more
|
||||
- **Fast Inference**: Optimized kernels, efficient scheduler loop, and Cache-DiT acceleration
|
||||
- **Ease of Use**: OpenAI-compatible API, CLI, and Python SDK
|
||||
- **Multi-Platform**: NVIDIA GPUs (H100, H200, A100, B200, 4090), AMD GPUs (MI300X, MI325X) and Ascend NPU (A2, A3)
|
||||
- **Multi-Platform**:
|
||||
- NVIDIA GPUs (H100, H200, A100, B200, 4090)
|
||||
- AMD GPUs (MI300X, MI325X)
|
||||
- Ascend NPU (A2, A3)
|
||||
- Apple Silicon (M-series via MPS)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -69,7 +69,7 @@ For detailed ROCm system configuration and installation from source, see [AMD GP
|
||||
|
||||
## Platform-Specific: MUSA (Moore Threads GPUs)
|
||||
|
||||
For Moore Threads GPUs (MTGPU) with the MUSA software stack:
|
||||
For Moore Threads GPUs (MTGPU) with the MUSA software stack, please follow the instructions below to install from source:
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
@@ -93,3 +93,28 @@ sglang generate --model-path black-forest-labs/FLUX.1-dev \
|
||||
--prompt "A logo With Bold Large text: SGL Diffusion" \
|
||||
--save-output
|
||||
```
|
||||
|
||||
## Platform-Specific: Apple MPS
|
||||
|
||||
For Apple MPS, please follow the instructions below to install from source:
|
||||
|
||||
```bash
|
||||
# Install ffmpeg
|
||||
brew install ffmpeg
|
||||
|
||||
# Install uv
|
||||
brew install uv
|
||||
|
||||
# Clone the repository
|
||||
git clone https://github.com/sgl-project/sglang.git
|
||||
cd sglang
|
||||
|
||||
# Create and activate a virtual environment
|
||||
uv venv -p 3.11 sglang-diffusion
|
||||
source sglang-diffusion/bin/activate
|
||||
|
||||
# Install the Python packages
|
||||
uv pip install --upgrade pip
|
||||
rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
|
||||
uv pip install -e "python[all_mps]"
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user