sgl-diffusion is an inference framework for accelerated image/video generation.
sgl-diffusion features an end-to-end unified pipeline for accelerating diffusion models. It is designed to be modular and extensible, allowing users to easily add new optimizations and techniques.
Key Features
sgl-diffusion has the following features:
- State-of-the-art performance optimizations for inference
- Video Sparse Attention
- Sliding Tile Attention
- TeaCache
- Sage Attention
- USP
- CFG Parallel
- Diverse hardware and OS support
- Supported hardware: H100, H200, A100, B200, 4090
- Supported OS: Linux, Windows, MacOS
Getting Started
uv pip install 'sglang[diffusion]' --prerelease=allow
For more information, check the docs.
Inference
Here's a minimal example to generate a video using the default settings:
from sglang.multimodal_gen import DiffGenerator
def main():
# Create a diff generator from a pre-trained model
generator = DiffGenerator.from_pretrained(
model_path="Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
num_gpus=1, # Adjust based on your hardware
)
# Provide a prompt for your video
prompt = "A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes wide with interest."
# Generate the video
video = generator.generate(
prompt,
return_frames=True, # Also return frames from this call (defaults to False)
output_path="my_videos/", # Controls where videos are saved
save_output=True
)
if __name__ == '__main__':
main()
Or, more simply, with the CLI:
sglang generate --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
--text-encoder-cpu-offload --pin-cpu-memory \
--prompt "A curious raccoon" \
--save-output
For more information, check the docs.
Contributing
All contributions are welcome.
Acknowledgement
We learnt and reused code from the following projects: