[Doc] Tiny update Cuda 13 environment instructions (#17174)

This commit is contained in:
Baizhou Zhang
2026-01-16 06:12:26 +08:00
committed by GitHub
parent 72e2f70ef7
commit 8b99af9af8
+5 -5
View File
@@ -21,10 +21,6 @@ For example, on GB200, you will need to do the following. Otherwise, it will ins
uv pip install "sglang" --extra-index-url https://download.pytorch.org/whl/cu129
```
On B300 (SM103), we recommend using the image at lmsysorg/sglang:dev-cu13.
Please, do not re-install the project as editable inside the docker image, since it will override the version of
libraries specified by the cu13 docker image.
**Quick fixes to common problems**
- If you encounter `OSError: CUDA_HOME environment variable is not set`. Please set it to your CUDA install root with either of the following solutions:
@@ -78,6 +74,10 @@ docker run --gpus all \
You can also find the nightly docker images [here](https://hub.docker.com/r/lmsysorg/sglang/tags?name=nightly).
On B300 (SM103) or Cuda 13 environment, we recommend using the nightly image at `lmsysorg/sglang:dev-cu13` or stable image at `lmsysorg/sglang:latest-cu130-runtime`.
Please, do not re-install the project as editable inside the docker image, since it will override the version of
libraries specified by the cu13 docker image.
## Method 4: Using Kubernetes
Please check out [OME](https://github.com/sgl-project/ome), a Kubernetes operator for enterprise-grade management and serving of large language models (LLMs).
@@ -208,4 +208,4 @@ echo "Build and push completed successfully!"
- [FlashInfer](https://github.com/flashinfer-ai/flashinfer) is the default attention kernel backend. It only supports sm75 and above. If you encounter any FlashInfer-related issues on sm75+ devices (e.g., T4, A10, A100, L4, L40S, H100), please switch to other kernels by adding `--attention-backend triton --sampling-backend pytorch` and open an issue on GitHub.
- To reinstall flashinfer locally, use the following command: `pip3 install --upgrade flashinfer-python --force-reinstall --no-deps` and then delete the cache with `rm -rf ~/.cache/flashinfer`.
- To run sglang in Cuda 13 environment, please use the following docker: `docker pull lmsysorg/sglang:latest-cu130-runtime`.
- When encountering `ptxas fatal : Value 'sm_103a' is not defined for option 'gpu-name'` in Cuda 13 image, please use this command `rm -f /usr/local/lib/python3.12/dist-packages/triton/backends/nvidia/bin/ptxas && ln -s /usr/local/cuda/bin/ptxas /usr/local/lib/python3.12/dist-packages/triton/backends/nvidia/bin/ptxas` to fix the triton ptxas path.