From 8b99af9af87df54154915068320303a1e7104020 Mon Sep 17 00:00:00 2001 From: Baizhou Zhang Date: Fri, 16 Jan 2026 06:12:26 +0800 Subject: [PATCH] [Doc] Tiny update Cuda 13 environment instructions (#17174) --- docs/get_started/install.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/get_started/install.md b/docs/get_started/install.md index 733cfb250..f1de87e64 100644 --- a/docs/get_started/install.md +++ b/docs/get_started/install.md @@ -21,10 +21,6 @@ For example, on GB200, you will need to do the following. Otherwise, it will ins uv pip install "sglang" --extra-index-url https://download.pytorch.org/whl/cu129 ``` -On B300 (SM103), we recommend using the image at lmsysorg/sglang:dev-cu13. -Please, do not re-install the project as editable inside the docker image, since it will override the version of -libraries specified by the cu13 docker image. - **Quick fixes to common problems** - If you encounter `OSError: CUDA_HOME environment variable is not set`. Please set it to your CUDA install root with either of the following solutions: @@ -78,6 +74,10 @@ docker run --gpus all \ You can also find the nightly docker images [here](https://hub.docker.com/r/lmsysorg/sglang/tags?name=nightly). +On B300 (SM103) or Cuda 13 environment, we recommend using the nightly image at `lmsysorg/sglang:dev-cu13` or stable image at `lmsysorg/sglang:latest-cu130-runtime`. +Please, do not re-install the project as editable inside the docker image, since it will override the version of +libraries specified by the cu13 docker image. + ## Method 4: Using Kubernetes Please check out [OME](https://github.com/sgl-project/ome), a Kubernetes operator for enterprise-grade management and serving of large language models (LLMs). @@ -208,4 +208,4 @@ echo "Build and push completed successfully!" - [FlashInfer](https://github.com/flashinfer-ai/flashinfer) is the default attention kernel backend. It only supports sm75 and above. If you encounter any FlashInfer-related issues on sm75+ devices (e.g., T4, A10, A100, L4, L40S, H100), please switch to other kernels by adding `--attention-backend triton --sampling-backend pytorch` and open an issue on GitHub. - To reinstall flashinfer locally, use the following command: `pip3 install --upgrade flashinfer-python --force-reinstall --no-deps` and then delete the cache with `rm -rf ~/.cache/flashinfer`. -- To run sglang in Cuda 13 environment, please use the following docker: `docker pull lmsysorg/sglang:latest-cu130-runtime`. +- When encountering `ptxas fatal : Value 'sm_103a' is not defined for option 'gpu-name'` in Cuda 13 image, please use this command `rm -f /usr/local/lib/python3.12/dist-packages/triton/backends/nvidia/bin/ptxas && ln -s /usr/local/cuda/bin/ptxas /usr/local/lib/python3.12/dist-packages/triton/backends/nvidia/bin/ptxas` to fix the triton ptxas path.