diff --git a/docs/get_started/install.md b/docs/get_started/install.md index f1de87e64..123b12904 100644 --- a/docs/get_started/install.md +++ b/docs/get_started/install.md @@ -21,6 +21,16 @@ For example, on GB200, you will need to do the following. Otherwise, it will ins uv pip install "sglang" --extra-index-url https://download.pytorch.org/whl/cu129 ``` +For CUDA 13, Docker is recommended (see Method 3 note on B300/CUDA 13). If you do not have Docker access, installing the matching `sgl_kernel` wheel from https://github.com/sgl-project/whl/releases after installing SGLang also works. Replace `X.Y.Z` with the `sgl_kernel` version required by your SGLang install. Examples: + +```bash +uv pip install "https://github.com/sgl-project/whl/releases/download/vX.Y.Z/sgl_kernel-X.Y.Z+cu130-cp310-abi3-manylinux2014_x86_64.whl" # x86_64 +``` + +```bash +uv pip install "https://github.com/sgl-project/whl/releases/download/vX.Y.Z/sgl_kernel-X.Y.Z+cu130-cp310-abi3-manylinux2014_aarch64.whl" # aarch64 +``` + **Quick fixes to common problems** - If you encounter `OSError: CUDA_HOME environment variable is not set`. Please set it to your CUDA install root with either of the following solutions: @@ -74,7 +84,7 @@ docker run --gpus all \ You can also find the nightly docker images [here](https://hub.docker.com/r/lmsysorg/sglang/tags?name=nightly). -On B300 (SM103) or Cuda 13 environment, we recommend using the nightly image at `lmsysorg/sglang:dev-cu13` or stable image at `lmsysorg/sglang:latest-cu130-runtime`. +On B300 (SM103) or CUDA 13 environment, we recommend using the nightly image at `lmsysorg/sglang:dev-cu13` or stable image at `lmsysorg/sglang:latest-cu130-runtime`. Please, do not re-install the project as editable inside the docker image, since it will override the version of libraries specified by the cu13 docker image. @@ -208,4 +218,4 @@ echo "Build and push completed successfully!" - [FlashInfer](https://github.com/flashinfer-ai/flashinfer) is the default attention kernel backend. It only supports sm75 and above. If you encounter any FlashInfer-related issues on sm75+ devices (e.g., T4, A10, A100, L4, L40S, H100), please switch to other kernels by adding `--attention-backend triton --sampling-backend pytorch` and open an issue on GitHub. - To reinstall flashinfer locally, use the following command: `pip3 install --upgrade flashinfer-python --force-reinstall --no-deps` and then delete the cache with `rm -rf ~/.cache/flashinfer`. -- When encountering `ptxas fatal : Value 'sm_103a' is not defined for option 'gpu-name'` in Cuda 13 image, please use this command `rm -f /usr/local/lib/python3.12/dist-packages/triton/backends/nvidia/bin/ptxas && ln -s /usr/local/cuda/bin/ptxas /usr/local/lib/python3.12/dist-packages/triton/backends/nvidia/bin/ptxas` to fix the triton ptxas path. +- When encountering `ptxas fatal : Value 'sm_103a' is not defined for option 'gpu-name'` on B300/GB300, fix it with `export TRITON_PTXAS_PATH=/usr/local/cuda/bin/ptxas`.