diff --git a/docs/get_started/install.md b/docs/get_started/install.md index 990a8f7f5..59aff71b3 100644 --- a/docs/get_started/install.md +++ b/docs/get_started/install.md @@ -1,7 +1,6 @@ # Install SGLang You can install SGLang using one of the methods below. - This page primarily applies to common NVIDIA GPU platforms. For other or newer platforms, please refer to the dedicated pages for [AMD GPUs](../platforms/amd_gpu.md), [Intel Xeon CPUs](../platforms/cpu_server.md), [TPU](../platforms/tpu.md), [NVIDIA DGX Spark](https://lmsys.org/blog/2025-11-03-gpt-oss-on-nvidia-dgx-spark/), [NVIDIA Jetson](../platforms/nvidia_jetson.md), [Ascend NPUs](../platforms/ascend_npu.md), and [Intel XPU](../platforms/xpu.md). @@ -18,7 +17,7 @@ uv pip install "sglang" **Quick fixes to common problems** - In some cases (e.g., GB200), the above command might install a wrong torch version (e.g., the CPU version) due to dependency resolution. To fix this, you can first run the above command and then force-reinstall the correct [PyTorch](https://pytorch.org/get-started/locally/) with the following: ``` - uv pip install "torch" --extra-index-url https://download.pytorch.org/whl/cu129 --force-reinstall + uv pip install "torch==2.9.1" "torchvision" --extra-index-url https://download.pytorch.org/whl/cu129 --force-reinstall ``` - For CUDA 13, Docker is recommended (see the Method 3 note on B300/GB300/CUDA 13). If you do not have Docker access, installing the matching `sgl_kernel` wheel from [the sgl-project whl releases](https://github.com/sgl-project/whl/releases) after installing SGLang also works. Replace `X.Y.Z` with the `sgl_kernel` version required by your SGLang (you can find this by running `uv pip show sgl_kernel`). Examples: ```bash diff --git a/python/sglang/srt/configs/lfm2.py b/python/sglang/srt/configs/lfm2.py index 147feed98..40b3cc208 100644 --- a/python/sglang/srt/configs/lfm2.py +++ b/python/sglang/srt/configs/lfm2.py @@ -99,4 +99,3 @@ class Lfm2Config(HFLfm2Config): # Cannot use .register() because lfm2 is already registered by transformers # Directly modify the internal _extra_content dict instead CONFIG_MAPPING._extra_content["lfm2"] = Lfm2Config -logger.info("Registered SGLang Lfm2Config to override HuggingFace's version") diff --git a/python/sglang/srt/layers/torchao_utils.py b/python/sglang/srt/layers/torchao_utils.py index 7de4bf157..4276e122a 100644 --- a/python/sglang/srt/layers/torchao_utils.py +++ b/python/sglang/srt/layers/torchao_utils.py @@ -42,6 +42,9 @@ def apply_torchao_config_to_model( quantize the model, e.g. int4wo-128 means int4 weight only quantization with group_size 128 """ + if torchao_config == "" or torchao_config is None: + return model + # Lazy import to suppress some warnings from torchao.quantization import ( float8_dynamic_activation_float8_weight, @@ -53,9 +56,7 @@ def apply_torchao_config_to_model( ) from torchao.quantization.observer import PerRow, PerTensor - if torchao_config == "" or torchao_config is None: - return model - elif "int8wo" in torchao_config: + if "int8wo" in torchao_config: quantize_(model, int8_weight_only(), filter_fn=proj_filter_conv3d) elif "int8dq" in torchao_config: quantize_(model, int8_dynamic_activation_int8_weight(), filter_fn=filter_fn)