chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Release sglang kernel 0.4.0 (#20440)

Browse Source 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
This commit is contained in:
Xiaoyu Zhang
2026-03-16 20:34:58 +08:00
committed by GitHub
parent 3d58cd16d9
commit 15097c5c3b
36 changed files with 188 additions and 146 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/release-whl-kernel.yml vendored
View File

@@ -72,7 +72,7 @@ jobs:
working-directory: sgl-kernel
run: |
pip install twine
python3 -m twine upload --skip-existing dist/* -u __token__ -p ${{ secrets.PYPI_TOKEN }}
python3 -m twine upload --skip-existing dist/* -u __token__ -p ${{ secrets.PYPI_TOKEN_SGLANG_KERNEL }}
- name: Upload artifacts
uses: actions/upload-artifact@v4

10
3rdparty/amd/wheel/README.md vendored
View File

@@ -1,10 +1,10 @@
# sgl-kernel
# sglang-kernel (prior sgl-kernel)
Building and releasing sgl-kernel as a wheel is a part of the release workflow. Check [release-whl-kernel.yml](https://github.com/sgl-project/sglang/blob/main/.github/workflows/release-whl-kernel.yml) for details.
Building and releasing `sglang-kernel` as a wheel is a part of the release workflow. Check [release-whl-kernel.yml](https://github.com/sgl-project/sglang/blob/main/.github/workflows/release-whl-kernel.yml) for details.
# sglang
`3rdparty/amd/wheel/sglang/pyproject.toml` is the AMD-specific pyproject for building the `amd-sglang` wheel. It extends `python/pyproject_other.toml` with two ROCm-version extras (`rocm700`, `rocm720`) that pin the matching torch/triton/torchaudio/torchvision/sgl-kernel wheels, and renames the package to `amd-sglang`.
`3rdparty/amd/wheel/sglang/pyproject.toml` is the AMD-specific pyproject for building the `amd-sglang` wheel. It extends `python/pyproject_other.toml` with two ROCm-version extras (`rocm700`, `rocm720`) that pin the matching torch/triton/torchaudio/torchvision/`sglang-kernel` wheels, and renames the package to `amd-sglang`.
## Operation to build sglang wheel
@@ -20,13 +20,13 @@ $ cd python && python -m build
ROCm 7.0.0:
```
pip uninstall sgl-kernel sglang amd-sglang
pip uninstall sglang-kernel sglang amd-sglang
pip install "amd-sglang[all-hip,rocm700]" -i https://pypi.amd.com/rocm-7.0.0/simple --extra-index-url https://pypi.org/simple
```
ROCm 7.2.0:
```
pip uninstall sgl-kernel sglang amd-sglang
pip uninstall sglang-kernel sglang amd-sglang
pip install "amd-sglang[all-hip,rocm720]" -i https://pypi.amd.com/rocm-7.2.0/simple --extra-index-url https://pypi.org/simple
```

4
3rdparty/amd/wheel/sglang/pyproject.toml vendored
View File

@@ -76,7 +76,7 @@ rocm700 = [
"torchaudio @ https://repo.radeon.com/rocm/manylinux/rocm-rel-7.0.2/torchaudio-2.9.0%2Brocm7.0.2.gite3c6ee2b-cp310-cp310-linux_x86_64.whl",
"torchvision @ https://repo.radeon.com/rocm/manylinux/rocm-rel-7.0.2/torchvision-0.24.0%2Brocm7.0.2.gitb919bd0c-cp310-cp310-linux_x86_64.whl",
"mooncake-transfer-engine-non-cuda==0.3.8.post1",
"sgl-kernel @ https://github.com/sgl-project/whl/releases/download/v0.3.21/sgl_kernel-0.3.21+rocm700-cp310-abi3-manylinux2014_x86_64.whl#sha256=2d56ee97d8e6dae29d35a2524de4bffb8e07752bcc74e73313462496427832bf",
"sglang-kernel @ https://github.com/sgl-project/whl/releases/download/v0.4.0/sglang_kernel-0.4.0+rocm700-cp310-abi3-manylinux2014_x86_64.whl",
]
rocm720 = [
@@ -85,7 +85,7 @@ rocm720 = [
"torchaudio @ https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchaudio-2.9.0%2Brocm7.2.0.gite3c6ee2b-cp310-cp310-linux_x86_64.whl",
"torchvision @ https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchvision-0.24.0%2Brocm7.2.0.gitb919bd0c-cp310-cp310-linux_x86_64.whl",
"mooncake-transfer-engine-non-cuda==0.3.8.post1",
"sgl-kernel @ https://github.com/sgl-project/whl/releases/download/v0.3.21/sgl_kernel-0.3.21+rocm720-cp310-abi3-manylinux2014_x86_64.whl#sha256=91a3f50ef0b9d2a140dbf6e2aabff45a50c45a4782b9215997bce6f71749180f",
"sglang-kernel @ https://github.com/sgl-project/whl/releases/download/v0.4.0/sglang_kernel-0.4.0+rocm720-cp310-abi3-manylinux2014_x86_64.whl",
]
# HIP (Heterogeneous-computing Interface for Portability) for AMD

8
docker/Dockerfile
View File

@@ -11,7 +11,7 @@ ARG GRACE_BLACKWELL_DEEPEP_BRANCH=gb200_blog_part_2
ARG HOPPER_SBO_DEEPEP_COMMIT=9f2fc4b3182a51044ae7ecb6610f7c9c3258c4d6
ARG DEEPEP_COMMIT=9af0e0d0e74f3577af1979c9b9e1ac2cad0104ee
ARG BUILD_AND_DOWNLOAD_PARALLEL=8
ARG SGL_KERNEL_VERSION=0.3.21
ARG SGL_KERNEL_VERSION=0.4.0
ARG SGL_VERSION
ARG USE_LATEST_SGLANG=0
ARG GDRCOPY_VERSION=2.5.1
@@ -201,13 +201,13 @@ RUN --mount=type=cache,target=/root/.cache/pip \
*) echo "Unsupported CUDA version: $CUDA_VERSION" && exit 1 ;; \
esac \
&& if [ "$CUDA_VERSION" = "12.6.1" ]; then \
python3 -m pip install https://${GITHUB_ARTIFACTORY}/sgl-project/whl/releases/download/v${SGL_KERNEL_VERSION}/sgl_kernel-${SGL_KERNEL_VERSION}+cu124-cp310-abi3-manylinux2014_$(uname -m).whl --force-reinstall --no-deps \
python3 -m pip install https://${GITHUB_ARTIFACTORY}/sgl-project/whl/releases/download/v${SGL_KERNEL_VERSION}/sglang_kernel-${SGL_KERNEL_VERSION}+cu124-cp310-abi3-manylinux2014_$(uname -m).whl --force-reinstall --no-deps \
; \
elif [ "$CUDA_VERSION" = "12.8.1" ] || [ "$CUDA_VERSION" = "12.9.1" ]; then \
python3 -m pip install sgl-kernel==${SGL_KERNEL_VERSION} \
python3 -m pip install sglang-kernel==${SGL_KERNEL_VERSION} \
; \
elif [ "$CUDA_VERSION" = "13.0.1" ]; then \
python3 -m pip install https://github.com/sgl-project/whl/releases/download/v${SGL_KERNEL_VERSION}/sgl_kernel-${SGL_KERNEL_VERSION}+cu130-cp310-abi3-manylinux2014_$(uname -m).whl --force-reinstall --no-deps \
python3 -m pip install https://github.com/sgl-project/whl/releases/download/v${SGL_KERNEL_VERSION}/sglang_kernel-${SGL_KERNEL_VERSION}+cu130-cp310-abi3-manylinux2014_$(uname -m).whl --force-reinstall --no-deps \
; \
else \
echo "Unsupported CUDA version: $CUDA_VERSION" && exit 1 \

10
docs/developer_guide/contribution_guide.md
View File

@@ -125,17 +125,17 @@ Users listed in [CI_PERMISSIONS.json](https://github.com/sgl-project/sglang/blob
- If you write multiple if/else blocks for new features, ensure the common path (e.g., NVIDIA hardware or the existing code path) is the first branch.
## How to update sgl-kernel
Since sglang and sgl-kernel are separate Python packages, our current GitHub CI infrastructure does not support updating a kernel and using it immediately within the same pull request (PR).
To add a new kernel or modify an existing one in the sgl-kernel package, you must use multiple PRs.
Since sglang and the `sglang-kernel` (prior `sgl-kernel`) distribution are separate Python packages, our current GitHub CI infrastructure does not support updating a kernel and using it immediately within the same pull request (PR).
To add a new kernel or modify an existing one in the `sgl-kernel/` source tree, you must use multiple PRs.
Follow these steps:
1. Submit a PR to update the sgl-kernel source code without using it in sglang python package (e.g., [#8884](https://github.com/sgl-project/sglang/pull/8884/files)).
2. Bump the version of sgl-kernel (e.g., [#9220](https://github.com/sgl-project/sglang/pull/9220/files)).
- Once merged, this will trigger an automatic release of the sgl-kernel wheel to PyPI.
2. Bump the version of the kernel package (e.g., [#9220](https://github.com/sgl-project/sglang/pull/9220/files)).
- Once merged, this will trigger an automatic release of the `sglang-kernel` wheel to PyPI.
- If not urgent, you can wait for other people to release the wheel. A new version will typically be released within one week.
3. Apply the changes:
- Update the sgl-kernel version in `sglang/python/pyproject.toml` to use the modified kernels.
- Update the `sglang-kernel` version in `sglang/python/pyproject.toml` to use the modified kernels.
- Update the related caller code in the sglang to use the new kernel.
## Tips for newcomers

6
docs/get_started/install.md
View File

@@ -29,13 +29,13 @@ uv pip install torch==X.Y.Z torchvision torchaudio --index-url https://download.
uv pip install sglang
```
3. Install the `sgl_kernel` wheel for CUDA 13 from [the sgl-project whl releases](https://github.com/sgl-project/whl/blob/gh-pages/cu130/sgl-kernel/index.html). Replace `X.Y.Z` with the `sgl_kernel` version required by your SGLang install (you can find this by running `uv pip show sgl_kernel`). Examples:
3. Install the `sglang-kernel` wheel for CUDA 13 from [the sgl-project whl releases](https://github.com/sgl-project/whl/blob/gh-pages/cu130/sglang-kernel/index.html). Replace `X.Y.Z` with the `sglang-kernel` version required by your SGLang install (you can find this by running `uv pip show sglang-kernel`). Examples:
```bash
# x86_64
uv pip install "https://github.com/sgl-project/whl/releases/download/vX.Y.Z/sgl_kernel-X.Y.Z+cu130-cp310-abi3-manylinux2014_x86_64.whl"
uv pip install "https://github.com/sgl-project/whl/releases/download/vX.Y.Z/sglang_kernel-X.Y.Z+cu130-cp310-abi3-manylinux2014_x86_64.whl"
# aarch64
uv pip install "https://github.com/sgl-project/whl/releases/download/vX.Y.Z/sgl_kernel-X.Y.Z+cu130-cp310-abi3-manylinux2014_aarch64.whl"
uv pip install "https://github.com/sgl-project/whl/releases/download/vX.Y.Z/sglang_kernel-X.Y.Z+cu130-cp310-abi3-manylinux2014_aarch64.whl"
```
### **Quick fixes to common problems**

8
docs/platforms/ascend_contribution_guide.md
View File

@@ -127,16 +127,16 @@ Users listed in [CI_PERMISSIONS.json](https://github.com/sgl-project/sglang/blob
## How to update sgl-kernel
Since sglang and sgl-kernel are separate Python packages, our current GitHub CI infrastructure does not support updating a kernel and using it immediately within the same pull request (PR).
To add a new kernel or modify an existing one in the sgl-kernel package, you must use multiple PRs.
To add a new kernel or modify an existing one in the `sgl-kernel/` source tree, you must use multiple PRs.
Follow these steps:
1. Submit a PR to update the sgl-kernel source code without using it in sglang python package (e.g., [#8884](https://github.com/sgl-project/sglang/pull/8884/files)).
2. Bump the version of sgl-kernel (e.g., [#9220](https://github.com/sgl-project/sglang/pull/9220/files)).
- Once merged, this will trigger an automatic release of the sgl-kernel wheel to PyPI.
2. Bump the version of the kernel package (e.g., [#9220](https://github.com/sgl-project/sglang/pull/9220/files)).
- Once merged, this will trigger an automatic release of the `sglang-kernel` wheel to PyPI.
- If not urgent, you can wait for other people to release the wheel. A new version will typically be released within one week.
3. Apply the changes:
- Update the sgl-kernel version in `sglang/python/pyproject.toml` to use the modified kernels.
- Update the `sglang-kernel` version in `sglang/python/pyproject.toml` to use the modified kernels.
- Update the related caller code in the sglang to use the new kernel.
## How to update sgl-kernel-npu

2
python/pyproject.toml
View File

@@ -60,7 +60,7 @@ dependencies = [
"sentencepiece",
"setproctitle",
"sgl-fa4==4.0.5",
"sgl-kernel==0.3.21",
"sglang-kernel==0.4.0",
"soundfile==0.13.1",
"tiktoken",
"timm==1.0.16",

2
python/sglang/check_env.py
View File

@@ -20,7 +20,7 @@ def is_cuda_v2():
# List of packages to check versions
PACKAGE_LIST = [
"sglang",
"sgl_kernel",
"sglang-kernel",
"flashinfer_python",
"flashinfer_cubin",
"flashinfer_jit_cache",

15
python/sglang/jit_kernel/benchmark/bench_awq_marlin_moe_repack.py
View File

@@ -10,12 +10,9 @@ from sglang.jit_kernel.awq_marlin_repack import (
from sglang.jit_kernel.benchmark.utils import is_in_ci, run_benchmark
from sglang.srt.layers.quantization.utils import pack_cols, quantize_weights
try:
from sgl_kernel import awq_marlin_moe_repack as aot_awq_marlin_moe_repack
AOT_AVAILABLE = True
except ImportError:
AOT_AVAILABLE = False
AOT_AVAILABLE = hasattr(torch.ops.sgl_kernel, "awq_marlin_moe_repack") and hasattr(
torch.ops.sgl_kernel.awq_marlin_moe_repack, "default"
)
IS_CI = is_in_ci()
@@ -66,7 +63,9 @@ def check_correctness():
)
out_jit = jit_awq_marlin_moe_repack(b_q_weight, perm, size_k, SIZE_N, NUM_BITS)
out_aot = aot_awq_marlin_moe_repack(b_q_weight, perm, size_k, SIZE_N, NUM_BITS)
out_aot = torch.ops.sgl_kernel.awq_marlin_moe_repack.default(
b_q_weight, perm, size_k, SIZE_N, NUM_BITS
)
torch.cuda.synchronize()
torch.testing.assert_close(out_jit, out_aot, rtol=0, atol=0)
print("Correctness check passed (JIT vs AOT)")
@@ -111,7 +110,7 @@ def benchmark(num_experts, size_k, size_n, num_bits, provider):
b_q_weight, perm, size_k, size_n, num_bits
)
elif provider == "aot":
fn = lambda: aot_awq_marlin_moe_repack(
fn = lambda: torch.ops.sgl_kernel.awq_marlin_moe_repack.default(
b_q_weight, perm, size_k, size_n, num_bits
)
else:

17
python/sglang/jit_kernel/benchmark/bench_awq_marlin_repack.py
View File

@@ -10,12 +10,9 @@ from sglang.jit_kernel.awq_marlin_repack import (
from sglang.jit_kernel.benchmark.utils import is_in_ci, run_benchmark
from sglang.srt.layers.quantization.utils import pack_cols, quantize_weights
try:
from sgl_kernel import awq_marlin_repack as aot_awq_marlin_repack
AOT_AVAILABLE = True
except ImportError:
AOT_AVAILABLE = False
AOT_AVAILABLE = hasattr(torch.ops.sgl_kernel, "awq_marlin_repack") and hasattr(
torch.ops.sgl_kernel.awq_marlin_repack, "default"
)
IS_CI = is_in_ci()
@@ -50,7 +47,9 @@ def check_correctness():
print("sgl_kernel AOT not available, skipping correctness check")
return
out_jit = jit_awq_marlin_repack(_q_w_awq, SIZE_K, SIZE_N, NUM_BITS)
out_aot = aot_awq_marlin_repack(_q_w_awq, SIZE_K, SIZE_N, NUM_BITS)
out_aot = torch.ops.sgl_kernel.awq_marlin_repack.default(
_q_w_awq, SIZE_K, SIZE_N, NUM_BITS
)
torch.cuda.synchronize()
torch.testing.assert_close(out_jit, out_aot, rtol=0, atol=0)
print("Correctness check passed (JIT vs AOT)")
@@ -96,7 +95,9 @@ def benchmark(size_k, size_n, num_bits, provider):
if provider == "jit":
fn = lambda: jit_awq_marlin_repack(q_w_awq, size_k, size_n, num_bits)
elif provider == "aot":
fn = lambda: aot_awq_marlin_repack(q_w_awq, size_k, size_n, num_bits)
fn = lambda: torch.ops.sgl_kernel.awq_marlin_repack.default(
q_w_awq, size_k, size_n, num_bits
)
else:
raise ValueError(f"Unknown provider: {provider}")

35
python/sglang/jit_kernel/benchmark/bench_gptq_marlin.py
View File

@@ -8,12 +8,9 @@ from sglang.jit_kernel.gptq_marlin import gptq_marlin_gemm as jit_gptq_marlin_ge
from sglang.srt.layers.quantization.marlin_utils import marlin_make_workspace
from sglang.test.test_marlin_utils import marlin_quantize
try:
from sgl_kernel import gptq_marlin_gemm as aot_gptq_marlin_gemm
AOT_AVAILABLE = True
except ImportError:
AOT_AVAILABLE = False
AOT_AVAILABLE = hasattr(torch.ops.sgl_kernel, "gptq_marlin_gemm") and hasattr(
torch.ops.sgl_kernel.gptq_marlin_gemm, "default"
)
IS_CI = is_in_ci()
@@ -51,13 +48,35 @@ def _run_gemm(fn, a):
)
def _run_gemm_aot(a):
return torch.ops.sgl_kernel.gptq_marlin_gemm.default(
a,
None,
_marlin_q_w,
_marlin_s,
None,
None,
_g_idx,
_sort_indices,
_workspace,
QUANT_TYPE.id,
a.shape[0],
SIZE_N,
SIZE_K,
True,
False,
False,
False,
)
def check_correctness():
if not AOT_AVAILABLE:
print("sgl_kernel AOT not available, skipping correctness check")
return
a = torch.randn((16, SIZE_K), dtype=torch.float16, device="cuda")
out_jit = _run_gemm(jit_gptq_marlin_gemm, a)
out_aot = _run_gemm(aot_gptq_marlin_gemm, a)
out_aot = _run_gemm_aot(a)
torch.testing.assert_close(out_jit, out_aot, rtol=1e-3, atol=1e-3)
print("Correctness check passed (JIT vs AOT)")
@@ -97,7 +116,7 @@ def benchmark(size_m, provider):
if provider == "jit":
fn = lambda: _run_gemm(jit_gptq_marlin_gemm, a)
elif provider == "aot":
fn = lambda: _run_gemm(aot_gptq_marlin_gemm, a)
fn = lambda: _run_gemm_aot(a)
else:
raise ValueError(f"Unknown provider: {provider}")

17
python/sglang/jit_kernel/benchmark/bench_gptq_marlin_repack.py
View File

@@ -7,12 +7,9 @@ from sglang.jit_kernel.benchmark.utils import is_in_ci, run_benchmark
from sglang.jit_kernel.gptq_marlin_repack import gptq_marlin_repack as jit_fn
from sglang.srt.layers.quantization.utils import gptq_quantize_weights, pack_rows
try:
from sgl_kernel import gptq_marlin_repack as aot_fn
AOT_AVAILABLE = True
except ImportError:
AOT_AVAILABLE = False
AOT_AVAILABLE = hasattr(torch.ops.sgl_kernel, "gptq_marlin_repack") and hasattr(
torch.ops.sgl_kernel.gptq_marlin_repack, "default"
)
IS_CI = is_in_ci()
@@ -44,7 +41,9 @@ def check_correctness():
size_k = 4096
q_w_gptq, sort_indices = _get_inputs(size_k)
out_jit = jit_fn(q_w_gptq, sort_indices, size_k, SIZE_N, NUM_BITS)
out_aot = aot_fn(q_w_gptq, sort_indices, size_k, SIZE_N, NUM_BITS)
out_aot = torch.ops.sgl_kernel.gptq_marlin_repack.default(
q_w_gptq, sort_indices, size_k, SIZE_N, NUM_BITS
)
torch.testing.assert_close(out_jit, out_aot, rtol=0, atol=0)
print("Correctness check passed (JIT vs AOT)")
@@ -83,7 +82,9 @@ def benchmark(size_k, provider):
if provider == "jit":
fn = lambda: jit_fn(q_w_gptq, sort_indices, size_k, SIZE_N, NUM_BITS)
elif provider == "aot":
fn = lambda: aot_fn(q_w_gptq, sort_indices, size_k, SIZE_N, NUM_BITS)
fn = lambda: torch.ops.sgl_kernel.gptq_marlin_repack.default(
q_w_gptq, sort_indices, size_k, SIZE_N, NUM_BITS
)
else:
raise ValueError(f"Unknown provider: {provider}")

9
python/sglang/jit_kernel/benchmark/bench_moe_wna16_marlin.py
View File

@@ -8,12 +8,9 @@ from sglang.jit_kernel.moe_wna16_marlin import moe_wna16_marlin_gemm as jit_fn
from sglang.srt.layers.moe.fused_moe_triton import moe_align_block_size
from sglang.test.test_marlin_utils import marlin_quantize
try:
from sgl_kernel import moe_wna16_marlin_gemm as _aot_import # noqa: F401
AOT_AVAILABLE = True
except (ImportError, AttributeError):
AOT_AVAILABLE = False
AOT_AVAILABLE = hasattr(torch.ops.sgl_kernel, "moe_wna16_marlin_gemm") and hasattr(
torch.ops.sgl_kernel.moe_wna16_marlin_gemm, "default"
)
IS_CI = is_in_ci()

16
python/sglang/jit_kernel/tests/test_awq_marlin_moe_repack.py
View File

@@ -8,12 +8,14 @@ from sglang.jit_kernel.awq_marlin_repack import (
)
from sglang.srt.layers.quantization.utils import pack_cols, quantize_weights
try:
from sgl_kernel import awq_marlin_moe_repack as aot_awq_marlin_moe_repack
AOT_AVAILABLE = True
except ImportError:
AOT_AVAILABLE = False
def _has_aot_awq_marlin_moe_repack() -> bool:
return hasattr(torch.ops.sgl_kernel, "awq_marlin_moe_repack") and hasattr(
torch.ops.sgl_kernel.awq_marlin_moe_repack, "default"
)
AOT_AVAILABLE = _has_aot_awq_marlin_moe_repack()
def awq_pack(
@@ -68,7 +70,9 @@ def test_awq_marlin_moe_repack_jit_vs_aot(
perm = torch.empty((num_experts, 0), dtype=torch.int32, device="cuda")
out_jit = jit_awq_marlin_moe_repack(b_q_weight, perm, size_k, size_n, num_bits)
out_aot = aot_awq_marlin_moe_repack(b_q_weight, perm, size_k, size_n, num_bits)
out_aot = torch.ops.sgl_kernel.awq_marlin_moe_repack.default(
b_q_weight, perm, size_k, size_n, num_bits
)
torch.cuda.synchronize()

16
python/sglang/jit_kernel/tests/test_awq_marlin_repack.py
View File

@@ -9,12 +9,14 @@ from sglang.jit_kernel.awq_marlin_repack import (
from sglang.srt.layers.quantization.utils import pack_cols, quantize_weights
from sglang.test.test_marlin_utils import get_weight_perm, marlin_weights
try:
from sgl_kernel import awq_marlin_repack as aot_awq_marlin_repack
AOT_AVAILABLE = True
except ImportError:
AOT_AVAILABLE = False
def _has_aot_awq_marlin_repack() -> bool:
return hasattr(torch.ops.sgl_kernel, "awq_marlin_repack") and hasattr(
torch.ops.sgl_kernel.awq_marlin_repack, "default"
)
AOT_AVAILABLE = _has_aot_awq_marlin_repack()
def awq_pack(
@@ -58,7 +60,9 @@ def test_awq_marlin_repack_jit_vs_aot(num_bits, k_tiles, n_tiles, group_size):
q_w_awq = awq_pack(q_w, num_bits, size_k, size_n)
out_jit = jit_awq_marlin_repack(q_w_awq, size_k, size_n, num_bits)
out_aot = aot_awq_marlin_repack(q_w_awq, size_k, size_n, num_bits)
out_aot = torch.ops.sgl_kernel.awq_marlin_repack.default(
q_w_awq, size_k, size_n, num_bits
)
torch.cuda.synchronize()

15
python/sglang/jit_kernel/tests/test_moe_wna16_marlin.py
View File

@@ -2,7 +2,6 @@ import itertools
import pytest
import torch
from sgl_kernel import moe_wna16_marlin_gemm as aot_moe_wna16_marlin_gemm
from sgl_kernel.scalar_type import scalar_types
from sglang.jit_kernel.moe_wna16_marlin import moe_wna16_marlin_gemm
@@ -10,6 +9,15 @@ from sglang.srt.layers.moe.fused_moe_triton import moe_align_block_size
from sglang.test.test_marlin_utils import awq_marlin_quantize, marlin_quantize
def _has_aot_moe_wna16_marlin_gemm() -> bool:
return hasattr(torch.ops.sgl_kernel, "moe_wna16_marlin_gemm") and hasattr(
torch.ops.sgl_kernel.moe_wna16_marlin_gemm, "default"
)
AOT_AVAILABLE = _has_aot_moe_wna16_marlin_gemm()
def stack_and_dev(tensors: list[torch.Tensor]):
dev = tensors[0].device
return torch.stack(tensors, dim=0).to(dev)
@@ -143,7 +151,7 @@ def _run_single_gemm_aot(
is_k_full,
use_atomic_add,
):
return aot_moe_wna16_marlin_gemm(
return torch.ops.sgl_kernel.moe_wna16_marlin_gemm.default(
a,
c,
qweight,
@@ -224,6 +232,9 @@ TEST_CASES = generate_test_cases()
def test_moe_wna16_marlin_gemm(
m, n, k, e, topk, dtype, group_size, act_order, quant_type
):
if not AOT_AVAILABLE:
pytest.skip("sgl_kernel moe_wna16_marlin_gemm AOT op not available")
torch.manual_seed(0)
has_zp = quant_type in [scalar_types.uint4, scalar_types.uint8]

6
python/sglang/srt/entrypoints/engine.py
View File

@@ -1142,9 +1142,9 @@ def _set_envs_and_config(server_args: ServerArgs):
)
if _is_cuda:
assert_pkg_version(
"sgl-kernel",
"0.3.21",
"Please reinstall the latest version with `pip install sgl-kernel --force-reinstall`",
"sglang-kernel",
"0.4.0",
"Please reinstall the latest version with `pip install sglang-kernel --force-reinstall`",
)
# Signal handlers can only be registered from the main thread.

1
scripts/ci/amd/amd_ci_install_dependency.sh
View File

@@ -110,6 +110,7 @@ if [ -n "$SKIP_SGLANG_BUILD" ]; then
echo "Didn't build checkout SGLang"
else
docker exec ci_sglang pip uninstall sgl-kernel -y || true
docker exec ci_sglang pip uninstall sglang-kernel -y || true
docker exec ci_sglang pip uninstall sglang -y || true
# Clear Python cache to ensure latest code is used
docker exec ci_sglang find /opt/venv -name "*.pyc" -delete || true

14
scripts/ci/cuda/ci_install_dependency.sh
View File

@@ -190,7 +190,7 @@ fi
# Install sgl-kernel
SGL_KERNEL_VERSION_FROM_KERNEL=$(grep -Po '(?<=^version = ")[^"]*' sgl-kernel/pyproject.toml)
SGL_KERNEL_VERSION_FROM_SRT=$(grep -Po -m1 '(?<=sgl-kernel==)[0-9A-Za-z\.\-]+' python/pyproject.toml)
SGL_KERNEL_VERSION_FROM_SRT=$(grep -Po -m1 '(?<=sglang-kernel==)[0-9A-Za-z\.\-]+' python/pyproject.toml)
echo "SGL_KERNEL_VERSION_FROM_KERNEL=${SGL_KERNEL_VERSION_FROM_KERNEL} SGL_KERNEL_VERSION_FROM_SRT=${SGL_KERNEL_VERSION_FROM_SRT}"
if [ "${CUSTOM_BUILD_SGL_KERNEL:-}" = "true" ] && [ -d "sgl-kernel/dist" ]; then
@@ -201,7 +201,7 @@ if [ "${CUSTOM_BUILD_SGL_KERNEL:-}" = "true" ] && [ -d "sgl-kernel/dist" ]; then
else
WHEEL_ARCH="x86_64"
fi
$PIP_CMD install sgl-kernel/dist/sgl_kernel-${SGL_KERNEL_VERSION_FROM_KERNEL}-cp310-abi3-manylinux2014_${WHEEL_ARCH}.whl --force-reinstall $PIP_INSTALL_SUFFIX
$PIP_CMD install sgl-kernel/dist/sglang_kernel-${SGL_KERNEL_VERSION_FROM_KERNEL}-cp310-abi3-manylinux2014_${WHEEL_ARCH}.whl --force-reinstall $PIP_INSTALL_SUFFIX
elif [ "${CUSTOM_BUILD_SGL_KERNEL:-}" = "true" ] && [ ! -d "sgl-kernel/dist" ]; then
# CUSTOM_BUILD_SGL_KERNEL was set but artifacts not available (e.g., stage rerun without wheel build)
# Fail instead of falling back to PyPI - we need to test the built kernel, not PyPI version
@@ -212,15 +212,15 @@ elif [ "${CUSTOM_BUILD_SGL_KERNEL:-}" = "true" ] && [ ! -d "sgl-kernel/dist" ];
else
# On Blackwell machines, skip reinstall if correct version already installed to avoid race conditions
if [ "$IS_BLACKWELL" = "1" ]; then
INSTALLED_SGL_KERNEL=$(pip show sgl-kernel 2>/dev/null | grep "^Version:" | awk '{print $2}' || echo "")
INSTALLED_SGL_KERNEL=$(pip show sglang-kernel 2>/dev/null | grep "^Version:" | awk '{print $2}' || echo "")
if [ "$INSTALLED_SGL_KERNEL" = "$SGL_KERNEL_VERSION_FROM_SRT" ]; then
echo "sgl-kernel==${SGL_KERNEL_VERSION_FROM_SRT} already installed, skipping reinstall"
echo "sglang-kernel==${SGL_KERNEL_VERSION_FROM_SRT} already installed, skipping reinstall"
else
echo "Installing sgl-kernel==${SGL_KERNEL_VERSION_FROM_SRT} (current: ${INSTALLED_SGL_KERNEL:-none})"
$PIP_CMD install sgl-kernel==${SGL_KERNEL_VERSION_FROM_SRT} $PIP_INSTALL_SUFFIX
echo "Installing sglang-kernel==${SGL_KERNEL_VERSION_FROM_SRT} (current: ${INSTALLED_SGL_KERNEL:-none})"
$PIP_CMD install sglang-kernel==${SGL_KERNEL_VERSION_FROM_SRT} $PIP_INSTALL_SUFFIX
fi
else
$PIP_CMD install sgl-kernel==${SGL_KERNEL_VERSION_FROM_SRT} --force-reinstall $PIP_INSTALL_SUFFIX
$PIP_CMD install sglang-kernel==${SGL_KERNEL_VERSION_FROM_SRT} --force-reinstall $PIP_INSTALL_SUFFIX
fi
fi

13
scripts/release/README.md
View File

@@ -25,17 +25,18 @@ python scripts/release/bump_sglang_version.py 0.5.3rc0
- `python/sglang/version.py`
### `bump_kernel_version.py`
Updates sgl-kernel version across all relevant files following the pattern from [PR #10732](https://github.com/sgl-project/sglang/pull/10732).
Updates the `sglang-kernel` release version across all relevant files following the pattern from [PR #10732](https://github.com/sgl-project/sglang/pull/10732).
**Usage:**
```bash
python scripts/release/bump_kernel_version.py 0.3.12
python scripts/release/bump_kernel_version.py 0.4.0
```
**Files updated:**
- `sgl-kernel/pyproject.toml`
- `sgl-kernel/pyproject_cpu.toml`
- `sgl-kernel/pyproject_rocm.toml`
- `sgl-kernel/pyproject_musa.toml`
- `sgl-kernel/python/sgl_kernel/version.py`
## Manual Testing Instructions
@@ -68,7 +69,7 @@ python scripts/release/bump_kernel_version.py 0.3.12
1. **Run the script:**
```bash
python scripts/release/bump_kernel_version.py 0.3.13
python scripts/release/bump_kernel_version.py 0.4.0
```
2. **Verify changes with git diff:**
@@ -78,8 +79,8 @@ python scripts/release/bump_kernel_version.py 0.3.12
3. **Check specific files contain the new version:**
```bash
grep -r "0.3.13" sgl-kernel/python/sgl_kernel/version.py
grep -r "0.3.13" sgl-kernel/pyproject.toml
grep -r "0.4.0" sgl-kernel/python/sgl_kernel/version.py
grep -r "0.4.0" sgl-kernel/pyproject.toml
```
4. **Reset changes (if testing):**
@@ -90,6 +91,6 @@ python scripts/release/bump_kernel_version.py 0.3.12
## Version Format Validation
- **SGLang versions:** `X.Y.Z` or `X.Y.ZrcN` (e.g., `0.5.3` or `0.5.3rc0`)
- **Kernel versions:** `X.Y.Z` (e.g., `0.3.12`)
- **Kernel versions:** `X.Y.Z` (e.g., `0.4.0`)
The scripts will validate the version format and exit with an error if invalid.

1
scripts/release/bump_kernel_version.py
View File

@@ -22,6 +22,7 @@ def main():
Path("sgl-kernel/pyproject.toml"),
Path("sgl-kernel/pyproject_cpu.toml"),
Path("sgl-kernel/pyproject_rocm.toml"),
Path("sgl-kernel/pyproject_musa.toml"),
Path("sgl-kernel/python/sgl_kernel/version.py"),
]

18
scripts/release/bump_kernel_version_to_sglang.py
View File

@@ -1,6 +1,6 @@
#!/usr/bin/env python3
"""
Bump sgl-kernel version in SGLang files to match the version in sgl-kernel/pyproject.toml.
Bump sglang-kernel version in SGLang files to match the version in sgl-kernel/pyproject.toml.
Updates:
- python/pyproject.toml
- python/sglang/srt/entrypoints/engine.py
@@ -37,7 +37,7 @@ def get_kernel_version_from_source() -> str:
def update_python_pyproject(new_version: str) -> bool:
"""Update sgl-kernel version in python/pyproject.toml"""
"""Update sglang-kernel version in python/pyproject.toml"""
pyproject_path = Path("python/pyproject.toml")
if not pyproject_path.exists():
@@ -46,10 +46,10 @@ def update_python_pyproject(new_version: str) -> bool:
content = pyproject_path.read_text()
# Replace "sgl-kernel==x.x.x" with new version
# Replace "sglang-kernel==x.x.x" with new version
new_content = re.sub(
r'"sgl-kernel==[^"]+"',
f'"sgl-kernel=={new_version}"',
r'"sglang-kernel==[^"]+"',
f'"sglang-kernel=={new_version}"',
content,
)
@@ -63,7 +63,7 @@ def update_python_pyproject(new_version: str) -> bool:
def update_engine_py(new_version: str) -> bool:
"""Update sgl-kernel version in python/sglang/srt/entrypoints/engine.py"""
"""Update sglang-kernel version in python/sglang/srt/entrypoints/engine.py"""
engine_path = Path("python/sglang/srt/entrypoints/engine.py")
if not engine_path.exists():
@@ -72,9 +72,9 @@ def update_engine_py(new_version: str) -> bool:
content = engine_path.read_text()
# Replace version in assert_pkg_version("sgl-kernel", "version", ...)
# Replace version in assert_pkg_version("sglang-kernel", "version", ...)
new_content = re.sub(
r'(assert_pkg_version\s*\(\s*"sgl-kernel"\s*,\s*)"[^"]+"',
r'(assert_pkg_version\s*\(\s*"sglang-kernel"\s*,\s*)"[^"]+"',
rf'\1"{new_version}"',
content,
)
@@ -117,7 +117,7 @@ def update_dockerfile(new_version: str) -> bool:
def main():
kernel_version = get_kernel_version_from_source()
print(f"Bumping sgl-kernel version to: {kernel_version}\n")
print(f"Bumping sglang-kernel version to: {kernel_version}\n")
updated_files = []

26
scripts/release/check_kernel_version_to_sglang.py
View File

@@ -1,6 +1,6 @@
#!/usr/bin/env python3
"""
Check if sgl-kernel version from sgl-kernel/pyproject.toml matches the versions
Check if sglang-kernel version from sgl-kernel/pyproject.toml matches the versions
used in SGLang files (python/pyproject.toml, engine.py, and Dockerfile).
Sets GitHub Actions output variables to indicate if sync is needed.
"""
@@ -36,7 +36,7 @@ def get_kernel_version_from_source() -> str:
def get_kernel_version_from_python_pyproject() -> str:
"""Extract sgl-kernel version from python/pyproject.toml"""
"""Extract sglang-kernel version from python/pyproject.toml"""
pyproject_path = Path("python/pyproject.toml")
if not pyproject_path.exists():
@@ -45,17 +45,17 @@ def get_kernel_version_from_python_pyproject() -> str:
content = pyproject_path.read_text()
# Match "sgl-kernel==x.x.x"
match = re.search(r'"sgl-kernel==([^"]+)"', content)
# Match "sglang-kernel==x.x.x"
match = re.search(r'"sglang-kernel==([^"]+)"', content)
if not match:
print("Error: Could not find sgl-kernel version in python/pyproject.toml")
print("Error: Could not find sglang-kernel version in python/pyproject.toml")
sys.exit(1)
return match.group(1)
def get_kernel_version_from_engine() -> str:
"""Extract sgl-kernel version from python/sglang/srt/entrypoints/engine.py"""
"""Extract sglang-kernel version from python/sglang/srt/entrypoints/engine.py"""
engine_path = Path("python/sglang/srt/entrypoints/engine.py")
if not engine_path.exists():
@@ -64,13 +64,13 @@ def get_kernel_version_from_engine() -> str:
content = engine_path.read_text()
# Find the assert_pkg_version call for sgl-kernel
# Look for the pattern: assert_pkg_version("sgl-kernel", "version", ...)
# Find the assert_pkg_version call for sglang-kernel
# Look for the pattern: assert_pkg_version("sglang-kernel", "version", ...)
match = re.search(
r'assert_pkg_version\s*\(\s*"sgl-kernel"\s*,\s*"([^"]+)"', content
r'assert_pkg_version\s*\(\s*"sglang-kernel"\s*,\s*"([^"]+)"', content
)
if not match:
print("Error: Could not find sgl-kernel version in engine.py")
print("Error: Could not find sglang-kernel version in engine.py")
sys.exit(1)
return match.group(1)
@@ -102,8 +102,10 @@ def main():
dockerfile_version = get_kernel_version_from_dockerfile()
print(f"Kernel version in sgl-kernel/pyproject.toml: {kernel_version}")
print(f"Kernel version in python/pyproject.toml: {pyproject_version}")
print(f"Kernel version in engine.py: {engine_version}")
print(
f"SGLang kernel dependency version in python/pyproject.toml: {pyproject_version}"
)
print(f"SGLang kernel dependency version in engine.py: {engine_version}")
print(f"Kernel version in Dockerfile: {dockerfile_version}")
# Check if any version differs from the source

10
scripts/release/commit_and_pr_kernel_to_sglang.sh
View File

@@ -25,9 +25,9 @@ COMMIT_FILES=$(git diff --name-only | sed 's/^/ - /')
# Commit changes
echo "Committing changes..."
git add -A
git commit -m "chore: bump sgl-kernel version to ${KERNEL_VERSION} in SGLang
git commit -m "chore: bump sglang-kernel version to ${KERNEL_VERSION} in SGLang
This commit updates the sgl-kernel version across SGLang files to match
This commit updates the sglang-kernel version across SGLang files to match
the version defined in sgl-kernel/pyproject.toml.
Files updated:
@@ -42,10 +42,10 @@ git push origin "${BRANCH_NAME}"
# Create pull request
echo "Creating pull request..."
PR_URL=$(gh pr create \
--title "chore: bump sgl-kernel version to ${KERNEL_VERSION}" \
--title "chore: bump sglang-kernel version to ${KERNEL_VERSION}" \
--body "## Summary
This PR bumps the \`sgl-kernel\` version to \`${KERNEL_VERSION}\` across SGLang files to match the version defined in \`sgl-kernel/pyproject.toml\`.
This PR bumps the \`sglang-kernel\` version to \`${KERNEL_VERSION}\` across SGLang files to match the version defined in \`sgl-kernel/pyproject.toml\`.
**Kernel Version:** \`${KERNEL_VERSION}\`
@@ -54,7 +54,7 @@ ${FILES_LIST}
## Context
The sgl-kernel version in \`sgl-kernel/pyproject.toml\` has been updated. This PR ensures that all SGLang files referencing the kernel version are updated accordingly:
The kernel version in \`sgl-kernel/pyproject.toml\` has been updated. This PR ensures that all SGLang files referencing the \`sglang-kernel\` dependency are updated accordingly:
- \`python/pyproject.toml\` - dependency specification
- \`python/sglang/srt/entrypoints/engine.py\` - version check
- \`docker/Dockerfile\` - Docker build argument

13
scripts/update_kernel_whl_index.py
View File

@@ -15,11 +15,11 @@ def check_wheel_cuda_version(path_name, target_cuda_version):
if re.search(f"rocm", path_name):
return False
# For other CUDA versions, the wheel path name will contain the cuda version suffix, e.g. sgl_kernel-0.3.16.post5+cu130-cp310-abi3-manylinux2014_x86_64.whl
# For other CUDA versions, the wheel path name will contain the cuda version suffix, e.g. sglang_kernel-0.4.0+cu130-cp310-abi3-manylinux2014_x86_64.whl
if target_cuda_version != DEFAULT_CUDA_VERSION:
return target_cuda_version in path_name
# For the default CUDA version, the wheel path name will not contain any cuda version suffix, e.g. sgl_kernel-0.3.16.post5-cp310-abi3-manylinux2014_x86_64.whl
# For the default CUDA version, the wheel path name will not contain any cuda version suffix, e.g. sglang_kernel-0.4.0-cp310-abi3-manylinux2014_x86_64.whl
# So we need to check if the wheel path name contains any other cuda version suffix
for cuda_version in SUPPORTED_CUDA_VERSIONS:
if cuda_version != DEFAULT_CUDA_VERSION and cuda_version in path_name:
@@ -28,7 +28,7 @@ def check_wheel_cuda_version(path_name, target_cuda_version):
def update_wheel_index(cuda_version=DEFAULT_CUDA_VERSION, rocm_version=None):
index_dir = pathlib.Path(f"sgl-whl/cu{cuda_version}/sgl-kernel")
index_dir = pathlib.Path(f"sgl-whl/cu{cuda_version}/sglang-kernel")
index_dir.mkdir(exist_ok=True, parents=True)
base_url = "https://github.com/sgl-project/whl/releases/download"
@@ -39,7 +39,7 @@ def update_wheel_index(cuda_version=DEFAULT_CUDA_VERSION, rocm_version=None):
with open(path, "rb") as f:
sha256 = hashlib.sha256(f.read()).hexdigest()
ver = re.findall(
r"sgl_kernel-([0-9.]+(?:\.post[0-9]+)?)(?:\+cu[0-9]+)?-", path.name
r"sglang_kernel-([0-9.]+(?:\.post[0-9]+)?)(?:\+cu[0-9]+)?-", path.name
)[0]
full_url = f"{base_url}/v{ver}/{path.name}#sha256={sha256}"
with (index_dir / "index.html").open("a") as f:
@@ -47,7 +47,7 @@ def update_wheel_index(cuda_version=DEFAULT_CUDA_VERSION, rocm_version=None):
def _update_non_cuda_wheel_index(backend, version):
index_dir = pathlib.Path(f"sgl-whl/{backend}{version}/sgl-kernel")
index_dir = pathlib.Path(f"sgl-whl/{backend}{version}/sglang-kernel")
index_dir.mkdir(exist_ok=True, parents=True)
base_url = "https://github.com/sgl-project/whl/releases/download"
@@ -58,7 +58,8 @@ def _update_non_cuda_wheel_index(backend, version):
with open(path, "rb") as f:
sha256 = hashlib.sha256(f.read()).hexdigest()
ver = re.findall(
rf"sgl_kernel-([0-9.]+(?:\.post[0-9]+)?)(?:\+{backend}[0-9]+)?-", path.name
rf"sglang_kernel-([0-9.]+(?:\.post[0-9]+)?)(?:\+{backend}[0-9]+)?-",
path.name,
)[0]
full_url = f"{base_url}/v{ver}/{path.name}#sha256={sha256}"
with (index_dir / "index.html").open("a") as f:

14
sgl-kernel/README.md
View File

@@ -1,22 +1,22 @@
# sgl-kernel
# sglang-kernel (prior sgl-kernel)
[Kernel Library](https://github.com/sgl-project/sglang/tree/main/sgl-kernel) for LLM inference engines
<div align="center">
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](https://github.com/sgl-project/sglang/blob/main/LICENSE)
[![PyPI](https://img.shields.io/pypi/v/sgl-kernel)](https://pypi.org/project/sgl-kernel)
[![PyPI](https://img.shields.io/pypi/v/sglang-kernel)](https://pypi.org/project/sglang-kernel)
</div>
sgl-kernel provides optimized compute primitives for LLM inference engines, enabling efficient inference for large language models and vision-language models through custom kernel operations. It has been used by [LightLLM](https://github.com/ModelTC/LightLLM), [SGLang](https://github.com/sgl-project/sglang) and so on.
`sglang-kernel` provides optimized compute primitives for LLM inference engines, enabling efficient inference for large language models and vision-language models through custom kernel operations. The source tree remains under the `sgl-kernel/` directory and the Python import path remains `sgl_kernel`.
## Installation
Requires torch == 2.9.1
```bash
# Latest version
pip3 install sgl-kernel --upgrade
pip3 install sglang-kernel --upgrade
```
## Building from Source
@@ -26,7 +26,7 @@ Requires
- scikit-build-core
- ninja(optional)
### Use Makefile to build sgl-kernel
### Use Makefile to build from the sgl-kernel source tree
```bash
make build
@@ -125,10 +125,10 @@ This tool requires `cubloaty` (install with `pip install cubloaty`) to work.
pip install cubloaty
# Analyze a wheel file
python analyze_whl_kernel_sizes.py path/to/sgl_kernel-*.whl
python analyze_whl_kernel_sizes.py path/to/sglang_kernel-*.whl
# Custom output file
python analyze_whl_kernel_sizes.py path/to/sgl_kernel-*.whl --output my_analysis.txt
python analyze_whl_kernel_sizes.py path/to/sglang_kernel-*.whl --output my_analysis.txt
```
The tool generates:

2
sgl-kernel/analyze_whl_kernel_sizes.py
View File

@@ -197,7 +197,7 @@ def generate_report(all_kernels, output_file):
def main():
parser = argparse.ArgumentParser(
description="Analyze CUDA kernel sizes in sgl-kernel whl file"
description="Analyze CUDA kernel sizes in sglang-kernel wheel files"
)
parser.add_argument("whl", type=str, help="Path to whl file")
parser.add_argument(

4
sgl-kernel/pyproject.toml
View File

@@ -7,8 +7,8 @@ requires = [
build-backend = "scikit_build_core.build"
[project]
name = "sgl-kernel"
version = "0.3.21"
name = "sglang-kernel"
version = "0.4.0"
authors = [
{ name="SGLang Kernel Team", email="sglang@lmsys.org" },
]

2
sgl-kernel/pyproject_cpu.toml
View File

@@ -8,7 +8,7 @@ build-backend = "scikit_build_core.build"
[project]
name = "sglang-kernel-cpu"
version = "0.3.21"
version = "0.4.0"
description = "Kernel Library for SGLang"
readme = "README.md"
requires-python = ">=3.10"

4
sgl-kernel/pyproject_musa.toml
View File

@@ -9,8 +9,8 @@ requires = [
build-backend = "setuptools.build_meta"
[project]
name = "sgl-kernel"
version = "0.3.20"
name = "sglang-kernel"
version = "0.4.0"
description = "Kernel Library for SGLang"
readme = "README.md"
requires-python = ">=3.10"

4
sgl-kernel/pyproject_rocm.toml
View File

@@ -8,8 +8,8 @@ requires = [
build-backend = "setuptools.build_meta"
[project]
name = "sgl-kernel"
version = "0.3.21"
name = "sglang-kernel"
version = "0.4.0"
description = "Kernel Library for SGLang"
readme = "README.md"
requires-python = ">=3.10"

4
sgl-kernel/python/sgl_kernel/load_utils.py
View File

@@ -169,10 +169,10 @@ def _load_architecture_specific_ops():
cuda_version = torch.version.cuda
if cuda_version and cuda_version.startswith("13"):
install_hint = (
"pip install sgl-kernel --index-url https://docs.sglang.ai/whl/cu130/"
"pip install sglang-kernel --index-url https://docs.sglang.ai/whl/cu130/"
)
else:
install_hint = "pip install --upgrade sgl_kernel"
install_hint = "pip install --upgrade sglang-kernel"
error_msg = f"""
[sgl_kernel] CRITICAL: Could not load any common_ops library!

2
sgl-kernel/python/sgl_kernel/version.py
View File

@@ -1 +1 @@
__version__ = "0.3.21"
__version__ = "0.4.0"

2
sgl-kernel/setup_musa.py
View File

@@ -197,7 +197,7 @@ class _CustomBuildExt(BuildExtension):
setup(
name="sgl-kernel",
name="sglang-kernel",
version=_get_version(),
packages=find_packages(where="python"),
package_dir={"": "python"},

2
sgl-kernel/setup_rocm.py
View File

@@ -118,7 +118,7 @@ ext_modules = [
]
setup(
name="sgl-kernel",
name="sglang-kernel",
version=_get_version(),
packages=find_packages(where="python"),
package_dir={"": "python"},