5 people fd16c91cb8 Handle Marlin weight restoration and shape recording (QAT INT4 Rollout Part1) (#15238) ...

Co-authored-by: Gao016 <yngao016@163.com>
Co-authored-by: yefei12 <xjtu_yefeichen@163.com>
Co-authored-by: yzlnew <yzlnew@gmail.com>
Co-authored-by: Peng Zhang <aniz1905@gmail.com>

2026-01-07 21:11:24 +08:00

..

compressed_tensors

Handle Marlin weight restoration and shape recording (QAT INT4 Rollout Part1) (#15238)

2026-01-07 21:11:24 +08:00

configs

Refactor tuning block wise kernel and opt Qwen/Qwen3-VL-32B-Instruct-FP8 (#14141)

2025-12-08 09:24:58 +08:00

quark

Fix the accuracy issue when running mxfp4 dsv3 model and enable ep (#15304)

2025-12-17 01:53:22 -08:00

__init__.py

[NPU][1/N] NPU basic functions refactor and new modelslim quant type (#13359)

2025-12-04 16:15:31 +08:00

auto_round.py

Add support for AutoRound quantized models (#10153)

2025-10-27 18:17:29 +08:00

awq_triton.py

Add AWQ quantization support for NPU. (#10158)

2025-10-23 12:08:05 -07:00

awq.py

[AMD] suppress warning for amd (#16620)

2026-01-07 01:37:40 -08:00

base_config.py

fix qwenvl compressed tensors quantization weight loader (#11914)

2025-12-17 18:01:43 +08:00

blockwise_int8.py

[7/N] MoE Refactor: the implementation of new framework (#9269)

2025-09-05 21:09:09 -07:00

fp8_kernel.py

[JIT kernel] Apply jit per_tensor_quant_fp8 kernel (#15836)

2026-01-05 10:15:00 +08:00

fp8_utils.py

[FP8] Fix weight_scale shape to match with x_scale shape for per-tensor quant under torch.compile (#16356)

2026-01-05 13:55:26 -08:00

fp8.py

Refactor fp8 nextn layer for DeepSeek nvfp4 checkpoint (#15353)

2025-12-28 11:57:09 +08:00

fpgemm_fp8.py

[lint] improve ruff check (#11922)

2025-10-22 11:32:50 +08:00

gguf.py

[AMD] suppress warning for amd (#16620)

2026-01-07 01:37:40 -08:00

gptq.py

Support piecewise cuda graph for fused marlin moe (#15100)

2025-12-16 20:05:32 +08:00

int8_kernel.py

[Lint] Add python/sglang to ruff F401 checks and remove unused imports in files (#11685)

2025-10-17 16:49:46 -07:00

int8_utils.py

Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730)

2025-02-24 05:43:35 -08:00

kv_cache.py

Support FlashAttention3 page_size > 1 and topk > 1 case with paged attn and spec decode (#7725)

2025-11-26 11:44:41 +08:00

kvfp4_tensor.py

feat: Add FP4 (E2M1) KV Cache Support with Quantization Utilities for MLA (#10078)

2025-11-01 22:24:58 -07:00

marlin_utils_fp8.py

[BugFix] fix gptq_marlin_gemm has no parameter called b_bias (#13571)

2025-12-17 10:03:55 +08:00

marlin_utils.py

[Refactor] Clean up custom op (#15995)

2026-01-01 21:41:56 +08:00

modelopt_quant.py

[Refactor] Clean up custom op (#15995)

2026-01-01 21:41:56 +08:00

moe_wna16.py

[7/N] MoE Refactor: the implementation of new framework (#9269)

2025-09-05 21:09:09 -07:00

mxfp4_tensor.py

[sgl-kernel] misc: update deepgemm version for sgl-kernel (#9340)

2025-08-27 12:01:30 -07:00

mxfp4.py

[Refactor] Clean up custom op (#15995)

2026-01-01 21:41:56 +08:00

petit_utils.py

Support NVFP4 quantized dense models on AMD CDNA2/CDNA3 GPUs (#7302)

2025-07-18 19:59:39 -07:00

petit.py

[Lint] Add python/sglang to ruff F401 checks and remove unused imports in files (#11685)

2025-10-17 16:49:46 -07:00

qoq.py

Refactor: move all quantization-related code to srt/layer/quantization (#7989)

2025-07-17 00:47:07 -07:00

rocm_mxfp4_utils.py

Optimized deepseek-v3/r1 model performance on mxfp4 run (#10008)

2025-09-04 15:11:22 -07:00

unquant.py

[Refactor] Rename CustomOp -> MultiPlatformOp (#16175)

2025-12-31 01:16:32 +08:00

utils.py

Mistral Large 3 NVFP4 TRTLLM MoE support (#15049)

2025-12-18 11:11:42 +08:00

w4afp8.py

[perf]optimize w4afp8 kernel on deepseek-v3-0324 (#12921)

2025-12-18 18:13:22 +08:00

w8a8_fp8.py

[Fix] MoE: fix w8a8_fp8 MoE and add tests to cover this code path (#10429)

2025-09-14 17:34:28 -07:00

w8a8_int8.py

[Piecewise CUDA Graph] Support INT8 (#14918)

2025-12-17 18:20:57 +08:00