chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

[quant] fix fp32 downcasting (#19844)

Browse Source 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
This commit is contained in:
StonyPort
2026-03-05 17:54:59 +08:00
committed by GitHub
parent 472eef4071
commit 806d41ab65
3 changed files with 4 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/references/environment_variables.md
View File

@@ -122,6 +122,7 @@ SGLang supports various environment variables that can be used to configure its
| `SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE` | Quantize moe of nextn layer from BF16 to FP8 when launching DeepSeek NVFP4 checkpoint | `false` |
| `SGLANG_ENABLE_FLASHINFER_FP8_GEMM` (deprecated) | Use flashinfer kernels when running blockwise fp8 GEMM on Blackwell GPUs. **DEPRECATED**: Please use `--fp8-gemm-backend=flashinfer_trtllm` (SM100/SM103) or `--fp8-gemm-backend=flashinfer_cutlass` (SM120/SM121 and newer) instead. | `false` |
| `SGLANG_SUPPORT_CUTLASS_BLOCK_FP8` (deprecated) | Use Cutlass kernels when running blockwise fp8 GEMM on Hopper or Blackwell GPUs. **DEPRECATED**: Please use `--fp8-gemm-backend=cutlass` instead. | `false` |
| `SGLANG_QUANT_ALLOW_DOWNCASTING` | Allow weights downcasting | `false` |
## Distributed Computing