chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Refactor KTransformers heterogeneous compute with unified GPU-quantization backend (#12834)

Browse Source 
Co-authored-by: Chen Hongtao <56470055+chenht2022@users.noreply.github.com>
Co-authored-by: chenht2022 <cht22@mails.tsinghua.edu.cn>
Co-authored-by: skqliao <skqliao@gmail.com>
Co-authored-by: ovowei <1913953267@qq.com>
This commit is contained in:
Atream
2025-11-10 13:06:32 +08:00
committed by GitHub
parent d1be60c3c5
commit ddd1440d0f
10 changed files with 494 additions and 507 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
python/sglang/srt/environ.py
View File

@@ -266,16 +266,6 @@ class Envs:
# Release & Resume Memory
SGLANG_MEMORY_SAVER_CUDA_GRAPH = EnvBool(False)
# Ktransformers
SGLANG_KT_MOE_NUM_GPU_EXPERTS = EnvInt(None)
SGLANG_KT_MOE_CPUINFER = EnvInt(None)
SGLANG_KT_THREADPOOL_COUNT = EnvInt(None)
SGLANG_KT_MOE_AMX_WEIGHT_PATH = EnvStr(None)
SGLANG_KT_AMX_METHOD = EnvStr(None)
SGLANG_KT_MOE_CHUNKED_PREFILL_SIZE = EnvInt(None)
SGLANG_KT_MOE_MAX_DEFERRED_EXPERTS_PER_TOKEN = EnvInt(None)
SGLANG_KT_MOE_TOTAL_LAYERS = EnvInt(None)
# Sparse Embeddings
SGLANG_EMBEDDINGS_SPARSE_HEAD = EnvStr(None)