chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[PP] Add a minimum chunk value for PP dynamic chunking (#16140)

Browse Source 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
This commit is contained in:
Shangming Cai
2025-12-30 16:35:39 +08:00
committed by GitHub
parent 8c5d91b884
commit db3821a9ef
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
python/sglang/srt/managers/scheduler_pp_mixin.py
View File

@@ -1343,7 +1343,8 @@ class ChunkSizePredictor:
smoothed_chunk_size = base_chunk_size + smooth_coeff * (
calculated_chunk_size_float - base_chunk_size
)
calculated_chunk_size = int(smoothed_chunk_size)
# Make sure the dynamic chunk size is at least 1/4 of the base chunk size
calculated_chunk_size = max(int(smoothed_chunk_size), base_chunk_size // 4)
# Align to page_size (minimum alignment size is 64)
alignment_size = max(page_size, 64)