chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Add Speculative Decoding Eagle3 topk > 1 (#5318)

Browse Source 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
This commit is contained in:
Qingquan Song
2025-04-20 22:58:28 -07:00
committed by GitHub
parent eef9433b46
commit 188f0955fa
6 changed files with 872 additions and 167 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
python/sglang/srt/utils.py
View File

@@ -1909,6 +1909,8 @@ def is_page_size_one(server_args):
return server_args.page_size == 1
# TODO(hebiao064): Accelerate FA3 Spec Decode with topk > 1.
# TODO(hebiao064): Improve the acc rate for FA3 Spec Decode with topk == 1 and page_size > 1.
def is_no_spec_infer_or_topk_one(server_args):
return server_args.speculative_eagle_topk is None or (
server_args.speculative_eagle_topk is not None