Commit Graph
39 Commits
Author SHA1 Message Date
laoyao0822 a27114d9dc Prevent spec-v2 decode warmup races
Port the fix-decode spec-v2 ownership and plan-stream fixes onto the current branch. Draft extend now keeps the scheduler-owned batch lengths committed until acceptance, binds the draft runner to the draft-extend attention backend, keeps speculative KV allocation monotonic, and lets non-graph target verify initialize metadata after DP padding. The worker also records rebound tensors on the forward stream and orders plan-stream metadata work after current-stream inputs are available.

Constraint: fix-decode commits cd8e47ed9c and 60e3956d9c address CUDA illegal-address failures in spec-v2 decode warmup paths.

Rejected: Cherry-pick the commits blindly | the current branch has intervening decode changes, so a minimal manual port kept the patch surface to the affected files.

Confidence: medium

Scope-risk: moderate

Directive: Do not move target-verify non-graph metadata initialization back into prepare_for_v2_verify without validating DP padding and NSA metadata ordering.

Tested: RED local pytest test/registered/spec/eagle/test_eagle_v2_draft_extend_contract.py -q failed 8/8 before production changes.

Tested: PYTHONPATH=python python3 -m pytest test/registered/spec/eagle/test_eagle_v2_draft_extend_contract.py -q passed 8/8 locally.

Tested: PYTHONPATH=python python3 -m py_compile python/sglang/srt/speculative/eagle_info_v2.py python/sglang/srt/speculative/eagle_worker_v2.py python/sglang/srt/speculative/spec_utils.py passed locally.

Tested: Remote g0034:cjy-glm5-new pytest for test/registered/spec/eagle/test_eagle_v2_draft_extend_contract.py passed 8/8, plus remote py_compile for the three speculative modules.

Tested: Local and remote sha256 sums matched for all four synced files.

Not-tested: Full spec-v2 decode server restart/warmup under production traffic.
2026-06-29 03:16:14 +08:00
Lianmin Zheng 814202704b ci: unify PR test suite naming (#21187) 2026-03-23 00:18:45 -07:00
kpham-sglandClaude Opus 4.6 bc4aaab6a1 [Spec][Ngram] 2/N: Rename branch length to max trie depth (#21181)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 23:35:25 -07:00
kpham-sgl 6d160b42bb [Spec][Ngram] 1/N: Reference based Speculative Decoding refactor (#20393) 2026-03-22 00:55:10 -07:00
Pai Liu 65dd08153d Fix Test* mixin classes being collected as standalone pytest tests (#20417) 2026-03-12 18:18:45 -07:00
Alison ShaoandAlison Shao 7b44bc923e Relax flaky B200 GSM8K accuracy thresholds (#20304)
Co-authored-by: Alison Shao <alisonshao@Mac.attlocal.net>
2026-03-11 12:35:21 -07:00
Alison ShaoandAlison Shao f32882cee5 [CI] Relax Eagle infer_b spec accept length threshold (#20300)
Co-authored-by: Alison Shao <alisonshao@Mac.attlocal.net>
2026-03-10 15:40:52 -07:00
09a118fafe Support return_logprob for spec v2 (overlap safe) (#19801)
Co-authored-by: Ratish1 <ratish1501@gmail.com>
Co-authored-by: Ratish1 <formula733@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-03-10 15:38:27 -07:00
kpham-sglandhnyls2002 346a4131cf [Spec] Refactor NaN/OOB checks to async maybe_detect_* with env-var control (#19899)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-03-05 13:51:05 -08:00
Hubert Lu 441045a7bf [AMD] Fix EAGLE3 speculative decoding with aiter attention backend (#19362) 2026-03-03 16:12:13 -08:00
Liangsheng Yin dcea74d63f Add timeout abort kits for normal / eagle. (#18815) 2026-02-13 17:57:30 -08:00
Alison Shao bedade1ef0 Merge stage-c-test-large-4-gpu suites into partitioned suites (#18325) 2026-02-06 15:32:33 -08:00
DiweiSunjunduGao, Pengfei <pengfei.gao@intel.com>
495290aefd enable ut test for xpu devices (#11712)
Co-authored-by: jundu <jun.du@intel.com>
Co-authored-by: Gao, Pengfei <pengfei.gao@intel.com>
2026-02-03 11:15:14 -08:00
Alison Shao 146b5fcc84 [CI] Reorganize stage-b 1-GPU tests for 5090 compatibility (#16826) 2026-01-15 15:23:35 -08:00
Bingxu Chenandyctseng0211 98096b5e02 [AMD CI] migrate and re-enable CI tests to new CI registry (#16949)
Co-authored-by: yctseng0211 <yctseng@amd.com>
2026-01-14 21:25:25 -08:00
Alison Shao b880607108 Add 5090 dry run stage to PR test workflow (#17022) 2026-01-13 14:12:33 -08:00
Alison Shao cc25f9df50 Update est_time for stage-b-test-small-1-gpu tests (#16835) 2026-01-11 12:03:43 +08:00
YAMY 959343795f [test] update acc len threshold to 2.7 for eagle dp attention tests (#16463) 2026-01-06 17:38:35 +08:00
Liangsheng Yin ba67e006a7 Refactor speculative algorithm registry. (#16168) 2025-12-31 01:24:22 +08:00
Gaoji Liuandliugaoji.lgj 7518dc3532 feat(SpecEagleV2): add standalone_worker_v2 (#12625)
Co-authored-by: liugaoji.lgj <liugaoji.lgj@alibaba-inc.com>
2025-12-30 17:55:04 +08:00
Qiaolin Yu 6a3e709253 Tiny rename test_deepseek_v3_fp4_mtp_stage_b.py (#16141) 2025-12-30 15:55:07 +08:00
Alison Shao f4ec6f8e17 ci: migrate remaining spec/eagle tests to test/registered/spec/ (#15800) 2025-12-29 14:00:00 -08:00
Liangsheng Yin c7e7bfa32c Add EAGLE3 test with MMLU dataset. (#15945) 2025-12-28 16:07:58 +08:00
Liangsheng Yin bf90ea9c5b Unify spec v2's naming manner. (#15990) 2025-12-28 14:14:52 +08:00
Liangsheng Yin 9ad546d7e8 Tiny cleanup the models' name in test_utils (#15920) 2025-12-27 14:13:23 +08:00
Liangsheng Yin f4e835af2f Cleanup ModelRunner (#15802) 2025-12-25 18:13:30 +08:00
Liangsheng Yin 6292c24437 Tiny fix test eagle infer b. (#15716) 2025-12-24 11:57:15 +08:00
Qiaolin Yu aef7ca7cf2 Raise the accept length bar in dpsk-r1-fp4 spec decoding tests (#15705) 2025-12-23 18:38:02 -08:00
Liangsheng Yin bd572360f3 Tiny apply gsm8k mixin to ngram test (#15606) 2025-12-24 01:30:26 +08:00
Yubo WangandLiangsheng Yin 762846531f Fix Illegal Memory Access when fa3 + spec + topk + page_size > 1 (#15469)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
2025-12-24 00:13:57 +08:00
Alison Shao 883747ced1 [CI] Migrate Attention Backend tests to test/registered/attention/ (#15563) 2025-12-22 22:17:52 -08:00
Alison Shao 989d4b3012 [CI] Migrate nightly tests to test/registered/ (#15582) 2025-12-22 22:16:32 -08:00
Liangsheng Yin ecb401ed42 Enhance runtime memory check in CI (#15192) 2025-12-16 21:40:38 +08:00
Ziming Huang 997ea57eaf Fix tensor mismatch error in sepc + topk > 1 + page_size > 1 (#14874) 2025-12-15 01:10:58 +08:00
Liangsheng Yin ed52d01b0b Fix spec info's filter when reqs are finished right after prefill (#14742) 2025-12-14 00:32:54 +08:00
Liangsheng Yin d977dd2e07 Fix IMA with flashinfer + spec + topk & Add radix attention test cases for eagle (#13740) 2025-12-13 23:04:41 +08:00
Liangsheng Yin 01e3b3f3a3 Fix decode OOM caused by retraction (#14939) 2025-12-13 12:59:17 +08:00
Liangsheng Yin c660d8dfd0 Re-org eagle unit tests (#14909) 2025-12-12 12:25:39 +09:00
Alison Shao e6f0ddda44 [CI] Migrate Eagle 1-GPU tests to test/registered/ (#14529) 2025-12-09 12:56:36 +09:00