chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

[Spec][Ngram] 2/N: Rename branch length to max trie depth (#21181)

Browse Source 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
kpham-sgl
2026-03-22 23:35:25 -07:00
committed by GitHub
co-authored by Claude Opus 4.6
parent d6b12c401c
commit bc4aaab6a1
11 changed files with 31 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/advanced_features/server_arguments.md
+1 -1
View File
@@ -299,7 +299,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
| `--speculative-ngram-min-bfs-breadth` | The minimum breadth for BFS (Breadth-First Search) in ngram speculative decoding. | `1` | Type: int |
| `--speculative-ngram-max-bfs-breadth` | The maximum breadth for BFS (Breadth-First Search) in ngram speculative decoding. | `10` | Type: int |
| `--speculative-ngram-match-type` | Ngram tree-building mode. `BFS` selects recency-based expansion and `PROB` selects frequency-based expansion. This setting is forwarded to the ngram cache implementation. | `BFS` | `BFS`, `PROB` |
| `--speculative-ngram-branch-length` | The branch length for ngram speculative decoding. | `18` | Type: int |
| `--speculative-ngram-max-trie-depth` | The max trie depth for ngram speculative decoding. | `18` | Type: int |
| `--speculative-ngram-capacity` | The cache capacity for ngram speculative decoding. | `10000000` | Type: int |
## Multi-layer Eagle speculative decoding
docs/advanced_features/speculative_decoding.md
+2 -2
View File
@@ -393,7 +393,7 @@ Enable it with:
| `--speculative-ngram-min-bfs-breadth` | Minimum BFS breadth. | `1` |
| `--speculative-ngram-max-bfs-breadth` | Maximum BFS breadth. | `10` |
| `--speculative-ngram-match-type` | Ngram tree-building mode: `"BFS"` for recency-based expansion or `"PROB"` for frequency-based expansion. | `"BFS"` |
| `--speculative-ngram-branch-length` | How many recent tokens to insert into the cache. | `18` |
| `--speculative-ngram-max-trie-depth` | The max trie depth for ngram speculative decoding. | `18` |
| `--speculative-ngram-capacity` | Cache capacity (number of entries). | `10,000,000` |
Notes:
@@ -469,7 +469,7 @@ Below is a comprehensive list of all speculative decoding parameters available i
| `--speculative-ngram-min-bfs-breadth` | `int` | `1` | Minimum BFS breadth |
| `--speculative-ngram-max-bfs-breadth` | `int` | `10` | Maximum BFS breadth |
| `--speculative-ngram-match-type` | `str` | `"BFS"` | Ngram tree-building mode: `"BFS"` for recency-based expansion or `"PROB"` for frequency-based expansion |
| `--speculative-ngram-branch-length` | `int` | `18` | Recent tokens to insert into cache |
| `--speculative-ngram-max-trie-depth` | `int` | `18` | Max trie depth for ngram speculative decoding |
| `--speculative-ngram-capacity` | `int` | `10,000,000` | Cache capacity |
### Environment variables
docs/platforms/ascend_npu_support_features.md
+1 -1
View File
@@ -229,7 +229,7 @@ click [Server Arguments](https://docs.sglang.io/advanced_features/server_argumen
| `--speculative-ngram-`<br/>`min-bfs-breadth` | `1` | Type: int | Experimental |
| `--speculative-ngram-`<br/>`max-bfs-breadth` | `10` | Type: int | Experimental |
| `--speculative-ngram-`<br/>`match-type` | `BFS` | `BFS`,<br/> `PROB` | Experimental. `BFS` uses recency-based expansion; `PROB` uses frequency-based expansion. |
| `--speculative-ngram-`<br/>`branch-length` | `18` | Type: int | Experimental |
| `--speculative-ngram-`<br/>`max-trie-depth` | `18` | Type: int | Experimental |
| `--speculative-ngram-`<br/>`capacity` | `10000000` | Type: int | Experimental |
## Expert parallelism