[DLLM] Add JointThreshold algorithm for joint M2T and T2T decoding (#18171)
Signed-off-by: Junlin Zhou <zhoujunlin.zjl@antgroup.com> Co-authored-by: Tiwei Bie <tiwei.btw@antgroup.com>
This commit is contained in:
@@ -6,6 +6,8 @@ Diffusion language models have shown promise for non-autoregressive text generat
|
||||
|
||||
## Example Launch Command
|
||||
|
||||
SGLang supports different DLLM algorithms such as `LowConfidence` and `JointThreshold`.
|
||||
|
||||
```shell
|
||||
python3 -m sglang.launch_server \
|
||||
--model-path inclusionAI/LLaDA2.0-mini \ # example HF/local path
|
||||
@@ -17,6 +19,10 @@ python3 -m sglang.launch_server \
|
||||
|
||||
## Example Configuration File
|
||||
|
||||
Depending on the algorithm selected, the configuration parameters vary.
|
||||
|
||||
LowConfidence Config:
|
||||
|
||||
```yaml
|
||||
# Confidence threshold for accepting predicted tokens
|
||||
# - Higher values: More conservative, better quality but slower
|
||||
@@ -28,6 +34,25 @@ threshold: 0.95
|
||||
block_size: 32
|
||||
```
|
||||
|
||||
JointThreshold Config:
|
||||
|
||||
```yaml
|
||||
# Decoding threshold for Mask-to-Token (M2T) phase
|
||||
# - Higher values: More conservative, better quality but slower
|
||||
# - Lower values: More aggressive, faster but potentially lower quality
|
||||
# Range: 0.0 - 1.0
|
||||
threshold: 0.5
|
||||
# Decoding threshold for Token-to-Token (T2T) phase
|
||||
# Range: 0.0 - 1.0
|
||||
# Setting to 0.0 allows full editing (recommended for most cases).
|
||||
edit_threshold: 0.0
|
||||
# Max extra T2T steps after all masks are removed. Prevents infinite loops.
|
||||
max_post_edit_steps: 16
|
||||
# 2-gram repetition penalty (default 0).
|
||||
# An empirical value of 3 is often sufficient to mitigate most repetitions.
|
||||
penalty_lambda: 0
|
||||
```
|
||||
|
||||
## Example Client Code Snippet
|
||||
|
||||
Just like other supported models, diffusion language models can be used via the REST API or Python client.
|
||||
|
||||
Reference in New Issue
Block a user