chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Remove hybrid_kvcache_ratio in server args (#16399)

Browse Source
This commit is contained in:
Ke Bao
2026-01-06 13:13:13 +08:00
committed by GitHub
parent 7be1a8c70c
commit 3aa11ca722
5 changed files with 16 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/basic_usage/llama4.md
View File

@@ -32,7 +32,7 @@ python3 -m sglang.launch_server \
- **Chat Template**: Add `--chat-template llama-4` for chat completion tasks.
- **Enable Multi-Modal**: Add `--enable-multimodal` for multi-modal capabilities.
- **Enable Hybrid-KVCache**: Add `--hybrid-kvcache-ratio` for hybrid kv cache. Details can be seen in [this PR](https://github.com/sgl-project/sglang/pull/6563)
- **Enable Hybrid-KVCache**: Set `--swa-full-tokens-ratio` to adjust the ratio of SWA layer (for Llama4, it's local attention layer) KV tokens / full layer KV tokens. (default: 0.8, range: 0-1)
### EAGLE Speculative Decoding