diff --git a/docs/advanced_features/lora.ipynb b/docs/advanced_features/lora.ipynb index 67bfdac32..c01e9fbfb 100644 --- a/docs/advanced_features/lora.ipynb +++ b/docs/advanced_features/lora.ipynb @@ -534,11 +534,10 @@ "\n", "SGLang supports two LoRA backends that you can choose from using the `--lora-backend` argument:\n", "\n", - "- `triton`: Default basic Triton-based backend.\n", - "- `csgmv`: Chunked SGMV backend optimized for high concurrency scenarios.\n", + "- `triton`: Basic Triton-based backend.\n", + "- `csgmv`: Default chunked SGMV backend optimized for high concurrency scenarios.\n", "\n", - "The `csgmv` backend was recently introduced to improve performance especially at high-concurrency scenarios. Our benchmark shows that it achieves 20% to 80% latency improvements over the basic triton backend.\n", - "Currently it is at preview phase, we expect to make it our the default LoRA backend in future release. Before that, you can adopt it by manually setting the `--lora-backend` server config." + "The `csgmv` backend was recently introduced to improve performance especially at high-concurrency scenarios. Our benchmark shows that it achieves 20% to 80% latency improvements over the basic triton backend." ] }, { diff --git a/docs/advanced_features/server_arguments.md b/docs/advanced_features/server_arguments.md index e554c0286..c2914f789 100644 --- a/docs/advanced_features/server_arguments.md +++ b/docs/advanced_features/server_arguments.md @@ -231,7 +231,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s | `--max-loras-per-batch` | Maximum number of adapters for a running batch, including base-only requests. | `8` | Type: int | | `--max-loaded-loras` | If specified, limits the maximum number of LoRA adapters loaded in CPU memory at a time. Must be ≥ `--max-loras-per-batch`. | `None` | Type: int | | `--lora-eviction-policy` | LoRA adapter eviction policy when the GPU memory pool is full. | `lru` | `lru`, `fifo` | -| `--lora-backend` | Choose the kernel backend for multi-LoRA serving. | `triton` | `triton`, `csgmv` | +| `--lora-backend` | Choose the kernel backend for multi-LoRA serving. | `csgmv` | `triton`, `csgmv` | | `--max-lora-chunk-size` | Maximum chunk size for the ChunkedSGMV LoRA backend. Only used when `--lora-backend` is `csgmv`. Larger values may improve performance. | `16` | `16`, `32`, `64`, `128` | ## Kernel Backends (Attention, Sampling, Grammar, GEMM)