From 7c498a6538346d08e89b7c5bac6f539bced78ae7 Mon Sep 17 00:00:00 2001
From: Teng Ma <sima.mt@alibaba-inc.com>
Date: Mon, 16 Mar 2026 07:44:21 +0800
Subject: [PATCH] [DOC] add documents for encoder global mm cache (#20636)

---
 docs/advanced_features/epd_disaggregation.md | 75 ++++++++++++++++++++
 docs/advanced_features/server_arguments.md   |  1 +
 2 files changed, 76 insertions(+)

diff --git a/docs/advanced_features/epd_disaggregation.md b/docs/advanced_features/epd_disaggregation.md
index c543c29bc..d07898361 100644
--- a/docs/advanced_features/epd_disaggregation.md
+++ b/docs/advanced_features/epd_disaggregation.md
@@ -16,6 +16,81 @@ When launching a language-only model, you must additionally specify the encoder
 
 We support multiple encoder transfer backends, including zmq_to_scheduler, zmq_to_tokenizer, and mooncake (the default is zmq_to_scheduler). The backend can be selected using `--encoder-transfer-backend`.
 
+### Encoder transfer with Mooncake
+
+`--encoder-transfer-backend mooncake` controls **how encoder outputs are transferred** between encoder and language/prefill services. It is an encoder transfer option and can be used independently of the global multimodal embedding cache.
+
+Example:
+
+```bash
+# encoder
+python -m sglang.launch_server \
+  --model-path Qwen/Qwen3-VL-8B-Instruct \
+  --encoder-only \
+  --encoder-transfer-backend mooncake \
+  --port 30000
+
+# language-only server
+python -m sglang.launch_server \
+  --model-path Qwen/Qwen3-VL-8B-Instruct \
+  --language-only \
+  --encoder-urls http://127.0.0.1:30000 \
+  --encoder-transfer-backend mooncake \
+  --port 30002
+```
+
+### Global multimodal embedding cache with Mooncake
+
+SGLang also supports a Mooncake-backed **global multimodal embedding cache** for EPD workloads. When enabled on encoder servers, repeated image inputs can reuse previously computed ViT embeddings across instances instead of running the vision encoder again.
+
+This feature is useful when:
+
+- the deployment serves repeated or overlapping image inputs,
+- encoder compute is the bottleneck, and
+- Mooncake is already available in the cluster.
+
+At a high level, the encoder checks whether the image embedding already exists in Mooncake. Cache hits are prefetched from the global store, while misses are encoded normally and inserted into the cache in the background.
+
+To enable it:
+
+- install and configure Mooncake in the same way as other SGLang Mooncake integrations,
+- add `--enable-mm-global-cache` on the encoder server.
+
+`--enable-mm-global-cache` controls **whether multimodal embeddings are looked up and stored in the global Mooncake cache**. It is separate from `--encoder-transfer-backend`, which only controls encoder output transport.
+
+For Mooncake deployment and configuration details, see [HiCache best practices](hicache_best_practices.md#deployment-with-mooncake) and the [Mooncake backend README](../../python/sglang/srt/mem_cache/storage/mooncake_store/README.md).
+
+Example:
+
+```bash
+# Shared Mooncake configuration
+export MOONCAKE_TE_META_DATA_SERVER="http://127.0.0.1:8080/metadata"
+export MOONCAKE_MASTER="127.0.0.1:50051"
+export MOONCAKE_PROTOCOL="rdma"
+export MOONCAKE_GLOBAL_SEGMENT_SIZE="4gb"
+
+# encoder with global multimodal cache enabled
+python -m sglang.launch_server \
+  --model-path Qwen/Qwen3-VL-8B-Instruct \
+  --encoder-only \
+  --enable-mm-global-cache \
+  --port 30000
+
+# language-only server
+python -m sglang.launch_server \
+  --model-path Qwen/Qwen3-VL-8B-Instruct \
+  --language-only \
+  --encoder-urls http://127.0.0.1:30000 \
+  --port 30002
+```
+
+Notes:
+
+- This cache is for **multimodal encoder embeddings**, not the language model KV cache.
+- The feature currently uses Mooncake as the shared backing store.
+- It can be enabled regardless of which `--encoder-transfer-backend` you use.
+- It is most relevant for EPD or encoder-disaggregated VLM deployments where the same images are likely to appear across requests or instances.
+
 #### Qwen VL
 
 - EP Disaggregation
diff --git a/docs/advanced_features/server_arguments.md b/docs/advanced_features/server_arguments.md
index 5b35c9215..730a22745 100644
--- a/docs/advanced_features/server_arguments.md
+++ b/docs/advanced_features/server_arguments.md
@@ -538,6 +538,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
 | `--mm-process-config` | Multimodal preprocessing config, a json config contains keys: `image`, `video`, `audio`. | `{}` | Type: JSON / Dict |
 | `--mm-enable-dp-encoder` | Enabling data parallelism for mm encoder. The dp size will be set to the tp size automatically. | `False` | bool flag (set to enable) |
 | `--limit-mm-data-per-request` | Limit the number of multimodal inputs per request. e.g. '{"image": 1, "video": 1, "audio": 1}' | `None` | Type: JSON / Dict |
+| `--enable-mm-global-cache` | Enable Mooncake-backed global multimodal embedding cache on encoder servers so repeated images can reuse cached ViT embeddings instead of recomputing them. | `False` | bool flag (set to enable) |
 
 ## For checkpoint decryption
 | Argument | Description | Defaults | Options |