model: support DeepSeek-OCR-2 (#17897)
This commit is contained in:
@@ -0,0 +1,54 @@
|
||||
# DeepSeek OCR (OCR-1 / OCR-2)
|
||||
|
||||
DeepSeek OCR models are multimodal (image + text) models for OCR and document understanding.
|
||||
|
||||
## Launch server
|
||||
|
||||
```shell
|
||||
python -m sglang.launch_server \
|
||||
--model-path deepseek-ai/DeepSeek-OCR-2 \
|
||||
--trust-remote-code \
|
||||
--host 0.0.0.0 \
|
||||
--port 30000
|
||||
```
|
||||
|
||||
> You can replace `deepseek-ai/DeepSeek-OCR-2` with `deepseek-ai/DeepSeek-OCR`.
|
||||
|
||||
## Prompt examples
|
||||
|
||||
Recommended prompts from the model card:
|
||||
|
||||
```
|
||||
<image>
|
||||
<|grounding|>Convert the document to markdown.
|
||||
```
|
||||
|
||||
```
|
||||
<image>
|
||||
Free OCR.
|
||||
```
|
||||
|
||||
## OpenAI-compatible request example
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
url = "http://localhost:30000/v1/chat/completions"
|
||||
|
||||
data = {
|
||||
"model": "deepseek-ai/DeepSeek-OCR-2",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "<image>\n<|grounding|>Convert the document to markdown."},
|
||||
{"type": "image_url", "image_url": {"url": "https://example.com/your_image.jpg"}},
|
||||
],
|
||||
}
|
||||
],
|
||||
"max_tokens": 512,
|
||||
}
|
||||
|
||||
response = requests.post(url, json=data)
|
||||
print(response.text)
|
||||
```
|
||||
@@ -33,6 +33,7 @@ Its core features include:
|
||||
basic_usage/ollama_api.md
|
||||
basic_usage/offline_engine_api.ipynb
|
||||
basic_usage/native_api.ipynb
|
||||
basic_usage/deepseek_ocr.md
|
||||
basic_usage/sampling_params.md
|
||||
basic_usage/popular_model_usage.rst
|
||||
|
||||
|
||||
@@ -30,6 +30,7 @@ in the GitHub search bar.
|
||||
|----------------------------|--------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------|
|
||||
| **Qwen-VL** | `Qwen/Qwen3-VL-235B-A22B-Instruct` | Alibaba's vision-language extension of Qwen; for example, Qwen2.5-VL (7B and larger variants) can analyze and converse about image content. | |
|
||||
| **DeepSeek-VL2** | `deepseek-ai/deepseek-vl2` | Vision-language variant of DeepSeek (with a dedicated image processor), enabling advanced multimodal reasoning on image and text inputs. | |
|
||||
| **DeepSeek-OCR / OCR-2** | `deepseek-ai/DeepSeek-OCR-2` | OCR-focused DeepSeek models for document understanding and text extraction. | Use `--trust-remote-code`. |
|
||||
| **Janus-Pro** (1B, 7B) | `deepseek-ai/Janus-Pro-7B` | DeepSeek's open-source multimodal model capable of both image understanding and generation. Janus-Pro employs a decoupled architecture for separate visual encoding paths, enhancing performance in both tasks. | |
|
||||
| **MiniCPM-V / MiniCPM-o** | `openbmb/MiniCPM-V-2_6` | MiniCPM-V (2.6, ~8B) supports image inputs, and MiniCPM-o adds audio/video; these multimodal LLMs are optimized for end-side deployment on mobile/edge devices. | |
|
||||
| **Llama 3.2 Vision** (11B) | `meta-llama/Llama-3.2-11B-Vision-Instruct` | Vision-enabled variant of Llama 3 (11B) that accepts image inputs for visual question answering and other multimodal tasks. | |
|
||||
|
||||
Reference in New Issue
Block a user