Update MiniMax-M2 ToolCall and add MiniMax-M2.1 in Docs (#15538)
Co-authored-by: xuebi <xuebi@minimaxi.com> Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
This commit is contained in:
Roger Young
co-authored by
xuebi
Xinyuan Tong
xuebi
Xinyuan Tong
parent
cf81737693
commit
5c64a20da7
@@ -0,0 +1,66 @@
|
||||
# MiniMax M2.1/M2 Usage
|
||||
|
||||
[MiniMax-M2.1](https://huggingface.co/MiniMaxAI/MiniMax-M2.1) and [MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2) are advanced large language models created by [MiniMax](https://www.minimax.io/).
|
||||
|
||||
MiniMax-M2 series redefines efficiency for agents. It's a compact, fast, and cost-effective MoE model (230 billion total parameters with 10 billion active parameters) built for elite performance in coding and agentic tasks, all while maintaining powerful general intelligence. With just 10 billion activated parameters, MiniMax-M2 provides the sophisticated, end-to-end tool use performance expected from today's leading models, but in a streamlined form factor that makes deployment and scaling easier than ever.
|
||||
|
||||
## Supported Models
|
||||
|
||||
This guide applies to the following models. You only need to update the model name during deployment. The following examples use **MiniMax-M2**:
|
||||
|
||||
- [MiniMaxAI/MiniMax-M2.1](https://huggingface.co/MiniMaxAI/MiniMax-M2.1)
|
||||
- [MiniMaxAI/MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2)
|
||||
|
||||
## System Requirements
|
||||
|
||||
The following are recommended configurations; actual requirements should be adjusted based on your use case:
|
||||
|
||||
- 4x 96GB GPUs: Supported context length of up to 400K tokens.
|
||||
- 8x 144GB GPUs: Supported context length of up to 3M tokens.
|
||||
|
||||
## Deployment with Python
|
||||
|
||||
4-GPU deployment command:
|
||||
|
||||
```bash
|
||||
python -m sglang.launch_server \
|
||||
--model-path MiniMaxAI/MiniMax-M2 \
|
||||
--tp-size 4 \
|
||||
--tool-call-parser minimax-m2 \
|
||||
--reasoning-parser minimax-append-think \
|
||||
--host 0.0.0.0 \
|
||||
--trust-remote-code \
|
||||
--port 8000 \
|
||||
--mem-fraction-static 0.85
|
||||
```
|
||||
|
||||
8-GPU deployment command:
|
||||
|
||||
```bash
|
||||
python -m sglang.launch_server \
|
||||
--model-path MiniMaxAI/MiniMax-M2 \
|
||||
--tp-size 8 \
|
||||
--ep-size 8 \
|
||||
--tool-call-parser minimax-m2 \
|
||||
--reasoning-parser minimax-append-think \
|
||||
--host 0.0.0.0 \
|
||||
--trust-remote-code \
|
||||
--port 8000 \
|
||||
--mem-fraction-static 0.85
|
||||
```
|
||||
|
||||
## Testing Deployment
|
||||
|
||||
After startup, you can test the SGLang OpenAI-compatible API with the following command:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "MiniMaxAI/MiniMax-M2",
|
||||
"messages": [
|
||||
{"role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}]},
|
||||
{"role": "user", "content": [{"type": "text", "text": "Who won the world series in 2020?"}]}
|
||||
]
|
||||
}'
|
||||
```
|
||||
@@ -1,4 +1,4 @@
|
||||
Popular Model Usage (DeepSeek, GPT-OSS, GLM, Llama, Qwen, and more)
|
||||
Popular Model Usage (DeepSeek, GPT-OSS, GLM, Llama, MiniMax, Qwen, and more)
|
||||
===============================================================
|
||||
|
||||
.. toctree::
|
||||
@@ -9,6 +9,7 @@ Popular Model Usage (DeepSeek, GPT-OSS, GLM, Llama, Qwen, and more)
|
||||
glm45.md
|
||||
glmv.md
|
||||
gpt_oss.md
|
||||
minimax_m2.md
|
||||
qwen3.md
|
||||
qwen3_vl.md
|
||||
llama4.md
|
||||
|
||||
@@ -35,7 +35,7 @@ in the GitHub search bar.
|
||||
| **MiniCPM** (v3, 4B) | `openbmb/MiniCPM3-4B` | OpenBMB’s series of compact LLMs for edge devices; MiniCPM 3 (4B) achieves GPT-3.5-level results in text tasks. |
|
||||
| **OLMo** (2, 3) | `allenai/OLMo-2-1124-7B-Instruct` | Allen AI’s series of Open Language Models designed to enable the science of language models. |
|
||||
| **OLMoE** (Open MoE) | `allenai/OLMoE-1B-7B-0924` | Allen AI’s open Mixture-of-Experts model (7B total, 1B active parameters) delivering state-of-the-art results with sparse expert activation. |
|
||||
| **MiniMax-M2** | `minimax/MiniMax-M2` | MiniMax’s SOTA LLM for coding & agentic workflows. |
|
||||
| **MiniMax-M2** (M2, M2.1) | `minimax/MiniMax-M2`, `minimax/MiniMax-M2.1` | MiniMax’s SOTA LLM for coding & agentic workflows. |
|
||||
| **StableLM** (3B, 7B) | `stabilityai/stablelm-tuned-alpha-7b` | StabilityAI’s early open-source LLM (3B & 7B) for general text generation; a demonstration model with basic instruction-following ability. |
|
||||
| **Command-R** (Cohere) | `CohereForAI/c4ai-command-r-v01` | Cohere’s open conversational LLM (Command series) optimized for long context, retrieval-augmented generation, and tool use. |
|
||||
| **DBRX** (Databricks) | `databricks/dbrx-instruct` | Databricks’ 132B-parameter MoE model (36B active) trained on 12T tokens; competes with GPT-3.5 quality as a fully open foundation model. |
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
import ast
|
||||
import html
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
@@ -16,17 +14,6 @@ from sglang.srt.function_call.core_types import (
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _safe_val(raw: str) -> Any:
|
||||
raw = html.unescape(raw.strip())
|
||||
try:
|
||||
return json.loads(raw)
|
||||
except Exception:
|
||||
try:
|
||||
return ast.literal_eval(raw)
|
||||
except Exception:
|
||||
return raw
|
||||
|
||||
|
||||
class MinimaxM2Detector(BaseFormatDetector):
|
||||
"""
|
||||
Detector for MiniMax M2 models.
|
||||
@@ -73,6 +60,169 @@ class MinimaxM2Detector(BaseFormatDetector):
|
||||
normal, calls = self._extract(text, tools)
|
||||
return StreamingParseResult(normal_text=normal, calls=calls)
|
||||
|
||||
def _convert_param_value(self, value: str, param_type: str) -> Any:
|
||||
"""Convert parameter value to the correct type (legacy single-type version)."""
|
||||
return self._convert_param_value_with_types(value, [param_type])
|
||||
|
||||
def _extract_types_from_schema(self, schema: Any) -> list[str]:
|
||||
"""
|
||||
Extract all possible types from a JSON schema definition.
|
||||
Handles anyOf, oneOf, allOf, type arrays, and enum fields.
|
||||
|
||||
Args:
|
||||
schema: The JSON schema definition for a parameter
|
||||
|
||||
Returns:
|
||||
List of type strings (e.g., ["string", "integer", "null"])
|
||||
"""
|
||||
if schema is None:
|
||||
return ["string"]
|
||||
|
||||
if not isinstance(schema, dict):
|
||||
return ["string"]
|
||||
|
||||
types: set[str] = set()
|
||||
|
||||
# Handle direct "type" field
|
||||
if "type" in schema:
|
||||
type_value = schema["type"]
|
||||
if isinstance(type_value, str):
|
||||
types.add(type_value)
|
||||
elif isinstance(type_value, list):
|
||||
for t in type_value:
|
||||
if isinstance(t, str):
|
||||
types.add(t)
|
||||
|
||||
# Handle enum - infer types from enum values
|
||||
if "enum" in schema and isinstance(schema["enum"], list) and schema["enum"]:
|
||||
for value in schema["enum"]:
|
||||
if value is None:
|
||||
types.add("null")
|
||||
elif isinstance(value, bool):
|
||||
types.add("boolean")
|
||||
elif isinstance(value, int):
|
||||
types.add("integer")
|
||||
elif isinstance(value, float):
|
||||
types.add("number")
|
||||
elif isinstance(value, str):
|
||||
types.add("string")
|
||||
elif isinstance(value, list):
|
||||
types.add("array")
|
||||
elif isinstance(value, dict):
|
||||
types.add("object")
|
||||
|
||||
# Handle anyOf, oneOf, allOf - recursively extract types
|
||||
for choice_field in ("anyOf", "oneOf", "allOf"):
|
||||
if choice_field in schema and isinstance(schema[choice_field], list):
|
||||
for choice in schema[choice_field]:
|
||||
extracted = self._extract_types_from_schema(choice)
|
||||
types.update(extracted)
|
||||
|
||||
# If no types found, default to string
|
||||
if not types:
|
||||
return ["string"]
|
||||
|
||||
return list(types)
|
||||
|
||||
def _convert_param_value_with_types(
|
||||
self, value: str, param_types: list[str]
|
||||
) -> Any:
|
||||
"""
|
||||
Convert parameter value to the correct type based on a list of possible types.
|
||||
Tries each type in order until one succeeds.
|
||||
|
||||
Args:
|
||||
value: The string value to convert
|
||||
param_types: List of possible type strings
|
||||
|
||||
Returns:
|
||||
The converted value
|
||||
"""
|
||||
if value.lower() == "null":
|
||||
return None
|
||||
|
||||
# Normalize types
|
||||
normalized_types = [t.lower() for t in param_types]
|
||||
|
||||
# Try null first if it's in the list
|
||||
if "null" in normalized_types or value.lower() in ("null", "none", "nil"):
|
||||
return None
|
||||
|
||||
# Try each type in order of preference (most specific first, string as fallback)
|
||||
# Priority: integer > number > boolean > object > array > string
|
||||
type_priority = [
|
||||
"integer",
|
||||
"int",
|
||||
"number",
|
||||
"float",
|
||||
"boolean",
|
||||
"bool",
|
||||
"object",
|
||||
"array",
|
||||
"string",
|
||||
"str",
|
||||
"text",
|
||||
]
|
||||
|
||||
for param_type in type_priority:
|
||||
if param_type not in normalized_types:
|
||||
continue
|
||||
|
||||
if param_type in ["string", "str", "text"]:
|
||||
return value
|
||||
elif param_type in ["integer", "int"]:
|
||||
try:
|
||||
return int(value)
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
elif param_type in ["number", "float"]:
|
||||
try:
|
||||
val = float(value)
|
||||
return val if val != int(val) else int(val)
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
elif param_type in ["boolean", "bool"]:
|
||||
lower_val = value.lower().strip()
|
||||
if lower_val in ["true", "1", "yes", "on"]:
|
||||
return True
|
||||
elif lower_val in ["false", "0", "no", "off"]:
|
||||
return False
|
||||
continue
|
||||
elif param_type in ["object", "array"]:
|
||||
try:
|
||||
return json.loads(value)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
|
||||
# Fallback: try JSON parse, then return as string
|
||||
try:
|
||||
return json.loads(value)
|
||||
except json.JSONDecodeError:
|
||||
return value
|
||||
|
||||
def _get_param_types_from_config(
|
||||
self, param_name: str, param_config: dict
|
||||
) -> list[str]:
|
||||
"""
|
||||
Get parameter types from parameter configuration.
|
||||
Handles anyOf, oneOf, allOf, and direct type definitions.
|
||||
|
||||
Args:
|
||||
param_name: The name of the parameter
|
||||
param_config: The properties dict from the tool schema
|
||||
|
||||
Returns:
|
||||
List of type strings
|
||||
"""
|
||||
if param_name not in param_config:
|
||||
return ["string"]
|
||||
|
||||
param_schema = param_config[param_name]
|
||||
if not isinstance(param_schema, dict):
|
||||
return ["string"]
|
||||
|
||||
return self._extract_types_from_schema(param_schema)
|
||||
|
||||
def parse_streaming_increment(
|
||||
self, new_text: str, tools: List[Tool]
|
||||
) -> StreamingParseResult:
|
||||
@@ -166,7 +316,7 @@ class MinimaxM2Detector(BaseFormatDetector):
|
||||
# Parse parameters incrementally
|
||||
if self._function_name_sent:
|
||||
# Process parameters and get any calls to emit
|
||||
parameter_calls = self._parse_and_stream_parameters(self._buf)
|
||||
parameter_calls = self._parse_and_stream_parameters(self._buf, tools)
|
||||
calls.extend(parameter_calls)
|
||||
|
||||
# Check if tool call is complete
|
||||
@@ -204,7 +354,9 @@ class MinimaxM2Detector(BaseFormatDetector):
|
||||
|
||||
return StreamingParseResult(normal_text=normal, calls=calls)
|
||||
|
||||
def _parse_and_stream_parameters(self, text_to_parse: str) -> List[ToolCallItem]:
|
||||
def _parse_and_stream_parameters(
|
||||
self, text_to_parse: str, tools: List[Tool]
|
||||
) -> List[ToolCallItem]:
|
||||
"""
|
||||
Parse complete parameter blocks from text and return any tool call items to emit.
|
||||
|
||||
@@ -236,7 +388,9 @@ class MinimaxM2Detector(BaseFormatDetector):
|
||||
for match in param_matches:
|
||||
param_name = match.group(1).strip()
|
||||
param_value = match.group(2)
|
||||
new_params[param_name] = _safe_val(param_value)
|
||||
new_params[param_name] = self._parse_parameter(
|
||||
self._current_function_name, param_name, param_value, tools
|
||||
)
|
||||
|
||||
# Calculate parameter diff to stream with proper incremental JSON building
|
||||
if new_params != self._current_parameters:
|
||||
@@ -337,7 +491,7 @@ class MinimaxM2Detector(BaseFormatDetector):
|
||||
pidx = ptxt.index('">')
|
||||
pname = ptxt[:pidx].strip()
|
||||
pval = ptxt[pidx + 2 :].lstrip("\n").rstrip("\n")
|
||||
params[pname] = _safe_val(pval)
|
||||
params[pname] = self._parse_parameter(fname, pname, pval, tools)
|
||||
raw = {"name": fname, "arguments": params}
|
||||
try:
|
||||
# TODO: fix idx in function call, the index for a function
|
||||
@@ -347,6 +501,20 @@ class MinimaxM2Detector(BaseFormatDetector):
|
||||
logger.warning("invalid tool call for %s dropped", fname)
|
||||
return res
|
||||
|
||||
def _parse_parameter(
|
||||
self, fname: str, pname: str, pval: str, tools: List[Tool]
|
||||
) -> Any:
|
||||
param_config = {}
|
||||
for tool in tools:
|
||||
if tool.function.name == fname and tool.function.parameters is not None:
|
||||
parameters = tool.function.parameters
|
||||
if isinstance(parameters, dict) and "properties" in parameters:
|
||||
param_config = parameters["properties"]
|
||||
break
|
||||
|
||||
param_type = self._get_param_types_from_config(pname, param_config)
|
||||
return self._convert_param_value_with_types(pval, param_type)
|
||||
|
||||
def supports_structural_tag(self) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
Reference in New Issue
Block a user