Harden OpenAI tool-call/chat-template + reasoning parsing

serving_chat:
- Tolerate a malformed historical tool_call `arguments` string (a valid JSON
  document followed by trailing content) instead of 400-ing the whole
  multi-turn request: salvage the leading JSON document via raw_decode, else
  keep the raw string. (A 112-message tool-history request was rejected with
  orjson "unexpected content after document".)
- Catch TypeError (not only jinja2.TemplateError) from the chat-template
  render so a `tojson` filter on a Jinja Undefined becomes a clean 400 instead
  of a 500 (upstream #20700 / 5e9bd21979).

reasoning_parser:
- Strip only LEADING think-start marker tokens; a global replace would delete a
  `<think>` token that legitimately appears inside reasoning content. Preserve
  model-generated whitespace in reasoning/normal text (drop .strip()/.rstrip())
  (upstream #24251 / dac78768f0).
- Add Glm45 detector tests: leading-only strip, token-inside-content preserved,
  repeated leading markers, streaming trailing whitespace.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-17 19:58:42 +00:00
co-authored by Claude Opus 4.8
parent e23168e7f5
commit 2212963a6c
3 changed files with 70 additions and 10 deletions
@@ -427,6 +427,40 @@ class TestGlm45Detector(CustomTestCase):
self.assertEqual(result.normal_text, text)
self.assertEqual(result.reasoning_text, "")
def test_detect_and_parse_preserves_whitespace(self):
"""Reasoning + normal text must keep model-generated whitespace (newlines,
indentation); the parser must not rewrite the payload."""
text = "<think>\n Let me think\n</think>\n\nThe answer is 42.\n"
result = self.detector.detect_and_parse(text)
self.assertEqual(result.reasoning_text, "\n Let me think\n")
self.assertEqual(result.normal_text, "\n\nThe answer is 42.\n")
def test_detect_and_parse_keeps_think_token_inside_content(self):
"""A think-start token appearing INSIDE the reasoning content is real
content, not a marker: only LEADING markers are stripped (a global
replace would corrupt the reasoning)."""
text = "<think>quote the <think> tag literally</think>answer"
result = self.detector.detect_and_parse(text)
self.assertEqual(result.reasoning_text, "quote the <think> tag literally")
self.assertEqual(result.normal_text, "answer")
def test_detect_and_parse_strips_repeated_leading_think_tokens(self):
"""Repeated leading think-start tokens are markers, not payload."""
text = "<think><think>Let me think</think>The answer is 42."
result = self.detector.detect_and_parse(text)
self.assertEqual(result.reasoning_text, "Let me think")
self.assertEqual(result.normal_text, "The answer is 42.")
def test_streaming_preserves_reasoning_trailing_whitespace(self):
"""Streaming reasoning text must preserve trailing whitespace before the
end token (no rstrip)."""
self.detector.parse_streaming_increment("<think>")
result = self.detector.parse_streaming_increment(
"reasoning \n</think>answer"
)
self.assertEqual(result.reasoning_text, "reasoning \n")
self.assertEqual(result.normal_text, "answer")
def test_streaming_normal_flow(self):
"""Test streaming with normal reasoning flow."""
# Start reasoning