3 people b1246c50f8 Fix chunked prefill and KV cache leaks for streaming sessions (#20476) ...

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>

2026-03-13 02:36:55 -07:00

..

4-gpu-models

[Test] Refactor KL divergence and prefix cache branching to kits (#19715)

2026-03-12 16:11:59 +08:00

8-gpu-models

Fix FA3 swa spec pg_size > 1 (#20369)

2026-03-12 11:42:01 +08:00

amd

[AMD] Fix FP8 assertion failure in aiter MLA decode by falling back to self.k_scale (#19935)

2026-03-12 01:48:51 -07:00

ascend

[NPU] [Quantization] w4a4 MoE layer support (#18924)

2026-03-11 16:52:35 +03:00

attention

[GDN] Change Attention State Layout from [N, HV, K, V] to [N, HV, V, K] (#20283)

2026-03-12 10:53:12 +08:00

backends

[FlashInfer v0.6.4] [RL] Integrate FlashInfer mxfp8 gemm, MoE, and routed MoE (#19537)

2026-03-10 15:37:57 -07:00

bench_fn

[Benchmark] Fix generated_shared_prefix attribute naming and remove args dependency (#19363)

2026-02-25 18:45:54 -08:00

constrained_decoding

Fix Test* mixin classes being collected as standalone pytest tests (#20417)

2026-03-12 18:18:45 -07:00

core

[Utils] Add NetworkAddress abstraction for IPv6-safe address handling (#20306)

2026-03-11 00:07:37 -07:00

debug_utils

Enhance error resilience in dump comparator (#19685)

2026-03-02 19:08:35 +08:00

disaggregation

[Performance] Decode Offload improves the long texts performance 100% through dynamic block offload. (#17216)

2026-03-08 17:16:53 +08:00

distributed

Fix Test* mixin classes being collected as standalone pytest tests (#20417)

2026-03-12 18:18:45 -07:00

dllm

[DLLM] Add initial radix cache support (#18724)

2026-03-04 23:24:09 -08:00

ep

chore: bump mooncake version to 0.3.9 (#18316)

2026-02-07 17:30:01 +08:00

eval

Fix nightly VLM accuracy: gemma3n TP fixes + removal, latency thresholds (#19401)

2026-02-27 14:24:02 -08:00

function_call

[Tool Call] Stream DeepSeek-V3.2 function call parameters in JSON format. (#16091)

2026-03-03 01:46:29 -08:00

hicache

[Benchmark] Remove re-exports from bench_serving.py (#19130)

2026-02-21 14:30:30 -08:00

kernels

[DSv32] [GLM5] Improve Model Quality by Avoiding FP32 Precision Loss in weights_proj (#19041)

2026-02-22 16:20:51 +08:00

layers

Return intermediate Mamba states (#19716)

2026-03-09 16:04:36 +08:00

lora

[Bug] Fix lora tp bug on H200 (#19769)

2026-03-04 20:11:02 -08:00

metrics

[Bug] Fix missing TTFT histogram for single-batch requests (#20122)

2026-03-08 15:18:51 -07:00

mla

Relax flaky test thresholds for MLA DeepSeek V3 and AutoRound (#20068)

2026-03-06 17:26:28 -08:00

model_loading

Fix/nemotron mtp quantaized (#19433)

2026-03-03 01:07:46 -08:00

models

[Test] Refactor KL divergence and prefix cache branching to kits (#19715)

2026-03-12 16:11:59 +08:00

moe

Merge stage-c-test-large-4-gpu suites into partitioned suites (#18325)

2026-02-06 15:32:33 -08:00

openai_server

Fix Test* mixin classes being collected as standalone pytest tests (#20417)

2026-03-12 18:18:45 -07:00

ops

[AMD] CI - Add MI35x nightly/PR tests for kv-cache-fp8 and allreduce-fusion (DeepSeek) (#19834)

2026-03-05 07:09:57 -08:00

parser

Add KimiK2Detector with tool interruption support (#19696)

2026-03-03 14:04:49 +08:00

perf

[TestFix] change LoRA tests to use NVIDIA adapter instead of Nutanix (#19642)

2026-03-02 12:55:41 -08:00

piecewise_cuda_graph

[CI] Refactor PCG related CI (#19994)

2026-03-10 14:49:25 +08:00

profiling

Migrate 4-GPU/8-GPU workflow jobs to stage-c and add CI registry decorators (#17299)

2026-01-31 22:37:22 -08:00

quant

Relax flaky B200 GSM8K accuracy thresholds (#20304)

2026-03-11 12:35:21 -07:00

radix_cache

[RadixTree][7/N Refactor]: Refactor mamba radix tree, release dup kvcache in insert func (#19429)

2026-03-13 12:28:32 +08:00

rl

[Scheduler] Unify idle checks into is_fully_idle() and fix weight update test (#20296)

2026-03-10 17:50:23 -07:00

rotary

feat: Support mrope_section with rope_type: "yarn" (#13313)

2026-02-14 22:51:44 +08:00

sampling

Fix flaky penalty tests by using higher temperature for effect comparison (#18380)

2026-02-12 21:08:37 +08:00

scheduler

add mixed chunk unit test and make small refactors (#18776)

2026-03-08 19:56:09 +08:00

sessions

Fix chunked prefill and KV cache leaks for streaming sessions (#20476)

2026-03-13 02:36:55 -07:00

spec

Fix Test* mixin classes being collected as standalone pytest tests (#20417)

2026-03-12 18:18:45 -07:00

stress

Add stress test workflow (#13937)

2025-11-26 13:24:39 -08:00

tokenizer

[CI] Reorganize stage-b 1-GPU tests for 5090 compatibility (#16826)

2026-01-15 15:23:35 -08:00

utils

[Utils] Add NetworkAddress abstraction for IPv6-safe address handling (#20306)

2026-03-11 00:07:37 -07:00

vlm

[VLM] Replace decord with torchcodec for video decoding (#20055)

2026-03-09 19:23:49 +08:00

test_hybrid_dp_ep_tp_mtp.py

Add hybrid parallelism test to nightly CI (#17444)

2026-01-20 17:43:50 -08:00

test_srt_backend.py

[AMD CI] Migrate and Add More Testcases (#17116)

2026-01-19 08:07:39 -08:00