PiteXChen 8b22deef5b fix【hicache】fix the KV cache resource occupation and invalid loading from prefetch when pending requests are aborted. (#16369) ...

Signed-off-by: CLFutureX <chenyongqyl@163.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

2026-01-15 15:14:38 -08:00

..

async_dynamic_batch_tokenizer.py

[Performance] Dynamic Batch Tokenizer (#9382)

2025-09-14 01:56:04 +08:00

async_mm_data_processor.py

[VLM] Optimize async mm data process mechanism (#12066)

2025-11-01 01:24:53 +08:00

cache_controller.py

fix【hicache】fix the KV cache resource occupation and invalid loading from prefetch when pending requests are aborted. (#16369)

2026-01-15 15:14:38 -08:00

configure_logging.py

Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632)

2025-06-29 23:16:19 -07:00

data_parallel_controller.py

[dpc]: unify DP controller load balancing and simplify dispatch logic (#16258)

2026-01-11 12:38:03 +08:00

detokenizer_manager.py

Tiny add CPU resource monitoring for overload diagnosis (#16852)

2026-01-10 12:48:18 +08:00

disagg_service.py

[NPU] upgrade npu mf_apater plugin (#15853)

2026-01-13 09:02:10 +08:00

io_struct.py

[API] Add /v1/loads endpoint for load metrics (#16976)

2026-01-14 09:13:20 -08:00

mm_utils.py

EVS Framework: Support NemotronH_Nano_VL_V2 (#14051)

2026-01-05 16:18:07 +08:00

multi_tokenizer_mixin.py

Revert "[FEAT] optimize tensor zmq transfer for multimodal inputs" (#16386)

2026-01-04 22:05:26 -08:00

multimodal_processor.py

feat: support external custom models (#13429)

2025-11-20 11:16:58 +08:00

overlap_utils.py

feat(SpecEagleV2): add standalone_worker_v2 (#12625)

2025-12-30 17:55:04 +08:00

prefill_delayer.py

Support token low usage watermark in prefill delayer (#16814)

2026-01-09 23:07:18 +08:00

request_metrics_exporter.py

[Auto Sync] Update request_metrics_exporter.py (20251230) (#16183)

2025-12-30 11:09:41 -08:00

schedule_batch.py

Refactor prefix cache type checking (#17028)

2026-01-15 11:28:13 +08:00

schedule_policy.py

Refactor prefix cache type checking (#17028)

2026-01-15 11:28:13 +08:00

scheduler_dp_attn_mixin.py

Move log_prefill_stats_late to correct location in PP mode (#15946)

2025-12-31 12:00:24 +08:00

scheduler_input_blocker.py

Tiny move files to utils folder (#11166)

2025-10-03 22:40:06 +08:00

scheduler_metrics_mixin.py

[API] Add /v1/loads endpoint for load metrics (#16976)

2026-01-14 09:13:20 -08:00

scheduler_output_processor_mixin.py

Fix log_decode_stats_every_iteration when having TP in attention (#16871)

2026-01-10 20:10:23 +08:00

scheduler_pp_mixin.py

Clarify the meaning of cpu_group / entry_rank when dp + tp is enabled. (#16876)

2026-01-11 13:04:43 +08:00

scheduler_profiler_mixin.py

Clarify the meaning of cpu_group / entry_rank when dp + tp is enabled. (#16876)

2026-01-11 13:04:43 +08:00

scheduler_recv_skipper.py

[Feat]Add scheduler recv skipper weights to environment configuration (#13855)

2025-11-27 18:16:11 +08:00

scheduler_runtime_checker_mixin.py

Refactor prefix cache type checking (#17028)

2026-01-15 11:28:13 +08:00

scheduler_update_weights_mixin.py

refactor: add type hints to scheduler mixins (#15913)

2025-12-26 16:50:07 -08:00

scheduler.py

fix【hicache】fix the KV cache resource occupation and invalid loading from prefetch when pending requests are aborted. (#16369)

2026-01-15 15:14:38 -08:00

session_controller.py

fix session request with None tokenizer (#16278)

2026-01-14 10:41:08 -08:00

template_manager.py

Move parsers under a single folder (#9912)

2025-09-02 18:25:04 -07:00

tokenizer_communicator_mixin.py

[API] Add /v1/loads endpoint for load metrics (#16976)

2026-01-14 09:13:20 -08:00

tokenizer_manager_multiitem_mixin.py

feat: support qwen3(-VL) rerank scoring&chat template (#16403)

2026-01-15 00:45:46 +08:00

tokenizer_manager.py

[logprob] Fix logprob + streaming for long concurrent decode by caching already processed logprob (#17005)

2026-01-13 12:39:14 -08:00

tp_worker.py

Clarify the meaning of cpu_group / entry_rank when dp + tp is enabled. (#16876)

2026-01-11 13:04:43 +08:00

utils.py

Support EPLB balancedness prometheus metric without GPU->CPU synchronize (#15401)

2025-12-18 22:24:23 +08:00