pansicheng f84db115b1 Add storage read/write bandwidth logs to monitor kvcache performance (#9965) ...

Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

2025-09-05 16:52:55 -07:00

..

cache_controller.py

Add storage read/write bandwidth logs to monitor kvcache performance (#9965)

2025-09-05 16:52:55 -07:00

configure_logging.py

Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632)

2025-06-29 23:16:19 -07:00

data_parallel_controller.py

[feat] support minimum token load balance in dp attention (#7379)

2025-08-03 00:46:47 -07:00

detokenizer_manager.py

Move multi-tokenizer event loop to better place (#9902)

2025-09-01 23:12:21 -07:00

io_struct.py

fix MultiTokenizerWrapper name (#10049)

2025-09-05 13:39:46 +08:00

mm_utils.py

[feat]Ascend NPU Gemma-3-12b and Gemma-3-27b support (#8909)

2025-08-31 00:25:07 -07:00

multi_tokenizer_mixin.py

fix MultiTokenizerWrapper name (#10049)

2025-09-05 13:39:46 +08:00

multimodal_processor.py

Fix CI (#9012)

2025-08-09 13:33:42 -07:00

schedule_batch.py

Add enable_flashinfer_mxfp4_bf16_moe for higher precision and slower moe backend (#9004)

2025-08-23 15:38:40 +08:00

schedule_policy.py

Ensure chunked request extension length respects both rem_chunk_tokens and rem_total_tokens limits (#10003)

2025-09-04 04:15:26 -07:00

scheduler_input_blocker.py

Support colocating requests (#7973)

2025-07-28 22:51:49 -07:00

scheduler_metrics_mixin.py

[1/N] DP-refactor: move dp balance code into scheduler's mixin class (#10004)

2025-09-04 16:53:58 +08:00

scheduler_output_processor_mixin.py

Fix input logprob index (#9841)

2025-08-31 02:56:47 -07:00

scheduler_profiler_mixin.py

Fix typo in scheduler (#9934)

2025-09-05 17:45:27 +08:00

scheduler_recv_skipper.py

Reduce scheduler recv requests overhead (#8947)

2025-08-08 00:10:05 -07:00

scheduler_update_weights_mixin.py

feat(draft_model): support draft_model for RemoteModelLoader (#6407)

2025-08-28 16:09:52 -07:00

scheduler.py

Add storage read/write bandwidth logs to monitor kvcache performance (#9965)

2025-09-05 16:52:55 -07:00

session_controller.py

Fix DP load for embedding (#9165)

2025-08-15 23:58:44 -07:00

template_manager.py

Move parsers under a single folder (#9912)

2025-09-02 18:25:04 -07:00

tokenizer_manager.py

[Vulnerability]feat(conn): set bootstrap server host (#9931)

2025-09-05 17:36:17 +08:00

tp_worker_overlap_thread.py

Support GPU pinning for LoRA (#8697)

2025-08-06 19:39:45 -07:00

tp_worker.py

[Fix] Add speculative_draft_model_revision to server_args (#5255)

2025-09-05 19:45:46 +08:00

utils.py

[PD] Support PD disaggregation with Prefill PP (#8846)

2025-08-16 18:31:31 -07:00