Sundara Raman Ramachandran a360511d7b [Generative Score API] Scoring(Prefill-only) optimizations. (#9748)

2025-09-14 01:57:06 +08:00

..

async_dynamic_batch_tokenizer.py

[Performance] Dynamic Batch Tokenizer (#9382)

2025-09-14 01:56:04 +08:00

cache_controller.py

support memory_pool_host page first direct layout (#10031)

2025-09-11 23:19:44 -07:00

configure_logging.py

Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632)

2025-06-29 23:16:19 -07:00

data_parallel_controller.py

[1/N]DP refactor: Improve dp rank scheduling in PD disaggregation mode. (#10169)

2025-09-09 12:27:55 +08:00

detokenizer_manager.py

[Auto Sync] Update io_struct.py (20250909) (#10236)

2025-09-09 14:15:21 -07:00

disagg_service.py

[1/2] Refactor multi-tokenizer manager (#10074)

2025-09-07 19:13:34 +08:00

io_struct.py

Support loading weights from remote instance (#8215)

2025-09-12 17:40:22 +08:00

mm_utils.py

Qwen2.5-VL eagle3 infer (#8801)

2025-09-07 20:44:34 -07:00

multi_tokenizer_mixin.py

[Auto Sync] Update io_struct.py (20250909) (#10236)

2025-09-09 14:15:21 -07:00

multimodal_processor.py

Fix CI (#9012)

2025-08-09 13:33:42 -07:00

schedule_batch.py

[Generative Score API] Scoring(Prefill-only) optimizations. (#9748)

2025-09-14 01:57:06 +08:00

schedule_policy.py

[Bug fix] Fix Gemma 2 and fix Gemma 3 multimodal with bs > 1 on NPU (#9871)

2025-09-08 01:19:40 -07:00

scheduler_input_blocker.py

Support colocating requests (#7973)

2025-07-28 22:51:49 -07:00

scheduler_metrics_mixin.py

Add graph runner support with torch compile on CPU (#7843)

2025-09-07 21:33:58 -07:00

scheduler_output_processor_mixin.py

[Generative Score API] Scoring(Prefill-only) optimizations. (#9748)

2025-09-14 01:57:06 +08:00

scheduler_profiler_mixin.py

Fix typo in scheduler (#9934)

2025-09-05 17:45:27 +08:00

scheduler_recv_skipper.py

Reduce scheduler recv requests overhead (#8947)

2025-08-08 00:10:05 -07:00

scheduler_update_weights_mixin.py

feat(draft_model): support draft_model for RemoteModelLoader (#6407)

2025-08-28 16:09:52 -07:00

scheduler.py

[Generative Score API] Scoring(Prefill-only) optimizations. (#9748)

2025-09-14 01:57:06 +08:00

session_controller.py

Fix DP load for embedding (#9165)

2025-08-15 23:58:44 -07:00

template_manager.py

Move parsers under a single folder (#9912)

2025-09-02 18:25:04 -07:00

tokenizer_communicator_mixin.py

Support loading weights from remote instance (#8215)

2025-09-12 17:40:22 +08:00

tokenizer_manager.py

[Generative Score API] Scoring(Prefill-only) optimizations. (#9748)

2025-09-14 01:57:06 +08:00

tp_worker_overlap_thread.py

[Generative Score API] Scoring(Prefill-only) optimizations. (#9748)

2025-09-14 01:57:06 +08:00

tp_worker.py

[Generative Score API] Scoring(Prefill-only) optimizations. (#9748)

2025-09-14 01:57:06 +08:00

utils.py

[PD] Support PD disaggregation with Prefill PP (#8846)

2025-08-16 18:31:31 -07:00