Lu Changqi 0e0ec70200 Hierarchical Caching supports MLA (#4009) ...

Signed-off-by: Changqi Lu <luchangqi.123@bytedance.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

2025-03-13 20:42:14 -07:00

..

image_processors

model: Support Janus-pro (#3203)

2025-03-12 11:02:11 -07:00

cache_controller.py

Hierarchical Caching supports MLA (#4009)

2025-03-13 20:42:14 -07:00

configure_logging.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00

data_parallel_controller.py

Improve DP attention (#4390)

2025-03-13 08:23:56 -07:00

detokenizer_manager.py

[Minor] more code cleanup (#4077)

2025-03-04 21:23:47 -08:00

image_processor.py

refactor: move image processors to separate files (#4229)

2025-03-11 12:35:35 -07:00

io_struct.py

Add Support for Qwen2-VL Multi-modal Embedding Models (#3694)

2025-03-06 16:46:20 -08:00

multi_modality_padding.py

refactor: move image processors to separate files (#4229)

2025-03-11 12:35:35 -07:00

schedule_batch.py

Fix the output of hidden states after HTTP requests (#4269)

2025-03-13 14:54:06 -07:00

schedule_policy.py

Support page size > 1 (#4356)

2025-03-12 22:22:39 -07:00

scheduler_output_processor_mixin.py

Fix the output of hidden states after HTTP requests (#4269)

2025-03-13 14:54:06 -07:00

scheduler.py

Hot fix for hicache with new page aligned radixtree (#4397)

2025-03-13 15:50:49 -07:00

session_controller.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00

tokenizer_manager.py

Add Support for Qwen2-VL Multi-modal Embedding Models (#3694)

2025-03-06 16:46:20 -08:00

tp_worker_overlap_thread.py

Support page size > 1 (#4356)

2025-03-12 22:22:39 -07:00

tp_worker.py

[Eagle] Refactor eagle speculative decoding (#3986)

2025-03-05 08:06:07 -08:00

utils.py

Frontend: better error message handling for FINISH_ABORT in scheduler.py (#2956)

2025-01-18 19:37:30 -08:00