Liangsheng Yin 5e1558f1f2 Update max_req_len and max_req_input_len (#1748)

2024-10-21 16:12:04 -07:00

..

data_parallel_controller.py

Add back data parallelism (#1635)

2024-10-11 07:22:48 -07:00

detokenizer_manager.py

[Minor] Rename no_eos_trim to no_stop_trim (#1661)

2024-10-13 20:30:03 -07:00

image_processor.py

Llama3.2 vision model support (#1551)

2024-10-21 15:01:21 -07:00

io_struct.py

Fix failed ci tests on long prompts; Better error messages for embedding models (#1700)

2024-10-17 09:23:29 -07:00

schedule_batch.py

Llama3.2 vision model support (#1551)

2024-10-21 15:01:21 -07:00

schedule_policy.py

Returning a per request metric for number of cached_tokens read (#1599)

2024-10-16 11:49:22 -07:00

scheduler.py

Update max_req_len and max_req_input_len (#1748)

2024-10-21 16:12:04 -07:00

tokenizer_manager.py

Llama3.2 vision model support (#1551)

2024-10-21 15:01:21 -07:00

tp_worker_overlap_thread.py

Faster overlap mode scheduler (#1738)

2024-10-21 04:30:52 -07:00

tp_worker.py

Update max_req_len and max_req_input_len (#1748)

2024-10-21 16:12:04 -07:00