Lu ChangqiandZhiqiang Xie 0e0ec70200 Hierarchical Caching supports MLA (#4009) ...

Signed-off-by: Changqi Lu <luchangqi.123@bytedance.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

2025-03-13 20:42:14 -07:00

..

configs

model: Support Janus-pro (#3203)

2025-03-12 11:02:11 -07:00

constrained

[Fix & Style] Refactor the grammar backend to reduce human errors and improve readability (#4030)

2025-03-04 03:56:45 -08:00

distributed

[Fix] Check the device backend before calling empty_cache function (#4212)

2025-03-12 21:28:48 -07:00

entrypoints

typo: Update http_server.py (#4350)

2025-03-12 15:05:30 -07:00

layers

Add some fused elementwise kernels for grok-1 (#4398)

2025-03-13 13:39:10 -07:00

lora

Lazily import lora backends (#4225)

2025-03-08 23:39:26 -08:00

managers

Hierarchical Caching supports MLA (#4009)

2025-03-13 20:42:14 -07:00

mem_cache

Hierarchical Caching supports MLA (#4009)

2025-03-13 20:42:14 -07:00

metrics

Split the __init__ of scheduler as smaller functions. Improve the eagle tests (#4128)

2025-03-06 00:13:20 -08:00

model_executor

Improve DP attention (#4390)

2025-03-13 08:23:56 -07:00

model_loader

Revert "[XPU][CPU] Enable the native path of DeepSeek" (#4367)

2025-03-12 23:45:52 -07:00

models

Add some fused elementwise kernels for grok-1 (#4398)

2025-03-13 13:39:10 -07:00

openai_api

Fix Llama3.3 tool call support (#4320)

2025-03-13 14:01:41 -07:00

sampling

[EAGLE] many fixes for eagle (#4195)

2025-03-07 22:12:13 -08:00

speculative

Support nextn for flashinfer mla attention backend (#4218)

2025-03-09 00:01:54 -08:00

_custom_ops.py

Simplify tests & Fix trtllm custom allreduce registration (#4252)

2025-03-10 01:24:22 -07:00

aio_rwlock.py

A better aio rwlock that guarantees the order (#2547)

2024-12-22 15:44:32 -08:00

conversation.py

model: Support Janus-pro (#3203)

2025-03-12 11:02:11 -07:00

custom_op.py

Remove vllm ops scaled fp8 quant and accelerate per token quant by 20-28% (#4215)

2025-03-12 00:08:03 -07:00

function_call_parser.py

Fix Llama3.3 tool call support (#4320)

2025-03-13 14:01:41 -07:00

hf_transformers_utils.py

model: Support Janus-pro (#3203)

2025-03-12 11:02:11 -07:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torchao (#2436)

2024-12-11 04:21:42 -08:00

reasoning_parser.py

Reasoning parser (#4000)

2025-03-03 21:16:36 -08:00

server_args.py

Improve DP attention (#4390)

2025-03-13 08:23:56 -07:00

server.py

Misc fixes for eagle (flush_cache, CPU overhead) (#3014)

2025-01-20 20:27:38 -08:00

torch_memory_saver_adapter.py

Add a new api configure_logging to allow dumping the requests (#2875)

2025-01-13 14:24:00 -08:00

utils.py

Revert "[XPU][CPU] Enable the native path of DeepSeek" (#4367)

2025-03-12 23:45:52 -07:00

warmup.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00