Chang Su 1940cdec61 [Bugfix] Fix Llama4 gibberish output with long context and CUDA graph (#6162)

2025-05-09 15:33:02 -07:00

..

configs

[Fix] Incorrect Memory Allocation on CUDA:0 by Non-Zero CUDA Processes in TP/DP (#5745)

2025-05-08 17:52:26 -07:00

connector

…

constrained

upgrade xgrammar to 0.1.19 (#6129)

2025-05-08 14:42:02 -07:00

disaggregation

Better PD initialization (#5751)

2025-05-07 01:12:57 +08:00

distributed

[AMD] switch to custom allreduce regardless of MSCCL setting on ROCm (#6097)

2025-05-08 13:46:58 -07:00

entrypoints

[PD] Add control to slow down a server (#5572)

2025-05-08 01:03:08 -07:00

layers

[Bugfix] Fix Llama4 gibberish output with long context and CUDA graph (#6162)

2025-05-09 15:33:02 -07:00

lora

Fix flaky issues of lora and add multi batch tests (#5957)

2025-05-04 13:11:40 -07:00

managers

[Fix] Incorrect Memory Allocation on CUDA:0 by Non-Zero CUDA Processes in TP/DP (#5745)

2025-05-08 17:52:26 -07:00

mem_cache

Fix prefill OOM error in the case of large page size (#5081)

2025-05-05 16:02:55 -07:00

metrics

[metrics] Add in queue metrics (#4444)

2025-04-09 17:19:27 -07:00

model_executor

feat: add thinking_budget (#6089)

2025-05-09 08:22:09 -07:00

model_loader

Tiny refactor weight loading logic (#5232)

2025-05-08 01:02:56 -07:00

models

opt flashinfer mla cat (#5822)

2025-05-08 23:17:14 -07:00

openai_api

feat: add thinking_budget (#6089)

2025-05-09 08:22:09 -07:00

platforms

[feat] interface for platforms abstraction (#4928)

2025-03-31 00:04:21 -07:00

sampling

feat: add thinking_budget (#6089)

2025-05-09 08:22:09 -07:00

speculative

[PP] Add pipeline parallelism (#5724)

2025-04-30 18:18:07 -07:00

_custom_ops.py

Clean up imports (#5467)

2025-04-16 15:26:49 -07:00

aio_rwlock.py

…

code_completion_parser.py

fix: Use is not None instead of != None for None checks. (#5687)

2025-04-26 19:26:57 -07:00

conversation.py

[Minor] Fix duplicate method definitions in conversation.py (#6012)

2025-05-04 13:02:53 -07:00

custom_op.py

Clean up imports (#5467)

2025-04-16 15:26:49 -07:00

function_call_parser.py

feat: support pythonic tool call and index in tool call streaming (#5725)

2025-04-29 17:30:44 -07:00

hf_transformers_utils.py

Support InternVL3 (#5350)

2025-05-01 22:38:59 -07:00

mm_utils.py

…

model_parallel.py

…

patch_torch.py

Fix torch.compile cacheing (#5259)

2025-04-10 18:08:45 -07:00

reasoning_parser.py

feat: add thinking_budget (#6089)

2025-05-09 08:22:09 -07:00

server_args.py

[Feature] Add FlashAttention3 as a backend for VisionAttention (#5764)

2025-05-08 10:01:19 -07:00

torch_memory_saver_adapter.py

[CI] Fix ci tests (#5769)

2025-04-27 07:18:10 -07:00

utils.py

Clean logs for DeepSeek-V3 launching (#6079)

2025-05-07 18:54:50 -07:00

warmup.py

…