Chunyuan WU 128f16a817 [CPU]convert topk_weights to fp32 for INT8 and FP8 paths (for llama4) and fix LmHead weight pack (#7818)

2025-07-08 19:27:24 -07:00

..

attention

Ascend attention backend(PA&MLA) (#7722)

2025-07-03 09:23:19 -07:00

moe

[CPU]convert topk_weights to fp32 for INT8 and FP8 paths (for llama4) and fix LmHead weight pack (#7818)

2025-07-08 19:27:24 -07:00

quantization

Enable ModelOpt Llama4 fp8 checkpoint deployment in SGLang (#7129)

2025-07-08 00:19:50 -07:00

activation.py

Move mem_fraction_static adjustment for multimodal models to server_args.py & Fix session control & Other cleanups (#7748)

2025-07-04 16:33:33 -07:00

amx_utils.py

[CPU] refine CPU integration code (#7647)

2025-07-03 09:51:09 -07:00

communicator.py

Fix illegal memory in trtllm allreduce fusion (#7864)

2025-07-08 11:47:17 -07:00

dp_attention.py

Fix incorrect spec_num_draft_tokens in draft_extend (#7757)

2025-07-05 02:18:16 -07:00

elementwise.py

Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632)

2025-06-29 23:16:19 -07:00

flashinfer_comm_fusion.py

Fix illegal memory in trtllm allreduce fusion (#7864)

2025-07-08 11:47:17 -07:00

layernorm.py

Fix illegal memory in trtllm allreduce fusion (#7864)

2025-07-08 11:47:17 -07:00

linear.py

[CPU] support the case where num_attention_heads or intermediate_size is not divisible by the TP size (#6771)

2025-07-03 09:51:38 -07:00

logits_processor.py

saving hidden_states.clone() (#7705)

2025-07-04 20:07:42 -07:00

multimodal.py

chore: upgrade flashinfer v0.2.6.post1 jit (#6958)

2025-06-09 09:22:39 -07:00

parameter.py

[CPU] support the case where num_attention_heads or intermediate_size is not divisible by the TP size (#6771)

2025-07-03 09:51:38 -07:00

pooler.py

Feat/support rerank (#6058)

2025-06-16 10:50:01 -07:00

radix_attention.py

Fix grammar abort & Minor style fixes (#7204)

2025-06-14 22:49:41 -07:00

rotary_embedding.py

Ascend attention backend(PA&MLA) (#7722)

2025-07-03 09:23:19 -07:00

sampler.py

Reintroduce tiny fix sampler error when prob is not contiguous (#7354)

2025-06-19 09:39:34 -07:00

torchao_utils.py

Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

utils.py

[Feature] Support Flashinfer fmha on Blackwell (#6930)

2025-06-06 12:57:50 -07:00

vocab_parallel_embedding.py

[CPU]convert topk_weights to fp32 for INT8 and FP8 paths (for llama4) and fix LmHead weight pack (#7818)

2025-07-08 19:27:24 -07:00