3 people cbbc82b7b8 Support qwen2 vl model (#1721) ...

Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: ispobock <ISPObaoke@163.com>

2024-10-19 21:44:38 -07:00

..

attention

Support qwen2 vl model (#1721)

2024-10-19 21:44:38 -07:00

fused_moe

[Performance, hardware] MoE tuning update to AMD MI300x GPUs (#1619)

2024-10-10 22:48:15 -07:00

quantization

Remove references to squeezellm (#1603)

2024-10-07 11:30:41 -07:00

activation.py

Optimize broadcast & Reorg code (#1598)

2024-10-07 13:19:23 -07:00

layernorm.py

Use is_flashinfer_available to replace is_hip for flashinfer check (#1596)

2024-10-06 22:54:05 -07:00

linear.py

Update vllm to 0.6.3 (#1711) (#1720)

2024-10-19 20:45:41 -07:00

logits_processor.py

Clean up batch data structures: Introducing ModelWorkerBatch (#1544)

2024-09-30 06:41:49 -07:00

pooler.py

Rename InputMetadata -> ForwardBatch (#1543)

2024-09-30 02:41:11 -07:00

radix_attention.py

Simplify flashinfer dispatch (#1552)

2024-10-01 00:28:42 -07:00

rotary_embedding.py

Support qwen2 vl model (#1721)

2024-10-19 21:44:38 -07:00

sampler.py

Simplify the nan detection and greedy check in sampler (#1709)

2024-10-18 20:21:24 -07:00

torchao_utils.py

Add float8 dynamic quant to torchao_utils (#1528)

2024-09-28 12:27:54 -07:00