[Feature, Hardware] Enable DeepseekV3 on AMD GPUs (#2601)

Co-authored-by: root <root@banff-cyxtera-s83-5.amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Bruce Xue <yigex@xilinx.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
This commit is contained in:
yigex
2025-01-02 16:23:19 -08:00
committed by GitHub
co-authored by root HAI Bruce Xue Yineng Zhang
parent bdf946bf81
commit c7ae474a49
2 changed files with 9 additions and 5 deletions
@@ -406,6 +406,10 @@ def _decode_grouped_att_m_fwd(
Lk = k_buffer.shape[-1]
Lv = v_buffer.shape[-1]
# [TODO] work around shmem limit on MI3xx
if is_hip_ and Lk >= 576:
BLOCK = 16
if Lk == 576:
BLOCK_DMODEL = 512
BLOCK_DPE = 64