Optimize layernorm_gated for Qwen3-Next (#16397)

Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
This commit is contained in:
Yuan Luo
2026-01-10 20:55:31 +08:00
committed by GitHub
parent bdb76b34db
commit d1ec93e3ac
5 changed files with 791 additions and 34 deletions

View File

@@ -2467,6 +2467,11 @@ def set_cuda_arch():
)
def cdiv(a: int, b: int) -> int:
"""Ceiling division."""
return -(a // -b)
def next_power_of_2(n: int):
return 1 << (n - 1).bit_length() if n > 0 else 1