[Piecewise CUDA Graph] Fix recompile issue for Mixtral and Grok2 (#13667)

Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Ke Bao <ISPObaoke@163.com>
Co-authored-by: Oasis-Git <ayw.sirius19@gmail.com>
This commit is contained in:
Stefan He
2025-11-20 14:20:11 -08:00
committed by GitHub
co-authored by Minglei Zhu Ke Bao Oasis-Git
parent 6b262ac839
commit b5344b31b8
3 changed files with 24 additions and 160 deletions
+1
View File
@@ -353,6 +353,7 @@ class MixtralForCausalLM(nn.Module):
)
self.logits_processor = LogitsProcessor(config)
@torch.no_grad()
def forward(
self,
input_ids: torch.Tensor,