[Piecewise CUDA Graph] Fix recompile issue for Mixtral and Grok2 (#13667)
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com> Co-authored-by: Ke Bao <ISPObaoke@163.com> Co-authored-by: Oasis-Git <ayw.sirius19@gmail.com>
This commit is contained in:
co-authored by
Minglei Zhu
Ke Bao
Oasis-Git
parent
6b262ac839
commit
b5344b31b8
@@ -353,6 +353,7 @@ class MixtralForCausalLM(nn.Module):
|
||||
)
|
||||
self.logits_processor = LogitsProcessor(config)
|
||||
|
||||
@torch.no_grad()
|
||||
def forward(
|
||||
self,
|
||||
input_ids: torch.Tensor,
|
||||
|
||||
Reference in New Issue
Block a user