[Bug fix] Fix Gemma 2 and fix Gemma 3 multimodal with bs > 1 on NPU (#9871)

Co-authored-by: Maksim <makcum888e@mail.ru>
2025-09-08 01:19:40 -07:00
parent ee21817c6b
commit 5dd8c6444b
3 changed files with 11 additions and 9 deletions
@@ -550,7 +550,7 @@ class PrefillAdder:
                )
            else:
                # Make sure at least one page is available
-                trunc_len = self.rem_chunk_tokens - self.page_size + 1
+                trunc_len = self.rem_chunk_tokens // self.page_size * self.page_size
                if trunc_len <= 0:
                    return AddReqResult.OTHER