[FEAT] Shared mem pool based cuda ipc for multi-modal data transport (#11917)

Co-authored-by: kousakawang <wanghanpei@bytedance.com> Co-authored-by: Yuan Luo <4908075+yuan-luo@users.noreply.github.com>
2025-11-02 16:46:37 +08:00
parent a920b9dace
commit 7efd8b3d1f
8 changed files with 424 additions and 20 deletions
@@ -340,6 +340,7 @@ class Qwen2Model(nn.Module):
        input_embeds: torch.Tensor = None,
        pp_proxy_tensors: Optional[PPProxyTensors] = None,
    ) -> Union[torch.Tensor, PPProxyTensors]:
+
        if self.pp_group.is_first_rank:
            if input_embeds is None:
                hidden_states = self.embed_tokens(input_ids)