[Feature] Add sampler custom logits processor (#2396)

Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
2025-01-19 14:46:53 -08:00
parent 3bcf5ecea7
commit e403d23757
12 changed files with 302 additions and 4 deletions
@@ -381,6 +381,7 @@ class TokenizerManager:
                lora_path=obj.lora_path,
                input_embeds=input_embeds,
                session_params=session_params,
+                custom_logit_processor=obj.custom_logit_processor,
            )
        elif isinstance(obj, EmbeddingReqInput):
            tokenized_obj = TokenizedEmbeddingReqInput(