fix: log warning when disable cuda graph (#5209)

This commit is contained in:
Yineng Zhang
2025-04-09 14:16:13 -07:00
committed by GitHub
parent 456b008bd8
commit 4cb53ecd0c
3 changed files with 13 additions and 7 deletions

View File

@@ -924,6 +924,12 @@ class ModelRunner:
return
if self.server_args.disable_cuda_graph:
logger.warning(
"\n\nCUDA Graph is DISABLED.\n"
"This will cause significant performance degradation.\n"
"CUDA Graph should almost never be disabled in most usage scenarios.\n"
"If you encounter OOM issues, please try setting --mem-fraction-static to a lower value (such as 0.8 or 0.7) instead of disabling CUDA Graph.\n"
)
return
tic = time.time()