Support releasing CUDA graph memory when paused (#7873)

Co-authored-by: ryang-max <y1cunhui.yang@gmail.com> Co-authored-by: ryang <38470282+ryang-max@users.noreply.github.com>
2025-10-28 14:40:50 +08:00
parent d2b8c4123e
commit 691c8534cf
7 changed files with 94 additions and 15 deletions
@@ -239,6 +239,9 @@ class Envs:
    SGLANG_IMAGE_MAX_PIXELS = EnvInt(16384 * 28 * 28)
    SGLANG_RESIZE_RESAMPLE = EnvStr("")

+    # Release & Resume Memory
+    SGLANG_MEMORY_SAVER_CUDA_GRAPH = EnvBool(False)
+
    # Ktransformers
    SGLANG_KT_MOE_NUM_GPU_EXPERTS = EnvInt(None)
    SGLANG_KT_MOE_CPUINFER = EnvInt(None)