Files
sglang/python
laoyao0822 a32050d1d8 Expose NSA index sharing as launch-time overrides
Index-topk sharing previously required editing model config or passing raw JSON overrides. Add first-class server args that merge into json_model_override_args before ModelConfig is read, preserving existing JSON overrides while letting launch scripts toggle index_topk_freq directly. Treat offset 0 as unset so wrappers can use 0 for default behavior without injecting a model-config override.

Constraint: Prefill and decode launch commands need the same effective model config without mutating /ssd model files.

Rejected: Require editing config.json | operationally fragile across g0034/g0035/g0036 model copies.

Rejected: Only use --json-model-override-args | works but is too error-prone for frequent launch-command tuning.

Confidence: high

Scope-risk: narrow

Directive: Keep these flags as model-config override shortcuts; apply them before any get_model_config() call.

Tested: Remote pytest in g0034 container for NSA index override parser tests: 2 passed.

Tested: py_compile for server_args.py and test_server_args.py.

Not-tested: Full prefill/decode ETE launch with --nsa-index-topk-freq enabled.
2026-06-10 03:24:33 +08:00
..