a32050d1d8
Index-topk sharing previously required editing model config or passing raw JSON overrides. Add first-class server args that merge into json_model_override_args before ModelConfig is read, preserving existing JSON overrides while letting launch scripts toggle index_topk_freq directly. Treat offset 0 as unset so wrappers can use 0 for default behavior without injecting a model-config override. Constraint: Prefill and decode launch commands need the same effective model config without mutating /ssd model files. Rejected: Require editing config.json | operationally fragile across g0034/g0035/g0036 model copies. Rejected: Only use --json-model-override-args | works but is too error-prone for frequent launch-command tuning. Confidence: high Scope-risk: narrow Directive: Keep these flags as model-config override shortcuts; apply them before any get_model_config() call. Tested: Remote pytest in g0034 container for NSA index override parser tests: 2 passed. Tested: py_compile for server_args.py and test_server_args.py. Not-tested: Full prefill/decode ETE launch with --nsa-index-topk-freq enabled.
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.