leavelet
dbc5ebbaa0
A single cudaHostRegister over a >~1 TB host buffer fails with cudaErrorMemoryAllocation on B300 (hard per-call ceiling: 512 GiB OK, 1024 GiB FAIL), crashing the hicache_size=1600 (~1.5 TB CP shared-L2 slab) prefill at startup. The registration cannot simply be chunked: a memcpy (cudaMemcpyBatchAsync, the CP-L2 H2D/D2H transfer) fails with cudaErrorInvalidValue when its host range straddles a registration boundary (verified empirically on b300-049). Fix: physically split the host cache into multiple page-aligned slabs, each <= a safe single-registration size (default 480 GiB, env SGLANG_CP_HICACHE_MAX_SLAB_GB), reusing the existing SharedHostTensorGroupAllocator + per-slab transfer splitting (_host_transfer_segments). Each slab is one whole registration and no transfer crosses a boundary; small configs (hicache_size<=400) stay single-slab unchanged. - memory_pool_host.py: add cp_hicache_max_single_register_bytes() + the fail-loud _check_single_cuda_host_register_size guard; revert the (transfer-unsafe) registration chunking back to one cudaHostRegister per buffer/slab. - hiradix_cache.py: _cp_shared_l2_slab_pages_by_payload auto-caps each payload's slab <= the ceiling so large caches auto-split. - cp_l3_slab_accessor.py: CpSharedL2SlabAccessor is now slab-count-aware (CpL3SlabSpan + per-slab dispatch via global_base_page; per-slab layer stride); _cp_l3_slab_spans rewires _maybe_init_cp_l3 off the single-slab assumption. L3 disk slabs / slot pool / LMDB index / GC are content-addressed and unchanged. Tests: multi-slab accessor incl. a non-circular torch.frombuffer-layout check; a real allocate_group + _cp_l3_slab_spans roundtrip; slab-cap auto-split; L3 store cross-slab spill/reload. Reviewed by 3 adversarial agents, no correctness bugs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Registered Tests
Tests under this directory are auto-discovered by run_suite.py via CI registration decorators.
Where Should I Put My New Test?
No server / engine launch required
| What you're testing | Directory | Requires |
|---|---|---|
| Component logic in isolation (cache, scheduler, config, parser, etc.) | unit/<module>/ |
CPU or GPU |
| CUDA kernel correctness | kernels/ |
GPU |
Server / engine launch required (E2E)
| What you're testing | Directory | Requires |
|---|---|---|
| Model inference correctness | models/, 4-gpu-models/, 8-gpu-models/ |
GPU |
| Feature-specific (OpenAI API, LoRA, speculative, distributed, VLM, etc.) | openai_server/, lora/, spec/, distributed/, ... |
GPU |
| Benchmarks (performance, accuracy, stress) | benchmark/ |
GPU |
| Platform-specific | amd/, ascend/ |
Vendor GPU |
See unit/README.md for unit test conventions.