Files
sglang/python/sglang
leavelet 2ae46013ac L3 3.0: wire CpL3Store into HiRadixCache (construct + drain + reset/clear)
_maybe_init_cp_l3: build the 3 CpSharedL2SlabAccessor (target_kv always; draft_kv if draft pool;
index_k if NSA) from the LIVE host pool slabs (verified attrs: host.allocator.mapping.mmap,
layer_num/page_num/kv_cache_dim/dtype.itemsize; index_host_tensor_allocator + indexer attrs), then
CpL3Store.from_config + connect (PLP gate). Single-slab only (default --cp-shared-l2-slab-size-gb 0);
fail loud on the multi-slab group case (future extension, not a silent stub).
_drain_l3_control_queues: per-tick CP-cpu-group MIN over the 2 ack-qsizes + drain MIN of each, idle
early-return on the replicated has_inflight (R1 CRIT-3/4) — wired in check_hicache_events (gated on
enable_cp_l3, NOT the enable_storage drain). reset() -> cp_l3_store.clear(); shutdown() -> close().
Default-off + behavior-neutral. Compiles. 3.0 CODE complete; flag-on smoke gated on cluster branch sync.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 21:31:12 +00:00
..

Code Structure

  • eval: The evaluation utilities.
  • lang: The frontend language.
  • multimodal_gen: Inference framework for accelerated image/video generation.
  • srt: The backend engine for running local models. (SRT = SGLang Runtime).
  • test: The test utilities.
  • api.py: The public APIs.
  • bench_offline_throughput.py: Benchmark the performance in the offline mode.
  • bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
  • bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
  • bench_serving.py: Benchmark online serving with dynamic requests.
  • check_env.py: Check the environment variables and dependencies.
  • global_config.py: The global configs and constants.
  • launch_server.py: The entry point for launching a local server.
  • profiler.py: The profiling entry point to send profile requests.
  • utils.py: Common utilities.
  • version.py: Version info.