Files
sglang/python/sglang
leavelet e4e0784387 CP HiCache trace: content-key in fwd_hash for L1-hit vs L2-reload differential
Adds a per-request content fingerprint (hash of extend input-ids + seq_len) to
fwd_hash so the SAME request forwarded from an L1-hit (known-good) and an
L2-reload (suspect) can be JOINED across the log without rid (the Rust PD
gateway strips the client rid). Gated to bs<=1 forwards (the join is only
meaningful single-request, and this skips the c=24 flood forwards so the
level-3 log stays small). The analyzer joins by ck and reports the first
DETERMINISTIC stage (topk/attn) that diverges = corruption localized.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:20:54 +00:00
..

Code Structure

  • eval: The evaluation utilities.
  • lang: The frontend language.
  • multimodal_gen: Inference framework for accelerated image/video generation.
  • srt: The backend engine for running local models. (SRT = SGLang Runtime).
  • test: The test utilities.
  • api.py: The public APIs.
  • bench_offline_throughput.py: Benchmark the performance in the offline mode.
  • bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
  • bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
  • bench_serving.py: Benchmark online serving with dynamic requests.
  • check_env.py: Check the environment variables and dependencies.
  • global_config.py: The global configs and constants.
  • launch_server.py: The entry point for launching a local server.
  • profiler.py: The profiling entry point to send profile requests.
  • utils.py: Common utilities.
  • version.py: Version info.