co-authored by
sglang-bot
parent
2a5773440e
commit
7e626d12b7
@@ -21,7 +21,7 @@ This error may result from kernel errors or out-of-memory issues:
|
||||
- If the server hangs during initialization or running, it can be memory issues (out of memory), network issues (nccl errors), or other bugs in sglang.
|
||||
- If it is out of memory, you might see that `avail mem` is very low during the initialization or right after initialization. In this case,
|
||||
you can try to decrease `--mem-fraction-static`, decrease `--cuda-graph-max-bs`, or decrease `--chunked-prefill-size`.
|
||||
- Other bugs, please raise a Github issue to us.
|
||||
- Other bugs, please file an issue on GitHub.
|
||||
|
||||
|
||||
## Frequently Asked Questions
|
||||
@@ -34,6 +34,6 @@ From our initial investigation, this indeterminism arises from two factors: dyna
|
||||
|
||||
To achieve more deterministic outputs in the current code, you can add `--disable-radix-cache` and send only one request at a time. The results will be mostly deterministic under this setting.
|
||||
|
||||
**Note**:
|
||||
Recently, we also introduced a deterministic mode, you can enable it with `--enable-deterministic-inference`. It might not work for all cases.
|
||||
**Update**:
|
||||
Recently, we also introduced a deterministic mode, you can enable it with `--enable-deterministic-inference`.
|
||||
Please find more details in this blog post: https://lmsys.org/blog/2025-09-22-sglang-deterministic/
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
# Learn more
|
||||
# Learn More and Join the Community
|
||||
|
||||
You can find more blogs, slides, and videos about SGLang at [https://github.com/sgl-project/sgl-learning-materials](https://github.com/sgl-project/sgl-learning-materials).
|
||||
|
||||
The latest SGLang features and updates are shared through the [LMSYS blog](https://lmsys.org/blog/).
|
||||
|
||||
The 2025 H2 roadmap can be found at this [issue](https://github.com/sgl-project/sglang/issues/7736).
|
||||
- The development roadmap: [2025 Q4](https://github.com/sgl-project/sglang/issues/12780)
|
||||
- The latest SGLang features and updates are shared through the [LMSYS blog](https://lmsys.org/blog/)
|
||||
- X (formerly Twitter): https://x.com/lmsysorg
|
||||
- LinkedIn: https://www.linkedin.com/company/sgl-project/
|
||||
- Join Slack: https://slack.sglang.ai/
|
||||
- More blogs, slides, and videos about SGLang at [https://github.com/sgl-project/sgl-learning-materials](https://github.com/sgl-project/sgl-learning-materials)
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
# Production Request Tracing
|
||||
|
||||
SGlang exports request trace data based on the OpenTelemetry Collector. You can enable tracing by adding the `--enable-trace` and configure the OpenTelemetry Collector endpoint using `--otlp-traces-endpoint` when launching the server.
|
||||
|
||||
You can find example screenshots of the visualization in https://github.com/sgl-project/sglang/issues/8965.
|
||||
|
||||
Reference in New Issue
Block a user