From f032c4f3d66605edd3177ac7d242c3d9704888bf Mon Sep 17 00:00:00 2001 From: Kun Lin <81014421+klhhhhh@users.noreply.github.com> Date: Tue, 3 Feb 2026 00:43:20 -0500 Subject: [PATCH] Support Markdown/Notebook-Friendly Documentation Export for Downstream Integration (#18131) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 赵晨阳 --- .github/workflows/release-docs.yml | 1 + docs/Makefile | 20 +++++++++ docs/README.md | 69 +++++++++++++++++++++++++++++- 3 files changed, 89 insertions(+), 1 deletion(-) diff --git a/.github/workflows/release-docs.yml b/.github/workflows/release-docs.yml index 07e6871a2..fc725d88b 100644 --- a/.github/workflows/release-docs.yml +++ b/.github/workflows/release-docs.yml @@ -47,6 +47,7 @@ jobs: run: | cd docs make html + make markdown python3 wrap_run_llm.py cd _build/html diff --git a/docs/Makefile b/docs/Makefile index 6b8792c42..3b437725c 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -38,6 +38,26 @@ compile: echo "Total execution time: $${TOTAL_ELAPSED}s" >> logs/timing.log; \ echo "All Notebook execution timings:" && cat logs/timing.log +# Convert Notebook files to Markdown artifacts (no execution) +markdown: + @set -e; \ + echo "Exporting notebooks to Markdown..."; \ + mkdir -p "$(BUILDDIR)/html/markdown"; \ + find $(SOURCEDIR) -path "*/_build/*" -prune -o -name "*.ipynb" -print0 | \ + parallel -0 -j3 --halt soon,fail=1 ' \ + NB_SRC="{}"; \ + REL_DIR=$$(dirname "$$NB_SRC"); \ + NB_NAME=$$(basename "$$NB_SRC"); \ + NB_BASE=$${NB_NAME%.ipynb}; \ + OUT_DIR="$(BUILDDIR)/html/markdown/$$REL_DIR"; \ + mkdir -p "$$OUT_DIR"; \ + jupyter nbconvert --to markdown "$$NB_SRC" \ + --output "$$NB_BASE.md" \ + --output-dir "$$OUT_DIR" \ + >/dev/null; \ + ' || exit 1; \ + echo "Markdown artifacts written to: $(BUILDDIR)/html/markdown" + # Serve documentation with auto-build and live reload serve: @echo "Starting auto-build server at http://0.0.0.0:$(PORT)" diff --git a/docs/README.md b/docs/README.md index f4cb9ce46..93ad5724c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -45,7 +45,6 @@ find . -name '*.ipynb' -exec nbstripout {} \; # After these checks pass, push your changes and open a PR on your branch pre-commit run --all-files ``` ---- ## Documentation Style Guidelines @@ -55,3 +54,71 @@ pre-commit run --all-files - Reuse the launched server as much as possible to reduce server launch time. - Do not use absolute links (e.g., `https://docs.sglang.io/get_started/install.html`). Always prefer relative links (e.g., `../get_started/install.md`). - Follow the existing examples to learn how to launch a server, send a query and other common styles. + +## Documentation Build, Deployment, and CI + +The SGLang documentation pipeline is based on **Sphinx** and supports rendering Jupyter notebooks (`.ipynb`) into HTML/Markdown for web display. Detailed logits can be found in the [Makefile](./Makefile). + +### Notebook Execution (`make compile`) + +The `make compile` target is responsible for executing notebooks before rendering: + +* Finds all `.ipynb` files under `docs/` (excluding `_build/`) +* Executes notebooks in parallel using GNU Parallel, with a relatively small `--mem-fraction-static` +* Wraps execution with `retry` to reduce flaky failures +* Executes notebooks via `jupyter nbconvert --execute --inplace` +* Records execution timing in `logs/timing.log` + +This step ensures notebooks contain up-to-date outputs with each commit in the main branch before rendering. + +### Web Rendering (`make html`) + +After compilation, Sphinx builds the website: + +* Reads Markdown, reStructuredText, and Jupyter notebooks +* Renders them into HTML pages +* Outputs the website into: + +``` +docs/_build/html/ +``` + +This directory is the source for online documentation hosting. + +### Markdown Export (`make markdown`) + +To support downstream consumers, we add a **new Makefile target**: + +```bash +make markdown +``` + +This target: + +* Does **not modify** `make compile` +* Scans all `.ipynb` files (excluding `_build/`) +* Converts notebooks directly to Markdown using `jupyter nbconvert --to markdown` +* Writes Markdown artifacts into the existing build directory: + +``` +docs/_build/html/markdown/.md +``` + +Example: + +``` +docs/advanced_features/lora.ipynb +→ docs/_build/html/markdown/advanced_features/lora.md +``` + +### CI Execution + +In our [CI](https://github.com/sgl-project/sglang/blob/main/.github/workflows/release-docs.yml), the documentation pipeline first gets all the executed results and renders HTML and Markdown by: + +```bash +make compile # execute notebooks (ensure outputs are up to date) +make html # build website as usual +make markdown # export markdown artifacts into _build/html/markdown +``` + +Then, the compiled results are forced pushed to [sgl-project.io](https://github.com/sgl-project/sgl-project.github.io) for rendering. In other words, sgl-project.io is push-only. All the changes of SGLang docs should be made directly in SGLang main repo, then push to the sgl-project.io.