Add SWIFT coding agent probe experiment scripts

This commit is contained in:
Codex
2026-06-24 21:57:11 +08:00
parent 73e9752331
commit 4063246ac2
3 changed files with 53 additions and 5 deletions

View File

@@ -29,13 +29,37 @@ git submodule update --init --recursive
./scripts/setup_env.sh
```
脚本会显式设置 B300 代理:
脚本会显式设置 B300 代理和国内 PyPI 源
```bash
http://100.72.0.101:8888
https://mirrors.aliyun.com/pypi/simple/
```
Python 依赖安装在仓库内 `.venv`,不会写入系统 Python。
Python 依赖安装在仓库内 `.venv`,不会写入系统 Python。安装逻辑是:
1. 先在 `.venv` 内安装 `uv`
2. 用 SWIFT 官方推荐的 `uv pip install -e third_party/modelscope-swift --torch-backend=auto` 安装 SWIFT 源码依赖。
3. 再安装本 probe repo。
4. 默认额外补充 `deepspeed<0.19``liger-kernel``tensorboard``nvitop`
如果要安装 SWIFT 的全量 optional 依赖:
```bash
INSTALL_SWIFT_ALL=1 ./scripts/setup_env.sh
```
如果只想安装 SWIFT core不补额外训练包
```bash
INSTALL_TRAINING_EXTRAS=0 ./scripts/setup_env.sh
```
如果要追加其他包:
```bash
EXTRA_UV_PACKAGES="flash-attn==2.8.3" ./scripts/setup_env.sh
```
## 下载数据集

View File

@@ -10,6 +10,9 @@ Use this skill when the user wants to run or modify the TI coding-agent SFT prob
- `https_proxy=http://100.72.0.101:8888`
- `HF_ENDPOINT=https://hf-mirror.com`
- Do not install Python packages globally. Use the repository-local `.venv` created by `scripts/setup_env.sh`.
- `scripts/setup_env.sh` installs `uv` inside `.venv`, then installs SWIFT from the submodule using SWIFT's documented source-install path: `uv pip install -e third_party/modelscope-swift --torch-backend=auto`.
- Default setup adds training extras: `deepspeed<0.19`, `liger-kernel`, `tensorboard`, and `nvitop`. Disable with `INSTALL_TRAINING_EXTRAS=0`.
- Install SWIFT optional all-dependencies only when explicitly needed: `INSTALL_SWIFT_ALL=1 ./scripts/setup_env.sh`.
- Do not start GPU training before checking GPU occupancy with `nvidia-smi`.
- Do not commit or upload `data/`, `models/`, `outputs/`, `runs/`, or `logs/`.

View File

@@ -9,12 +9,33 @@ export https_proxy="${https_proxy:-http://100.72.0.101:8888}"
export HTTP_PROXY="${HTTP_PROXY:-${http_proxy}}"
export HTTPS_PROXY="${HTTPS_PROXY:-${https_proxy}}"
export PIP_INDEX_URL="${PIP_INDEX_URL:-https://mirrors.aliyun.com/pypi/simple/}"
export UV_DEFAULT_INDEX="${UV_DEFAULT_INDEX:-${PIP_INDEX_URL}}"
export UV_INDEX_URL="${UV_INDEX_URL:-${PIP_INDEX_URL}}"
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -U pip setuptools wheel
python -m pip install -e .
python -m pip install -e third_party/modelscope-swift
python -m pip install -U pip setuptools wheel uv
UV=(python -m uv pip)
"${UV[@]}" install -U pip setuptools wheel
if [[ "${INSTALL_SWIFT_ALL:-0}" == "1" ]]; then
"${UV[@]}" install -e 'third_party/modelscope-swift[all]' --torch-backend=auto
else
"${UV[@]}" install -e third_party/modelscope-swift --torch-backend=auto
fi
"${UV[@]}" install -e .
if [[ "${INSTALL_TRAINING_EXTRAS:-1}" == "1" ]]; then
"${UV[@]}" install -U 'deepspeed<0.19' liger-kernel tensorboard nvitop
fi
if [[ -n "${EXTRA_UV_PACKAGES:-}" ]]; then
# shellcheck disable=SC2206
EXTRA_PACKAGES=(${EXTRA_UV_PACKAGES})
"${UV[@]}" install -U "${EXTRA_PACKAGES[@]}"
fi
mkdir -p data/raw data/processed models outputs runs logs
python - <<'PY'