ti_coding_agent_probe/SKILL.md

TI Coding Agent Probe Skill

Use this skill when the user wants to run or modify the TI coding-agent SFT probe experiments based on the ModelScope dataset eigentom/ti_coding_agent_training_probe_20260624. Hugging Face remains available as a fallback backend.

Repository Contract

• Work in /ssd/workspace/yi/ti_coding_agent_probe on B300 unless the user says otherwise.
• Use the B300 proxy for all network access:
  • http_proxy=http://100.72.0.101:8888
  • https_proxy=http://100.72.0.101:8888
  • HF_ENDPOINT=https://hf-mirror.com
• Do not install Python packages globally. Use the repository-local .venv created by scripts/setup_env.sh.
• scripts/setup_env.sh installs uv inside .venv, then installs SWIFT from the submodule using SWIFT's documented source-install path: uv pip install -e third_party/modelscope-swift --torch-backend=auto.
• Default setup adds training extras: deepspeed<0.19, liger-kernel, tensorboard, and nvitop. Disable with INSTALL_TRAINING_EXTRAS=0.
• Install SWIFT optional all-dependencies only when explicitly needed: INSTALL_SWIFT_ALL=1 ./scripts/setup_env.sh.
• Do not start GPU training before checking GPU occupancy with nvidia-smi.
• Do not commit or upload data/, models/, outputs/, runs/, or logs/.

Key Commands

Initialize the repo and environment:

git submodule update --init --recursive
./scripts/setup_env.sh

Download the training/eval probe dataset from ModelScope:

export MODELSCOPE_API_TOKEN=<modelscope-api-key>
./scripts/download_dataset.py

The downloader writes:

• data/raw/training_probe/{train,validation}.jsonl.gz
• data/raw/training_probe/{train,validation}.parquet
• data/processed/training_probe/{train,validation}.jsonl
• data/processed/training_probe/{train,validation}.parquet

Use Hugging Face fallback only when requested:

export DATASET_BACKEND=huggingface
export HF_ENDPOINT=https://hf-mirror.com
export HF_DATASET_REPO_ID=<owner>/ti_coding_agent_training_probe_20260624
./scripts/download_dataset.py

Download base models:

./scripts/download_models.sh

Run the full ordered experiment:

./scripts/run_all_experiments.sh

Run only one stage:

./scripts/train_qwen35_9b_lora.sh
./scripts/train_qwen35_9b_full.sh
./scripts/train_qwen36_27b_lora.sh
./scripts/train_qwen36_27b_full.sh

Open TensorBoard:

./scripts/tensorboard.sh

Training Semantics

The dataset uses SWIFT-style chat messages. system, user, and tool messages should remain masked with loss=false; only assistant trajectories should contribute to loss. This keeps scaffold prompts and tool outputs as conditioning context rather than targets to memorize.

The default experiment uses:

• 1 epoch
• LoRA rank 32 for LoRA runs
• bf16 full fine-tuning for full runs
• max_length=262144
• checkpoint save every 1000 steps
• validation every 1000 steps
• TensorBoard logging under runs/

Override model IDs or paths with:

export QWEN35_9B_MODEL_ID=<hf-id>
export QWEN36_27B_MODEL_ID=<hf-id>
export QWEN35_9B_MODEL_PATH=<local-path>
export QWEN36_27B_MODEL_PATH=<local-path>