119 lines
3.8 KiB
Markdown
119 lines
3.8 KiB
Markdown
# TI Coding Agent Data Prep Skill
|
|
|
|
Use this repo for Open-SWE-Traces data preparation before coding-agent SFT experiments.
|
|
|
|
## Purpose
|
|
|
|
This project organizes the existing Open-SWE-Traces probing, native-trace filtering, and trajectory repurposing scripts into a Python project. It is intended for work on `nvidia/Open-SWE-Traces`, especially:
|
|
|
|
- inspecting dataset structure and source splits,
|
|
- auditing trajectories for SFT-quality hard filters,
|
|
- measuring Qwen tokenizer/token statistics,
|
|
- decomposing trajectories into task-oriented phases,
|
|
- exporting balanced ModelScope-SWIFT training/validation probes,
|
|
- experimenting with pi-mono-style conversion.
|
|
|
|
## Repository Layout
|
|
|
|
- `scripts/probing/`: dataset inspection, unique-count reports, tokenizer comparison, and token counting.
|
|
- `scripts/filtering/`: native trajectory audit and hard filtering.
|
|
- `scripts/repurposing/`: subproblem decomposition, coarse task-stage conversion, SWIFT export, and pi-mono conversion.
|
|
- `src/ti_coding_agent_data_prep/openswe/`: shared constants and path helpers.
|
|
- `data/`: local datasets; ignored by git.
|
|
- `runs/`: script outputs; ignored by git.
|
|
- `docs/`: retained legacy notes from the original probe workspace.
|
|
|
|
## Expected Data Location
|
|
|
|
By default scripts read:
|
|
|
|
```text
|
|
data/Open-SWE-Traces
|
|
```
|
|
|
|
On B300, prefer a symlink instead of copying the dataset:
|
|
|
|
```bash
|
|
ln -s /ssd/workspace/yi/openswetraces_probe/Open-SWE-Traces data/Open-SWE-Traces
|
|
```
|
|
|
|
## Environment
|
|
|
|
Use a repo-local environment. On B300, set proxy variables before installing or downloading:
|
|
|
|
```bash
|
|
export http_proxy=http://100.72.0.101:8888
|
|
export https_proxy=http://100.72.0.101:8888
|
|
export HTTP_PROXY=http://100.72.0.101:8888
|
|
export HTTPS_PROXY=http://100.72.0.101:8888
|
|
export HF_ENDPOINT=https://hf-mirror.com
|
|
uv venv .venv --python 3.10
|
|
source .venv/bin/activate
|
|
uv pip install -e '.[dev]'
|
|
```
|
|
|
|
## Main Entrypoints
|
|
|
|
Probe one sample per split:
|
|
|
|
```bash
|
|
python scripts/probing/inspect_sample.py
|
|
```
|
|
|
|
Generate unique-count and random-20 decomposition reports:
|
|
|
|
```bash
|
|
python scripts/probing/analyze_and_decompose.py
|
|
```
|
|
|
|
Run native hard-filter audit:
|
|
|
|
```bash
|
|
python scripts/filtering/audit_native_traces.py --input-root data/Open-SWE-Traces --output-dir runs/native_trace_audit
|
|
```
|
|
|
|
Run exact Qwen token counting:
|
|
|
|
```bash
|
|
python scripts/probing/count_qwen_tokens_exact_parallel.py --input-root data/Open-SWE-Traces --output runs/native_trace_audit/qwen_exact_token_count.json --model Qwen/Qwen3-32B --workers 12
|
|
```
|
|
|
|
Build the balanced 5k SWIFT training probe:
|
|
|
|
```bash
|
|
python scripts/repurposing/build_swift_training_probe_5k.py
|
|
```
|
|
|
|
Build the balanced 500-row validation probe:
|
|
|
|
```bash
|
|
python scripts/repurposing/build_swift_validation_500.py
|
|
```
|
|
|
|
Build the full hard-filter-kept SWIFT training set, approximately 190k rows with the current audit policy:
|
|
|
|
```bash
|
|
python scripts/repurposing/build_swift_full_kept.py --input-root data/Open-SWE-Traces --output-dir runs/training_full_kept_swift
|
|
```
|
|
|
|
Use `build_swift_full_kept.py` for real full-data SFT preparation. Use
|
|
`build_swift_training_probe_5k.py` plus `build_swift_validation_500.py` only for small probe runs and pipeline tests.
|
|
|
|
Try pi-mono-style conversion:
|
|
|
|
```bash
|
|
python scripts/repurposing/convert_openswe_to_pi_mono.py --input-root data/Open-SWE-Traces --output-root runs/pi_mono_converted
|
|
```
|
|
|
|
## Filtering Semantics
|
|
|
|
The hard filter rejects trajectories with malformed tool-call JSON, tool-call/tool-result mismatch, polluted tool names or arguments, empty or inconsistent final patches, excessive length, repeated no-progress loops, and unresolved off-track traces.
|
|
|
|
## Training Format Notes
|
|
|
|
For SWIFT exports:
|
|
|
|
- MiniMax trajectories are treated as thinking-mode data and `reasoning_content` is wrapped with `<think>...</think>`.
|
|
- Qwen trajectories are treated as non-thinking-mode data.
|
|
- `system`, `user`, and `tool` messages use `loss=false`; assistant messages use `loss=true`.
|