OpenSWETraces_cleanup/changelog.md

Changelog

This file records strategy changes that materially affect dataset decisions or training-data semantics. Generated audit manifests are not treated as stable API contracts.

3.0.1 - 2026-08-18

CPU profiling

• Replaced directory-level Python thread concurrency with Parquet shard-level processes so CPU-bound tokenization and metric extraction bypass the GIL.
• Each process streams batches of 16 records to an atomic shard part; the parent merges completed parts in source order without loading them into memory.
• profile --resume now reuses completed shard parts, while interrupted temporary parts are safely recomputed.
• Review sampling uses stored Parquet provenance and reads only shards that contain selected trajectories.

3.0.0 - 2026-08-18

Strategy replacement

• Removed the GLM client, prompts, schemas, model-based classification, boundary selection, quality scoring, repair planning, and repair review.
• Replaced model judgments with deterministic length, token, tool-failure, failure-position, structure, and streak metrics.
• Added explicit 80K, 128K, 256K, and over-256K context buckets.
• Added dataset-level quantiles, token-tail contribution statistics, p99 tool error outlier detection, and stratified human-review sampling.
• Reserved automatic rejection for broken tool structure, five consecutive failures, or a failure rate of at least 50%. Distributed failures, relative error outliers, and early failure clusters require review.
• Token counts now require an explicit local tokenizer.json; the code never substitutes a character-based approximation.
• Reduced the public CLI to profile, summarize, and sample.

Motivation

• Multi-call LLM auditing was operationally unstable and could hallucinate safe boundaries or miss incorrect final patches.
• Model-generated trajectory repair had no deterministic invariant capable of proving that the rewritten trace remained correct.
• The new pipeline is reproducible, inspectable, CPU-only, and does not mutate source trajectories.

2.1.0 - 2026-08-09

Strategy correction

• Removed the unconditional cutoff before the first state-changing turn for failed and unverified trajectories. A write or edit is not evidence of an error by itself, and this cap discarded normal implementation, recovery, and verification work.
• The semantic boundary now remains the earliest unrecovered major or critical assistant behavior selected by the boundary stage. The isolated prefix-quality stage remains the safety gate. The first stateful turn is retained as telemetry for paired evaluation against the former policy.

2.0.0 - 2026-08-07

Strategy

• Replaced the single-call trajectory audit with two isolated model calls: boundary selection followed by prefix-only quality scoring.
• The boundary call receives an outcome-blind and reference-patch-blind view. It does receive the trajectory's own final model patch to check coverage and final diff pollution. Long trajectories are presented as ordered blocks targeting 32 turns, with absolute turn IDs and immediate assistant/tool-result pairs kept together.
• Python materializes the exact prefix before the selected assistant turn. The quality call is then built again from that prefix, so it cannot see the removed suffix, external outcome, model patch, reference patch, or boundary rationale.
• Simplified both prompts and added concrete positive, recovery, and late-claim examples.
• Replaced the weighted six-dimension score and issue penalties with five equally weighted 0-20 dimensions. Python computes the final 0-100 score and tier.
• Distinguished a safe full process trajectory (KEEP_FULL) from an uncertain trajectory (HOLD) so the boundary stage does not reject useful data merely because no truncation is needed.
• Kept an explicit list of erroneous and inefficient assistant actions as a quality surrogate; Python recomputes its counts locally.
• Each listed issue records whether it was recovered. Python rejects a model response that claims a prefix is safe while also listing an unrecovered major/critical issue.

Validation

• A boundary must be an assistant turn inside the selected block.
• Boundary evidence must include that assistant turn ID; Python attaches the exact immutable source excerpt rather than trusting model-copied quotes.
• The model does not repeat a block ID or prefix-safe boolean. Python derives the containing block from the validated boundary turn, while the isolated second call judges the materialized prefix itself.
• KEEP_FULL now requires explicit review of task coverage, exact interfaces, final repository state, user scope constraints, and claim/tool consistency; passing tests alone are not treated as proof.
• Complete-trajectory scoring receives the final model patch, while truncated prefix scoring remains suffix-isolated.
• Failed and unverified (resolved=0/-1) records are capped before the first statically detected state-changing assistant turn. The earlier of that cap and the GLM boundary is materialized, preferring shorter safe investigation traces over late cuts.
• Removed redundant model-written prefix-validity booleans. Python derives safety directly from unrecovered major/critical behavior issues.
• A prefix scorer cannot cite a turn outside the materialized prefix.
• HIGH, MEDIUM, LOW, and REJECT are computed locally rather than trusted from model output.

Stress pilot

• A difficulty-weighted 15-record set was used to expose late-boundary errors; it is not an estimate of the full dataset acceptance rate.
• Raw semantic boundaries were often too late or incorrectly returned KEEP_FULL. After applying the non-success stateful cap, two independent human reviewers found all 15 effective prefixes safe and no late boundary.
• Retained-turn ratios ranged from 11.14% to 39.74%, with an 18.73% median. The final quality distribution was 10 HIGH and 5 MEDIUM.
• The final low-reasoning, five-worker run completed 15/15 records without an error in 90.05 seconds. High-reasoning mode was substantially slower and had gateway timeouts, so it is not the practical batch default.

Motivation

• In the previous 500-record pilot, 360 of 423 prefix candidates included suffix behavior in the score despite prompt instructions to score only the retained prefix.
• Manual stress review found useful conservative cutoffs, but also late cutoffs that retained failed patches and normal experiments incorrectly treated as permanent errors.
• Prompt wording alone could not guarantee suffix isolation, so the new design enforces it in the data passed to each call.

1.0.0 - 2026-08-06

• Initial static classification, one-call trajectory audit, allowlisted repair planning, deterministic repair application, and independent repair review.
• Added API retries and concurrent JSONL processing.