# Model Building

This folder owns construction of the tokenizer-swapped base model.

It contains only the tokenizer swap v2 algorithm. Dataset construction, training, and evaluation live in the other workflow folders.

## Main Files

```text
build_qwen3_dsv4_remap_checkpoint_v2.py
run_remap_v2.sh
```

## Inputs

The remap script needs:

- source Qwen model checkpoint
- source Qwen tokenizer
- target DSV4 tokenizer

Default paths in `run_remap_v2.sh` are environment-variable driven and can be overridden:

```bash
BASE_MODEL=/path/to/Qwen3-0.6B \
DSV_TOKENIZER=/path/to/dsv4_tokenizer \
OUT=/path/to/output_checkpoint \
bash model_building/run_remap_v2.sh
```

## Output

By default, generated checkpoints go to:

```text
model_building/generated_models/
```

This directory is ignored by git. Do not commit checkpoint weights.

## Algorithm Summary

The v2 remap builds DSV4-sized input embedding and LM-head matrices from the source Qwen checkpoint.

For each DSV4 token row, initialization is selected in this priority order:

1. Exact same token surface exists in the Qwen vocab.
2. Functional special-token mapping is available, such as DSV BOS to Qwen `<|im_start|>` and DSV EOS to Qwen EOS.
3. Byte-level token can be decoded, re-tokenized with Qwen, and initialized by averaging the corresponding Qwen rows.
4. Raw token decomposition can be tokenized with Qwen and averaged.
5. Global embedding/head mean fallback.

The script writes the remapped checkpoint plus `tokenizer_remap_v2_report.json` for auditability.

## Output Contract

The output checkpoint is consumed by `model_training/` scripts as `MODEL`.