REPRO_CHECKS.md
2.0 KB · 22 lines · markdown Raw
1 # Reproducibility checks (2026-11-13, before upload)
2
3 Setup, as on a new server: the code copied to a clean directory (no access to the original `vtm_cosmos/` tree), a **fresh**
4 conda env built by `scripts/setup_env.sh` (`ENV_NAME=acts_so_test`: torch 2.10.0+cu128, cuDNN 91002, imageio-ffmpeg 0.6.0),
5 cosmos-framework freshly cloned @ 3f94d25, `nvidia/Cosmos3-Nano` and `Wan2.2_VAE.pth` freshly downloaded with
6 `scripts/download.py --what nano wan_vae`, `data/` and `checkpoints/` = exactly the files uploaded to this repo. 1× H100 80 GB per job.
7
8 | check | what | result |
9 |---|---|---|
10 | training smoke, all 8 configs | `NGPU=1 scripts/train.sh configs/<cfg>.yaml run_name=smoke_<cfg> training.max_steps=30` | all rc=0, train + val pass, 30.5 GiB peak |
11 | windows | train / val windows per task | mb 71,386 / 109, pushT 55,199 / 92, rope 77,188 / 126 (= original runs) |
12 | 884599 config | action normaliser vs `runs/nano_action_causal_umi_sensoronly_noforce_v1` (job 884599); trainable params | byte-identical; 45,301,952 (view_action2llm9 frozen) |
13 | TODO 1a / 2a (sensor-only, force) | normaliser vs pusht_v1 / rope_v1 | identical (pushT); rope differs from rope_noforce only at `dp_y`, as intended |
14 | TODO 1b / 2b (sensor-only, no force) | normaliser vs the earlier `pusht_noforce_v1` / `rope_noforce_v1` runs | byte-identical / identical |
15 | inference pusht_v1 e019 | `eval/run_test_case.py --task pusht`: case 0 (short + 15-window long), cases 1-3 short | PSNR/SSIM/LPIPS Δ = 0 vs `eval/cases/pusht_reference_metrics.json` |
16 | inference rope_v1 e019 | same, `--task rope` | Δ = 0 |
17 | inference 884599 e013 | same, `--task umi_sensoronly_noforce` | Δ = 0 vs the original cluster eval of epoch_013 |
18 | summarize | `eval/summarize.py --task … --reference` | runs; paired Δ = 0.000 |
19
20 Not re-checked in this round (unchanged code, checked for the 11-08 release `WitneyWW/acts-world-model`): re-encoding latents from
21 `data/raw` (bit-identical) and rebuilding the dforce actions of all 166 episodes from `data/raw/parquet` (bit-identical).
22