text_both_base_0728Checkpoint epoch004-global_step17000, on 5 val.jsonl and
5 train.jsonl clips. Every clip has generated audio as well as video —
unmute to judge the full result.
a video with {object};
the model must put the object and its sound back.a video without {object}; the model must take it out.--prepend-ref, stripped again before decoding);
no first frame forces it off (--no-prepend-ref).val.jsonl rows were held out;
train.jsonl rows were seen during training. A large train-over-val
advantage would mean memorisation rather than a learned edit.add
direction was found to overshoot at --cfg-scale 5.0 — the latents leave
the VAE's range and decode to a flat pink wash, monotonically in cfg and across seeds,
while remove survives it. This matrix is generated at cfg 1.0.with first frame is out-of-distribution for this checkpoint.
Training used ref_prepend_warmup_steps=5000 with
p_ref = max(0, 1 - step/5000), so from step 5,000 onward the anchor was
stripped from both target and source latents. At step 17,000 this checkpoint trained
anchor-free for 12,000 of its 17,000 steps — no first frame is the mode
that matches training.Mean of (L1 → target − L1 → input) over the 5 clips of each cell. Negative is good: the generation sits closer to the ground-truth target than to the clip it was given.