Visuo-tactile world model · contact-aware loss

What the contact loss sees — and what the thresholds mean

The tactile diffusion target is dense MSE, which is ~blind to contact (contact is a tiny fraction of the signal). Two x0-space aux losses fix this by acting only on masked regions defined by the GT contact energy e = |z_tac − z_ref| averaged over the 16 latent channels, per 16×16 latent cell:

① contact reconstruction — masked MSE where e ≥ 0.35 (real contact): pull the prediction onto the GT contact patch.
② false-contact suppression — hinge where e ≤ 0.076 (real background): penalize predicted energy above the 0.11 margin, so the model can't hallucinate contact.
Both are α̅(t)-weighted (only applied at low-noise steps where x0 is meaningful) and added on top of the base objective.

Per-sample anatomy

contact mask (e ≥ 0.35) background mask (e ≤ 0.076) false-contact hit (pred energy > 0.11 in background)
sample 0 · sensor L · frame ti=4
sample 1 · sensor L · frame ti=4
sample 2 · sensor L · frame ti=4
sample 3 · sensor L · frame ti=3

Thresholds: latent ↔ image space

threshold (latent)role≈ pixel Δ (0–255)% of cells
0.076background0.660%
0.110false-contact margin1.033%
0.350contact4.67%

Each 16×16 latent cell maps to an 8×8 image patch (Cosmos is 8× spatial). The scatter plots every cell's latent energy (x) against the actual pixel change in its patch (y). The three dashed lines are the loss thresholds — read across to see how much visible contact each represents: 0.35 is a clearly visible contact patch, 0.076 is the background noise floor, 0.11 is the "don't hallucinate above this" margin.