Separable vs non-separable spatio-temporal disaggregation

Simulation study results — 50-seed run plus an independent 10-seed replication.

Downloads

fernando-50seeds.zip
50-seed run · 2,250 comparisons · code + results + SUMMARY.md 221 KB fernando-results-10seeds.zip
10-seed run · 450 comparisons · code + results 67 KB SUMMARY.md
Full write-up — read this first 6 KB hyperparameters-50seeds.csv
Posterior hyperparameter summaries for all 4,500 fits, with true values and a coverage flag 4.2 MB fields-50seeds.tar.gz
The 150 simulated continuous fields (model inputs) — lets you regenerate datasets exactly 24.5 MB

Headline

Data were generated from the non-separable model, then fitted with both models on the identical dataset. The non-separable model achieves lower RMSE in 56.8% of 2,250 paired comparisons (95% CI 54.7–58.8%, sign test p = 1.6×10⁻¹⁰). Median difference +0.000195 — about 0.1% of an RMSE of ~0.20. Real, but small.

The 10-seed run overstated the effect. It gave 64.0% [59.4–68.4], which lies outside the 50-seed interval. Config 1's effect disappeared entirely: 58.7% (p=0.041) → 50.5% (p=0.80).

By config

config10-seed50-seedp (50-seed)
1 (rt=2)58.7%50.5%0.80
2 (rt=6)58.0%54.8%0.0095
3 (rt=24)75.3%64.9%2×10⁻¹⁶
overall64.0%56.8%1.6×10⁻¹⁰

Spatial aggregation dominates

Non-separable win rate by spatial coarsening (n=450 each) — monotone, crossing 50% between nsf 4 and 6:

nsf23468
NS wins86.9%68.0%59.3%37.1%32.4%

Temporal coarsening matters far less: ntf 2/3/4 → 60.4% / 57.3% / 52.5%.

The non-separable model is occasionally catastrophic at nsf=8

The mean difference is negative (−0.0033) while the median is positive — a heavy left tail. In five cases the non-separable fit gave RMSE of 1.2–3.0 against the separable model's ~0.24, roughly 10× worse. Ten of the twelve extreme cases favour separable, all at nsf=8.

Quote the sign test and median, not the mean. The paired t-test is non-significant (p=0.073) purely because the tail dominates the mean. 10% trimmed mean = +0.000221, close to the median.

The only two INLA convergence failures in the whole run were also at nsf=8 (2 of 350 fits there; 0 of 1,404 elsewhere). The cells that fail to converge are the cells that blow up.

Method changes

The warm start (control.mode) was removed — measured at 1.80× slower than a cold fit on the identical dataset (893.1 s vs 495.7 s), with no effect on where the optimiser converges. A bug in runa_s.R was fixed: a bare load() overwrote the separable chain's theta.ini with the non-separable one's. Field generation is now seeded. Details in SUMMARY.md.