20260224 meeting

Background

This experiment uses the same MERRA-2 dataset as the previous session (20260224 meeting).

In the previous experiment, surface temperature was used as the sample variable, while in this experiment, ozone concentration is used. The diffusion parameters are set as follows:

• T: 500
• beta_0: 0.0008
• beta_T: 0.08

All experiments in this session use the above parameters. Other parameters remain unchanged except for s4_dropout, which is adjusted to 0.3, 0.5, and 0.8 to evaluate how different dropout rates affect model prediction performance. Additionally, three different iteration counts, 500, 4,000, and 6,000, are used to examine whether dropout rates influence prediction accuracy under different training iterations.

Naming Scheme

Experiments are named using the format XXX-XXXX-XX, where XXX means whether $autoFRK$ is used during training; XXXX for number of training iterations; XX means whether spatial standardization is applied. For example, NoFRK-4000-NoSP means no $autoFRK$ during training, 4,000 iterations, no spatial normalization during inference.

s4_dropout 0.3

FRK-500-SP

FRK-4000-SP

FRK-6000-SP

NoFRK-500-SP

NoFRK-4000-SP

NoFRK-6000-SP

s4_dropout 0.5

FRK-4000-SP

NoFRK-4000-SP

s4_dropout 0.8

FRK-500-SP

FRK-4000-SP

FRK-6000-SP

NoFRK-500-SP

NoFRK-4000-SP

NoFRK-6000-SP

Conclusion

The conclusions of this experiment are as follows:

• Best Performance

  The best result appears in the combination of Dropout 0.8 and 4,000 iterations (FRK-4000-SP), achieving a full spatiotemporal MSPE of 271.44.

• Key Mechanism

  A high dropout rate (0.8) effectively suppresses overfitting to meteorological noise, significantly enhancing the model’s generalization ability for future predictions.

• Convergence Characteristics

  For the more complex ozone dataset, 4,000 iterations serve as the performance saturation point. In contrast, 500 iterations are entirely insufficient for diffusion-model convergence, resulting in numerical divergence.

Effect of s4_dropout

By fixing the iteration count at 4,000, we compare how different s4_dropout rates influence prediction performance:

As dropout increases from 0.3 to 0.8, the overall MSPE decreases by approximately 5.6%. This indicates that ozone concentration exhibits substantial stochastic perturbations, and applying a strong dropout rate forces the S4 structure to learn more intrinsic spatiotemporal patterns. Moreover, Dropout 0.8 achieves the lowest error across all Future prediction metrics (both known and unknown locations), confirming that although a high dropout rate slows training convergence, it yields improved performance on the test set.

Effect of Iterations

We compare different iteration counts (500, 4,000, 6,000) under the same dropout setting:

• 500 Iterations: Numerical Explosion

  • Regardless of FRK or NoFRK configurations, the MSPE at 500 iterations is abnormally high.
  • Ozone concentration exhibits far more complex value distributions and spatiotemporal dynamics compared to temperature data. In diffusion models, 500 iterations are insufficient for learning the reverse-diffusion denoising process, causing generated values to deviate from physically meaningful magnitudes.

• 4,000 vs. 6,000 Iterations: Diminishing Returns

  Using Dropout 0.8 as an example:

  • 4,000 Iterations: MSPE = 271.44
  • 6,000 Iterations: MSPE = 274.14
  • Observation: Increasing to 6,000 iterations slightly worsens performance. This suggests that the model reaches its optimal convergence around 4,000 iterations, and further training may cause the model to start memorizing specific noise patterns in the training set.

References

• Zhu X, Xiong Y, Wu M, et al. Weather2K: A Multivariate Spatio-Temporal Benchmark Dataset for Meteorological Forecasting Based on Real-Time Observation Data from Ground Weather Stations[C]//International Conference on Artificial Intelligence and Statistics. PMLR, 2023: 2704-2722.
• Juan Lopez Alcaraz, Nils Strodthoff (2022). Diffusion-based time series imputation and forecasting with structured state space models. Transactions on Machine Learning Research. Retrieved from https://openreview.net/forum?id=hHiIbk7ApW
• SSSD (2022). GitHub. Retrieved from https://github.com/AI4HealthUOL/SSSD