20250923 meeting

This week, the Weather2K dataset was used. After preprocessing, it was confirmed that the dataset contains no missing values. The data spans from January 2017 to August 2021, with a recording frequency of every 3 hours, resulting in 13,632 time steps across 2,130 observation stations, all recorded in China Standard Time (CST, UTC+8). The dataset originates from the China Meteorological Administration (CMA) ground weather stations, collected in compliance with the standards of Specifications for Surface Meteorological Observation—General (GB/T 35221-2017) and Quality Control of Surface Meteorological Observation Data (QX/T 118-2010).

According to the original paper, the full dataset, named Weather2K-N, contains all weather station data but was not released due to confidentiality. The open-source version, Weather2K-R, is stored in NumPy format with a shape of (1866, 13, 13632). In addition, the paper provides a special version, Weather2K-S, which includes data from 15 representative weather stations distributed across different regions, stored in CSV format.

In this experiment, Weather2K-R was used. Its stored variables are as follows:

After simple data preprocessing, the following summary statistics were obtained:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

shape = data_train_known_real.shape
data_slice = data_train_known_real.reshape(shape[0], shape[1] * shape[2])
df = pd.DataFrame(data_slice)

summary_df = pd.DataFrame({
    'min': df.min(axis=1),
    'max': df.max(axis=1),
    'range': df.max(axis=1) - df.min(axis=1),
    'mean': df.mean(axis=1),
    'median': df.median(axis=1),
    'std': df.std(axis=1),
    'nan_count': df.isna().sum(axis=1),
    'mode': df.mode(axis=1).iloc[:, 0]
})

row_names = [
    'Air pressure (ap, hpa)',
    'Air Temperature (t, °C)',
    'Maximum temperature (mxt, °C)',
    'Minimum temperature (mnt, °C)',
    'Relative humidity (rh, %)',
    'Precipitation in 3h (p3, mm)',
    'Wind direction (wd, °)',
    'Wind speed (ws, ms^-1)',
    'Maximum wind direction (mwd, °)',
    'Maximum wind speed (mws, ms^-1)'
]

summary_df.index = row_names
pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)

print(summary_df)

The time span for this experiment is March 5, 2021 00:00 to July 26, 2021 21:00, covering 1152 time steps, providing sufficient historical information for the model to capture both seasonal and daily variations.

The test period is July 27, 2021 00:00 to August 31, 2021 21:00, with 288 time steps. Among them, August 24, 2021 21:00 to August 31, 2021 21:00 contains 57 missing steps, which serve as a challenge to the model’s spatial imputation and temporal forecasting capability.

The dataset contains 1492 known stations for training and validation, and 374 unknown stations requiring prediction or imputation. The experiment covers both time-series forecasting and spatial interpolation to evaluate model performance in a multi-station, multivariate environment.

1
2
3
4
5
6
7
8
9

# config for time series
time_duration = 1440  # 8 times a day * 30 days * 6 months = 1440
data = data[:, -time_duration:, :]
time_index = time_index[-time_duration:]

# config for split & save
ratio_station_known = 0.8  # ratio of known stations in training set
ratio_time_past = 0.8  # ratio of known stations in training set
ratio_missing = 0.2

Due to the large scale differences among variables, each time series was standardized before training. For example, using RegressionEnsemble, when applying time-series forecasting and spatial imputation with autoFRK, variables such as relative humidity, wind direction, and maximum wind speed/direction showed significantly higher mean squared prediction errors (MSPE). These variables should therefore be excluded in subsequent experiments.

References

• Zhu X, Xiong Y, Wu M, et al. Weather2K: A Multivariate Spatio-Temporal Benchmark Dataset for Meteorological Forecasting Based on Real-Time Observation Data from Ground Weather Stations[C]//International Conference on Artificial Intelligence and Statistics. PMLR, 2023: 2704-2722.