mindearth.data.Era5Data

class mindearth.data.Era5Data(data_params, run_mode='train', kno_patch=False)[source]

This class is used to process ERA5 re-analyze data, and is used to generate the dataset generator supported by MindSpore. This class inherits the Data class.

Parameters

data_params (dict) – dataset-related configuration of the model.
run_mode (str, optional) – whether the dataset is used for training, evaluation or testing. Supports [“train”, “test”, “valid”]. Default: 'train'.
kno_patch (bool, optional) – Indicates whether the data is already partitioned into patches. If True, the data is assumed to be pre-processed and no further patching is performed. If False, the data will be processed into patches as per the specified parameters. Default: False.

Supported Platforms:: Ascend GPU

Examples

>>> from mindearth.data import Era5Data
>>> data_params = {
...     'name': 'era5',
...     'root_dir': './dataset',
...     'feature_dims': 69,
...     't_in': 1,
...     't_out_train': 1,
...     't_out_valid': 20,
...     't_out_test': 20,
...     'valid_interval': 1,
...     'test_interval': 1,
...     'train_interval': 1,
...     'pred_lead_time': 6,
...     'data_frequency': 6,
...     'train_period': [2015, 2015],
...     'valid_period': [2016, 2016],
...     'test_period': [2017, 2017],
...     'patch': True,
...     'patch_size': 8,
...     'batch_size': 8,
...     'num_workers': 1,
...     'grid_resolution': 1.4,
...     'h_size': 128,
...     'w_size': 256
... }
>>> dataset_generator = Era5Data(data_params)