mindspore.dataset.sync_wait_for_dataset

mindspore.dataset.sync_wait_for_dataset(rank_id, rank_size, current_epoch)[source]

Wait util the dataset files required by all devices are downloaded.

Note

It should be used together with mindspore.dataset.OBSMindDataset and be called before each epoch.

Parameters

rank_id (int) – Rank ID of the device.
rank_size (int) – Rank size.
current_epoch (int) – Number of current epochs.

Examples

>>> # Create a synchronization callback
>>>
>>> from mindspore.dataset import sync_wait_for_dataset
>>> from mindspore.train.callback import Callback
>>>
>>> class SyncForDataset(Callback):
...     def __init__(self):
...         super(SyncForDataset, self).__init__()
...     def epoch_begin(self, run_context):
...         cb_params = run_context.original_args()
...         epoch_num = cb_params.cur_epoch_num
...         sync_wait_for_dataset(rank_id, rank_size, epoch_num)