mindspore.dataset.DSCallback

class mindspore.dataset.DSCallback(step_size=1)

Abstract base class used to build dataset callback classes.

Users can obtain the dataset pipeline context through ds_run_context , including cur_epoch_num , cur_step_num_in_epoch and cur_step_num .

Parameters

step_size (int, optional) – The number of steps between adjacent ds_step_begin/ds_step_end calls. Default: 1, will be called at each step.

Examples

>>> import mindspore.dataset as ds
>>> from mindspore.dataset import DSCallback
>>>
>>> class PrintInfo(DSCallback):
...     def ds_begin(self, ds_run_context):
...         print("callback: start dataset pipeline", flush=True)
...
...     def ds_epoch_begin(self, ds_run_context):
...         print("callback: epoch begin, we are in epoch", ds_run_context.cur_epoch_num, flush=True)
...
...     def ds_epoch_end(self, ds_run_context):
...         print("callback: epoch end, we are in epoch", ds_run_context.cur_epoch_num, flush=True)
...
...     def ds_step_begin(self, ds_run_context):
...         print("callback: step begin, step", ds_run_context.cur_step_num_in_epoch, flush=True)
...
...     def ds_step_end(self, ds_run_context):
...         print("callback: step end, step", ds_run_context.cur_step_num_in_epoch, flush=True)
>>>
>>> dataset = ds.GeneratorDataset([1, 2], "col1", shuffle=False, num_parallel_workers=1)
>>> dataset = dataset.map(operations=lambda x: x, callbacks=PrintInfo())
>>>
>>> # Start dataset pipeline
>>> iterator = dataset.create_tuple_iterator(num_epochs=2)
>>> for i in range(2):
...     for d in iterator:
...         pass
callback: start dataset pipeline
callback: epoch begin, we are in epoch 1
callback: step begin, step 1
callback: step begin, step 2
callback: step end, step 1
callback: step end, step 2
callback: epoch end, we are in epoch 1
callback: epoch begin, we are in epoch 2
callback: step begin, step 1
callback: step begin, step 2
callback: step end, step 1
callback: step end, step 2
callback: epoch end, we are in epoch 2

ds_begin(ds_run_context)

Called before the data pipeline is started.

Parameters

ds_run_context (RunContext) – Include some information of the data pipeline.

ds_epoch_begin(ds_run_context)

Called before a new epoch is started.

Parameters

ds_run_context (RunContext) – Include some information of the data pipeline.

ds_epoch_end(ds_run_context)

Called after an epoch is finished.

Parameters

ds_run_context (RunContext) – Include some information of the data pipeline.

ds_step_begin(ds_run_context)

Called before a step start.

Parameters

ds_run_context (RunContext) – Include some information of the data pipeline.

ds_step_end(ds_run_context)

Called after a step finished.

Parameters

ds_run_context (RunContext) – Include some information of the data pipeline.