mindspore.dataset.DSCallback
- class mindspore.dataset.DSCallback(step_size=1)[source]
Abstract base class used to build dataset callback classes.
Users can obtain the dataset pipeline context through ds_run_context , including cur_epoch_num , cur_step_num_in_epoch and cur_step_num .
- Parameters
step_size (int, optional) – The number of steps between adjacent ds_step_begin/ds_step_end calls. Default:
1
, will be called at each step.
Examples
>>> import mindspore.dataset as ds >>> from mindspore.dataset import DSCallback >>> >>> class PrintInfo(DSCallback): ... def ds_begin(self, ds_run_context): ... print("callback: start dataset pipeline", flush=True) ... ... def ds_epoch_begin(self, ds_run_context): ... print("callback: epoch begin, we are in epoch", ds_run_context.cur_epoch_num, flush=True) ... ... def ds_epoch_end(self, ds_run_context): ... print("callback: epoch end, we are in epoch", ds_run_context.cur_epoch_num, flush=True) ... ... def ds_step_begin(self, ds_run_context): ... print("callback: step begin, step", ds_run_context.cur_step_num_in_epoch, flush=True) ... ... def ds_step_end(self, ds_run_context): ... print("callback: step end, step", ds_run_context.cur_step_num_in_epoch, flush=True) >>> >>> dataset = ds.GeneratorDataset([1, 2], "col1", shuffle=False, num_parallel_workers=1) >>> dataset = dataset.map(operations=lambda x: x, callbacks=PrintInfo()) >>> >>> # Start dataset pipeline >>> iterator = dataset.create_tuple_iterator(num_epochs=2) >>> for i in range(2): ... for d in iterator: ... pass callback: start dataset pipeline callback: epoch begin, we are in epoch 1 callback: step begin, step 1 callback: step begin, step 2 callback: step end, step 1 callback: step end, step 2 callback: epoch end, we are in epoch 1 callback: epoch begin, we are in epoch 2 callback: step begin, step 1 callback: step begin, step 2 callback: step end, step 1 callback: step end, step 2 callback: epoch end, we are in epoch 2
- ds_begin(ds_run_context)[source]
Called before the data pipeline is started.
- Parameters
ds_run_context (RunContext) – Include some information of the data pipeline.
- ds_epoch_begin(ds_run_context)[source]
Called before a new epoch is started.
- Parameters
ds_run_context (RunContext) – Include some information of the data pipeline.
- ds_epoch_end(ds_run_context)[source]
Called after an epoch is finished.
- Parameters
ds_run_context (RunContext) – Include some information of the data pipeline.
- ds_step_begin(ds_run_context)[source]
Called before a step start.
- Parameters
ds_run_context (RunContext) – Include some information of the data pipeline.
- ds_step_end(ds_run_context)[source]
Called after a step finished.
- Parameters
ds_run_context (RunContext) – Include some information of the data pipeline.