mindspore.SummaryRecord

class mindspore.SummaryRecord(log_dir, file_prefix='events', file_suffix='_MS', num_process=32, network=None, max_file_size=None, raise_exception=False, export_options=None)

SummaryRecord is used to record the summary data and lineage data.

The API will create a summary file and lineage files lazily in a given directory and writes data to them. It writes the data to files by executing the record method. In addition to recording the data bubbled up from the network by defining the summary operators, SummaryRecord also supports to record extra data which can be added by calling add_value.

Note

1. Make sure to close the SummaryRecord at the end, otherwise the process will not exit. Please see the Example section below to learn how to close properly in two ways.

2. Only one SummaryRecord instance is allowed at a time, otherwise it will cause data writing problems.

3. SummaryRecord only supports Linux systems.

4. The Summary is not supported when compile source with -s on option.

Parameters

• log_dir (str) – The log_dir is a directory location to save the summary.

• file_prefix (str) – The prefix of file. Default: "events" .

• file_suffix (str) – The suffix of file. Default: "_MS" .

• num_process (int) – Number of processes saving summary data. The more processes there are, the better the performance, but there may be host memory overflow issues. Default: 32 .

• network (Cell) – Obtain a pipeline through network for saving graph summary. Default: None .

• max_file_size (int, optional) – The maximum size of each file that can be written to disk (in bytes). For example, to write not larger than 4GB, specify max_file_size=4*1024**3. Default: None , which means no limit.

• raise_exception (bool, optional) – Sets whether to throw an exception when a RuntimeError or OSError exception occurs in recording data. Default: False , this means that error logs are printed and no exception is thrown.

• export_options (Union[None, dict]) –

  Perform custom operations on the export data. Note that the size of export files is not limited by the max_file_size. You can customize the export data with a dictionary. For example, you can set {‘tensor_format’: ‘npy’} to export tensor as npy file. The data that supports control is shown below. Default: None , it means that the data is not exported.

  • tensor_format (Union[str, None]): Customize the export tensor format. Supports [“npy”, None]. Default: None , it means that the tensor is not exported.

    • npy: export tensor as npy file.

Raises

• TypeError – max_file_size is not int or file_prefix and file_suffix is not string.

• ValueError – The Summary is not supported, please without -s on and recompile source.

Examples

>>> import mindspore as ms
>>> if __name__ == '__main__':
...     # use in with statement to auto close
...     with ms.SummaryRecord(log_dir="./summary_dir") as summary_record:
...         pass
...
...     # use in try .. finally .. to ensure closing
...     try:
...         summary_record = ms.SummaryRecord(log_dir="./summary_dir")
...     finally:
...         summary_record.close()

add_value(plugin, name, value)

Add value to be recorded later.

Parameters

• plugin (str) –

  The plugin of the value.

  • graph: the value is a computational graph.

  • scalar: the value is a scalar.

  • image: the value is an image.

  • tensor: the value is a tensor.

  • histogram: the value is a histogram.

  • train_lineage: the value is a lineage data for the training phase.

  • eval_lineage: the value is a lineage data for the evaluation phase.

  • dataset_graph: the value is a dataset graph.

  • custom_lineage_data: the value is a customized lineage data.

  • LANDSCAPE: the value is a landscape.

• name (str) – The value of the name.

• value (Union[Tensor, GraphProto, TrainLineage, EvaluationLineage, DatasetGraph, UserDefinedInfo, LossLandscape]) –

  The value to store.

  • The data type of value should be ‘GraphProto’ (see mindspore/ccsrc/anf_ir.proto) object when the plugin is ‘graph’.

  • The data type of value should be ‘Tensor’ object when the plugin is ‘scalar’, ‘image’, ‘tensor’ or ‘histogram’.

  • The data type of value should be a ‘TrainLineage’ object when the plugin is ‘train_lineage’, see mindspore/ccsrc/lineage.proto.

  • The data type of value should be a ‘EvaluationLineage’ object when the plugin is ‘eval_lineage’, see mindspore/ccsrc/lineage.proto.

  • The data type of value should be a ‘DatasetGraph’ object when the plugin is ‘dataset_graph’, see mindspore/ccsrc/lineage.proto.

  • The data type of value should be a ‘UserDefinedInfo’ object when the plugin is ‘custom_lineage_data’, see mindspore/ccsrc/lineage.proto.

  • The data type of value should be a ‘LossLandscape’ object when the plugin is ‘LANDSCAPE’, see mindspore/ccsrc/summary.proto.

Raises

• ValueError – plugin is not in the optional value.

• TypeError – name is not non-empty string, or the data type of value is not ‘Tensor’ object when the plugin is ‘scalar’, ‘image’, ‘tensor’ or ‘histogram’.

Examples

>>> import mindspore as ms
>>> from mindspore import Tensor
>>> if __name__ == '__main__':
...     with ms.SummaryRecord(log_dir="./summary_dir", file_prefix="xx_", file_suffix="_yy") \
...             as summary_record:
...         summary_record.add_value('scalar', 'loss', Tensor(0.1))

close()

Flush the buffer and write files to disk and close summary records. Please use the statement to autoclose.

Examples

>>> import mindspore as ms
>>> if __name__ == '__main__':
...     try:
...         summary_record = ms.SummaryRecord(log_dir="./summary_dir")
...     finally:
...         summary_record.close()

flush()

Flush the buffer and write buffer data to disk.

Call it to make sure that all pending events have been written to disk.

Examples

>>> import mindspore as ms
>>> if __name__ == '__main__':
...     with ms.SummaryRecord(log_dir="./summary_dir", file_prefix="xx_", file_suffix="_yy") \
...             as summary_record:
...         summary_record.flush()

property log_dir

Get the full path of the log file.

Returns

str, the full path of log file.

Examples

>>> import mindspore as ms
>>> if __name__ == '__main__':
...     with ms.SummaryRecord(log_dir="./summary_dir", file_prefix="xx_", file_suffix="_yy") \
...             as summary_record:
...         log_dir = summary_record.log_dir

record(step, train_network=None, plugin_filter=None)

Record the summary.

Parameters

• step (int) – Represents training step number.

• train_network (Cell) – The spare network for saving graph. Default: None, it means just do not save the graph summary when the original network graph is None.

• plugin_filter (Callable[[str], bool], optional) – The filter function, which is used to filter out which plugin should be written. Default: None.

Returns

bool, whether the record process is successful or not.

Raises

TypeError – step is not int, or train_network is not mindspore.nn.Cell .

Examples

>>> import mindspore as ms
>>> if __name__ == '__main__':
...     with ms.SummaryRecord(log_dir="./summary_dir", file_prefix="xx_", file_suffix="_yy") \
...             as summary_record:
...         result = summary_record.record(step=2)

set_mode(mode)

Set the model running phase. Different phases affect data recording.

Parameters

mode (str) –

The mode to be set, which should be ‘train’ or ‘eval’. When the mode is ‘eval’, summary_record will not record the data of summary operators.

• train: the model running phase is train mode.

• eval: the model running phase is eval mode, When the mode is ‘eval’, summary_record will not record the data of summary operators.

Raises

ValueError – mode is not in the optional value.

Examples

>>> import mindspore as ms
>>> if __name__ == '__main__':
...     with ms.SummaryRecord(log_dir="./summary_dir", file_prefix="xx_", file_suffix="_yy") \
...             as summary_record:
...         summary_record.set_mode('eval')