mindspore.SummaryLandscape

class mindspore.SummaryLandscape(summary_dir)

SummaryLandscape can help you to collect loss landscape information. It can create landscape in PCA direction or random direction by calculating loss.

Note

SummaryLandscape only supports Linux systems.

Parameters

summary_dir (str) – The path of summary is used to save the model weight, metadata and other data required to create landscape.

Examples

>>> import mindspore as ms
>>> import mindspore.nn as nn
>>> from mindspore.train import Model, Accuracy, Loss
>>> from mindspore import SummaryCollector, SummaryLandscape
>>>
>>> if __name__ == '__main__':
...     # If the device_target is Ascend, set the device_target to "Ascend"
...     ms.set_context(mode=ms.GRAPH_MODE, device_target="GPU")
...     # Create the dataset taking MNIST as an example. Refer to
...     # https://gitee.com/mindspore/docs/blob/r2.3.q1/docs/mindspore/code/mnist.py
...     ds_train = create_dataset()
...     # Define the network structure of LeNet5. Refer to
...     # https://gitee.com/mindspore/docs/blob/r2.3.q1/docs/mindspore/code/lenet.py
...     network = LeNet5()
...     net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean")
...     net_opt = nn.Momentum(network.trainable_params(), 0.01, 0.9)
...     model = Model(network, net_loss, net_opt, metrics={"Accuracy": Accuracy()})
...     # Simple usage for collect landscape information:
...     interval_1 = [1, 2, 3, 4, 5]
...     summary_collector = SummaryCollector(summary_dir='./summary/lenet_interval_1',
...                                          collect_specified_data={'collect_landscape':{"landscape_size": 4,
...                                                                                        "unit": "step",
...                                                                          "create_landscape":{"train":True,
...                                                                                             "result":False},
...                                                                          "num_samples": 2048,
...                                                                          "intervals": [interval_1]}
...                                                                    })
...     model.train(1, ds_train, callbacks=[summary_collector], dataset_sink_mode=False)
...
...     # Simple usage for visualization landscape:
...     def callback_fn():
...         # Define the network structure of LeNet5. Refer to
...         # https://gitee.com/mindspore/docs/blob/r2.3.q1/docs/mindspore/code/lenet.py
...         network = LeNet5()
...         net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean")
...         metrics = {"Loss": Loss()}
...         model = Model(network, net_loss, metrics=metrics)
...         # Create the dataset taking MNIST as an example. Refer to
...         # https://gitee.com/mindspore/docs/blob/r2.3.q1/docs/mindspore/code/mnist.py
...         ds_eval = create_dataset()
...         return model, network, ds_eval, metrics
...
...     summary_landscape = SummaryLandscape('./summary/lenet_interval_1')
...     # parameters of collect_landscape can be modified or unchanged
...     summary_landscape.gen_landscapes_with_multi_process(callback_fn,
...                                                        collect_landscape={"landscape_size": 4,
...                                                                         "create_landscape":{"train":False,
...                                                                                            "result":False},
...                                                                          "num_samples": 2048,
...                                                                          "intervals": [interval_1]},
...                                                         device_ids=[1])

clean_ckpt()

Clean the checkpoint.

Tutorial Examples:

• Training Optimization Process Visualization

gen_landscapes_with_multi_process(callback_fn, collect_landscape=None, device_ids=None, output=None)

Use the multi process to generate landscape.

Parameters

• callback_fn (python function) –

  A python function object. User needs to write a function, it has no input, and the return requirements are as follows.

  • mindspore.train.Model: User’s model object.

  • mindspore.nn.Cell: User’s network object.

  • mindspore.dataset: User’s dataset object for create loss landscape.

  • mindspore.train.Metrics: User’s metrics object.

• collect_landscape (Union[dict, None]) –

  The meaning of the parameters when creating loss landscape is consistent with the fields with the same name in SummaryCollector. The purpose of setting here is to allow users to freely modify creating parameters. Default: None .

  • landscape_size (int): Specify the image resolution of the generated loss landscape. For example, if it is set to 128 , the resolution of the landscape is 128 * 128. The calculation time increases with the increase of resolution. Default: 40 . Optional values: between 3 and 256.

  • create_landscape (dict): Select how to create loss landscape. Training process loss landscape(train) and training result loss landscape(result). Default: {"train": True, "result": True}. Optional: True / False .

  • num_samples (int): The size of the dataset used to create the loss landscape. For example, in image dataset, You can set num_samples is 2048, which means that 2048 images are used to create loss landscape. Default: 2048 .

  • intervals (List[List[int]]): Specifies the interval in which the loss landscape. For example: If the user wants to create loss landscape of two training processes, they are 1-5 epoch and 6-10 epoch respectively. They can set [[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]]. Note: Each interval have at least three epochs.

• device_ids (List(int)) – Specifies which devices are used to create loss landscape. For example: [0, 1] refers to creating loss landscape with device 0 and device 1. Default: None .

• output (str) – Specifies the path to save the loss landscape. Default: None . The default save path is the same as the summary file.