mindspore.SummaryLandscape

class mindspore.SummaryLandscape(summary_dir)

SummaryLandscape can help you to collect loss landscape information. It can create landscape in PCA direction or random direction by calculating loss.

Note

1. When using SummaryLandscape, you need to run the code in if __name__ == “__main__” .

2. SummaryLandscape only supports Linux systems.

Parameters

summary_dir (str) – The path of summary is used to save the model weight, metadata and other data required to create landscape.

Examples

>>> import mindspore as ms
>>> import mindspore.nn as nn
>>> from mindspore.nn import Loss, Accuracy
>>>
>>> if __name__ == '__main__':
...     # If the device_target is Ascend, set the device_target to "Ascend"
...     ms.set_context(mode=ms.GRAPH_MODE, device_target="GPU")
...     mnist_dataset_dir = '/path/to/mnist_dataset_directory'
...     # The detail of create_dataset method shown in model_zoo.official.cv.lenet.src.dataset.py
...     ds_train = create_dataset(mnist_dataset_dir, 32)
...     # The detail of LeNet5 shown in model_zoo.official.cv.lenet.src.lenet.py
...     network = LeNet5(10)
...     net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean")
...     net_opt = nn.Momentum(network.trainable_params(), 0.01, 0.9)
...     model = ms.Model(network, net_loss, net_opt, metrics={"Accuracy": Accuracy()})
...     # Simple usage for collect landscape information:
...     interval_1 = [1, 2, 3, 4, 5]
...     summary_collector = ms.SummaryCollector(summary_dir='./summary/lenet_interval_1',
...                                          collect_specified_data={'collect_landscape':{"landscape_size": 4,
...                                                                                        "unit": "step",
...                                                                          "create_landscape":{"train":True,
...                                                                                             "result":False},
...                                                                          "num_samples": 2048,
...                                                                          "intervals": [interval_1]}
...                                                                    })
...     model.train(1, ds_train, callbacks=[summary_collector], dataset_sink_mode=False)
...
...     # Simple usage for visualization landscape:
...     def callback_fn():
...         network = LeNet5(10)
...         net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean")
...         metrics = {"Loss": Loss()}
...         model = Model(network, net_loss, metrics=metrics)
...         mnist_dataset_dir = '/path/to/mnist_dataset_directory'
...         ds_eval = create_dataset(mnist_dataset_dir, 32)
...         return model, network, ds_eval, metrics
...
...     summary_landscape = ms.SummaryLandscape('./summary/lenet_interval_1')
...     # parameters of collect_landscape can be modified or unchanged
...     summary_landscape.gen_landscapes_with_multi_process(callback_fn,
...                                                        collect_landscape={"landscape_size": 4,
...                                                                         "create_landscape":{"train":False,
...                                                                                            "result":False},
...                                                                          "num_samples": 2048,
...                                                                          "intervals": [interval_1]},
...                                                         device_ids=[1])

clean_ckpt()

Clean the checkpoint.

gen_landscapes_with_multi_process(callback_fn, collect_landscape=None, device_ids=None, output=None)

Use the multi process to generate landscape.

Parameters

• callback_fn (python function) –

  A python function object. User needs to write a function, it has no input, and the return requirements are as follows.

  • mindspore.Model: User’s model object.

  • mindspore.nn.Cell: User’s network object.

  • mindspore.dataset: User’s dataset object for create loss landscape.

  • mindspore.nn.Metrics: User’s metrics object.

• collect_landscape (Union[dict, None]) –

  The meaning of the parameters when creating loss landscape is consistent with the fields with the same name in SummaryCollector. The purpose of setting here is to allow users to freely modify creating parameters. Default: None.

  • landscape_size (int): Specify the image resolution of the generated loss landscape. For example, if it is set to 128, the resolution of the landscape is 128 * 128. The calculation time increases with the increase of resolution. Default: 40. Optional values: between 3 and 256.

  • create_landscape (dict): Select how to create loss landscape. Training process loss landscape(train) and training result loss landscape(result). Default: {“train”: True, “result”: True}. Optional: True/False.

  • num_samples (int): The size of the dataset used to create the loss landscape. For example, in image dataset, You can set num_samples is 2048, which means that 2048 images are used to create loss landscape. Default: 2048.

  • intervals (List[List[int]): Specifies the interval in which the loss landscape. For example: If the user wants to create loss landscape of two training processes, they are 1-5 epoch and 6-10 epoch respectively. They can set [[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]]. Note: Each interval have at least three epochs.

• device_ids (List(int)) – Specifies which devices are used to create loss landscape. For example: [0, 1] refers to creating loss landscape with device 0 and device 1. Default: None.

• output (str) – Specifies the path to save the loss landscape. Default: None. The default save path is the same as the summary file.