Introduction || Quick Start || Tensor || Data Loading and Processing || Model || Autograd || Train || Save and Load || Accelerating with Static Graphs || Mixed Precision

Saving and Loading the Model

The previous section describes how to adjust hyperparameters and train network models. During network model training, we want to save the intermediate and final results for fine-tuning and subsequent model deployment and inference. This section describes how to save and load a model.

import numpy as np
import mindspore
from mindspore import nn
from mindspore import Tensor

def network():
    model = nn.SequentialCell(
                nn.Flatten(),
                nn.Dense(28*28, 512),
                nn.ReLU(),
                nn.Dense(512, 512),
                nn.ReLU(),
                nn.Dense(512, 10))
    return model

Saving and Loading the Model Weight

Saving model by using the save_checkpoint interface, and the specified saving path of passing in the network:

model = network()
mindspore.save_checkpoint(model, "model.ckpt")

To load the model weights, you need to create instances of the same model and then load the parameters by using the load_checkpoint and load_param_into_net methods.

model = network()
param_dict = mindspore.load_checkpoint("model.ckpt")
param_not_load, _ = mindspore.load_param_into_net(model, param_dict)
print(param_not_load)

[]

• param_not_load is an unloaded parameter list, and empty means all parameters are loaded successfully.

• When MindX DL (Ascend deep learning component) version 6.0 or later is installed in the environment, the MindIO acceleration CheckPoint function is enabled by default. For details, please refer to MindIO Introduction. MindX DL can be downloaded here.

Saving and Loading MindIR

In addition to Checkpoint, MindSpore provides a unified Intermediate Representation (IR) for cloud side (training) and end side (inference). Models can be saved as MindIR directly by using the export interface (only support strict graph mode).

mindspore.set_context(mode=mindspore.GRAPH_MODE, jit_syntax_level=mindspore.STRICT)
model = network()
inputs = Tensor(np.ones([1, 1, 28, 28]).astype(np.float32))
mindspore.export(model, inputs, file_name="model", file_format="MINDIR")

MindIR saves both Checkpoint and model structure, so it needs to define the input Tensor to get the input shape.

The existing MindIR model can be easily loaded through the load interface and passed into nn.GraphCell for inference.

nn.GraphCell only supports graph mode.

graph = mindspore.load("model.mindir")
model = nn.GraphCell(graph)
outputs = model(inputs)
print(outputs.shape)

(1, 10)

Syntax Support Scope

Not all Python syntax and data types are supported for MindIR export. Unsupported cases will raise errors during export.

1. MindIR export only supports basic syntax at the STRICT level. For detailed coverage, refer to Static Graph Syntax Support Documentation.

2. Return value data types are limited to:

   • Python built-in types: int, float, bool, str, tuple, list.

   • MindSpore framework types: Tensor, Parameter, COOTensor, CSRTensor.

   For example, in the following program, the return value type is mindspore.dtype, which is not supported. As a result, an error is reported when MindIR is exported.

   import mindspore
   from mindspore import nn, Tensor
   
   class Model(nn.Cell):
   
       def construct(self, x: Tensor) -> mindspore.dtype:
           return x.dtype
   

3. In nn.Cell's construct() method, random number generators from mindspore.mint (e.g., mint.rand, mint.randn, mint.randint, mint.randperm) are prohibited. Use equivalent mindspore.ops interfaces instead.

4. Parameter objects must be defined either in nn.Cell's __init__() method or as function input arguments. Otherwise, MindIR export will fail. For instance, a globally defined Parameter (as shown below) triggers an unsupported error.

   import mindspore
   from mindspore import Parameter, nn
   
   # The Parameter is created outside nn.Cell and used by the Model as a global variable.
   global_param = Parameter([1, 2, 3], name='global_param')
   
   class Model(nn.Cell):
   
       def __init__(self):
           super().__init__()
           # Parameters defined within nn.Cell.__init__() are exportable.
           self.bias = Parameter([0, 1, -1])
   
       def construct(self, x: Parameter):  # Parameters passed as function arguments are exportable.
           # The global_param is a global variable and will cause an error during export.
           return x + global_param + self.bias
   
   model = Model()
   param = Parameter([1, 2, 3], name='input_param')
   mindspore.export(model, param, file_name="model", file_format="MINDIR")