mindspore.nn.Cell
- class mindspore.nn.Cell(auto_prefix=True, flags=None)[source]
Base class for all neural networks.
A ‘Cell’ could be a single neural network cell, such as conv2d, relu, batch_norm, etc. or a composition of cells to constructing a network.
Note
In general, the autograd algorithm will automatically generate the implementation of the gradient function, but if back-propagation(bprop) method is implemented, the gradient function will be replaced by the bprop. The bprop implementation will receive a tensor dout containing the gradient of the loss w.r.t. the output, and a tensor out containing the forward result. The bprop needs to compute the gradient of the loss w.r.t. the inputs, gradient of the loss w.r.t. Parameter variables are not supported currently. The bprop method must contain the self parameter.
- Parameters
- Supported Platforms:
Ascend
GPU
CPU
Examples
>>> import mindspore.nn as nn >>> import mindspore.ops as ops >>> class MyCell(nn.Cell): ... def __init__(self): ... super(MyCell, self).__init__() ... self.relu = ops.ReLU() ... ... def construct(self, x): ... return self.relu(x)
- add_flags(**flags)[source]
Add customized attributes for cell.
This method is also called when the cell class is instantiated and the class parameter ‘flag’ is set to True.
- add_flags_recursive(**flags)[source]
If a cell contains child cells, this method can recursively customize attributes of all cells.
- auto_parallel_compile_and_run()[source]
Whether or not to execute compile and run.
- Returns
bool, _auto_parallel_compile_and_run value.
- property bprop_debug
Get whether cell custom bprop debug is enabled.
- cast_param(param)[source]
Cast parameter according to auto mix precision level in pynative mode.
This interface is currently used in the case of auto mix precision and usually need not to be used explicitly.
- Parameters
param (Parameter) – Parameters, the type of which should be cast.
- Returns
Parameter, the input parameter with type automatically cast.
- cells()[source]
Returns an iterator over immediate cells.
- Returns
Iteration, all the child cells in the cell.
- cells_and_names(cells=None, name_prefix="")[source]
Returns an iterator over all cells in the network.
Includes the cell’s name and itself.
- Parameters
- Returns
Iteration, all the child cells and corresponding names in the cell.
Examples
>>> n = Net() >>> names = [] >>> for m in n.cells_and_names(): ... if m[0]: ... names.append(m[0])
- compile_and_run(*inputs)[source]
Compiles and runs cell.
- Parameters
inputs (tuple) – Inputs of the Cell object.
- Returns
Object, the result of executing.
- construct(*inputs, **kwargs)[source]
Defines the computation to be performed. This method must be overridden by all subclasses.
- Returns
Tensor, returns the computed result.
- extend_repr()[source]
Sets the extended representation of the Cell.
To print customized extended information, re-implement this method in your own cells.
- get_parameters(expand=True)[source]
Returns an iterator over cell parameters.
Yields parameters of this cell. If expand is true, yield parameters of this cell and all subcells.
- Parameters
expand (bool) – If true, yields parameters of this cell and all subcells. Otherwise, only yield parameters that are direct members of this cell. Default: True.
- Returns
Iteration, all parameters at the cell.
Examples
>>> net = Net() >>> parameters = [] >>> for item in net.get_parameters(): ... parameters.append(item)
- get_scope()[source]
Returns the scope of a cell object in one network.
- Returns
String, scope of the cell.
- infer_param_pipeline_stage()[source]
Infer pipeline stages of all parameters in the cell.
Note
If a parameter does not belong to any cell which has been set pipeline_stage, the parameter should use add_pipeline_stage to add it’s pipeline_stage information.
If a parameter P has been used by two operator in different stages “stageA” and “stageB”, the parameter P should use P.add_pipeline_stage(stageA) and P.add_pipeline_stage(stageB) to add it’s stage information before use infer_param_pipeline_stage.
- Returns
The params belong to current stage in pipeline parallel.
- Raises
RuntimeError – If there is a parameter does not belong to any stage.
- init_parameters_data(auto_parallel_mode=False)[source]
Initialize all parameters and replace the original saved parameters in cell.
Note
trainable_params() and other similar interfaces may return different parameter instance after init_parameters_data, do not save these result.
- Parameters
auto_parallel_mode (bool) – If running in auto_parallel_mode.
- Returns
Dict[Parameter, Parameter], returns a dict of original parameter and replaced parameter.
- insert_child_to_cell(child_name, child_cell)[source]
Adds a child cell to the current cell with a given name.
- insert_param_to_cell(param_name, param, check_name_contain_dot=True)[source]
Adds a parameter to the current cell.
Inserts a parameter with given name to the cell. Please refer to the usage in source code of mindspore.nn.Cell.__setattr__.
- Parameters
- Raises
KeyError – If the name of parameter is null or contains dot.
AttributeError – If user did not call init() first.
TypeError – If the type of parameter is not Parameter.
- load_parameter_slice(params)[source]
Replace parameters with sliced tensors by parallel strategies.
Please refer to the usage in source code of mindspore.common._CellGraphExecutor.compile.
- Parameters
params (dict) – The parameters dictionary used for initializing the data graph.
- name_cells()[source]
Returns an iterator over all cells in the network.
Include name of the cell and cell itself.
- Returns
Dict[String, Cell], all the child cells and corresponding names in the cell.
- property param_prefix
Param prefix is the prefix of current cell’s direct child parameter.
- property parameter_layout_dict
parameter_layout_dict represents the tensor layout of a parameter, which is inferred by shard strategy and distributed operator information.
- parameters_and_names(name_prefix="", expand=True)[source]
Returns an iterator over cell parameters.
Includes the parameter’s name and itself.
- Parameters
- Returns
Iteration, all the names and corresponding parameters in the cell.
Examples
>>> n = Net() >>> names = [] >>> for m in n.parameters_and_names(): ... if m[0]: ... names.append(m[0])
- parameters_broadcast_dict(recurse=True)[source]
Gets the parameters broadcast dictionary of this cell.
- Parameters
recurse (bool) – Whether contains the parameters of subcells. Default: True.
- Returns
OrderedDict, return parameters broadcast dictionary.
- parameters_dict(recurse=True)[source]
Gets parameters dictionary.
Gets the parameters dictionary of this cell.
- Parameters
recurse (bool) – Whether contains the parameters of subcells. Default: True.
- Returns
OrderedDict, return parameters dictionary.
- recompute(**kwargs)[source]
Set the cell recomputed. All the primitive in the cell will be set recomputed. If a primitive set recomputed feeds into some backward nodes for computing gradient, rather than storing the intermediate activation computed in forward pass, we will recompute it in backward pass.
Note
If the computation involves something like randomization or global variable, the equivalence is not guaranteed currently.
If the recompute api of a primitive in this cell is also called, the recompute mode of this primitive is subject to the recompute api of the primitive.
The interface can be configured only once. Therefore, when the parent cell is configured, the child cell should not be configured.
When the memory remains after applying the recompute, configuring ‘mp_comm_recompute=False’ to improve performance if necessary.
When the memory still not enough after applying the recompute, configuring ‘parallel_optimizer_comm_recompute=True’ to save more memory if necessary. Cells in the same fusion group should has the same parallel_optimizer_comm_recompute configures.
- Parameters
mp_comm_recompute (bool) – Specifies whether the model parallel communication operators in the cell are recomputed in auto parallel or semi auto parallel mode. Default: True.
parallel_optimizer_comm_recompute (bool) – Specifies whether the communication operator allgathers introduced by optimizer shard are recomputed in auto parallel or semi auto parallel mode. Default: False.
- register_backward_hook(fn)[source]
Set the cell backward hook function. Note that this function is only supported in pynative mode.
Note
fn must be defined as the following code. cell_name is the name of registered cell. grad_input is gradient passed to the cell. grad_output is the gradient computed and passed to the next cell or primitive, which may be modified and returned. hook_fn(cell_name, grad_input, grad_output) -> Tensor or None.
- Parameters
fn (function) – Specifies the hook function with grad as input.
- remove_redundant_parameters()[source]
Remove the redundant parameters.
This interface usually need not to be used explicitly.
- set_auto_parallel()[source]
Set the cell to auto parallel mode.
Note
If a cell needs to use the auto parallel or semi auto parallel mode for training, evaluation or prediction, this interface needs to be called by the cell.
- set_boost(boost_type)[source]
In order to improve the network performance, configure the network auto enable to accelerate the algorithm in the algorithm library.
If boost_type is not in the algorithm library, Please view the algorithm in the algorithm library through algorithm library.
Note
Some acceleration algorithms may affect the accuracy of the network, please choose carefully.
- Parameters
boost_type (str) – accelerate algorithm.
- Returns
Cell, the cell itself.
- Raises
ValueError – If boost_type is not in the algorithm library.
- set_broadcast_flag(mode=True)[source]
Set the cell to data_parallel mode.
The cell can be accessed as an attribute using the given name.
- Parameters
mode (bool) – Specifies whether the model is data_parallel. Default: True.
- set_comm_fusion(fusion_type, recurse=True)[source]
Set comm_fusion for all the parameters in the Net. Please refer to the description of mindspore.common.parameter.comm_fusion.
Note
The value of attribute will be overwritten when the function is called multiply.
- set_grad(requires_grad=True)[source]
Sets the cell flag for gradient. In pynative mode, this parameter specifies whether the network require gradients. If true, the backward network needed to compute the gradients will be generated when the forward network is executed.
- Parameters
requires_grad (bool) – Specifies if the net need to grad, if it is true, the cell will construct backward network in pynative mode. Default: True.
- Returns
Cell, the cell itself.
- set_parallel_input_with_inputs(*inputs)[source]
Slice inputs tensors by parallel strategies, and set the sliced inputs to _parallel_input_run
- Parameters
inputs (tuple) – inputs of construct method.
- set_param_fl(push_to_server=False, pull_from_server=False, requires_aggr=True)[source]
Set the way of parameter and server interaction.
- set_param_ps(recurse=True, init_in_server=False)[source]
Set whether the trainable parameters are updated by parameter server and whether the trainable parameters are initialized on server.
Note
It only works when a running task is in the parameter server mode.
- set_train(mode=True)[source]
Sets the cell to training mode.
The cell itself and all children cells will be set to training mode. Layers that have different constructions for training and predicting, such as BatchNorm, will distinguish between the branches by this attribute. If set to true, the training branch will be executed, otherwise another branch.
- Parameters
mode (bool) – Specifies whether the model is training. Default: True.
- Returns
Cell, the cell itself.
- to_float(dst_type)[source]
Add cast on all inputs of cell and child cells to run with certain float type.
If dst_type is mindspore.dtype.float16, all the inputs of Cell including input, Parameter, Tensor as const will be cast to float16. Please refer to the usage in source code of mindspore.train.amp.build_train_network.
Note
Multiple calls will overwrite.
- Parameters
dst_type (
mindspore.dtype
) – Transfer cell to run with dst_type. dst_type can be mindspore.dtype.float16 or mindspore.dtype.float32.- Returns
Cell, the cell itself.
- Raises
ValueError – If dst_type is not float32 or float16.
- trainable_params(recurse=True)[source]
Returns all trainable parameters.
Returns a list of all trainable parameters.
- Parameters
recurse (bool) – Whether contains the trainable parameters of subcells. Default: True.
- Returns
List, the list of trainable parameters.
- untrainable_params(recurse=True)[source]
Returns all untrainable parameters.
Returns a list of all untrainable parameters.
- Parameters
recurse (bool) – Whether contains the untrainable parameters of subcells. Default: True.
- Returns
List, the list of untrainable parameters.
- update_cell_prefix()[source]
Update the all child cells’ self.param_prefix.
After being invoked, it can get all the cell’s children’s name prefix by ‘_param_prefix’.