mindspore.nn
Neural Networks Cells.
Pre-defined building blocks or computing units to construct neural networks.
Cell
API Name |
Description |
Supported Platforms |
|
Base class for all neural networks. |
|
|
Base class for GraphKernel. |
To Be Developed |
Containers
API Name |
Description |
Supported Platforms |
|
Holds Cells in a list. |
|
|
Sequential cell container. |
|
Convolution Layers
API Name |
Description |
Supported Platforms |
|
1D convolution layer. |
|
|
1D transposed convolution layer. |
|
|
2D convolution layer. |
|
|
2D transposed convolution layer. |
|
Recurrent Layers
API Name |
Description |
Supported Platforms |
|
LSTM (Long Short-Term Memory) layer. |
|
|
Stacked LSTM (Long Short-Term Memory) layers. |
|
Sparse Layers
API Name |
Description |
Supported Platforms |
|
A simple lookup table that stores embeddings of a fixed dictionary and size. |
|
|
Returns a slice of the input tensor based on the specified indices. |
|
|
Returns a slice of input tensor based on the specified indices and the field ids. |
|
Non-linear Activations
API Name |
Description |
Supported Platforms |
|
Exponential Linear Uint activation function. |
|
|
Fast Gaussian error linear unit activation function. |
|
|
Gaussian error linear unit activation function. |
|
|
Gets the activation function. |
To Be Developed |
|
Hard sigmoid activation function. |
|
|
Hard swish activation function. |
|
|
Leaky ReLU activation function. |
|
|
Logsigmoid activation function. |
|
|
LogSoftmax activation function. |
|
|
PReLU activation function. |
|
|
Rectified Linear Unit activation function. |
|
|
Compute ReLU6 activation function. |
|
|
Sigmoid activation function. |
|
|
Softmax activation function. |
|
|
Tanh activation function. |
|
Utilities
API Name |
Description |
Supported Platforms |
|
Clips tensor values to a maximum \(L_2\)-norm. |
|
|
The dense connected layer. |
|
|
Dropout layer for the input. |
|
|
Flatten layer for the input. |
|
|
Apply l1 regularization to weights |
|
|
Computes the norm of vectors, currently including Euclidean norm, i.e., \(L_2\)-norm. |
|
|
Returns a one-hot tensor. |
|
|
Pads the input tensor according to the paddings and mode. |
|
|
Creates a sequence of numbers in range [start, limit) with step size delta. |
|
|
Samples the input tensor to the given size or scale_factor by using bilinear interpolate. |
|
|
Returns a tensor with elements above the kth diagonal zeroed. |
|
|
Returns a tensor with elements below the kth diagonal zeroed. |
|
|
Extract patches from images. |
|
Images Functions
API Name |
Description |
Supported Platforms |
|
Crop the centeral region of the images with the central_fraction. |
|
|
Returns two tensors, the first is along the height dimension and the second is along the width dimension. |
|
|
Returns MS-SSIM index between two images. |
|
|
Returns Peak Signal-to-Noise Ratio of two image batches. |
|
|
Returns SSIM index between two images. |
|
Normalization Layers
API Name |
Description |
Supported Platforms |
|
Batch normalization layer over a 2D input. |
|
|
Batch normalization layer over a 4D input. |
|
|
Global normalization layer over a N-dimension input. |
|
|
Group Normalization over a mini-batch of inputs. |
|
|
Applies Layer Normalization over a mini-batch of inputs. |
|
|
Returns a batched diagonal tensor with a given batched diagonal values. |
|
|
Returns the batched diagonal part of a batched tensor. |
|
|
Modifies the batched diagonal part of a batched tensor. |
|
Pooling layers
API Name |
Description |
Supported Platforms |
|
1D average pooling for temporal data. |
|
|
2D average pooling for temporal data. |
|
|
1D max pooling operation for temporal data. |
|
|
2D max pooling operation for temporal data. |
|
Quantized Functions
API Name |
Description |
Supported Platforms |
|
Quantization aware training activation function. |
|
|
A combination of convolution, Batchnorm, and activation layer. |
|
|
2D convolution with BatchNormal operation folded construct. |
|
|
2D convolution which use the convolution layer statistics once to calculate BatchNormal operation folded construct. |
To Be Developed |
|
2D convolution and batchnorm without fold with fake quantized construct. |
|
|
2D convolution with fake quantized operation layer. |
|
|
A combination of Dense, Batchnorm, and the activation layer. |
|
|
The fully connected layer with fake quantized operation. |
|
|
Quantization aware operation which provides the fake quantization observer function on data with min and max. |
To Be Developed |
|
Add fake quantized operation after Mul operation. |
|
|
Add fake quantized operation after TensorAdd operation. |
|
Loss Functions
API Name |
Description |
Supported Platforms |
|
BCELoss creates a criterion to measure the binary cross entropy between the true labels and predicted labels. |
|
|
Computes the similarity between two tensors using cosine distance. |
|
|
L1Loss creates a criterion to measure the mean absolute error (MAE) between \(x\) and \(y\) element-wise, where \(x\) is the input Tensor and \(y\) is the target Tensor. |
|
|
MSELoss creates a criterion to measure the mean squared error (squared L2-norm) between \(x\) and \(y\) element-wise, where \(x\) is the input and \(y\) is the target. |
|
|
Computes the sampled softmax training loss. |
|
|
A loss class for learning region proposals. |
|
|
Computes softmax cross entropy between logits and labels. |
|
Optimizer Functions
API Name |
Description |
Supported Platforms |
|
Implements the Adagrad algorithm with ApplyAdagrad Operator. |
|
|
Updates gradients by the Adaptive Moment Estimation (Adam) algorithm. |
|
|
This optimizer will offload Adam optimizer to host CPU and keep parameters being updated on the device, to minimize the memory cost. |
|
|
Implements the Adam algorithm to fix the weight decay. |
|
|
Implements the FTRL algorithm with ApplyFtrl Operator. |
|
|
Lamb Dynamic Learning Rate. |
|
|
Implements the LARS algorithm with LARSUpdate Operator. |
|
|
This optimizer will apply a lazy adam algorithm when gradient is sparse. |
|
|
Implements the Momentum algorithm. |
|
|
Base class for all optimizers. |
|
|
Implements the ProximalAdagrad algorithm with ApplyProximalAdagrad Operator. |
|
|
Implements Root Mean Squared Propagation (RMSProp) algorithm. |
|
|
Implements stochastic gradient descent. |
|
Wrapper Functions
API Name |
Description |
Supported Platforms |
|
A distributed optimizer. |
|
|
Dynamic Loss scale update cell. |
|
|
Static scale update cell, the loss scaling value will not be updated. |
|
|
Cell to run for getting the next operation. |
|
|
Cell that updates parameters. |
|
|
Network training package class. |
|
|
Network training with loss scaling. |
|
|
Cell that returns loss, output and label for evaluation. |
|
|
Cell that returns the gradients. |
|
|
Cell with loss function. |
|
Math Functions
API Name |
Description |
Supported Platforms |
|
Calculates Digamma using Lanczos' approximation referring to "A Precision Approximation of the Gamma Function". |
|
|
Calculates lower regularized incomplete Gamma function. |
|
|
This is semantically equal to lgamma(x) + lgamma(y) - lgamma(x + y). |
|
|
Calculates LGamma using Lanczos' approximation referring to "A Precision Approximation of the Gamma Function". |
|
|
Calculates the determinant of Positive-Definite Hermitian matrix using Cholesky decomposition. |
|
|
Calculates the inverse of Positive-Definite Hermitian matrix using Cholesky decomposition. |
|
|
Multiplies matrix x1 by matrix x2. |
|
|
Calculates the mean and variance of x. |
|
|
Reduces a dimension of a tensor by calculating exponential for all elements in the dimension, then calculate logarithm of the sum. |
|
Metrics
Calculates the accuracy for classification and multilabel data. |
|
Calculates the F1 score. |
|
Calculates the fbeta score. |
|
Gets the metric method based on the input name. |
|
Calculates the average of the loss. |
|
Calculates the mean absolute error. |
|
Base class of metric. |
|
Measures the mean squared error. |
|
Gets the names of the metric methods. |
|
Calculates precision for classification and multilabel data. |
|
Calculates recall for classification and multilabel data. |
|
Calculates the top-1 categorical accuracy. |
|
Calculates the top-5 categorical accuracy. |
|
Calculates the top-k categorical accuracy. |
Dynamic Learning Rate
LearningRateSchedule
The dynamic learning rates in this module are all subclasses of LearningRateSchedule. Pass the instance of LearningRateSchedule to an optimizer. During the training process, the optimizer calls the instance taking current step as input to get the current learning rate.
import mindspore.nn as nn
min_lr = 0.01
max_lr = 0.1
decay_steps = 4
cosine_decay_lr = nn.CosineDecayLR(min_lr, max_lr, decay_steps)
net = Net()
optim = nn.Momentum(net.trainable_params(), learning_rate=cosine_decay_lr, momentum=0.9)
Calculates learning rate base on cosine decay function. |
|
Calculates learning rate base on exponential decay function. |
|
Calculates learning rate base on inverse-time decay function. |
|
Calculates learning rate base on natural exponential decay function. |
|
Calculates learning rate base on polynomial decay function. |
|
Gets learning rate warming up. |
Dynamic LR
The dynamic learning rates in this module are all functions. Call the function and pass the result to an optimizer. During the training process, the optimizer takes result[current step] as current learning rate.
import mindspore.nn as nn
min_lr = 0.01
max_lr = 0.1
total_step = 6
step_per_epoch = 1
decay_epoch = 4
lr= nn.cosine_decay_lr(min_lr, max_lr, total_step, step_per_epoch, decay_epoch)
net = Net()
optim = nn.Momentum(net.trainable_params(), learning_rate=lr, momentum=0.9)
Calculate learning rate base on cosine decay function. |
|
Calculate learning rate base on exponential decay function. |
|
Calculate learning rate base on inverse-time decay function. |
|
Calculate learning rate base on natural exponential decay function. |
|
Get piecewise constant learning rate. |
|
Calculate learning rate base on polynomial decay function. |
|
Get learning rate warming up. |