sciai.architecture.MLPAAF

View Source On Gitee
class sciai.architecture.MLPAAF(layers, weight_init='xavier_trunc_normal', bias_init='zeros', activation='tanh', last_activation=None, a_value=1.0, scale=1.0, share_type='layer_wise')[source]

Multi-layer perceptron with adaptive activation function. The last layer is without activation function.

The first value in layers in Args should be equal to the size of last axis in_channels in input Tensor.

More information about the improvement for MLP, please refer to Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks.

Parameters

  • layers (Union(tuple[int], list[int])) – List of numbers of neurons in each layer, e.g., [2, 10, 10, 1].

  • weight_init (Union[str, Initializer]) – The weight_init parameter for Dense. The dtype is the same as x. The values of str refer to the function initializer. Default: ‘xavier_trunc_normal’.

  • bias_init (Union[str, Initializer]) – The bias_init parameter for Dense. The dtype is same as x. The values of str refer to the function initializer. Default: ‘zeros’.

  • activation (Union[str, Cell, Primitive, FunctionType, None]) – Activation function applied to the output of each fully connected layer excluding the last layer. Both activation name, e.g. ‘relu’, and mindspore activation function, e.g. nn.ReLU(), are supported. Default: ‘tanh’.

  • last_activation (Union[str, Cell, Primitive, FunctionType, None]) – Activation function applied to the output of the last dense layer. The type rule is the same as activation.

  • a_value (Union[Number, Tensor, Parameter]) – Adaptive trainable parameter a. Default: 1.0.

  • scale (Union[Number, Tensor]) – Fixed scale parameter scale. Default: 1.0.

  • share_type (str) – The sharing level of trainable parameter of adaptive function, can be layer_wise, global. default: layer_wise.

Inputs:

  • x (Tensor) - Tensor of shape (∗,in_channels).

Outputs:

Union(Tensor, tuple[Tensor]), Output of the network.

Raises

  • TypeError – If layers is not one of list, tuple, or elements in layers are not ints.

  • TypeError – If activation is not one of str, Cell, Primitive, None.

  • TypeError – If last_activation is not one of str, Cell, Primitive, None.

  • TypeError – If weight_init is not one of str, Initializer.

  • TypeError – If bias_init is not one of str, Initializer.

  • TypeError – If a_value is not one of Number, Tensor, Parameter.

  • TypeError – If scale is not one of Number, Tensor.

  • TypeError – If share_type is not str.

  • ValueError – If share_type is not supported.

Supported Platforms:

GPU CPU Ascend

Examples

>>> import mindspore as ms
>>> import numpy as np
>>> from sciai.architecture import MLPAAF
>>> x = ms.Tensor(np.array([[180, 234, 154], [244, 48, 247]]), ms.float32)
>>> net = MLPAAF((3, 10, 4))
>>> output = net(x)
>>> print(output.shape)
(2, 4)
a_value()[source]

Get trainable local adaptive parameter value(s) of the MLP.

Returns

Union(Parameter, tuple[Parameter]), The common trainable Parameter a if share_type is “global”;

a list of Parameters a in all layers if “layer_wise”.