mindspore.common.initializer

Initializer for cell parameters.

class mindspore.common.initializer.Constant(value)

Generates an array with constant value in order to initialize a tensor.

Parameters

value (Union[int, numpy.ndarray]) – The value to initialize.

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, Constant
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(Constant(3), [1, 2, 3], mindspore.float32))

class mindspore.common.initializer.Dirac(groups=1)

Generates an array with the Dirac delta function in order to initialize a tensor. It's usually used in convolution layers, preserves as many identities of the inputs as possible.

Parameters

groups (int) – The number of groups in convolution layer. Each group applies the same initialization. Default: 1 .

Raises

• ValueError – If the dimension of the initialized tensor is not in [3, 4, 5].

• ValueError – The first dimension of the initialized tensor cannot be divisible by group.

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, Dirac
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(Dirac(groups=2), [6, 4, 3, 3], mindspore.float32))
>>> w2 = Parameter(initializer("dirac", [6, 4, 3, 3], mindspore.float32))

class mindspore.common.initializer.HeNormal(negative_slope=0, mode='fan_in', nonlinearity='leaky_relu')

Generates an array with values sampled from HeKaiming Normal distribution \({N}(0, \text{sigma}^2)\) in order to initialize a tensor, where

\[sigma = \frac{gain} {\sqrt{fan\_mode}}\]

where \(gain\) is an optional scaling factor. \(fan\_mode\) is the number of input or output units of the weight tensor, depending on the mode is 'fan_in' or 'fan_out'.

For details of HeNormal algorithm, please check https://arxiv.org/abs/1502.01852.

Parameters

• negative_slope (int, float) – The negative slope of the rectifier used after this layer (only used when nonlinearity is 'leaky_relu'). Default: 0 .

• mode (str) – Either 'fan_in' or 'fan_out' . Choosing 'fan_in' preserves the magnitude of the variance of the weights in the forward pass. Choosing 'fan_out' preserves the magnitudes in the backwards pass. Default: 'fan_in' .

• nonlinearity (str) – The non-linear function, recommended to use only with 'relu' or 'leaky_relu' . Default: 'leaky_relu' .

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, HeNormal
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(HeNormal(), [1, 2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('he_normal', [1, 2, 3], mindspore.float32))

class mindspore.common.initializer.HeUniform(negative_slope=0, mode='fan_in', nonlinearity='leaky_relu')

Generates an array with values sampled from HeKaiming Uniform distribution \({U}(-\text{boundary}, \text{boundary})\) in order to initialize a tensor, where

\[boundary = \text{gain} \times \sqrt{\frac{3}{fan\_mode}}\]

where \(gain\) is an optional scaling factor. If \(fan\_mode\) is 'fan_in', it is the number of input units of the weight tensor. If \(fan\_mode\) is 'fan_out', it is the number of output units of the weight tensor.

For details of HeUniform algorithm, please check https://arxiv.org/abs/1502.01852.

Parameters

• negative_slope (int, float, bool) – The negative slope of the rectifier used after this layer (only used when nonlinearity is 'leaky_relu'). Default: 0 .

• mode (str) – Either 'fan_in' or 'fan_out' . Choosing 'fan_in' preserves the magnitude of the variance of the weights in the forward pass. Choosing 'fan_out' preserves the magnitudes in the backwards pass. Default: 'fan_in' .

• nonlinearity (str) – The non-linear function, recommended to use only with 'relu' or 'leaky_relu' . Default: 'leaky_relu' .

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, HeUniform
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(HeUniform(), [1, 2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('he_uniform', [1, 2, 3], mindspore.float32))

class mindspore.common.initializer.Identity(**kwargs)

Generates a 2 dimension identity matrix array in order to initialize a tensor.

Raises

ValueError – If the dimension of input tensor is not equal to 2.

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, Identity
>>> from mindspore import Parameter
>>> w1 = initializer(Identity(), [2, 3], mindspore.float32)
>>> w2 = initializer('identity', [2, 3], mindspore.float32)

class mindspore.common.initializer.Initializer(**kwargs)

The abstract base class of the initializer.

Note

Initializers are intended to be used for delayed initialization in parallel mode rather than Tensor initialization. If you have to use Initializers to create a Tensor, mindspore.Tensor.init_data() should be followed in most of the cases. For more information, please refer to mindspore.Tensor.init_data .

Parameters

kwargs (dict) – Keyword arguments for Initializer.

class mindspore.common.initializer.Normal(sigma=0.01, mean=0.0)

Generates an array with values sampled from Normal distribution \({N}(\text{sigma}, \text{mean})\) in order to initialize a tensor.

\[f(x) = \frac{1} {\sqrt{2*π} * sigma}exp(-\frac{(x - mean)^2} {2*{sigma}^2})\]

Parameters

• sigma (float) – The standard deviation of Normal distribution. Default: 0.01 .

• mean (float) – The mean of Normal distribution. Default: 0.0 .

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, Normal
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(Normal(), [1, 2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('normal', [1, 2, 3], mindspore.float32))

class mindspore.common.initializer.One(**kwargs)

Generates an array with constant value of one in order to initialize a tensor.

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, One
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(One(), [1, 2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('ones', [1, 2, 3], mindspore.float32))

class mindspore.common.initializer.Orthogonal(gain=1.)

Generates a (semi) orthogonal matrix array in order to initialize a tensor. The dimension of input tensor must have at least 2 dimensions. If the dimension is greater than 2, the trailing dimensions will be flattened.

Parameters

gain (float) – An optional scaling factor. Default: 1.0 .

Raises

ValueError – If the dimension of input tensor is less than 2.

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, Orthogonal
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(Orthogonal(gain=2.), [2, 3, 4], mindspore.float32))
>>> w2 = Parameter(initializer('orthogonal', [2, 3, 4], mindspore.float32))

class mindspore.common.initializer.Sparse(sparsity, sigma=0.01)

Generates a 2 dimension sparse matrix array in order to initialize a tensor. The non-zero positions will be filled with the value sampled from the normal distribution \({N}(0, sigma)\).

Parameters

• sparsity (float) – The fraction of elements being set to zero in each column.

• sigma (float) – The standard deviation of the normal distribution. Default: 0.01 .

Raises

ValueError – If the dimension of input tensor is not equal to 2.

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, Sparse
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(Sparse(sparsity=0.1, sigma=0.01), [5, 8], mindspore.float32))

class mindspore.common.initializer.TruncatedNormal(sigma=0.01, mean=0.0, a=- 2.0, b=2.0)

Generates an array with values sampled from Truncated Normal distribution in order to initialize a tensor.

Parameters

• sigma (float) – The standard deviation of Truncated Normal distribution. Default: 0.01 .

• mean (float) – The mean of Truncated Normal distribution. Default: 0.0 .

• a (float) – The lower bound of the truncated interval. Default: -2.0 .

• b (float) – The upper bound of the truncated interval. Default: 2.0 .

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, TruncatedNormal
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(TruncatedNormal(), [1, 2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('truncatedNormal', [1, 2, 3], mindspore.float32))

class mindspore.common.initializer.Uniform(scale=0.07)

Generates an array with values sampled from Uniform distribution \({U}(-\text{scale}, \text{scale})\) in order to initialize a tensor.

Parameters

scale (float) – The bound of the Uniform distribution. Default: 0.07 .

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, Uniform
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(Uniform(), [1, 2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('uniform', [1, 2, 3], mindspore.float32))

class mindspore.common.initializer.VarianceScaling(scale=1.0, mode='fan_in', distribution='truncated_normal')

Generates an random array with scaling in order to initialize a tensor. When distribution is 'truncated_normal' or 'untruncated_normal', the value will be sampled from truncated or untruncated normal distribution with a mean of 0 and a scaled standard deviation \(stddev = \sqrt{\frac{scale}{n}}\). \(n\) will be the number of input units if mode is 'fan_in', while \(n\) will be the number of output units if mode is 'fan_out'. \(n\) will be the average of 'fan_in' and 'fan_out' if mode is 'fan_avg'. When distribution is 'uniform', the value will be sampled from a uniform distribution within the limit of \([-\sqrt{\frac{3*scale}{n}}, \sqrt{\frac{3*scale}{n}}]\).

Parameters

• scale (float) – The scaling factor. Default: 1.0 .

• mode (str) – Should be 'fan_in' , 'fan_out' or 'fan_avg' . Default: 'fan_in' .

• distribution (str) – The type of distribution chose to sample values. It should be 'uniform' , 'truncated_normal' or 'untruncated_normal' . Default: 'truncated_normal' .

Raises

• ValueError – If scale is not greater than 0.

• ValueError – If mode is not 'fan_in', 'fan_out' or 'fan_avg'.

• ValueError – If distribution is not 'uniform', 'truncated_normal' or 'untruncated_normal'.

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, VarianceScaling
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(VarianceScaling(scale=1.0, mode='fan_out',
...                                            distribution='untruncated_normal'), [2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('varianceScaling', [2, 3], mindspore.float32))

class mindspore.common.initializer.XavierNormal(gain=1)

Generates an array with values sampled from Xavier normal distribution \({N}(0, \text{sigma}^2)\) in order to initialize a tensor, where

\[sigma = gain * \sqrt{\frac{2}{n_{in} + n_{out}}}\]

where \(gain\) is an optional scaling factor, \(n_{in}\) is the number of input units in the weight tensor, \(n_{out}\) is the number of output units in the weight tensor.

Parameters

gain (float) – An optional scaling factor. Default: 1 .

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, XavierNormal
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(XavierNormal(), [1, 2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('xavier_normal', [1, 2, 3], mindspore.float32))

class mindspore.common.initializer.XavierUniform(gain=1)

Generates an array with values sampled from Xavier uniform distribution \({U}(-\text{boundary}, \text{boundary})\) in order to initialize a tensor, where

\[boundary = gain * \sqrt{\frac{6}{n_{in} + n_{out}}}\]

where \(gain\) is an optional scaling factor. \(n_{in}\) is the number of input units in the weight tensor, \(n_{out}\) is the number of output units in the weight tensor.

For details of XavierUniform algorithm, please check http://proceedings.mlr.press/v9/glorot10a.html.

Parameters

gain (float) – An optional scaling factor. Default: 1 .

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, XavierUniform
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(XavierUniform(), [1, 2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('xavier_uniform', [1, 2, 3], mindspore.float32))

class mindspore.common.initializer.Zero(**kwargs)

Generates an array with constant value of zero in order to initialize a tensor.

Examples

>>> import mindspore
>>> from mindspore.common.initializer import initializer, Zero
>>> from mindspore import Parameter
>>> w1 = Parameter(initializer(Zero(), [1, 2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('zeros', [1, 2, 3], mindspore.float32))

mindspore.common.initializer.initializer(init, shape=None, dtype=mstype.float32)

Create and initialize a tensor.

Parameters

• init (Union[Tensor, str, Initializer, numbers.Number]) –

  Initialize value.

  • str: The init should be the alias of the class inheriting from Initializer and the corresponding class will be called in practice. The value of init can be "normal", "ones" or "zeros", etc.

  • Initializer: The init should be the class inheriting from Initializer to initialize tensor.

  • numbers.Number: The Constant will be called to initialize tensor.

  • Tensor: The tensor will be called to initialize tensor.

• shape (Union[tuple, list, int]) – The shape of the initialized tensor. Default: None .

• dtype (mindspore.dtype) – The type of data in initialized tensor. Default: mstype.float32 .

Returns

Returns a Tensor with the shape specified by the input shape. If shape is None, the returned Tensor will have the same shape as init.

Raises

• TypeError – The type of the argument 'init' is not correct.

• ValueError – The shape of the tensor which is passed through 'init' is not the same as that passed by 'shape'.

Examples

>>> import numpy as np
>>> import mindspore
>>> from mindspore import Tensor
>>> from mindspore.common.initializer import initializer, One
>>> from mindspore import Parameter
>>> data = Tensor(np.zeros([1, 2, 3]), mindspore.float32)
>>> w1 = Parameter(initializer(data, [1, 2, 3], mindspore.float32))
>>> w2 = Parameter(initializer('ones', [1, 2, 3], mindspore.float32))
>>> w3 = Parameter(initializer(One(), [1, 2, 3], mindspore.float32))
>>> w4 = Parameter(initializer(0, [1, 2, 3], mindspore.float32))