mindspore.common.initializer
Initializer for cell parameters.
- class mindspore.common.initializer.Constant(value)[source]
Generates an array with constant value in order to initialize a tensor.
- Parameters
value (Union[int, numpy.ndarray]) – The value to initialize.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, Constant >>> from mindspore import Parameter >>> w1 = Parameter(initializer(Constant(3), [1, 2, 3], mindspore.float32))
- class mindspore.common.initializer.Dirac(groups=1)[source]
Generates an array with the Dirac delta function in order to initialize a tensor. It's usually used in convolution layers, preserves as many identities of the inputs as possible.
- Parameters
groups (int) – The number of groups in convolution layer. Each group applies the same initialization. Default:
1.- Raises
ValueError – If the dimension of the initialized tensor is not in [3, 4, 5].
ValueError – The first dimension of the initialized tensor cannot be divisible by group.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, Dirac >>> from mindspore import Parameter >>> w1 = Parameter(initializer(Dirac(groups=2), [6, 4, 3, 3], mindspore.float32)) >>> w2 = Parameter(initializer("dirac", [6, 4, 3, 3], mindspore.float32))
- class mindspore.common.initializer.HeNormal(negative_slope=0, mode='fan_in', nonlinearity='leaky_relu')[source]
Generates an array with values sampled from HeKaiming Normal distribution \({N}(0, \text{sigma}^2)\) in order to initialize a tensor, where
\[sigma = \frac{gain} {\sqrt{fan\_mode}}\]where \(gain\) is an optional scaling factor. \(fan\_mode\) is the number of input or output units of the weight tensor, depending on the mode is 'fan_in' or 'fan_out'.
For details of HeNormal algorithm, please check https://arxiv.org/abs/1502.01852.
- Parameters
negative_slope (int, float) – The negative slope of the rectifier used after this layer (only used when nonlinearity is 'leaky_relu'). Default:
0.mode (str) – Either
'fan_in'or'fan_out'. Choosing'fan_in'preserves the magnitude of the variance of the weights in the forward pass. Choosing'fan_out'preserves the magnitudes in the backwards pass. Default:'fan_in'.nonlinearity (str) – The non-linear function, recommended to use only with
'relu'or'leaky_relu'. Default:'leaky_relu'.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, HeNormal >>> from mindspore import Parameter >>> w1 = Parameter(initializer(HeNormal(), [1, 2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('he_normal', [1, 2, 3], mindspore.float32))
- class mindspore.common.initializer.HeUniform(negative_slope=0, mode='fan_in', nonlinearity='leaky_relu')[source]
Generates an array with values sampled from HeKaiming Uniform distribution \({U}(-\text{boundary}, \text{boundary})\) in order to initialize a tensor, where
\[boundary = \text{gain} \times \sqrt{\frac{3}{fan\_mode}}\]where \(gain\) is an optional scaling factor. If \(fan\_mode\) is
'fan_in', it is the number of input units of the weight tensor. If \(fan\_mode\) is'fan_out', it is the number of output units of the weight tensor.For details of HeUniform algorithm, please check https://arxiv.org/abs/1502.01852.
- Parameters
negative_slope (int, float, bool) – The negative slope of the rectifier used after this layer (only used when nonlinearity is 'leaky_relu'). Default:
0.mode (str) – Either
'fan_in'or'fan_out'. Choosing'fan_in'preserves the magnitude of the variance of the weights in the forward pass. Choosing'fan_out'preserves the magnitudes in the backwards pass. Default:'fan_in'.nonlinearity (str) – The non-linear function, recommended to use only with
'relu'or'leaky_relu'. Default:'leaky_relu'.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, HeUniform >>> from mindspore import Parameter >>> w1 = Parameter(initializer(HeUniform(), [1, 2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('he_uniform', [1, 2, 3], mindspore.float32))
- class mindspore.common.initializer.Identity(**kwargs)[source]
Generates a 2 dimension identity matrix array in order to initialize a tensor.
- Raises
ValueError – If the dimension of input tensor is not equal to 2.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, Identity >>> from mindspore import Parameter >>> w1 = initializer(Identity(), [2, 3], mindspore.float32) >>> w2 = initializer('identity', [2, 3], mindspore.float32)
- class mindspore.common.initializer.Initializer(**kwargs)[source]
The abstract base class of the initializer.
Note
Initializers are intended to be used for delayed initialization in parallel mode rather than Tensor initialization. If you have to use Initializers to create a Tensor,
mindspore.Tensor.init_data()should be followed in most of the cases. For more information, please refer to mindspore.Tensor.init_data .- Parameters
kwargs (dict) – Keyword arguments for Initializer.
- class mindspore.common.initializer.Normal(sigma=0.01, mean=0.0)[source]
Generates an array with values sampled from Normal distribution \({N}(\text{sigma}, \text{mean})\) in order to initialize a tensor.
\[f(x) = \frac{1} {\sqrt{2*π} * sigma}exp(-\frac{(x - mean)^2} {2*{sigma}^2})\]- Parameters
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, Normal >>> from mindspore import Parameter >>> w1 = Parameter(initializer(Normal(), [1, 2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('normal', [1, 2, 3], mindspore.float32))
- class mindspore.common.initializer.One(**kwargs)[source]
Generates an array with constant value of one in order to initialize a tensor.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, One >>> from mindspore import Parameter >>> w1 = Parameter(initializer(One(), [1, 2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('ones', [1, 2, 3], mindspore.float32))
- class mindspore.common.initializer.Orthogonal(gain=1.0)[source]
Generates a (semi) orthogonal matrix array in order to initialize a tensor. The dimension of input tensor must have at least 2 dimensions. If the dimension is greater than 2, the trailing dimensions will be flattened.
- Parameters
gain (float) – An optional scaling factor. Default:
1.0.- Raises
ValueError – If the dimension of input tensor is less than 2.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, Orthogonal >>> from mindspore import Parameter >>> w1 = Parameter(initializer(Orthogonal(gain=2.), [2, 3, 4], mindspore.float32)) >>> w2 = Parameter(initializer('orthogonal', [2, 3, 4], mindspore.float32))
- class mindspore.common.initializer.Sparse(sparsity, sigma=0.01)[source]
Generates a 2 dimension sparse matrix array in order to initialize a tensor. The non-zero positions will be filled with the value sampled from the normal distribution \({N}(0, sigma)\).
- Parameters
- Raises
ValueError – If the dimension of input tensor is not equal to 2.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, Sparse >>> from mindspore import Parameter >>> w1 = Parameter(initializer(Sparse(sparsity=0.1, sigma=0.01), [5, 8], mindspore.float32))
- class mindspore.common.initializer.TruncatedNormal(sigma=0.01, mean=0.0, a=- 2.0, b=2.0)[source]
Generates an array with values sampled from Truncated Normal distribution in order to initialize a tensor.
- Parameters
sigma (float) – The standard deviation of Truncated Normal distribution. Default:
0.01.mean (float) – The mean of Truncated Normal distribution. Default:
0.0.a (float) – The lower bound of the truncated interval. Default:
-2.0.b (float) – The upper bound of the truncated interval. Default:
2.0.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, TruncatedNormal >>> from mindspore import Parameter >>> w1 = Parameter(initializer(TruncatedNormal(), [1, 2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('truncatedNormal', [1, 2, 3], mindspore.float32))
- class mindspore.common.initializer.Uniform(scale=0.07)[source]
Generates an array with values sampled from Uniform distribution \({U}(-\text{scale}, \text{scale})\) in order to initialize a tensor.
- Parameters
scale (float) – The bound of the Uniform distribution. Default:
0.07.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, Uniform >>> from mindspore import Parameter >>> w1 = Parameter(initializer(Uniform(), [1, 2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('uniform', [1, 2, 3], mindspore.float32))
- class mindspore.common.initializer.VarianceScaling(scale=1.0, mode='fan_in', distribution='truncated_normal')[source]
Generates an random array with scaling in order to initialize a tensor. When distribution is 'truncated_normal' or 'untruncated_normal', the value will be sampled from truncated or untruncated normal distribution with a mean of 0 and a scaled standard deviation \(stddev = \sqrt{\frac{scale}{n}}\). \(n\) will be the number of input units if mode is
'fan_in', while \(n\) will be the number of output units if mode is'fan_out'. \(n\) will be the average of'fan_in'and'fan_out'if mode is'fan_avg'. When distribution is'uniform', the value will be sampled from a uniform distribution within the limit of \([-\sqrt{\frac{3*scale}{n}}, \sqrt{\frac{3*scale}{n}}]\).- Parameters
- Raises
ValueError – If scale is not greater than 0.
ValueError – If mode is not
'fan_in','fan_out'or'fan_avg'.ValueError – If distribution is not
'uniform','truncated_normal'or'untruncated_normal'.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, VarianceScaling >>> from mindspore import Parameter >>> w1 = Parameter(initializer(VarianceScaling(scale=1.0, mode='fan_out', ... distribution='untruncated_normal'), [2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('varianceScaling', [2, 3], mindspore.float32))
- class mindspore.common.initializer.XavierNormal(gain=1)[source]
Generates an array with values sampled from Xavier normal distribution \({N}(0, \text{sigma}^2)\) in order to initialize a tensor, where
\[sigma = gain * \sqrt{\frac{2}{n_{in} + n_{out}}}\]where \(gain\) is an optional scaling factor, \(n_{in}\) is the number of input units in the weight tensor, \(n_{out}\) is the number of output units in the weight tensor.
- Parameters
gain (float) – An optional scaling factor. Default:
1.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, XavierNormal >>> from mindspore import Parameter >>> w1 = Parameter(initializer(XavierNormal(), [1, 2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('xavier_normal', [1, 2, 3], mindspore.float32))
- class mindspore.common.initializer.XavierUniform(gain=1)[source]
Generates an array with values sampled from Xavier uniform distribution \({U}(-\text{boundary}, \text{boundary})\) in order to initialize a tensor, where
\[boundary = gain * \sqrt{\frac{6}{n_{in} + n_{out}}}\]where \(gain\) is an optional scaling factor. \(n_{in}\) is the number of input units in the weight tensor, \(n_{out}\) is the number of output units in the weight tensor.
For details of XavierUniform algorithm, please check http://proceedings.mlr.press/v9/glorot10a.html.
- Parameters
gain (float) – An optional scaling factor. Default:
1.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, XavierUniform >>> from mindspore import Parameter >>> w1 = Parameter(initializer(XavierUniform(), [1, 2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('xavier_uniform', [1, 2, 3], mindspore.float32))
- class mindspore.common.initializer.Zero(**kwargs)[source]
Generates an array with constant value of zero in order to initialize a tensor.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, Zero >>> from mindspore import Parameter >>> w1 = Parameter(initializer(Zero(), [1, 2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('zeros', [1, 2, 3], mindspore.float32))
- mindspore.common.initializer.initializer(init, shape=None, dtype=mstype.float32)[source]
Create and initialize a tensor.
- Parameters
init (Union[Tensor, str, Initializer, numbers.Number]) –
Initialize value.
str: The init should be the alias of the class inheriting from Initializer and the corresponding class will be called in practice. The value of init can be
"normal","ones"or"zeros", etc.Initializer: The init should be the class inheriting from Initializer to initialize tensor.
numbers.Number: The Constant will be called to initialize tensor.
Tensor: The tensor will be called to initialize tensor.
shape (Union[tuple, list, int]) – The shape of the initialized tensor. Default:
None.dtype (
mindspore.dtype) – The type of data in initialized tensor. Default:mstype.float32.
- Returns
Tensor, return is Tensor object.
- Raises
TypeError – The type of the argument 'init' is not correct.
ValueError – The shape of the tensor which is passed through 'init' is not the same as that passed by 'shape'.
Examples
>>> import numpy as np >>> import mindspore >>> from mindspore import Tensor >>> from mindspore.common.initializer import initializer, One >>> from mindspore import Parameter >>> data = Tensor(np.zeros([1, 2, 3]), mindspore.float32) >>> w1 = Parameter(initializer(data, [1, 2, 3], mindspore.float32)) >>> w2 = Parameter(initializer('ones', [1, 2, 3], mindspore.float32)) >>> w3 = Parameter(initializer(One(), [1, 2, 3], mindspore.float32)) >>> w4 = Parameter(initializer(0, [1, 2, 3], mindspore.float32))