mindspore.ops.FusedSparseFtrl

class mindspore.ops.FusedSparseFtrl(lr, l1, l2, lr_power, use_locking=False)[source]

Merges the duplicate value of the gradient and then updates relevant entries according to the FTRL-proximal scheme.

All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.

Parameters

  • lr (float) – The learning rate value, must be positive.

  • l1 (float) – l1 regularization strength, must be greater than or equal to zero.

  • l2 (float) – l2 regularization strength, must be greater than or equal to zero.

  • lr_power (float) – Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero.

  • use_locking (bool) – Use locks for updating operation if true . Default: False.

Inputs:

  • var (Parameter) - The variable to be updated. The data type must be float32. The shape is \((N, *)\) where \(*\) means, any number of additional dimensions.

  • accum (Parameter) - The accumulation to be updated, must be same type and shape as var.

  • linear (Parameter) - the linear coefficient to be updated, must be same type and shape as var.

  • grad (Tensor) - A tensor of the same type as var and grad.shape[1:] = var.shape[1:] if var.shape > 1.

  • indices (Tensor) - A vector of indices into the first dimension of var and accum. The type must be int32 and indices.shape[0] = grad.shape[0].

Outputs:

Tuple of 3 Tensor, this operator will update the input parameters directly, the outputs are useless.

  • var (Tensor) - A Tensor with shape \((1, )\).

  • accum (Tensor) - A Tensor with shape \((1, )\).

  • linear (Tensor) - A Tensor with shape \((1, )\).

Raises

  • TypeError – If lr, l1, l2 or lr_power is not a float.

  • ValueError – If shape of lr_power less than or equal to zero.

  • TypeError – If dtype of var is not float32.

  • TypeError – If dtype of indices is not int32.

  • TypeError – If shape of accum, linear or grad is not same as var.

  • TypeError – If shape of indices is not same as shape of first dimension of grad.

Supported Platforms:

Ascend CPU

Examples

>>> class SparseApplyFtrlNet(nn.Cell):
...     def __init__(self):
...         super(SparseApplyFtrlNet, self).__init__()
...         self.sparse_apply_ftrl = ops.FusedSparseFtrl(lr=0.01, l1=0.0, l2=0.0, lr_power=-0.5)
...         self.var = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="var")
...         self.accum = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="accum")
...         self.linear = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="linear")
...
...     def construct(self, grad, indices):
...         out = self.sparse_apply_ftrl(self.var, self.accum, self.linear, grad, indices)
...         return out
...
>>> net = SparseApplyFtrlNet()
>>> grad = Tensor(np.array([[[0.1, 0.1]], [[0.1, 0.1]]]).astype(np.float32))
>>> indices = Tensor(np.array([0, 1]).astype(np.int32))
>>> output = net(grad, indices)
>>> print(net.var.asnumpy())
[[[-0.00598256 -0.00598256]]
 [[-0.00598256 -0.00598256]]
 [[ 1.          1.        ]]]