mindspore.amp.FixedLossScaleManager

class mindspore.amp.FixedLossScaleManager(loss_scale=128.0, drop_overflow_update=True)

Loss scale(Magnification factor of gradients when mix precision is used) manager with a fixed loss scale value, inherits from mindspore.amp.LossScaleManager.

Parameters

• loss_scale (float) – Magnification factor of gradients. Note that if drop_overflow_update is set to False, the value of loss_scale in optimizer should be set to the same as here. Default: 128.0.

• drop_overflow_update (bool) – Whether to execute optimizer if there is an overflow. If True, the optimizer will not executed when overflow occurs. Default: True.

Examples

>>> import mindspore as ms
>>> from mindspore import nn
>>>
>>> net = Net()
>>> #1) Drop the parameter update if there is an overflow
>>> loss_scale_manager = ms.FixedLossScaleManager()
>>> optim = nn.Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9)
>>> model = ms.Model(net, loss_scale_manager=loss_scale_manager, optimizer=optim)
>>>
>>> #2) Execute parameter update even if overflow occurs
>>> loss_scale = 1024.0
>>> loss_scale_manager = ms.FixedLossScaleManager(loss_scale, False)
>>> optim = nn.Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9, loss_scale=loss_scale)
>>> model = ms.Model(net, loss_scale_manager=loss_scale_manager, optimizer=optim)

get_drop_overflow_update()

Get drop_overflow_update, whether to drop optimizer update for current step when there is an overflow.

Returns

bool, drop_overflow_update value.

get_loss_scale()

Get loss scale value.

Returns

bool, loss_scale value.

get_update_cell()

Returns the instance of mindspore.nn.Cell that used to update the loss scale which will be called at mindspore.nn.TrainOneStepWithLossScaleCell. As the loss scale is fixed in this class, the instance will do nothing.

Returns

None or mindspore.nn.FixedLossScaleUpdateCell. Instance of mindspore.nn.FixedLossScaleUpdateCell when drop_overflow_update is True. None when drop_overflow_update is False.

update_loss_scale(overflow)

Update loss scale value. The interface at FixedLossScaleManager will do nothing.

Parameters

overflow (bool) – Whether it overflows.