Function Differences with tf.compat.v1.train.exponential_decay

View Source On Gitee

tf.compat.v1.train.exponential_decay

tf.compat.v1.train.exponential_decay(
    learning_rate,
    global_step,
    decay_steps,
    decay_rate,
    staircase=False,
    name=None
) -> Tensor

For more information, see tf.compat.v1.train.exponential_decay.

mindspore.nn.exponential_decay_lr

mindspore.nn.exponential_decay_lr(
    learning_rate,
    decay_rate,
    total_step,
    step_per_epoch,
    decay_epoch,
    is_stair=False
) -> list[float]

For more information, see mindspore.nn.exponential_decay_lr.

Differences

TensorFlow: calculate the learning rate based on the exponential decay function.

MindSpore: MindSpore API basically implements the same function as TensorFlow.

Categories

Subcategories

TensorFlow

MindSpore

Differences

Parameters

Parameter 1

learning_rate

learning_rate

-

Parameter 2

global_step

total_step

Same function, different parameter names

Parameter 3

decay_steps

decay_epoch

Same function, different parameter names

Parameter 4

decay_rate

decay_rate

-

Parameter 5

staircase

is_stair

Same function, different parameter names

Parameter 6

name

-

Not involved

Parameter 7

-

step_per_epoch

The number of steps per epoch, TensorFlow does not have this parameter

Code Example

The two APIs achieve the same function and have the same usage.

# TensorFlow
import tensorflow as tf

learning_rate = 1.0
decay_rate = 0.9
step_per_epoch = 2
epochs = 3
lr = []
for epoch in range(epochs):
    learning_rate = tf.compat.v1.train.exponential_decay(learning_rate, epoch, step_per_epoch, decay_rate, staircase=True)
    learning_rate = learning_rate().numpy().item()
    lr.append(round(float(learning_rate), 2))
print(lr)
# [1.0, 1.0, 0.9]

# MindSpore
import mindspore.nn as nn

learning_rate = 1.0
decay_rate = 0.9
total_step = 3
step_per_epoch = 2
decay_epoch = 1
output = nn.exponential_decay_lr(learning_rate, decay_rate, total_step, step_per_epoch, decay_epoch)
print(output)
# [1.0, 1.0, 0.9]