Function Differences with tf.nn.dropout

View Source On Gitee

tf.nn.dropout

tf.nn.dropout(
    x,
    rate,
    noise_shape=None,
    seed=None,
    name=None
) -> Tensor

For more information, see tf.nn.dropout.

mindspore.ops.dropout

mindspore.ops.dropout(x, p=0.5, seed0=0, seed1=0) -> Tensor

For more information, see mindspore.ops.dropout.

Differences

TensorFlow: dropout is a function used to prevent or mitigate overfitting by dropping a random portion of neurons at different training sessions. That is, the neuron output is randomly set to 0 with a certain probability p, which serves to reduce the neuron correlation. The remaining parameters that are not set to 0 will be scaled with \(\frac{1}{1-rate}\).

MindSpore: MindSpore API basically implements the same function as TensorFlow, but TensorFlow has an additional noise_shape parameter to control the retention/discard dimension.

Categories

Subcategories

TensorFlow

MindSpore

Differences

Parameters

Parameter 1

x

x

-

Parameter 2

rate

p

Same function, different parameter names

Parameter 3

noise_shape

-

A 1 int32 tensor representing a randomly generated “keep/drop” flag for the shape. MindSpore does not have this parameter

Parameter 4

seed

seed0

Same function, different parameter names

Parameter 5

name

Not involved

Parameter 6

-

seed1

The global random seed, which together with the random seed of the operator layer determines the final generated random number. Default value: 0

Code Example 1

When the value of noise_shape is None, the two APIs functions are the same.

# TensorFlow
import tensorflow as tf
import numpy as np
neuros = np.array([[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                   [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]],dtype=np.float32)
neuros_drop = tf.nn.dropout(neuros, rate=0.2)
print(neuros_drop.shape)
# (10, 10)

# MindSpore
import mindspore
x = mindspore.Tensor([[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                      [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]], mindspore.float32)
output, mask = mindspore.ops.dropout(x, p=0.2)
print(output.shape)
# (10, 10)