Function Differences with tf.keras.layers.PReLU

View Source On Gitee

tf.keras.layers.PReLU

tf.keras.layers.PReLU(
  alpha_initializer='zeros',
  alpha_regularizer=None,
  alpha_constraint=None,
  shared_axes=None
)(x) -> Tensor

For more information, see tf.keras.layers.PReLU.

mindspore.nn.PReLU

class mindspore.nn.PReLU(channel=1, w=0.25)(x) -> Tensor

For more information, see mindspore.nn.PReLU.

Differences

TensorFlow: PReLU activation function.

MindSpore: MindSpore API basically implements the same function as TensorFlow, but the parameter setting is different.

Categories

Subcategories

TensorFlow

MindSpore

Differences

Parameters

parameter 1

alpha_initializer

w

Initialization function of weights, same function of parameters, different default values, different parameter names

parameter 2

alpha_regularizer

-

Regularizer of weights. MindSpore does not have this parameter.

parameter 3

alpha_constraint

-

Constraints of Weights. MindSpore does not have this parameter.

parameter 4

shared_axes

-

Shared axes of learnable parameters of the activation function. MindSpore does not have this parameter.

parameter 5

-

channel

TensorFlow does not have this parameter.

Input

Single input

x

x

-

Code Example 1

TensorFlow alpha_initializer parameter is functionally identical to MindSpore parameter, with different default values and different parameter names. Default alpha of TensorFlow is 0.0, so using MindSpore, you only need to set w to 0.0 to achieve the same function.

# TensorFlow
import tensorflow as tf
from keras.layers import PReLU
import numpy as np

x = tf.constant([[-1.0, 2.2], [3.3, -4.0]], dtype=tf.float32)
m = PReLU()
out = m(x)
print(out.numpy())
# [[0.  2.2]
#  [3.3 0. ]]

# MindSpore
import mindspore
from mindspore import Tensor
import mindspore.nn as nn
import numpy as np

x = Tensor(np.array([[-1.0, 2.2], [3.3, -4.0]]), mindspore.float32)
prelu = nn.PReLU(w=0.0)
output = prelu(x)
print(output)
# [[0.  2.2]
#  [3.3 0. ]]

Code Example 2

TensorFlow alpha_initializer parameter can change the alpha value through the initialization function, and MindSpore simply sets w to the corresponding value to achieve the same function.

# TensorFlow
import tensorflow as tf
from keras.layers import PReLU
import numpy as np
x = tf.constant([[-1.0, 2.2], [3.3, -4.0]], dtype=tf.float32)
m = PReLU(alpha_initializer=tf.constant_initializer(0.5))
out = m(x)
print(out.numpy())
# [[-0.5  2.2]
#  [ 3.3 -2. ]]

# MindSpore
import mindspore
from mindspore import Tensor
import mindspore.nn as nn
import numpy as np
x = Tensor(np.array([[-1.0, 2.2], [3.3, -4.0]]), mindspore.float32)
prelu = nn.PReLU(w=0.5)
output = prelu(x)
print(output)
# [[-0.5  2.2]
#  [ 3.3 -2. ]]