Differences with torch.nn.LeakyReLU
torch.nn.LeakyReLU
class torch.nn.LeakyReLU(negative_slope=0.01, inplace=False)(input) -> Tensor
For more information, see torch.nn.LeakyReLU.
mindspore.nn.LeakyReLU
class mindspore.nn.LeakyReLU(alpha=0.2)(x) -> Tensor
For more information, see mindspore.nn.LeakyReLU.
Differences
PyTorch: Implements the basic functions of the Leaky ReLU activation function, where the parameter negative_slope
is used to control the slope of the activation function, and the parameter inplace
is used to control whether to choose to perform the operation of the activation function in-place.
MindSpore: MindSpore API basically implements the same function as PyTorch, where the parameter alpha
is the same as the parameter negative_slope
in PyTorch, with different parameter names and different default values. However, MindSpore does not have the inplace
parameter.
Categories |
Subcategories |
PyTorch |
MindSpore |
Difference |
---|---|---|---|---|
Parameters |
Parameter 1 |
negative_slope |
alpha |
Same function, different parameter names, different default values |
Parameter 2 |
inplace |
- |
This parameter is used in PyTorch to control whether to choose to perform the activation function in-place. MindSpore does not have this parameter |
|
Input |
Single input |
input |
x |
Same function, different parameter names |
Code Example 1
PyTorch parameter
negative_slope
and MindSpore parameteralpha
have the same function, with different parameter names and different default values, and get the same result when both values are the same.
# PyTorch
import torch
import torch.nn as nn
m = nn.LeakyReLU(0.2)
inputs = torch.tensor([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]], dtype=float)
output = m(inputs).to(torch.float32).detach().numpy()
print(output)
# [[-0.2 4. -1.6]
# [ 2. -1. 9. ]]
# MindSpore
import mindspore
from mindspore import Tensor
import mindspore.nn as nn
x = Tensor([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]).astype('float32')
m = nn.LeakyReLU()
output = m(x)
print(output)
# [[-0.2 4. -1.6]
# [ 2. -1. 9. ]]
Code Example 2
PyTorch parameter
inplace
is used to control whether to perform the activation function operation in-place, that is, to perform the activation function operation directly on the input, where the input is changed. MindSpore does not have this parameter, but can assign the output to the input to achieve similar function.
# PyTorch
import torch
import torch.nn as nn
m = nn.LeakyReLU(0.2, inplace=True)
inputs = torch.tensor([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]], dtype=torch.float32)
output = m(inputs)
print(inputs.detach().numpy())
# [[-0.2 4. -1.6]
# [ 2. -1. 9. ]]
# MindSpore
import mindspore
from mindspore import Tensor
import mindspore.nn as nn
x = Tensor([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]).astype('float32')
m = nn.LeakyReLU()
x = m(x)
print(x)
# [[-0.2 4. -1.6]
# [ 2. -1. 9. ]]