Differences with torch.nn.GELU

View Source On Gitee

torch.nn.GELU

class torch.nn.GELU()(input) -> Tensor

For more information, see torch.nn.GELU.

mindspore.nn.GELU

class mindspore.nn.GELU(approximate=True)(x) -> Tensor

For more information, see mindspore.nn.GELU.

Differences

PyTorch: This function represents the Gaussian error linear unit function \(GELU(X)=X\times \Phi(x)\), where \(\Phi(x)\) is the cumulative distribution function of the Gaussian distribution. The input x denotes an arbitrary number of dimensions.

MindSpore: MindSpore API implements basically the same function as PyTorch.

Categories

Subcategories

PyTorch

MindSpore

Difference

Parameter

Parameter 1

-

approximate

Determines whether approximation is enabled or not, and the default value is True. After testing, the output is more similar to Pytorch when approximate is False

Input

Single input

input

x

Same function, different parameter names

Code Example 1

The two APIs achieve the same function and have the same usage.

# PyTorch
import torch
input_x = torch.Tensor([[2, 4], [1, 2]])
output = torch.nn.GELU()(input_x)
print(output.detach().numpy())
# [[1.9544997 3.9998734]
#  [0.8413447 1.9544997]]

# MindSpore
import mindspore
import numpy as np
x = mindspore.Tensor(np.array([[2, 4], [1, 2]]), mindspore.float32)
output = mindspore.nn.GELU(approximate=False)(x)
print(output)
# [[1.9544997 3.9998732]
#  [0.8413447 1.9544997]]