mindspore.nn.GLU

class mindspore.nn.GLU(axis=- 1)[source]

The gated linear unit function.

\[{GLU}(a, b)= a \otimes \sigma(b)\]

where \(a\) is the first half of the input matrices and \(b\) is the second half.

Here \(\sigma\) is the sigmoid function, and \(\otimes\) is the Hadamard product.

Parameters

axis (int) – the axis to split the input. Default: -1 , the last axis in x.

Inputs:
  • x (Tensor) - \((\ast_1, N, \ast_2)\) where * means, any number of additional dimensions.

Outputs:

Tensor, the same dtype as the x, with the shape \((\ast_1, M, \ast_2)\) where \(M=N/2\).

Supported Platforms:

Ascend GPU CPU

Examples

>>> import mindspore as ms
>>> m = ms.nn.GLU()
>>> input = ms.Tensor([[0.1,0.2,0.3,0.4],[0.5,0.6,0.7,0.8]])
>>> output = m(input)
>>> print(output)
[[0.05744425 0.11973753]
 [0.33409387 0.41398472]]