mindspore.nn.GLU

class mindspore.nn.GLU(axis=- 1)

The gated linear unit function.

$$GLU(a,b)=a\otimes \sigma (b)$$

where $a$ is the first half of the input matrices and $b$ is the second half.

Here $\sigma$ is the sigmoid function, and $\otimes$ is the Hadamard product.

Parameters

axis (int) – the axis to split the input. Default: -1 , the last axis in x.

Inputs:

• x (Tensor) - $({\ast }_1,N,{\ast }_2)$ where * means, any number of additional dimensions.

Outputs:

Tensor, the same dtype as the x, with the shape $({\ast }_1,M,{\ast }_2)$ where $M=N/2$.

Supported Platforms:

Ascend GPU CPU

Examples

>>> import mindspore as ms
>>> m = ms.nn.GLU()
>>> input = ms.Tensor([[0.1,0.2,0.3,0.4],[0.5,0.6,0.7,0.8]])
>>> output = m(input)
>>> print(output)
[[0.05744425 0.11973753]
 [0.33409387 0.41398472]]