mindspore.ops.DynamicGRUV2

class mindspore.ops.DynamicGRUV2(direction='UNIDIRECTIONAL', cell_depth=1, keep_prob=1.0, cell_clip=- 1.0, num_proj=0, time_major=True, activation='tanh', gate_order='rzh', reset_after=True, is_training=True)

Applies a single-layer gated recurrent unit (GRU) to an input sequence.

$$\begin{array}{r}\begin{array}{l}{r}_{t+1}=\sigma (W_{ir}{x}_{t+1}+b_{ir}+W_{hr}{h}_{(t)}+b_{hr})\\ z_{t+1}=\sigma (W_{iz}{x}_{t+1}+b_{iz}+W_{hz}{h}_{(t)}+b_{hz})\\ n_{t+1}=\tanh (W_{in}{x}_{t+1}+b_{in}+r_{t+1}\ast (W_{hn}{h}_{(t)}+b_{hn}))\\ h_{t+1}=(1-z_{t+1})\ast n_{t+1}+z_{t+1}\ast h_{(t)}\end{array}\end{array}$$

where $h_{t+1}$ is the hidden state at time t+1, $x_{t+1}$ is the input at time t+1, $h_t$ is the hidden state of the layer at time t or the initial hidden state at time 0. $r_{t+1}$, $z_{t+1}$, $n_{t+1}$ are the reset, update, and new gates, respectively. $W$, $b$ are the weight parameter and the deviation parameter respectively. $\sigma$ is the sigmoid function, and $\ast$ is the Hadamard product.

Parameters

• direction (str, optional) – A string identifying the direction in the operator. Default: 'UNIDIRECTIONAL' . Only 'UNIDIRECTIONAL' is currently supported.

• cell_depth (int, optional) – An integer identifying the cell depth in the operator. Default: 1 .

• keep_prob (float, optional) – A float identifying the keep prob in the operator. Default: 1.0 .

• cell_clip (float, optional) – A float identifying the cell clip in the operator. Default: -1.0 .

• num_proj (int, optional) – An integer identifying the number projection in the operator. Default: 0 .

• time_major (bool, optional) – A bool identifying the time major in the operator. Default: True .

• activation (str, optional) – A string identifying the type of activation function in the operator. Default: 'tanh' . Only 'tanh' is currently supported.

• gate_order (str, optional) – A string identifying the gate order in weight and bias. Default: 'rzh' . 'zrh' is another option. Here, 'rzh' means the gate order is: reset gate, update gate, hidden gate. 'zrh' means the gate order is: update gate, reset gate, hidden gate.

• reset_after (bool, optional) – A bool identifying whether to apply reset gate after matrix multiplication. Default: True .

• is_training (bool, optional) – A bool identifying is training in the operator. Default: True .

Inputs:

• x (Tensor) - Current words. Tensor of shape $(\text{num\_step},\text{batch\_size},\text{input\_size})$. The data type must be float16.

• weight_input (Tensor) - Input-hidden weight $W_{\{ir,iz,in\}}$. Tensor of shape $(\text{input\_size},3\times \text{hidden\_size})$. The data type must be float16.

• weight_hidden (Tensor) - Hidden-hidden weight $W_{\{hr,hz,hn\}}$. Tensor of shape $(\text{hidden\_size},3\times \text{hidden\_size})$. The data type must be float16.

• bias_input (Tensor) - Input-hidden bias $b_{\{ir,iz,in\}}$. Tensor of shape $(3\times \text{hidden\_size})$, or None. Has the same data type with input init_h.

• bias_hidden (Tensor) - Hidden-hidden bias $b_{\{hr,hz,hn\}}$. Tensor of shape $(3\times \text{hidden\_size})$, or None. Has the same data type with input init_h.

• seq_length (Tensor) - The length of each batch. Tensor of shape $(\text{batch\_size})$. Only None is currently supported.

• init_h (Tensor) - Hidden state of initial time. Tensor of shape $(\text{batch\_size},\text{hidden\_size})$. The data type must be float16 or float32.

Outputs:

• y (Tensor) - A Tensor of shape:

  • y_shape = $(num\mathrm{\_}step,batch\mathrm{\_}size,min(hidden\mathrm{\_}size,num\mathrm{\_}proj))$: If num_proj > 0,

  • y_shape = $(num\mathrm{\_}step,batch\mathrm{\_}size,hidden\mathrm{\_}size)$: If num_proj = 0.

  Has the same data type with input bias_type.

• output_h (Tensor) - A Tensor of shape $(\text{num\_step},\text{batch\_size},\text{hidden\_size})$. Has the same data type with input bias_type.

• update (Tensor) - A Tensor of shape $(\text{num\_step},\text{batch\_size},\text{hidden\_size})$. Has the same data type with input bias_type.

• reset (Tensor) - A Tensor of shape $(\text{num\_step},\text{batch\_size},\text{hidden\_size})$. Has the same data type with input bias_type.

• new (Tensor) - A Tensor of shape $(\text{num\_step},\text{batch\_size},\text{hidden\_size})$. Has the same data type with input bias_type.

• hidden_new (Tensor) - A Tensor of shape $(\text{num\_step},\text{batch\_size},\text{hidden\_size})$. Has the same data type with input bias_type.

A note about the bias_type:

• If bias_input and bias_hidden both are None, bias_type is the data type of init_h.

• If bias_input is not None, bias_type is the data type of bias_input.

• If bias_input is None and bias_hidden is not None, bias_type is the data type of bias_hidden.

Raises

• TypeError – If direction, activation or gate_order is not a str.

• TypeError – If cell_depth or num_proj is not an int.

• TypeError – If keep_prob or cell_clip is not a float.

• TypeError – If time_major, reset_after or is_training is not a bool.

• TypeError – If x, weight_input, weight_hidden, bias_input, bias_hidden, seq_length or ini_h is not a Tensor.

• TypeError – If dtype of x, weight_input or weight_hidden is not float16.

• TypeError – If dtype of init_h is neither float16 nor float32.

Supported Platforms:

Ascend

Examples

>>> import numpy as np
>>> from mindspore import Tensor, ops
>>> x = Tensor(np.random.rand(2, 8, 64).astype(np.float16))
>>> weight_i = Tensor(np.random.rand(64, 48).astype(np.float16))
>>> weight_h = Tensor(np.random.rand(16, 48).astype(np.float16))
>>> bias_i = Tensor(np.random.rand(48).astype(np.float16))
>>> bias_h = Tensor(np.random.rand(48).astype(np.float16))
>>> init_h = Tensor(np.random.rand(8, 16).astype(np.float16))
>>> dynamic_gru_v2 = ops.DynamicGRUV2()
>>> output = dynamic_gru_v2(x, weight_i, weight_h, bias_i, bias_h, None, init_h)
>>> print(output[0].shape)
(2, 8, 16)