mindspore.ops.DynamicGRUV2
- class mindspore.ops.DynamicGRUV2(direction='UNIDIRECTIONAL', cell_depth=1, keep_prob=1.0, cell_clip=- 1.0, num_proj=0, time_major=True, activation='tanh', gate_order='rzh', reset_after=True, is_training=True)[source]
Applies a single-layer gated recurrent unit (GRU) to an input sequence.
\[\begin{split}\begin{array}{ll} r_{t+1} = \sigma(W_{ir} x_{t+1} + b_{ir} + W_{hr} h_{(t)} + b_{hr}) \\ z_{t+1} = \sigma(W_{iz} x_{t+1} + b_{iz} + W_{hz} h_{(t)} + b_{hz}) \\ n_{t+1} = \tanh(W_{in} x_{t+1} + b_{in} + r_{t+1} * (W_{hn} h_{(t)}+ b_{hn})) \\ h_{t+1} = (1 - z_{t+1}) * n_{t+1} + z_{t+1} * h_{(t)} \end{array}\end{split}\]where \(h_{t+1}\) is the hidden state at time t+1, \(x_{t+1}\) is the input at time t+1, \(h_{t}\) is the hidden state of the layer at time t or the initial hidden state at time 0, and \(r_{t+1}\), \(z_{t+1}\), \(n_{t+1}\) are the reset, update, and new gates, respectively. \(W\), \(b\) are the weight parameter and the deviation parameter respectively. \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product.
- Parameters
direction (str) – A string identifying the direction in the operator. Default: ‘UNIDIRECTIONAL’. Only ‘UNIDIRECTIONAL’ is currently supported.
cell_depth (int) – An integer identifying the cell depth in the operator. Default: 1.
keep_prob (float) – A float identifying the keep prob in the operator. Default: 1.0.
cell_clip (float) – A float identifying the cell clip in the operator. Default: -1.0.
num_proj (int) – An integer identifying the number projection in the operator. Default: 0.
time_major (bool) – A bool identifying the time major in the operator. Default: True.
activation (str) – A string identifying the type of activation function in the operator. Default: ‘tanh’. Only ‘tanh’ is currently supported.
gate_order (str) – A string identifying the gate order in weight and bias. Default: ‘rzh’. ‘zrh’ is another option. Here, ‘rzh’ means the gate order is: reset gate, update gate, hidden gate. ‘zrh’ means the gate order is: update gate, reset gate, hidden gate.
reset_after (bool) – A bool identifying whether to apply reset gate after matrix multiplication. Default: True.
is_training (bool) – A bool identifying is training in the operator. Default: True.
- Inputs:
x (Tensor) - Current words. Tensor of shape \((\text{num_step}, \text{batch_size}, \text{input_size})\). The data type must be float16.
weight_input (Tensor) - Input-hidden weight \(W_{\{ir,iz,in\}}\). Tensor of shape \((\text{input_size}, 3 \times \text{hidden_size})\). The data type must be float16.
weight_hidden (Tensor) - Hidden-hidden weight \(W_{\{hr,hz,hn\}}\). Tensor of shape \((\text{hidden_size}, 3 \times \text{hidden_size})\). The data type must be float16.
bias_input (Tensor) - Input-hidden bias \(b_{\{ir,iz,in\}}\). Tensor of shape \((3 \times \text{hidden_size})\), or None. Has the same data type with input init_h.
bias_hidden (Tensor) - Hidden-hidden bias \(b_{\{hr,hz,hn\}}\). Tensor of shape \((3 \times \text{hidden_size})\), or None. Has the same data type with input init_h.
seq_length (Tensor) - The length of each batch. Tensor of shape \((\text{batch_size})\). Only None is currently supported.
init_h (Tensor) - Hidden state of initial time. Tensor of shape \((\text{batch_size}, \text{hidden_size})\). The data type must be float16 or float32.
- Outputs:
y (Tensor) - A Tensor of shape:
y_shape = \((num\_step, batch\_size, min(hidden\_size, num\_proj))\): If num_proj > 0,
y_shape = \((num\_step, batch\_size, hidden\_size)\): If num_proj = 0.
Has the same data type with input bias_type.
output_h (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
update (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
reset (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
new (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
hidden_new (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
A note about the bias_type:
If bias_input and bias_hidden both are None, bias_type is the data type of init_h.
If bias_input is not None, bias_type is the data type of bias_input.
If bias_input is None and bias_hidden is not None, bias_type is the data type of bias_hidden.
- Raises
TypeError – If direction, activation or gate_order is not a str.
TypeError – If cell_depth or num_proj is not an int.
TypeError – If keep_prob or cell_clip is not a float.
TypeError – If time_major, reset_after or is_training is not a bool.
TypeError – If x, weight_input, weight_hidden, bias_input, bias_hidden, seq_length or ini_h is not a Tensor.
TypeError – If dtype of x, weight_input or weight_hidden is not float16.
TypeError – If dtype of init_h is neither float16 nor float32.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(2, 8, 64).astype(np.float16)) >>> weight_i = Tensor(np.random.rand(64, 48).astype(np.float16)) >>> weight_h = Tensor(np.random.rand(16, 48).astype(np.float16)) >>> bias_i = Tensor(np.random.rand(48).astype(np.float16)) >>> bias_h = Tensor(np.random.rand(48).astype(np.float16)) >>> init_h = Tensor(np.random.rand(8, 16).astype(np.float16)) >>> dynamic_gru_v2 = ops.DynamicGRUV2() >>> output = dynamic_gru_v2(x, weight_i, weight_h, bias_i, bias_h, None, init_h) >>> print(output[0].shape) (2, 8, 16)