mindspore.ops.DynamicRNN

View Source On Gitee
class mindspore.ops.DynamicRNN(cell_type='LSTM', direction='UNIDIRECTIONAL', cell_depth=1, use_peephole=False, keep_prob=1.0, cell_clip=- 1.0, num_proj=0, time_major=True, activation='tanh', forget_bias=0.0, is_training=True)[source]

Applies a recurrent neural network to the input. Only long short-term memory (LSTM) is supported currently.

\[\begin{split}\begin{array}{ll} \\ i_{t+1} = \sigma(W_{ix} x_{t+1} + b_{ix} + W_{ih} h_{(t)} + b_{ih}) \\ f_{t+1} = \sigma(W_{fx} x_{t+1} + b_{fx} + W_{fh} h_{(t)} + b_{fh}) \\ \tilde{c}_{t+1} = \tanh(W_{cx} x_{t+1} + b_{cx} + W_{ch} h_{(t)} + b_{ch}) \\ o_{t+1} = \sigma(W_{ox} x_{t+1} + b_{ox} + W_{oh} h_{(t)} + b_{oh}) \\ c_{t+1} = f_{t+1} * c_{(t)} + i_t * \tilde{c}_{t+1} \\ h_{t+1} = o_{t+1} * \tanh(c_{t+1}) \\ \end{array}\end{split}\]

\(h_{t+1}\) is the hidden state at time t+1. \(x_{t+1}\) is the input at time t+1. \(h_{t}\) is the hidden state of the layer at time t or the initial hidden state at time 0. \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product. \(W, b\) are learnable weights between the output and the input in the formula. For instance, \(W_{ix}, b_{ix}\) are the weight and bias used to transform from input \(x\) to \(i\).

Parameters

  • cell_type (str, optional) – A string identifying the cell type in the operator. Default: 'LSTM' . Only 'LSTM' is currently supported.

  • direction (str, optional) – A string identifying the direction in the operator. Default: 'UNIDIRECTIONAL' . Only 'UNIDIRECTIONAL' is currently supported.

  • cell_depth (int, optional) – An integer identifying the cell depth in the operator. Default: 1 .

  • use_peephole (bool, optional) – A bool identifying if use peephole in the operator. Default: False .

  • keep_prob (float, optional) – A float identifying the keep prob in the operator. Default: 1.0 .

  • cell_clip (float, optional) – A float identifying the cell clip in the operator. Default: -1.0 .

  • num_proj (int, optional) – An integer identifying the number projection in the operator. Default: 0 .

  • time_major (bool, optional) – A bool specify the data format of x. If it is set to True , the format is \((num\_step, batch\_size, input\_size)\), if it is set to False, the format is \((batch\_size, num\_step, input\_size)\). Default: True . Only supports True at present.

  • activation (str, optional) – A string identifying the type of activation function in the operator. Default: 'tanh' . Only 'tanh' is currently supported.

  • forget_bias (float, optional) – A float identifying the forget bias in the operator. Default: 0.0 .

  • is_training (bool, optional) – A bool identifying is training in the operator. Default: True .

Inputs:

  • x (Tensor) - Current words. Tensor of shape \((num\_step, batch\_size, input\_size)\). The data type must be float16.

  • w (Tensor) - Weight. Tensor of shape \((input\_size + hidden\_size, 4 * hidden\_size)\). The data type must be float16.

  • b (Tensor) - Bias. Tensor of shape \((4 * hidden\_size)\). The data type must be float16.

  • seq_length (Tensor) - The length of each batch. Tensor of shape \((batch\_size, )\). Only None is currently supported.

  • init_h (Tensor) - Hidden state of initial time. Tensor of shape \((1, batch\_size, hidden\_size)\). The data type must be float16.

  • init_c (Tensor) - Cell state of initial time. Tensor of shape \((1, batch\_size, hidden\_size)\). The data type must be float16.

Outputs:

  • y (Tensor) - A Tensor of shape \((num\_step, batch\_size, hidden\_size)\). Has the same type with input b.

  • output_h (Tensor) - A Tensor of shape \((num\_step, batch\_size, hidden\_size)\). With data type of float16.

  • output_c (Tensor) - A Tensor of shape \((num\_step, batch\_size, hidden\_size)\). Has the same type with input b.

  • i (Tensor) - A Tensor of shape \((num\_step, batch\_size, hidden\_size)\). Has the same type with input b.

  • j (Tensor) - A Tensor of shape \((num\_step, batch\_size, hidden\_size)\). Has the same type with input b.

  • f (Tensor) - A Tensor of shape \((num\_step, batch\_size, hidden\_size)\). Has the same type with input b.

  • o (Tensor) - A Tensor of shape \((num\_step, batch\_size, hidden\_size)\). Has the same type with input b.

  • tanhct (Tensor) - A Tensor of shape \((num\_step, batch\_size, hidden\_size)\). Has the same type with input b.

Raises

  • TypeError – If cell_type, direction or activation is not a str.

  • TypeError – If cell_depth or num_proj is not an int.

  • TypeError – If keep_prob, cell_clip or forget_bias is not a float.

  • TypeError – If use_peehpole, time_major or is_training is not a bool.

  • TypeError – If x, w, b, seq_length, init_h or init_c is not a Tensor.

  • TypeError – If dtype of x, w, init_h or init_c is not float16.

  • TypeError – If dtype of b is neither float16 nor float32.

Supported Platforms:

Ascend

Examples

>>> import numpy as np
>>> from mindspore import Tensor, ops
>>> x = Tensor(np.random.rand(2, 16, 64).astype(np.float16))
>>> w = Tensor(np.random.rand(96, 128).astype(np.float16))
>>> b = Tensor(np.random.rand(128).astype(np.float16))
>>> init_h = Tensor(np.random.rand(1, 16, 32).astype(np.float16))
>>> init_c = Tensor(np.random.rand(1, 16, 32).astype(np.float16))
>>> dynamic_rnn = ops.DynamicRNN()
>>> output = dynamic_rnn(x, w, b, None, init_h, init_c)
>>> print(output[0].shape)
(2, 16, 32)