mindspore.ops.DynamicRNN

class mindspore.ops.DynamicRNN(*args, **kwargs)[source]

Applies a recurrent neural network to the input. Only long short-term memory (LSTM) currently supported.

\[\begin{split}\begin{array}{ll} \\ i_{t+1} = \sigma(W_{ix} x_{t+1} + b_{ix} + W_{ih} h_{(t)} + b_{ih}) \\ f_{t+1} = \sigma(W_{fx} x_{t+1} + b_{fx} + W_{fh} h_{(t)} + b_{fh}) \\ \tilde{c}_{t+1} = \tanh(W_{cx} x_{t+1} + b_{cx} + W_{ch} h_{(t)} + b_{ch}) \\ o_{t+1} = \sigma(W_{ox} x_{t+1} + b_{ox} + W_{oh} h_{(t)} + b_{oh}) \\ c_{t+1} = f_{t+1} * c_{(t)} + i_t * \tilde{c}_{t+1} \\ h_{t+1} = o_{t+1} * \tanh(c_{t+1}) \\ \end{array}\end{split}\]

Here \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product. \(W, b\) are learnable weights between the output and the input in the formula. For instance, \(W_{ix}, b_{ix}\) are the weight and bias used to transform from input \(x\) to \(i\).

Note

The hidden_size in shape of inputs must be multiple of 16.

Parameters

  • cell_type (str) – A string identifying the cell type in the op. Default: ‘LSTM’. Only ‘LSTM’ is currently supported.

  • direction (str) – A string identifying the direction in the op. Default: ‘UNIDIRECTIONAL’. Only ‘UNIDIRECTIONAL’ is currently supported.

  • cell_depth (int) – An integer identifying the cell depth in the op. Default: 1.

  • use_peephole (bool) – A bool identifying if use peephole in the op. Default: False.

  • keep_prob (float) – A float identifying the keep prob in the op. Default: 1.0.

  • cell_clip (float) – A float identifying the cell clip in the op. Default: -1.0.

  • num_proj (int) – An integer identifying the num proj in the op. Default: 0.

  • time_major (bool) – A bool identifying the time major in the op. Default: True. Only True is currently supported.

  • activation (str) – A string identifying the type of activation function in the op. Default: ‘tanh’. Only ‘tanh’ is currently supported.

  • forget_bias (float) – A float identifying the forget bias in the op. Default: 0.0.

  • is_training (bool) – A bool identifying is training in the op. Default: True.

Inputs:

  • x (Tensor) - Current words. Tensor of shape (num_step, batch_size, input_size). The data type must be float16.

  • w (Tensor) - Weight. Tensor of shape (input_size + hidden_size, 4 x hidden_size). The data type must be float16.

  • b (Tensor) - Bias. Tensor of shape (4 x hidden_size). The data type must be float16 or float32.

  • seq_length (Tensor) - The length of each batch. Tensor of shape (batch_size). Only None is currently supported.

  • init_h (Tensor) - Hidden state of initial time. Tensor of shape (1, batch_size, hidden_size). The data type must be float16.

  • init_c (Tensor) - Cell state of initial time. Tensor of shape (1, batch_size, hidden_size). The data type must be float16.

Outputs:

  • y (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.

  • output_h (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). With data type of float16.

  • output_c (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.

  • i (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.

  • j (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.

  • f (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.

  • o (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.

  • tanhct (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.

Raises

  • TypeError – If cell_type, direction or activation is not a str.

  • TypeError – If cell_depth or num_proj is not an int.

  • TypeError – If keep_prob, cell_clip or forget_bias is not a float.

  • TypeError – If use_peehpole, time_major or is_training is not a bool.

  • TypeError – If x, w, b, seq_length, init_h or init_c is not a Tensor.

  • TypeError – If dtype of x, w, init_h or nit_c is not float16.

  • TypeError – If dtype of b is neither float16 nor float32.

Supported Platforms:

Ascend

Examples

>>> x = Tensor(np.random.rand(2, 16, 64).astype(np.float16))
>>> w = Tensor(np.random.rand(96, 128).astype(np.float16))
>>> b = Tensor(np.random.rand(128).astype(np.float16))
>>> init_h = Tensor(np.random.rand(1, 16, 32).astype(np.float16))
>>> init_c = Tensor(np.random.rand(1, 16, 32).astype(np.float16))
>>> dynamic_rnn = ops.DynamicRNN()
>>> output = dynamic_rnn(x, w, b, None, init_h, init_c)
>>> print(output[0].shape)
(2, 16, 32)