An implementation of a faster version of the Fast LSTM network layer. More...

Public Types
typedef OutputDataType::elem_type	ElemType

typedef InputDataType::elem_type	InputElemType

Public Member Functions
	FastLSTM ()
	Create the Fast LSTM object. More...

	FastLSTM (const size_t inSize, const size_t outSize, const size_t rho=std::numeric_limits< size_t >::max())
	Create the Fast LSTM layer object using the specified parameters. More...

template
void	Backward (const InputType &input, const ErrorType &gy, GradientType &g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...

OutputDataType const &	Delta () const
	Get the delta. More...

OutputDataType &	Delta ()
	Modify the delta. More...

template
void	Forward (const InputType &input, OutputType &output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

template
void	Gradient (const InputType &input, const ErrorType &error, GradientType &gradient)

OutputDataType const &	Gradient () const
	Get the gradient. More...

OutputDataType &	Gradient ()
	Modify the gradient. More...

size_t	InSize () const
	Get the number of input units. More...

OutputDataType const &	OutputParameter () const
	Get the output parameter. More...

OutputDataType &	OutputParameter ()
	Modify the output parameter. More...

size_t	OutSize () const
	Get the number of output units. More...

OutputDataType const &	Parameters () const
	Get the parameters. More...

OutputDataType &	Parameters ()
	Modify the parameters. More...

void	Reset ()

void	ResetCell (const size_t size)

size_t	Rho () const
	Get the maximum number of steps to backpropagate through time (BPTT). More...

size_t &	Rho ()
	Modify the maximum number of steps to backpropagate through time (BPTT). More...

template
void	serialize (Archive &ar, const unsigned int)
	Serialize the layer. More...

Detailed Description

template
class mlpack::ann::FastLSTM< InputDataType, OutputDataType >

An implementation of a faster version of the Fast LSTM network layer.

Basically by combining the calculation of the input, forget, output gates and hidden state in a single step. The standard formula changes as follows:

$\begin{eqnarray} i &=& sigmoid(W \cdot x + W \cdot h + b) \\ f &=& sigmoid(W \cdot x + W \cdot h + b) \\ z &=& tanh(W \cdot x + W \cdot h + b) \\ c &=& f \cdot c + i \cdot z \\ o &=& sigmoid(W \cdot x + W \cdot h + b) \\ h &=& o \cdot tanh(c) \end{eqnarray}$

Note that FastLSTM network layer does not use peephole connections between the cell and gates.

Note also that if a FastLSTM layer is desired as the first layer of a neural network, an IdentityLayer should be added to the network as the first layer, and then the FastLSTM layer should be added.

For more information, see the following.

@article{Hochreiter1997,
  author  = {Hochreiter, Sepp and Schmidhuber, J\"{u}rgen},
  title   = {Long Short-term Memory},
  journal = {Neural Comput.},
  year    = {1997},
  url     = {https://www.bioinf.jku.at/publications/older/2604.pdf}
}

See also: LSTM for a standard implementation of the LSTM layer.

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 66 of file fast_lstm.hpp.

Member Typedef Documentation

◆ ElemType

typedef OutputDataType::elem_type ElemType

Definition at line 71 of file fast_lstm.hpp.

◆ InputElemType

typedef InputDataType::elem_type InputElemType

Definition at line 70 of file fast_lstm.hpp.

Constructor & Destructor Documentation

◆ FastLSTM() [1/2]

FastLSTM ( )

Create the Fast LSTM object.

◆ FastLSTM() [2/2]

FastLSTM	(	const size_t	inSize,
		const size_t	outSize,
		const size_t	rho = `std::numeric_limits< size_t >::max()`
	)

Create the Fast LSTM layer object using the specified parameters.

Parameters

inSize	The number of input units.
outSize	The number of output units.
rho	Maximum number of steps to backpropagate through time (BPTT).

Member Function Documentation

◆ Backward()

void Backward	(	const InputType &	input,
		const ErrorType &	gy,
		GradientType &	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters

input	The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const

inline

Get the delta.

Definition at line 152 of file fast_lstm.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )

inline

Modify the delta.

Definition at line 154 of file fast_lstm.hpp.

◆ Forward()

void Forward	(	const InputType &	input,
		OutputType &	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

◆ Gradient() [1/3]

void Gradient	(	const InputType &	input,
		const ErrorType &	error,
		GradientType &	gradient
	)

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const

inline

Get the gradient.

Definition at line 157 of file fast_lstm.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )

inline

Modify the gradient.

Definition at line 159 of file fast_lstm.hpp.

◆ InSize()

size_t InSize ( ) const

inline

Get the number of input units.

Definition at line 162 of file fast_lstm.hpp.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 147 of file fast_lstm.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )

inline

Modify the output parameter.

Definition at line 149 of file fast_lstm.hpp.

◆ OutSize()

size_t OutSize ( ) const

inline

Get the number of output units.

Definition at line 165 of file fast_lstm.hpp.

References FastLSTM< InputDataType, OutputDataType >::serialize().

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const

inline

Get the parameters.

Definition at line 142 of file fast_lstm.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )

inline

Modify the parameters.

Definition at line 144 of file fast_lstm.hpp.

◆ Reset()

void Reset ( )

◆ ResetCell()

void ResetCell ( const size_t size )

◆ Rho() [1/2]

size_t Rho ( ) const

inline

Get the maximum number of steps to backpropagate through time (BPTT).

Definition at line 137 of file fast_lstm.hpp.

◆ Rho() [2/2]

size_t& Rho ( )

inline

Modify the maximum number of steps to backpropagate through time (BPTT).

Definition at line 139 of file fast_lstm.hpp.

◆ serialize()

void serialize	(	Archive &	ar,
		const unsigned	int
	)

Serialize the layer.

Referenced by FastLSTM< InputDataType, OutputDataType >::OutSize().

The documentation for this class was generated from the following file:

/home/jenkins-mlpack/mlpack.org/_src/mlpack-3.4.2/src/mlpack/methods/ann/layer/fast_lstm.hpp

Public Types

Public Member Functions

Detailed Description

template class mlpack::ann::FastLSTM< InputDataType, OutputDataType >

Member Typedef Documentation

◆ ElemType

◆ InputElemType

Constructor & Destructor Documentation

◆ FastLSTM() [1/2]

◆ FastLSTM() [2/2]

Member Function Documentation

◆ Backward()

◆ Delta() [1/2]

◆ Delta() [2/2]

◆ Forward()

◆ Gradient() [1/3]

◆ Gradient() [2/3]

◆ Gradient() [3/3]

◆ InSize()

◆ OutputParameter() [1/2]

◆ OutputParameter() [2/2]

◆ OutSize()

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ Reset()

◆ ResetCell()

◆ Rho() [1/2]

◆ Rho() [2/2]

◆ serialize()

template
class mlpack::ann::FastLSTM< InputDataType, OutputDataType >