An implementation of a faster version of the Fast LSTM network layer. More...
Public Types | |
typedef OutputDataType::elem_type | ElemType |
typedef InputDataType::elem_type | InputElemType |
Public Member Functions | |
FastLSTM () | |
Create the Fast LSTM object. More... | |
FastLSTM (const size_t inSize, const size_t outSize, const size_t rho=std::numeric_limits< size_t >::max()) | |
Create the Fast LSTM layer object using the specified parameters. More... | |
template | |
void | Backward (const InputType &input, const ErrorType &gy, GradientType &g) |
Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More... | |
OutputDataType const & | Delta () const |
Get the delta. More... | |
OutputDataType & | Delta () |
Modify the delta. More... | |
template | |
void | Forward (const InputType &input, OutputType &output) |
Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More... | |
template | |
void | Gradient (const InputType &input, const ErrorType &error, GradientType &gradient) |
OutputDataType const & | Gradient () const |
Get the gradient. More... | |
OutputDataType & | Gradient () |
Modify the gradient. More... | |
size_t | InSize () const |
Get the number of input units. More... | |
OutputDataType const & | OutputParameter () const |
Get the output parameter. More... | |
OutputDataType & | OutputParameter () |
Modify the output parameter. More... | |
size_t | OutSize () const |
Get the number of output units. More... | |
OutputDataType const & | Parameters () const |
Get the parameters. More... | |
OutputDataType & | Parameters () |
Modify the parameters. More... | |
void | Reset () |
void | ResetCell (const size_t size) |
size_t | Rho () const |
Get the maximum number of steps to backpropagate through time (BPTT). More... | |
size_t & | Rho () |
Modify the maximum number of steps to backpropagate through time (BPTT). More... | |
template | |
void | serialize (Archive &ar, const unsigned int) |
Serialize the layer. More... | |
An implementation of a faster version of the Fast LSTM network layer.
Basically by combining the calculation of the input, forget, output gates and hidden state in a single step. The standard formula changes as follows:
Note that FastLSTM network layer does not use peephole connections between the cell and gates.
Note also that if a FastLSTM layer is desired as the first layer of a neural network, an IdentityLayer should be added to the network as the first layer, and then the FastLSTM layer should be added.
For more information, see the following.
InputDataType | Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube). |
OutputDataType | Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube). |
Definition at line 66 of file fast_lstm.hpp.
typedef OutputDataType::elem_type ElemType |
Definition at line 71 of file fast_lstm.hpp.
typedef InputDataType::elem_type InputElemType |
Definition at line 70 of file fast_lstm.hpp.
FastLSTM | ( | const size_t | inSize, |
const size_t | outSize, | ||
const size_t | rho = std::numeric_limits< size_t >::max() |
||
) |
Create the Fast LSTM layer object using the specified parameters.
inSize | The number of input units. |
outSize | The number of output units. |
rho | Maximum number of steps to backpropagate through time (BPTT). |
void Backward | ( | const InputType & | input, |
const ErrorType & | gy, | ||
GradientType & | g | ||
) |
Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.
Using the results from the feed forward pass.
input | The propagated input activation. |
gy | The backpropagated error. |
g | The calculated gradient. |
|
inline |
Get the delta.
Definition at line 152 of file fast_lstm.hpp.
|
inline |
Modify the delta.
Definition at line 154 of file fast_lstm.hpp.
void Forward | ( | const InputType & | input, |
OutputType & | output | ||
) |
Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.
input | Input data used for evaluating the specified function. |
output | Resulting output activation. |
void Gradient | ( | const InputType & | input, |
const ErrorType & | error, | ||
GradientType & | gradient | ||
) |
|
inline |
Get the gradient.
Definition at line 157 of file fast_lstm.hpp.
|
inline |
Modify the gradient.
Definition at line 159 of file fast_lstm.hpp.
|
inline |
Get the number of input units.
Definition at line 162 of file fast_lstm.hpp.
|
inline |
Get the output parameter.
Definition at line 147 of file fast_lstm.hpp.
|
inline |
Modify the output parameter.
Definition at line 149 of file fast_lstm.hpp.
|
inline |
Get the number of output units.
Definition at line 165 of file fast_lstm.hpp.
References FastLSTM< InputDataType, OutputDataType >::serialize().
|
inline |
Get the parameters.
Definition at line 142 of file fast_lstm.hpp.
|
inline |
Modify the parameters.
Definition at line 144 of file fast_lstm.hpp.
void Reset | ( | ) |
void ResetCell | ( | const size_t | size | ) |
|
inline |
Get the maximum number of steps to backpropagate through time (BPTT).
Definition at line 137 of file fast_lstm.hpp.
|
inline |
Modify the maximum number of steps to backpropagate through time (BPTT).
Definition at line 139 of file fast_lstm.hpp.
void serialize | ( | Archive & | ar, |
const unsigned | int | ||
) |
Serialize the layer.
Referenced by FastLSTM< InputDataType, OutputDataType >::OutSize().