DicitonaryEnocdingPolicy is used as a helper class for StringEncoding. More...
Public Member Functions | |
| template | |
| void | serialize (Archive &, const unsigned int) |
| Serialize the class to the given archive. More... | |
Static Public Member Functions | |
| template | |
| static void | Encode (MatType &output, const size_t value, const size_t line, const size_t index) |
| The function performs the dictionary encoding algorithm i.e. More... | |
| template | |
| static void | Encode (std::vector< ElemType > &output, size_t value) |
| The function performs the dictionary encoding algorithm i.e. More... | |
| template | |
| static void | InitMatrix (MatType &output, const size_t datasetSize, const size_t maxNumTokens, const size_t) |
| The function initializes the output matrix. More... | |
| static void | PreprocessToken (const size_t, const size_t, const size_t) |
| The function is not used by the dictionary encoding policy. More... | |
| static void | Reset () |
| Clear the necessary internal variables. More... | |
Detailed Description
DicitonaryEnocdingPolicy is used as a helper class for StringEncoding.
The encoder assigns a positive integer number to each unique token and treats the dataset as categorical. The numbers are assigned sequentially starting from one. The order in which the tokens are labeled is defined by the dictionary used by the StringEncoding class. The encoder writes data either in the column-major order or in the row-major order depending on the output data type.
Definition at line 32 of file dictionary_encoding_policy.hpp.
Member Function Documentation
◆ Encode() [1/2]
|
inlinestatic |
The function performs the dictionary encoding algorithm i.e.
it writes the encoded token to the output. The encoder writes data in the column-major order.
- Template Parameters
-
MatType The output matrix type.
- Parameters
-
output Output matrix to store the encoded results (sp_mat or mat). value The encoded token. line The line number at which the encoding is performed. index The token index in the line.
Definition at line 77 of file dictionary_encoding_policy.hpp.
◆ Encode() [2/2]
|
inlinestatic |
The function performs the dictionary encoding algorithm i.e.
it writes the encoded token to the output. This is an overloaded function which saves the result into the given vector to avoid padding. The encoder writes data in the row-major order.
- Template Parameters
-
ElemType Type of the output values.
- Parameters
-
output Output vector to store the encoded line. value The encoded token.
Definition at line 97 of file dictionary_encoding_policy.hpp.
◆ InitMatrix()
|
inlinestatic |
The function initializes the output matrix.
The encoder writes data in the column-major order.
- Template Parameters
-
MatType The output matrix type.
- Parameters
-
output Output matrix to store the encoded results (sp_mat or mat). datasetSize The number of strings in the input dataset. maxNumTokens The maximum number of tokens in the strings of the input dataset. * (dictionarySize) The size of the dictionary (not used).
Definition at line 56 of file dictionary_encoding_policy.hpp.
◆ PreprocessToken()
|
inlinestatic |
The function is not used by the dictionary encoding policy.
- Parameters
-
* (line) The line number at which the encoding is performed. * (index) The token sequence number in the line. * (value) The encoded token.
Definition at line 109 of file dictionary_encoding_policy.hpp.
◆ Reset()
|
inlinestatic |
Clear the necessary internal variables.
Definition at line 38 of file dictionary_encoding_policy.hpp.
◆ serialize()
|
inline |
Serialize the class to the given archive.
Definition at line 118 of file dictionary_encoding_policy.hpp.
The documentation for this class was generated from the following file:
- /home/jenkins-mlpack/mlpack.org/_src/mlpack-3.4.2/src/mlpack/core/data/string_encoding_policies/dictionary_encoding_policy.hpp