overview

High-quality documentation is a development goal of mlpack. mlpack’s documentation is split into two parts: documentation for the bindings, and documentation for the C++ library. Generally, working with the bindings is a good choice for simple machine learning and data science tasks, and writing C++ is a good idea when complex or custom functionality is desired.

All interfaces are heavily documented, and if you find a documentation issue, please report it.

quickstart

Just getting started with mlpack? Try these quickstart tutorials for the bindings to other languages.

Once you’re comfortable with the quickstart guides for the language of your choice, full documentation for every binding can be found below. Quick links are in the left sidebar.

The C++ interfaces of mlpack are carefully documented in the source code, which can be browsed for more details.

A number of tutorials are available covering individual algorithms and functionality inside of mlpack, both for bindings to other languages and for the C++ interface.

mlpack 4.3.0 binding documentation

data formats

mlpack bindings for CLI take and return a restricted set of types, for simplicity. These include primitive types, matrix/vector types, categorical matrix types, and model types. Each type is detailed below.

  • int: An integer (i.e., “1”).
  • double: A floating-point number (i.e., “0.5”).
  • flag: A boolean flag option. If not specified, it is false; if specified, it is true.
  • string: A character string (i.e., “hello”).
  • int vector: A vector of integers, separated by commas (i.e., “1,2,3”).
  • string vector: A vector of strings, separated by commas (i.e., “hello”,”goodbye”).
  • 2-d matrix file: A data matrix filename. The file can be CSV (.csv), TSV (.csv), ASCII (space-separated values, .txt), Armadillo ASCII (.txt), PGM (.pgm), PPM (.ppm), Armadillo binary (.bin), or HDF5 (.h5, .hdf, .hdf5, or .he5), if mlpack was compiled with HDF5 support. The type of the data is detected by the extension of the filename. The storage should be such that one row corresponds to one point, and one column corresponds to one dimension (this is the typical storage format for on-disk data). CSV files will be checked for a header; if no header is found, the first row will be loaded as a data point. All values of the matrix will be loaded as double-precision floating point data.
  • 2-d index matrix file: A data matrix filename, where the matrix holds only non-negative integer values. This type is often used for labels or indices. The file can be CSV (.csv), TSV (.csv), ASCII (space-separated values, .txt), Armadillo ASCII (.txt), PGM (.pgm), PPM (.ppm), Armadillo binary (.bin), or HDF5 (.h5, .hdf, .hdf5, or .he5), if mlpack was compiled with HDF5 support. The type of the data is detected by the extension of the filename. The storage should be such that one row corresponds to one point, and one column corresponds to one dimension (this is the typical storage format for on-disk data). CSV files will be checked for a header; if no header is found, the first row will be loaded as a data point. All values of the matrix will be loaded as unsigned integers.
  • 1-d matrix file: A one-dimensional vector filename. This file can take the same formats as the data matrix filenames; however, it must either contain one row and many columns, or one column and many rows.
  • 1-d index matrix file: A one-dimensional vector filename, where the matrix holds only non-negative integer values. This type is typically used for labels or predictions or other indices. This file can take the same formats as the data matrix filenames; however, it must either contain one row and many columns, or one column and many rows.
  • 2-d categorical matrix file: A filename for a data matrix that can contain categorical (non-numeric) data. If the file contains only numeric data, then the same formats for regular data matrices can be used. If the file contains strings or other values that can’t be parsed as numbers, then the type to be loaded must be CSV (.csv) or ARFF (.arff). Any non-numeric data will be converted to an unsigned integer value, and dimensions where the data is converted will be treated as categorical dimensions. When using this format, there is no need for one-hot encoding of categorical data.
  • mlpackModel file: A filename containing an mlpack model. These can have one of three formats: binary (.bin), text (.txt), and XML (.xml). The XML format produces the largest (but most human-readable) files, while the binary format can be significantly more compact and quicker to load and save.

mlpack_approx_kfn

$ mlpack_approx_kfn [--algorithm 'ds'] [--calculate_error]
        [--exact_distances_file <string>] [--help] [--info <string>]
        [--input_model_file <string>] [--k 0] [--num_projections 5]
        [--num_tables 5] [--query_file <string>] [--reference_file <string>]
        [--verbose] [--version] [--distances_file <string>] [--neighbors_file
        <string>] [--output_model_file <string>]

An implementation of two strategies for furthest neighbor search. This can be used to compute the furthest neighbor of query point(s) from a set of points; furthest neighbor models can be saved and reused with future query point(s). Detailed documentation.

Input options

name type description default
--algorithm (-a) string Algorithm to use: ‘ds’ or ‘qdafn’. 'ds'
--calculate_error (-e) flag If set, calculate the average distance error for the first furthest neighbor only.  
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--exact_distances_file (-x) 2-d matrix file Matrix containing exact distances to furthest neighbors; this can be used to avoid explicit calculation when –calculate_error is set. ''
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) ApproxKFNModel file File containing input model. ''
--k (-k) int Number of furthest neighbors to search for. 0
--num_projections (-p) int Number of projections to use in each hash table. 5
--num_tables (-t) int Number of hash tables to use. 5
--query_file (-q) 2-d matrix file Matrix containing query points. ''
--reference_file (-r) 2-d matrix file Matrix containing the reference dataset. ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--distances_file (-d) 2-d matrix file Matrix to save furthest neighbor distances to.
--neighbors_file (-n) 2-d index matrix file Matrix to save neighbor indices to.
--output_model_file (-M) ApproxKFNModel file File to save output model to.

Detailed documentation

This program implements two strategies for furthest neighbor search. These strategies are:

  • The ‘qdafn’ algorithm from “Approximate Furthest Neighbor in High Dimensions” by R. Pagh, F. Silvestri, J. Sivertsen, and M. Skala, in Similarity Search and Applications 2015 (SISAP).
  • The ‘DrusillaSelect’ algorithm from “Fast approximate furthest neighbors with data-dependent candidate selection”, by R.R. Curtin and A.B. Gardner, in Similarity Search and Applications 2016 (SISAP).

These two strategies give approximate results for the furthest neighbor search problem and can be used as fast replacements for other furthest neighbor techniques such as those found in the mlpack_kfn program. Note that typically, the ‘ds’ algorithm requires far fewer tables and projections than the ‘qdafn’ algorithm.

Specify a reference set (set to search in) with --reference_file (-r), specify a query set with --query_file (-q), and specify algorithm parameters with --num_tables (-t) and --num_projections (-p) (or don’t and defaults will be used). The algorithm to be used (either ‘ds’—the default—or ‘qdafn’) may be specified with --algorithm (-a). Also specify the number of neighbors to search for with --k (-k).

Note that for ‘qdafn’ in lower dimensions, --num_projections (-p) may need to be set to a high value in order to return results for each query point.

If no query set is specified, the reference set will be used as the query set. The --output_model_file (-M) output parameter may be used to store the built model, and an input model may be loaded instead of specifying a reference set with the --input_model_file (-m) option.

Results for each query point can be stored with the --neighbors_file (-n) and --distances_file (-d) output parameters. Each row of these output matrices holds the k distances or neighbor indices for each query point.

Example

For example, to find the 5 approximate furthest neighbors with 'reference_set.csv' as the reference set and 'query_set.csv' as the query set using DrusillaSelect, storing the furthest neighbor indices to 'neighbors.csv' and the furthest neighbor distances to 'distances.csv', one could call

$ mlpack_approx_kfn --query_file query_set.csv --reference_file
  reference_set.csv --k 5 --algorithm ds --neighbors_file neighbors.csv
  --distances_file distances.csv

and to perform approximate all-furthest-neighbors search with k=1 on the set 'data.csv' storing only the furthest neighbor distances to 'distances.csv', one could call

$ mlpack_approx_kfn --reference_file reference_set.csv --k 1 --distances_file
  distances.csv

A trained model can be re-used. If a model has been previously saved to 'model.bin', then we may find 3 approximate furthest neighbors on a query set 'new_query_set.csv' using that model and store the furthest neighbor indices into 'neighbors.csv' by calling

$ mlpack_approx_kfn --input_model_file model.bin --query_file
  new_query_set.csv --k 3 --neighbors_file neighbors.csv

See also

mlpack_bayesian_linear_regression

BayesianLinearRegression

$ mlpack_bayesian_linear_regression [--center] [--help] [--info
        <string>] [--input_file <string>] [--input_model_file <string>]
        [--responses_file <string>] [--scale] [--test_file <string>] [--verbose]
        [--version] [--output_model_file <string>] [--predictions_file <string>]
        [--stds_file <string>]

An implementation of the bayesian linear regression. Detailed documentation.

Input options

name type description default
--center (-c) flag Center the data and fit the intercept if enabled.  
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Matrix of covariates (X). ''
--input_model_file (-m) BayesianLinearRegression file Trained BayesianLinearRegression model to use. ''
--responses_file (-r) 1-d matrix file Matrix of responses/observations (y). ''
--scale (-s) flag Scale each feature by their standard deviations if enabled.  
--test_file (-t) 2-d matrix file Matrix containing points to regress on (test points). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) BayesianLinearRegression file Output BayesianLinearRegression model.
--predictions_file (-o) 2-d matrix file If –test_file is specified, this file is where the predicted responses will be saved.
--stds_file (-u) 2-d matrix file If specified, this is where the standard deviations of the predictive distribution will be saved.

Detailed documentation

An implementation of the bayesian linear regression. This model is a probabilistic view and implementation of the linear regression. The final solution is obtained by computing a posterior distribution from gaussian likelihood and a zero mean gaussian isotropic prior distribution on the solution. Optimization is AUTOMATIC and does not require cross validation. The optimization is performed by maximization of the evidence function. Parameters are tuned during the maximization of the marginal likelihood. This procedure includes the Ockham’s razor that penalizes over complex solutions.

This program is able to train a Bayesian linear regression model or load a model from file, output regression predictions for a test set, and save the trained model to a file.

To train a BayesianLinearRegression model, the --input_file (-i) and --responses_file (-r)parameters must be given. The --center (-c)and --scale (-s) parameters control the centering and the normalizing options. A trained model can be saved with the --output_model_file (-M). If no training is desired at all, a model can be passed via the --input_model_file (-m) parameter.

The program can also provide predictions for test data using either the trained model or the given input model. Test points can be specified with the --test_file (-t) parameter. Predicted responses to the test points can be saved with the --predictions_file (-o) output parameter. The corresponding standard deviation can be save by precising the --stds_file (-u) parameter.

Example

For example, the following command trains a model on the data 'data.csv' and responses 'responses.csv'with center set to true and scale set to false (so, Bayesian linear regression is being solved, and then the model is saved to 'blr_model.bin':

$ mlpack_bayesian_linear_regression --input_file data.csv --responses_file
  responses.csv --center --scale --output_model_file blr_model.bin

The following command uses the 'blr_model.bin' to provide predicted responses for the data 'test.csv' and save those responses to 'test_predictions.csv':

$ mlpack_bayesian_linear_regression --input_model_file blr_model.bin
  --test_file test.csv --predictions_file test_predictions.csv

Because the estimator computes a predictive distribution instead of a simple point estimate, the --stds_file (-u) parameter allows one to save the prediction uncertainties:

$ mlpack_bayesian_linear_regression --input_model_file blr_model.bin
  --test_file test.csv --predictions_file test_predictions.csv --stds_file
  stds.csv

See also

mlpack_cf

Collaborative Filtering

$ mlpack_cf [--algorithm 'NMF'] [--all_user_recommendations] [--help]
        [--info <string>] [--input_model_file <string>] [--interpolation
        'average'] [--iteration_only_termination] [--max_iterations 1000]
        [--min_residue 1e-05] [--neighbor_search 'euclidean'] [--neighborhood 5]
        [--normalization 'none'] [--query_file <string>] [--rank 0]
        [--recommendations 5] [--seed 0] [--test_file <string>] [--training_file
        <string>] [--verbose] [--version] [--output_file <string>]
        [--output_model_file <string>]

An implementation of several collaborative filtering (CF) techniques for recommender systems. This can be used to train a new CF model, or use an existing CF model to compute recommendations. Detailed documentation.

Input options

name type description default
--algorithm (-a) string Algorithm used for matrix factorization. 'NMF'
--all_user_recommendations (-A) flag Generate recommendations for all users.  
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) CFModel file Trained CF model to load. ''
--interpolation (-i) string Algorithm used for weight interpolation. 'average'
--iteration_only_termination (-I) flag Terminate only when the maximum number of iterations is reached.  
--max_iterations (-N) int Maximum number of iterations. If set to zero, there is no limit on the number of iterations. 1000
--min_residue (-r) double Residue required to terminate the factorization (lower values generally mean better fits). 1e-05
--neighbor_search (-S) string Algorithm used for neighbor search. 'euclidean'
--neighborhood (-n) int Size of the neighborhood of similar users to consider for each query user. 5
--normalization (-z) string Normalization performed on the ratings. 'none'
--query_file (-q) 2-d index matrix file List of query users for which recommendations should be generated. ''
--rank (-R) int Rank of decomposed matrices (if 0, a heuristic is used to estimate the rank). 0
--recommendations (-c) int Number of recommendations to generate for each query user. 5
--seed (-s) int Set the random seed (0 uses std::time(NULL)). 0
--test_file (-T) 2-d matrix file Test set to calculate RMSE on. ''
--training_file (-t) 2-d matrix file Input dataset to perform CF on. ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d index matrix file Matrix that will store output recommendations.
--output_model_file (-M) CFModel file Output for trained CF model.

Detailed documentation

This program performs collaborative filtering (CF) on the given dataset. Given a list of user, item and preferences (the --training_file (-t) parameter), the program will perform a matrix decomposition and then can perform a series of actions related to collaborative filtering. Alternately, the program can load an existing saved CF model with the --input_model_file (-m) parameter and then use that model to provide recommendations or predict values.

The input matrix should be a 3-dimensional matrix of ratings, where the first dimension is the user, the second dimension is the item, and the third dimension is that user’s rating of that item. Both the users and items should be numeric indices, not names. The indices are assumed to start from 0.

A set of query users for which recommendations can be generated may be specified with the --query_file (-q) parameter; alternately, recommendations may be generated for every user in the dataset by specifying the --all_user_recommendations (-A) parameter. In addition, the number of recommendations per user to generate can be specified with the --recommendations (-c) parameter, and the number of similar users (the size of the neighborhood) to be considered when generating recommendations can be specified with the --neighborhood (-n) parameter.

For performing the matrix decomposition, the following optimization algorithms can be specified via the --algorithm (-a) parameter:

  • ‘RegSVD’ – Regularized SVD using a SGD optimizer
  • ‘NMF’ – Non-negative matrix factorization with alternating least squares update rules
  • ‘BatchSVD’ – SVD batch learning
  • ‘SVDIncompleteIncremental’ – SVD incomplete incremental learning
  • ‘SVDCompleteIncremental’ – SVD complete incremental learning
  • ‘BiasSVD’ – Bias SVD using a SGD optimizer
  • ‘SVDPP’ – SVD++ using a SGD optimizer
  • ‘RandSVD’ – RandomizedSVD learning
  • ‘QSVD’ – QuicSVD learning
  • ‘BKSVD’ – Block Krylov SVD learning

The following neighbor search algorithms can be specified via the --neighbor_search (-S) parameter:

  • ‘cosine’ – Cosine Search Algorithm
  • ‘euclidean’ – Euclidean Search Algorithm
  • ‘pearson’ – Pearson Search Algorithm

The following weight interpolation algorithms can be specified via the --interpolation (-i) parameter:

  • ‘average’ – Average Interpolation Algorithm
  • ‘regression’ – Regression Interpolation Algorithm
  • ‘similarity’ – Similarity Interpolation Algorithm

The following ranking normalization algorithms can be specified via the --normalization (-z) parameter:

  • ‘none’ – No Normalization
  • ‘item_mean’ – Item Mean Normalization
  • ‘overall_mean’ – Overall Mean Normalization
  • ‘user_mean’ – User Mean Normalization
  • ‘z_score’ – Z-Score Normalization

A trained model may be saved to with the --output_model_file (-M) output parameter.

Example

To train a CF model on a dataset 'training_set.csv' using NMF for decomposition and saving the trained model to 'model.bin', one could call:

$ mlpack_cf --training_file training_set.csv --algorithm NMF
  --output_model_file model.bin

Then, to use this model to generate recommendations for the list of users in the query set 'users.csv', storing 5 recommendations in 'recommendations.csv', one could call

$ mlpack_cf --input_model_file model.bin --query_file users.csv
  --recommendations 5 --output_file recommendations.csv

See also

mlpack_dbscan

DBSCAN clustering

$ mlpack_dbscan [--epsilon 1] [--help] [--info <string>] --input_file
        <string> [--min_size 5] [--naive] [--selection_type 'ordered']
        [--single_mode] [--tree_type 'kd'] [--verbose] [--version]
        [--assignments_file <string>] [--centroids_file <string>]

An implementation of DBSCAN clustering. Given a dataset, this can compute and return a clustering of that dataset. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--epsilon (-e) double Radius of each range search. 1
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Input dataset to cluster. required
--min_size (-m) int Minimum number of points for a cluster. 5
--naive (-N) flag If set, brute-force range search (not tree-based) will be used.  
--selection_type (-s) string If using point selection policy, the type of selection to use (‘ordered’, ‘random’). 'ordered'
--single_mode (-S) flag If set, single-tree range search (not dual-tree) will be used.  
--tree_type (-t) string If using single-tree or dual-tree search, the type of tree to use (‘kd’, ‘r’, ‘r-star’, ‘x’, ‘hilbert-r’, ‘r-plus’, ‘r-plus-plus’, ‘cover’, ‘ball’). 'kd'
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--assignments_file (-a) 1-d index matrix file Output matrix for assignments of each point.
--centroids_file (-C) 2-d matrix file Matrix to save output centroids to.

Detailed documentation

This program implements the DBSCAN algorithm for clustering using accelerated tree-based range search. The type of tree that is used may be parameterized, or brute-force range search may also be used.

The input dataset to be clustered may be specified with the --input_file (-i) parameter; the radius of each range search may be specified with the --epsilon (-e) parameters, and the minimum number of points in a cluster may be specified with the --min_size (-m) parameter.

The --assignments_file (-a) and --centroids_file (-C) output parameters may be used to save the output of the clustering. --assignments_file (-a) contains the cluster assignments of each point, and --centroids_file (-C) contains the centroids of each cluster.

The range search may be controlled with the --tree_type (-t), --single_mode (-S), and --naive (-N) parameters. --tree_type (-t) can control the type of tree used for range search; this can take a variety of values: ‘kd’, ‘r’, ‘r-star’, ‘x’, ‘hilbert-r’, ‘r-plus’, ‘r-plus-plus’, ‘cover’, ‘ball’. The --single_mode (-S) parameter will force single-tree search (as opposed to the default dual-tree search), and ‘--naive (-N) will force brute-force range search.

Example

An example usage to run DBSCAN on the dataset in 'input.csv' with a radius of 0.5 and a minimum cluster size of 5 is given below:

$ mlpack_dbscan --input_file input.csv --epsilon 0.5 --min_size 5

See also

mlpack_decision_tree

Decision tree

$ mlpack_decision_tree [--help] [--info <string>] [--input_model_file
        <string>] [--labels_file <string>] [--maximum_depth 0]
        [--minimum_gain_split 1e-07] [--minimum_leaf_size 20]
        [--print_training_accuracy] [--print_training_error] [--test_file
        <string>] [--test_labels_file <string>] [--training_file <string>]
        [--verbose] [--version] [--weights_file <string>] [--output_model_file
        <string>] [--predictions_file <string>] [--probabilities_file <string>]

An implementation of an ID3-style decision tree for classification, which supports categorical data. Given labeled data with numeric or categorical features, a decision tree can be trained and saved; or, an existing decision tree can be used for classification on new points. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) DecisionTreeModel file Pre-trained decision tree, to be used with test points. ''
--labels_file (-l) 1-d index matrix file Training labels. ''
--maximum_depth (-D) int Maximum depth of the tree (0 means no limit). 0
--minimum_gain_split (-g) double Minimum gain for node splitting. 1e-07
--minimum_leaf_size (-n) int Minimum number of points in a leaf. 20
--print_training_accuracy (-a) flag Print the training accuracy.  
--print_training_error (-e) flag Print the training error (deprecated; will be removed in mlpack 4.0.0).  
--test_file (-T) 2-d categorical matrix file Testing dataset (may be categorical). ''
--test_labels_file (-L) 1-d index matrix file Test point labels, if accuracy calculation is desired. ''
--training_file (-t) 2-d categorical matrix file Training dataset (may be categorical). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  
--weights_file (-w) 2-d matrix file The weight of labels ''

Output options

name type description
--output_model_file (-M) DecisionTreeModel file Output for trained decision tree.
--predictions_file (-p) 1-d index matrix file Class predictions for each test point.
--probabilities_file (-P) 2-d matrix file Class probabilities for each test point.

Detailed documentation

Train and evaluate using a decision tree. Given a dataset containing numeric or categorical features, and associated labels for each point in the dataset, this program can train a decision tree on that data.

The training set and associated labels are specified with the --training_file (-t) and --labels_file (-l) parameters, respectively. The labels should be in the range [0, num_classes - 1]. Optionally, if --labels_file (-l) is not specified, the labels are assumed to be the last dimension of the training dataset.

When a model is trained, the --output_model_file (-M) output parameter may be used to save the trained model. A model may be loaded for predictions with the --input_model_file (-m) parameter. The --input_model_file (-m) parameter may not be specified when the --training_file (-t) parameter is specified. The --minimum_leaf_size (-n) parameter specifies the minimum number of training points that must fall into each leaf for it to be split. The --minimum_gain_split (-g) parameter specifies the minimum gain that is needed for the node to split. The --maximum_depth (-D) parameter specifies the maximum depth of the tree. If --print_training_error (-e) is specified, the training error will be printed.

Test data may be specified with the --test_file (-T) parameter, and if performance numbers are desired for that test set, labels may be specified with the --test_labels_file (-L) parameter. Predictions for each test point may be saved via the --predictions_file (-p) output parameter. Class probabilities for each prediction may be saved with the --probabilities_file (-P) output parameter.

Example

For example, to train a decision tree with a minimum leaf size of 20 on the dataset contained in 'data.csv' with labels 'labels.csv', saving the output model to 'tree.bin' and printing the training error, one could call

$ mlpack_decision_tree --training_file data.arff --labels_file labels.csv
  --output_model_file tree.bin --minimum_leaf_size 20 --minimum_gain_split 0.001
  --print_training_accuracy

Then, to use that model to classify points in 'test_set.csv' and print the test error given the labels 'test_labels.csv' using that model, while saving the predictions for each point to 'predictions.csv', one could call

$ mlpack_decision_tree --input_model_file tree.bin --test_file test_set.arff
  --test_labels_file test_labels.csv --predictions_file predictions.csv

See also

mlpack_det

Density Estimation With Density Estimation Trees

$ mlpack_det [--folds 10] [--help] [--info <string>] [--input_model_file
        <string>] [--max_leaf_size 10] [--min_leaf_size 5] [--path_format 'lr']
        [--skip_pruning] [--test_file <string>] [--training_file <string>]
        [--verbose] [--version] [--output_model_file <string>]
        [--tag_counters_file <string>] [--tag_file <string>]
        [--test_set_estimates_file <string>] [--training_set_estimates_file
        <string>] [--vi_file <string>]

An implementation of density estimation trees for the density estimation task. Density estimation trees can be trained or used to predict the density at locations given by query points. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--folds (-f) int The number of folds of cross-validation to perform for the estimation (0 is LOOCV) 10
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) DTree<> file Trained density estimation tree to load. ''
--max_leaf_size (-L) int The maximum size of a leaf in the unpruned, fully grown DET. 10
--min_leaf_size (-l) int The minimum size of a leaf in the unpruned, fully grown DET. 5
--path_format (-p) string The format of path printing: ‘lr’, ‘id-lr’, or ‘lr-id’. 'lr'
--skip_pruning (-s) flag Whether to bypass the pruning process and output the unpruned tree only.  
--test_file (-T) 2-d matrix file A set of test points to estimate the density of. ''
--training_file (-t) 2-d matrix file The data set on which to build a density estimation tree. ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) DTree<> file Output to save trained density estimation tree to.
--tag_counters_file (-c) string The file to output the number of points that went to each leaf.
--tag_file (-g) string The file to output the tags (and possibly paths) for each sample in the test set.
--test_set_estimates_file (-E) 2-d matrix file The output estimates on the test set from the final optimally pruned tree.
--training_set_estimates_file (-e) 2-d matrix file The output density estimates on the training set from the final optimally pruned tree.
--vi_file (-i) 2-d matrix file The output variable importance values for each feature.

Detailed documentation

This program performs a number of functions related to Density Estimation Trees. The optimal Density Estimation Tree (DET) can be trained on a set of data (specified by --training_file (-t)) using cross-validation (with number of folds specified with the --folds (-f) parameter). This trained density estimation tree may then be saved with the --output_model_file (-M) output parameter.

The variable importances (that is, the feature importance values for each dimension) may be saved with the --vi_file (-i) output parameter, and the density estimates for each training point may be saved with the --training_set_estimates_file (-e) output parameter.

Enabling path printing for each node outputs the path from the root node to a leaf for each entry in the test set, or training set (if a test set is not provided). Strings like ‘LRLRLR’ (indicating that traversal went to the left child, then the right child, then the left child, and so forth) will be output. If ‘lr-id’ or ‘id-lr’ are given as the --path_format (-p) parameter, then the ID (tag) of every node along the path will be printed after or before the L or R character indicating the direction of traversal, respectively.

This program also can provide density estimates for a set of test points, specified in the --test_file (-T) parameter. The density estimation tree used for this task will be the tree that was trained on the given training points, or a tree given as the parameter --input_model_file (-m). The density estimates for the test points may be saved using the --test_set_estimates_file (-E) output parameter.

See also

mlpack_emst

Fast Euclidean Minimum Spanning Tree

$ mlpack_emst [--help] [--info <string>] --input_file <string>
        [--leaf_size 1] [--naive] [--verbose] [--version] [--output_file
        <string>]

An implementation of the Dual-Tree Boruvka algorithm for computing the Euclidean minimum spanning tree of a set of input points. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Input data matrix. required
--leaf_size (-l) int Leaf size in the kd-tree. One-element leaves give the empirically best performance, but at the cost of greater memory requirements. 1
--naive (-n) flag Compute the MST using O(n^2) naive algorithm.  
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d matrix file Output data. Stored as an edge list.

Detailed documentation

This program can compute the Euclidean minimum spanning tree of a set of input points using the dual-tree Boruvka algorithm.

The set to calculate the minimum spanning tree of is specified with the --input_file (-i) parameter, and the output may be saved with the --output_file (-o) output parameter.

The --leaf_size (-l) parameter controls the leaf size of the kd-tree that is used to calculate the minimum spanning tree, and if the --naive (-n) option is given, then brute-force search is used (this is typically much slower in low dimensions). The leaf size does not affect the results, but it may have some effect on the runtime of the algorithm.

Example

For example, the minimum spanning tree of the input dataset 'data.csv' can be calculated with a leaf size of 20 and stored as 'spanning_tree.csv' using the following command:

$ mlpack_emst --input_file data.csv --leaf_size 20 --output_file
  spanning_tree.csv

The output matrix is a three-dimensional matrix, where each row indicates an edge. The first dimension corresponds to the lesser index of the edge; the second dimension corresponds to the greater index of the edge; and the third column corresponds to the distance between the two points.

See also

mlpack_fastmks

$ mlpack_fastmks [--bandwidth 1] [--base 2] [--degree 2] [--help]
        [--info <string>] [--input_model_file <string>] [--k 0] [--kernel
        'linear'] [--naive] [--offset 0] [--query_file <string>]
        [--reference_file <string>] [--scale 1] [--single] [--verbose]
        [--version] [--indices_file <string>] [--kernels_file <string>]
        [--output_model_file <string>]

An implementation of the single-tree and dual-tree fast max-kernel search (FastMKS) algorithm. Given a set of reference points and a set of query points, this can find the reference point with maximum kernel value for each query point; trained models can be reused for future queries. Detailed documentation.

Input options

name type description default
--bandwidth (-w) double Bandwidth (for Gaussian, Epanechnikov, and triangular kernels). 1
--base (-b) double Base to use during cover tree construction. 2
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--degree (-d) double Degree of polynomial kernel. 2
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) FastMKSModel file Input FastMKS model to use. ''
--k (-k) int Number of maximum kernels to find. 0
--kernel (-K) string Kernel type to use: ‘linear’, ‘polynomial’, ‘cosine’, ‘gaussian’, ‘epanechnikov’, ‘triangular’, ‘hyptan’. 'linear'
--naive (-N) flag If true, O(n^2) naive mode is used for computation.  
--offset (-o) double Offset of kernel (for polynomial and hyptan kernels). 0
--query_file (-q) 2-d matrix file The query dataset. ''
--reference_file (-r) 2-d matrix file The reference dataset. ''
--scale (-s) double Scale of kernel (for hyptan kernel). 1
--single (-S) flag If true, single-tree search is used (as opposed to dual-tree search.  
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--indices_file (-i) 2-d index matrix file Output matrix of indices.
--kernels_file (-p) 2-d matrix file Output matrix of kernels.
--output_model_file (-M) FastMKSModel file Output for FastMKS model.

Detailed documentation

This program will find the k maximum kernels of a set of points, using a query set and a reference set (which can optionally be the same set). More specifically, for each point in the query set, the k points in the reference set with maximum kernel evaluations are found. The kernel function used is specified with the --kernel (-K) parameter.

Example

For example, the following command will calculate, for each point in the query set 'query.csv', the five points in the reference set 'reference.csv' with maximum kernel evaluation using the linear kernel. The kernel evaluations may be saved with the 'kernels.csv' output parameter and the indices may be saved with the 'indices.csv' output parameter.

$ mlpack_fastmks --k 5 --reference_file reference.csv --query_file query.csv
  --indices_file indices.csv --kernels_file kernels.csv --kernel linear

The output matrices are organized such that row i and column j in the indices matrix corresponds to the index of the point in the reference set that has j’th largest kernel evaluation with the point in the query set with index i. Row i and column j in the kernels matrix corresponds to the kernel evaluation between those two points.

This program performs FastMKS using a cover tree. The base used to build the cover tree can be specified with the --base (-b) parameter.

See also

mlpack_gmm_train

Gaussian Mixture Model (GMM) Training

$ mlpack_gmm_train [--diagonal_covariance] --gaussians 0 [--help]
        [--info <string>] --input_file <string> [--input_model_file <string>]
        [--kmeans_max_iterations 1000] [--max_iterations 250]
        [--no_force_positive] [--noise 0] [--percentage 0.02] [--refined_start]
        [--samplings 100] [--seed 0] [--tolerance 1e-10] [--trials 1]
        [--verbose] [--version] [--output_model_file <string>]

An implementation of the EM algorithm for training Gaussian mixture models (GMMs). Given a dataset, this can train a GMM for future use with other tools. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--diagonal_covariance (-d) flag Force the covariance of the Gaussians to be diagonal. This can accelerate training time significantly.  
--gaussians (-g) int Number of Gaussians in the GMM. required
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file The training data on which the model will be fit. required
--input_model_file (-m) GMM file Initial input GMM model to start training with. ''
--kmeans_max_iterations (-k) int Maximum number of iterations for the k-means algorithm (used to initialize EM). 1000
--max_iterations (-n) int Maximum number of iterations of EM algorithm (passing 0 will run until convergence). 250
--no_force_positive (-P) flag Do not force the covariance matrices to be positive definite.  
--noise (-N) double Variance of zero-mean Gaussian noise to add to data. 0
--percentage (-p) double If using –refined_start, specify the percentage of the dataset used for each sampling (should be between 0.0 and 1.0). 0.02
--refined_start (-r) flag During the initialization, use refined initial positions for k-means clustering (Bradley and Fayyad, 1998).  
--samplings (-S) int If using –refined_start, specify the number of samplings used for initial points. 100
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--tolerance (-T) double Tolerance for convergence of EM. 1e-10
--trials (-t) int Number of trials to perform in training GMM. 1
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) GMM file Output for trained GMM model.

Detailed documentation

This program takes a parametric estimate of a Gaussian mixture model (GMM) using the EM algorithm to find the maximum likelihood estimate. The model may be saved and reused by other mlpack GMM tools.

The input data to train on must be specified with the --input_file (-i) parameter, and the number of Gaussians in the model must be specified with the --gaussians (-g) parameter. Optionally, many trials with different random initializations may be run, and the result with highest log-likelihood on the training data will be taken. The number of trials to run is specified with the --trials (-t) parameter. By default, only one trial is run.

The tolerance for convergence and maximum number of iterations of the EM algorithm are specified with the --tolerance (-T) and --max_iterations (-n) parameters, respectively. The GMM may be initialized for training with another model, specified with the --input_model_file (-m) parameter. Otherwise, the model is initialized by running k-means on the data. The k-means clustering initialization can be controlled with the --kmeans_max_iterations (-k), --refined_start (-r), --samplings (-S), and --percentage (-p) parameters. If --refined_start (-r) is specified, then the Bradley-Fayyad refined start initialization will be used. This can often lead to better clustering results.

The ‘diagonal_covariance’ flag will cause the learned covariances to be diagonal matrices. This significantly simplifies the model itself and causes training to be faster, but restricts the ability to fit more complex GMMs.

If GMM training fails with an error indicating that a covariance matrix could not be inverted, make sure that the --no_force_positive (-P) parameter is not specified. Alternately, adding a small amount of Gaussian noise (using the --noise (-N) parameter) to the entire dataset may help prevent Gaussians with zero variance in a particular dimension, which is usually the cause of non-invertible covariance matrices.

The --no_force_positive (-P) parameter, if set, will avoid the checks after each iteration of the EM algorithm which ensure that the covariance matrices are positive definite. Specifying the flag can cause faster runtime, but may also cause non-positive definite covariance matrices, which will cause the program to crash.

Example

As an example, to train a 6-Gaussian GMM on the data in 'data.csv' with a maximum of 100 iterations of EM and 3 trials, saving the trained GMM to 'gmm.bin', the following command can be used:

$ mlpack_gmm_train --input_file data.csv --gaussians 6 --trials 3
  --output_model_file gmm.bin

To re-train that GMM on another set of data 'data2.csv', the following command may be used:

$ mlpack_gmm_train --input_model_file gmm.bin --input_file data2.csv
  --gaussians 6 --output_model_file new_gmm.bin

See also

mlpack_gmm_generate

GMM Sample Generator

$ mlpack_gmm_generate [--help] [--info <string>] --input_model_file
        <string> --samples 0 [--seed 0] [--verbose] [--version] [--output_file
        <string>]

A sample generator for pre-trained GMMs. Given a pre-trained GMM, this can sample new points randomly from that distribution. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) GMM file Input GMM model to generate samples from. required
--samples (-n) int Number of samples to generate. required
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d matrix file Matrix to save output samples in.

Detailed documentation

This program is able to generate samples from a pre-trained GMM (use gmm_train to train a GMM). The pre-trained GMM must be specified with the --input_model_file (-m) parameter. The number of samples to generate is specified by the --samples (-n) parameter. Output samples may be saved with the --output_file (-o) output parameter.

Example

The following command can be used to generate 100 samples from the pre-trained GMM 'gmm.bin' and store those generated samples in 'samples.csv':

$ mlpack_gmm_generate --input_model_file gmm.bin --samples 100 --output_file
  samples.csv

See also

mlpack_gmm_probability

GMM Probability Calculator

$ mlpack_gmm_probability [--help] [--info <string>] --input_file
        <string> --input_model_file <string> [--verbose] [--version]
        [--output_file <string>]

A probability calculator for GMMs. Given a pre-trained GMM and a set of points, this can compute the probability that each point is from the given GMM. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Input matrix to calculate probabilities of. required
--input_model_file (-m) GMM file Input GMM to use as model. required
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d matrix file Matrix to store calculated probabilities in.

Detailed documentation

This program calculates the probability that given points came from a given GMM (that is, P(X | gmm)). The GMM is specified with the --input_model_file (-m) parameter, and the points are specified with the --input_file (-i) parameter. The output probabilities may be saved via the --output_file (-o) output parameter.

Example

So, for example, to calculate the probabilities of each point in 'points.csv' coming from the pre-trained GMM 'gmm.bin', while storing those probabilities in 'probs.csv', the following command could be used:

$ mlpack_gmm_probability --input_model_file gmm.bin --input_file points.csv
  --output_file probs.csv

See also

mlpack_hmm_train

Hidden Markov Model (HMM) Training

$ mlpack_hmm_train [--batch] [--gaussians 0] [--help] [--info <string>]
        --input_file <string> [--input_model_file <string>] [--labels_file
        <string>] [--seed 0] [--states 0] [--tolerance 1e-05] [--type
        'gaussian'] [--verbose] [--version] [--output_model_file <string>]

An implementation of training algorithms for Hidden Markov Models (HMMs). Given labeled or unlabeled data, an HMM can be trained for further use with other mlpack HMM tools. Detailed documentation.

Input options

name type description default
--batch (-b) flag If true, input_file (and if passed, labels_file) are expected to contain a list of files to use as input observation sequences (and label sequences).  
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--gaussians (-g) int Number of gaussians in each GMM (necessary when type is ‘gmm’). 0
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) string File containing input observations. required
--input_model_file (-m) HMMModel file Pre-existing HMM model to initialize training with. ''
--labels_file (-l) string Optional file of hidden states, used for labeled training. ''
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--states (-n) int Number of hidden states in HMM (necessary, unless model_file is specified). 0
--tolerance (-T) double Tolerance of the Baum-Welch algorithm. 1e-05
--type (-t) string Type of HMM: discrete | gaussian | diag_gmm | gmm. 'gaussian'
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) HMMModel file Output for trained HMM.

Detailed documentation

This program allows a Hidden Markov Model to be trained on labeled or unlabeled data. It supports four types of HMMs: Discrete HMMs, Gaussian HMMs, GMM HMMs, or Diagonal GMM HMMs

Either one input sequence can be specified (with --input_file (-i)), or, a file containing files in which input sequences can be found (when --input_file (-i)and--batch (-b) are used together). In addition, labels can be provided in the file specified by --labels_file (-l), and if --batch (-b) is used, the file given to --labels_file (-l) should contain a list of files of labels corresponding to the sequences in the file given to --input_file (-i).

The HMM is trained with the Baum-Welch algorithm if no labels are provided. The tolerance of the Baum-Welch algorithm can be set with the --tolerance (-T)option. By default, the transition matrix is randomly initialized and the emission distributions are initialized to fit the extent of the data.

Optionally, a pre-created HMM model can be used as a guess for the transition matrix and emission probabilities; this is specifiable with --output_model_file (-M).

See also

mlpack_hmm_generate

Hidden Markov Model (HMM) Sequence Generator

$ mlpack_hmm_generate [--help] [--info <string>] --length 0 --model_file
        <string> [--seed 0] [--start_state 0] [--verbose] [--version]
        [--output_file <string>] [--state_file <string>]

A utility to generate random sequences from a pre-trained Hidden Markov Model (HMM). The length of the desired sequence can be specified, and a random sequence of observations is returned. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--length (-l) int Length of sequence to generate. required
--model_file (-m) HMMModel file Trained HMM to generate sequences with. required
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--start_state (-t) int Starting state of sequence. 0
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d matrix file Matrix to save observation sequence to.
--state_file (-S) 2-d index matrix file Matrix to save hidden state sequence to.

Detailed documentation

This utility takes an already-trained HMM, specified as the --model_file (-m) parameter, and generates a random observation sequence and hidden state sequence based on its parameters. The observation sequence may be saved with the --output_file (-o) output parameter, and the internal state sequence may be saved with the --state_file (-S) output parameter.

The state to start the sequence in may be specified with the --start_state (-t) parameter.

Example

For example, to generate a sequence of length 150 from the HMM 'hmm.bin' and save the observation sequence to 'observations.csv' and the hidden state sequence to 'states.csv', the following command may be used:

$ mlpack_hmm_generate --model_file hmm.bin --length 150 --output_file
  observations.csv --state_file states.csv

See also

mlpack_hmm_loglik

Hidden Markov Model (HMM) Sequence Log-Likelihood

$ mlpack_hmm_loglik [--help] [--info <string>] --input_file <string>
        --input_model_file <string> [--verbose] [--version] [--log_likelihood
        0]

A utility for computing the log-likelihood of a sequence for Hidden Markov Models (HMMs). Given a pre-trained HMM and an observation sequence, this computes and returns the log-likelihood of that sequence being observed from that HMM. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file File containing observations, required
--input_model_file (-m) HMMModel file File containing HMM. required
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--log_likelihood double Log-likelihood of the sequence.

Detailed documentation

This utility takes an already-trained HMM, specified with the --input_model_file (-m) parameter, and evaluates the log-likelihood of a sequence of observations, given with the --input_file (-i) parameter. The computed log-likelihood is given as output.

Example

For example, to compute the log-likelihood of the sequence 'seq.csv' with the pre-trained HMM 'hmm.bin', the following command may be used:

$ mlpack_hmm_loglik --input_file seq.csv --input_model_file hmm.bin

See also

mlpack_hmm_viterbi

Hidden Markov Model (HMM) Viterbi State Prediction

$ mlpack_hmm_viterbi [--help] [--info <string>] --input_file <string>
        --input_model_file <string> [--verbose] [--version] [--output_file
        <string>]

A utility for computing the most probable hidden state sequence for Hidden Markov Models (HMMs). Given a pre-trained HMM and an observed sequence, this uses the Viterbi algorithm to compute and return the most probable hidden state sequence. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Matrix containing observations, required
--input_model_file (-m) HMMModel file Trained HMM to use. required
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d index matrix file File to save predicted state sequence to.

Detailed documentation

This utility takes an already-trained HMM, specified as --input_model_file (-m), and evaluates the most probable hidden state sequence of a given sequence of observations (specified as ‘--input_file (-i), using the Viterbi algorithm. The computed state sequence may be saved using the --output_file (-o) output parameter.

Example

For example, to predict the state sequence of the observations 'obs.csv' using the HMM 'hmm.bin', storing the predicted state sequence to 'states.csv', the following command could be used:

$ mlpack_hmm_viterbi --input_file obs.csv --input_model_file hmm.bin
  --output_file states.csv

See also

mlpack_hoeffding_tree

Hoeffding trees

$ mlpack_hoeffding_tree [--batch_mode] [--bins 10] [--confidence 0.95]
        [--help] [--info <string>] [--info_gain] [--input_model_file <string>]
        [--labels_file <string>] [--max_samples 5000] [--min_samples 100]
        [--numeric_split_strategy 'binary'] [--observations_before_binning 100]
        [--passes 1] [--test_file <string>] [--test_labels_file <string>]
        [--training_file <string>] [--verbose] [--version] [--output_model_file
        <string>] [--predictions_file <string>] [--probabilities_file <string>]

An implementation of Hoeffding trees, a form of streaming decision tree for classification. Given labeled data, a Hoeffding tree can be trained and saved for later use, or a pre-trained Hoeffding tree can be used for predicting the classifications of new points. Detailed documentation.

Input options

name type description default
--batch_mode (-b) flag If true, samples will be considered in batch instead of as a stream. This generally results in better trees but at the cost of memory usage and runtime.  
--bins (-B) int If the ‘domingos’ split strategy is used, this specifies the number of bins for each numeric split. 10
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--confidence (-c) double Confidence before splitting (between 0 and 1). 0.95
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--info_gain (-i) flag If set, information gain is used instead of Gini impurity for calculating Hoeffding bounds.  
--input_model_file (-m) HoeffdingTreeModel file Input trained Hoeffding tree model. ''
--labels_file (-l) 1-d index matrix file Labels for training dataset. ''
--max_samples (-n) int Maximum number of samples before splitting. 5000
--min_samples (-I) int Minimum number of samples before splitting. 100
--numeric_split_strategy (-N) string The splitting strategy to use for numeric features: ‘domingos’ or ‘binary’. 'binary'
--observations_before_binning (-o) int If the ‘domingos’ split strategy is used, this specifies the number of samples observed before binning is performed. 100
--passes (-s) int Number of passes to take over the dataset. 1
--test_file (-T) 2-d categorical matrix file Testing dataset (may be categorical). ''
--test_labels_file (-L) 1-d index matrix file Labels of test data. ''
--training_file (-t) 2-d categorical matrix file Training dataset (may be categorical). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) HoeffdingTreeModel file Output for trained Hoeffding tree model.
--predictions_file (-p) 1-d index matrix file Matrix to output label predictions for test data into.
--probabilities_file (-P) 2-d matrix file In addition to predicting labels, provide rediction probabilities in this matrix.

Detailed documentation

This program implements Hoeffding trees, a form of streaming decision tree suited best for large (or streaming) datasets. This program supports both categorical and numeric data. Given an input dataset, this program is able to train the tree with numerous training options, and save the model to a file. The program is also able to use a trained model or a model from file in order to predict classes for a given test set.

The training file and associated labels are specified with the --training_file (-t) and --labels_file (-l) parameters, respectively. Optionally, if --labels_file (-l) is not specified, the labels are assumed to be the last dimension of the training dataset.

The training may be performed in batch mode (like a typical decision tree algorithm) by specifying the --batch_mode (-b) option, but this may not be the best option for large datasets.

When a model is trained, it may be saved via the --output_model_file (-M) output parameter. A model may be loaded from file for further training or testing with the --input_model_file (-m) parameter.

Test data may be specified with the --test_file (-T) parameter, and if performance statistics are desired for that test set, labels may be specified with the --test_labels_file (-L) parameter. Predictions for each test point may be saved with the --predictions_file (-p) output parameter, and class probabilities for each prediction may be saved with the --probabilities_file (-P) output parameter.

Example

For example, to train a Hoeffding tree with confidence 0.99 with data 'dataset.csv', saving the trained tree to 'tree.bin', the following command may be used:

$ mlpack_hoeffding_tree --training_file dataset.arff --confidence 0.99
  --output_model_file tree.bin

Then, this tree may be used to make predictions on the test set 'test_set.csv', saving the predictions into 'predictions.csv' and the class probabilities into 'class_probs.csv' with the following command:

$ mlpack_hoeffding_tree --input_model_file tree.bin --test_file test_set.arff
  --predictions_file predictions.csv --probabilities_file class_probs.csv

See also

mlpack_kde

Kernel Density Estimation

$ mlpack_kde [--abs_error 0] [--algorithm 'dual-tree'] [--bandwidth 1]
        [--help] [--info <string>] [--initial_sample_size 100]
        [--input_model_file <string>] [--kernel 'gaussian'] [--mc_break_coef
        0.4] [--mc_entry_coef 3] [--mc_probability 0.95] [--monte_carlo]
        [--query_file <string>] [--reference_file <string>] [--rel_error 0.05]
        [--tree 'kd-tree'] [--verbose] [--version] [--output_model_file
        <string>] [--predictions_file <string>]

An implementation of kernel density estimation with dual-tree algorithms. Given a set of reference points and query points and a kernel function, this can estimate the density function at the location of each query point using trees; trees that are built can be saved for later use. Detailed documentation.

Input options

name type description default
--abs_error (-E) double Relative error tolerance for the prediction. 0
--algorithm (-a) string Algorithm to use for the prediction.(‘dual-tree’, ‘single-tree’). 'dual-tree'
--bandwidth (-b) double Bandwidth of the kernel. 1
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--initial_sample_size (-s) int Initial sample size for Monte Carlo estimations. 100
--input_model_file (-m) KDEModel file Contains pre-trained KDE model. ''
--kernel (-k) string Kernel to use for the prediction.(‘gaussian’, ‘epanechnikov’, ‘laplacian’, ‘spherical’, ‘triangular’). 'gaussian'
--mc_break_coef (-c) double Controls what fraction of the amount of node’s descendants is the limit for the sample size before it recurses. 0.4
--mc_entry_coef (-C) double Controls how much larger does the amount of node descendants has to be compared to the initial sample size in order to be a candidate for Monte Carlo estimations. 3
--mc_probability (-P) double Probability of the estimation being bounded by relative error when using Monte Carlo estimations. 0.95
--monte_carlo (-S) flag Whether to use Monte Carlo estimations when possible.  
--query_file (-q) 2-d matrix file Query dataset to KDE on. ''
--reference_file (-r) 2-d matrix file Input reference dataset use for KDE. ''
--rel_error (-e) double Relative error tolerance for the prediction. 0.05
--tree (-t) string Tree to use for the prediction.(‘kd-tree’, ‘ball-tree’, ‘cover-tree’, ‘octree’, ‘r-tree’). 'kd-tree'
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) KDEModel file If specified, the KDE model will be saved here.
--predictions_file (-p) 1-d matrix file Vector to store density predictions.

Detailed documentation

This program performs a Kernel Density Estimation. KDE is a non-parametric way of estimating probability density function. For each query point the program will estimate its probability density by applying a kernel function to each reference point. The computational complexity of this is O(N^2) where there are N query points and N reference points, but this implementation will typically see better performance as it uses an approximate dual or single tree algorithm for acceleration.

Dual or single tree optimization avoids many barely relevant calculations (as kernel function values decrease with distance), so it is an approximate computation. You can specify the maximum relative error tolerance for each query value with --rel_error (-e) as well as the maximum absolute error tolerance with the parameter --abs_error (-E). This program runs using an Euclidean metric. Kernel function can be selected using the --kernel (-k) option. You can also choose what which type of tree to use for the dual-tree algorithm with --tree (-t). It is also possible to select whether to use dual-tree algorithm or single-tree algorithm using the --algorithm (-a) option.

Monte Carlo estimations can be used to accelerate the KDE estimate when the Gaussian Kernel is used. This provides a probabilistic guarantee on the the error of the resulting KDE instead of an absolute guarantee.To enable Monte Carlo estimations, the --monte_carlo (-S) flag can be used, and success probability can be set with the --mc_probability (-P) option. It is possible to set the initial sample size for the Monte Carlo estimation using --initial_sample_size (-s). This implementation will only consider a node, as a candidate for the Monte Carlo estimation, if its number of descendant nodes is bigger than the initial sample size. This can be controlled using a coefficient that will multiply the initial sample size and can be set using --mc_entry_coef (-C). To avoid using the same amount of computations an exact approach would take, this program recurses the tree whenever a fraction of the amount of the node’s descendant points have already been computed. This fraction is set using --mc_break_coef (-c).

Example

For example, the following will run KDE using the data in 'ref_data.csv' for training and the data in 'qu_data.csv' as query data. It will apply an Epanechnikov kernel with a 0.2 bandwidth to each reference point and use a KD-Tree for the dual-tree optimization. The returned predictions will be within 5% of the real KDE value for each query point.

$ mlpack_kde --reference_file ref_data.csv --query_file qu_data.csv
  --bandwidth 0.2 --kernel epanechnikov --tree kd-tree --rel_error 0.05
  --predictions_file out_data.csv

the predicted density estimations will be stored in 'out_data.csv'. If no --query_file (-q) is provided, then KDE will be computed on the --reference_file (-r) dataset. It is possible to select either a reference dataset or an input model but not both at the same time. If an input model is selected and parameter values are not set (e.g. --bandwidth (-b)) then default parameter values will be used.

In addition to the last program call, it is also possible to activate Monte Carlo estimations if a Gaussian kernel is used. This can provide faster results, but the KDE will only have a probabilistic guarantee of meeting the desired error bound (instead of an absolute guarantee). The following example will run KDE using a Monte Carlo estimation when possible. The results will be within a 5% of the real KDE value with a 95% probability. Initial sample size for the Monte Carlo estimation will be 200 points and a node will be a candidate for the estimation only when it contains 700 (i.e. 3.5200) points. If a node contains 700 points and 420 (i.e. 0.6700) have already been sampled, then the algorithm will recurse instead of keep sampling.

$ mlpack_kde --reference_file ref_data.csv --query_file qu_data.csv
  --bandwidth 0.2 --kernel gaussian --tree kd-tree --rel_error 0.05
  --predictions_file out_data.csv --monte_carlo --mc_probability 0.95
  --initial_sample_size 200 --mc_entry_coef 3.5 --mc_break_coef 0.6

See also

mlpack_kernel_pca

Kernel Principal Components Analysis

$ mlpack_kernel_pca [--bandwidth 1] [--center] [--degree 1] [--help]
        [--info <string>] --input_file <string> --kernel <string>
        [--kernel_scale 1] [--new_dimensionality 0] [--nystroem_method]
        [--offset 0] [--sampling 'kmeans'] [--verbose] [--version]
        [--output_file <string>]

An implementation of Kernel Principal Components Analysis (KPCA). This can be used to perform nonlinear dimensionality reduction or preprocessing on a given dataset. Detailed documentation.

Input options

name type description default
--bandwidth (-b) double Bandwidth, for ‘gaussian’ and ‘laplacian’ kernels. 1
--center (-c) flag If set, the transformed data will be centered about the origin.  
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--degree (-D) double Degree of polynomial, for ‘polynomial’ kernel. 1
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Input dataset to perform KPCA on. required
--kernel (-k) string The kernel to use; see the above documentation for the list of usable kernels. required
--kernel_scale (-S) double Scale, for ‘hyptan’ kernel. 1
--new_dimensionality (-d) int If not 0, reduce the dimensionality of the output dataset by ignoring the dimensions with the smallest eigenvalues. 0
--nystroem_method (-n) flag If set, the Nystroem method will be used.  
--offset (-O) double Offset, for ‘hyptan’ and ‘polynomial’ kernels. 0
--sampling (-s) string Sampling scheme to use for the Nystroem method: ‘kmeans’, ‘random’, ‘ordered’ 'kmeans'
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d matrix file Matrix to save modified dataset to.

Detailed documentation

This program performs Kernel Principal Components Analysis (KPCA) on the specified dataset with the specified kernel. This will transform the data onto the kernel principal components, and optionally reduce the dimensionality by ignoring the kernel principal components with the smallest eigenvalues.

For the case where a linear kernel is used, this reduces to regular PCA.

The kernels that are supported are listed below:

  • ‘linear’: the standard linear dot product (same as normal PCA): K(x, y) = x^T y

  • ‘gaussian’: a Gaussian kernel; requires bandwidth: K(x, y) = exp(-(|| x - y || ^ 2) / (2 * (bandwidth ^ 2)))

  • ‘polynomial’: polynomial kernel; requires offset and degree: K(x, y) = (x^T y + offset) ^ degree

  • ‘hyptan’: hyperbolic tangent kernel; requires scale and offset: K(x, y) = tanh(scale * (x^T y) + offset)

  • ‘laplacian’: Laplacian kernel; requires bandwidth: K(x, y) = exp(-(|| x - y ||) / bandwidth)

  • ‘epanechnikov’: Epanechnikov kernel; requires bandwidth: K(x, y) = max(0, 1 - || x - y ||^2 / bandwidth^2)

  • ‘cosine’: cosine distance: K(x, y) = 1 - (x^T y) / (|| x || * || y ||)

The parameters for each of the kernels should be specified with the options --bandwidth (-b), --kernel_scale (-S), --offset (-O), or --degree (-D) (or a combination of those parameters).

Optionally, the Nystroem method (“Using the Nystroem method to speed up kernel machines”, 2001) can be used to calculate the kernel matrix by specifying the --nystroem_method (-n) parameter. This approach works by using a subset of the data as basis to reconstruct the kernel matrix; to specify the sampling scheme, the --sampling (-s) parameter is used. The sampling scheme for the Nystroem method can be chosen from the following list: ‘kmeans’, ‘random’, ‘ordered’.

Example

For example, the following command will perform KPCA on the dataset 'input.csv' using the Gaussian kernel, and saving the transformed data to 'transformed.csv':

$ mlpack_kernel_pca --input_file input.csv --kernel gaussian --output_file
  transformed.csv

See also

mlpack_kmeans

K-Means Clustering

$ mlpack_kmeans [--algorithm 'naive'] [--allow_empty_clusters]
        --clusters 0 [--help] [--in_place] [--info <string>]
        [--initial_centroids_file <string>] --input_file <string>
        [--kill_empty_clusters] [--kmeans_plus_plus] [--labels_only]
        [--max_iterations 1000] [--percentage 0.02] [--refined_start]
        [--samplings 100] [--seed 0] [--verbose] [--version] [--centroid_file
        <string>] [--output_file <string>]

An implementation of several strategies for efficient k-means clustering. Given a dataset and a value of k, this computes and returns a k-means clustering on that data. Detailed documentation.

Input options

name type description default
--algorithm (-a) string Algorithm to use for the Lloyd iteration (‘naive’, ‘pelleg-moore’, ‘elkan’, ‘hamerly’, ‘dualtree’, or ‘dualtree-covertree’). 'naive'
--allow_empty_clusters (-e) flag Allow empty clusters to be persist.  
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--clusters (-c) int Number of clusters to find (0 autodetects from initial centroids). required
--help (-h) flag Default help info. Only exists in CLI binding.  
--in_place (-P) flag If specified, a column containing the learned cluster assignments will be added to the input dataset file. In this case, –output_file is overridden. (Do not use in Python.)  
--info string Print help on a specific option. Only exists in CLI binding. ''
--initial_centroids_file (-I) 2-d matrix file Start with the specified initial centroids. ''
--input_file (-i) 2-d matrix file Input dataset to perform clustering on. required
--kill_empty_clusters (-E) flag Remove empty clusters when they occur.  
--kmeans_plus_plus (-K) flag Use the k-means++ initialization strategy to choose initial points.  
--labels_only (-l) flag Only output labels into output file.  
--max_iterations (-m) int Maximum number of iterations before k-means terminates. 1000
--percentage (-p) double Percentage of dataset to use for each refined start sampling (use when –refined_start is specified). 0.02
--refined_start (-r) flag Use the refined initial point strategy by Bradley and Fayyad to choose initial points.  
--samplings (-S) int Number of samplings to perform for refined start (use when –refined_start is specified). 100
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--centroid_file (-C) 2-d matrix file If specified, the centroids of each cluster will be written to the given file.
--output_file (-o) 2-d matrix file Matrix to store output labels or labeled data to.

Detailed documentation

This program performs K-Means clustering on the given dataset. It can return the learned cluster assignments, and the centroids of the clusters. Empty clusters are not allowed by default; when a cluster becomes empty, the point furthest from the centroid of the cluster with maximum variance is taken to fill that cluster.

Optionally, the strategy to choose initial centroids can be specified. The k-means++ algorithm can be used to choose initial centroids with the --kmeans_plus_plus (-K) parameter. The Bradley and Fayyad approach (“Refining initial points for k-means clustering”, 1998) can be used to select initial points by specifying the --refined_start (-r) parameter. This approach works by taking random samplings of the dataset; to specify the number of samplings, the --samplings (-S) parameter is used, and to specify the percentage of the dataset to be used in each sample, the --percentage (-p) parameter is used (it should be a value between 0.0 and 1.0).

There are several options available for the algorithm used for each Lloyd iteration, specified with the --algorithm (-a) option. The standard O(kN) approach can be used (‘naive’). Other options include the Pelleg-Moore tree-based algorithm (‘pelleg-moore’), Elkan’s triangle-inequality based algorithm (‘elkan’), Hamerly’s modification to Elkan’s algorithm (‘hamerly’), the dual-tree k-means algorithm (‘dualtree’), and the dual-tree k-means algorithm using the cover tree (‘dualtree-covertree’).

The behavior for when an empty cluster is encountered can be modified with the --allow_empty_clusters (-e) option. When this option is specified and there is a cluster owning no points at the end of an iteration, that cluster’s centroid will simply remain in its position from the previous iteration. If the --kill_empty_clusters (-E) option is specified, then when a cluster owns no points at the end of an iteration, the cluster centroid is simply filled with DBL_MAX, killing it and effectively reducing k for the rest of the computation. Note that the default option when neither empty cluster option is specified can be time-consuming to calculate; therefore, specifying either of these parameters will often accelerate runtime.

Initial clustering assignments may be specified using the --initial_centroids_file (-I) parameter, and the maximum number of iterations may be specified with the --max_iterations (-m) parameter.

Example

As an example, to use Hamerly’s algorithm to perform k-means clustering with k=10 on the dataset 'data.csv', saving the centroids to 'centroids.csv' and the assignments for each point to 'assignments.csv', the following command could be used:

$ mlpack_kmeans --input_file data.csv --clusters 10 --output_file
  assignments.csv --centroid_file centroids.csv

To run k-means on that same dataset with initial centroids specified in 'initial.csv' with a maximum of 500 iterations, storing the output centroids in 'final.csv' the following command may be used:

$ mlpack_kmeans --input_file data.csv --initial_centroids_file initial.csv
  --clusters 10 --max_iterations 500 --centroid_file final.csv

See also

mlpack_lars

LARS

$ mlpack_lars [--help] [--info <string>] [--input_file <string>]
        [--input_model_file <string>] [--lambda1 0] [--lambda2 0]
        [--no_intercept] [--no_normalize] [--responses_file <string>]
        [--test_file <string>] [--use_cholesky] [--verbose] [--version]
        [--output_model_file <string>] [--output_predictions_file <string>]

An implementation of Least Angle Regression (Stagewise/laSso), also known as LARS. This can train a LARS/LASSO/Elastic Net model and use that model or a pre-trained model to output regression predictions for a test set. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Matrix of covariates (X). ''
--input_model_file (-m) LARS file Trained LARS model to use. ''
--lambda1 (-l) double Regularization parameter for l1-norm penalty. 0
--lambda2 (-L) double Regularization parameter for l2-norm penalty. 0
--no_intercept (-n) flag Do not fit an intercept in the model.  
--no_normalize (-N) flag Do not normalize data to unit variance before modeling.  
--responses_file (-r) 2-d matrix file Matrix of responses/observations (y). ''
--test_file (-t) 2-d matrix file Matrix containing points to regress on (test points). ''
--use_cholesky (-c) flag Use Cholesky decomposition during computation rather than explicitly computing the full Gram matrix.  
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) LARS file Output LARS model.
--output_predictions_file (-o) 2-d matrix file If –test_file is specified, this file is where the predicted responses will be saved.

Detailed documentation

An implementation of LARS: Least Angle Regression (Stagewise/laSso). This is a stage-wise homotopy-based algorithm for L1-regularized linear regression (LASSO) and L1+L2-regularized linear regression (Elastic Net).

This program is able to train a LARS/LASSO/Elastic Net model or load a model from file, output regression predictions for a test set, and save the trained model to a file. The LARS algorithm is described in more detail below:

Let X be a matrix where each row is a point and each column is a dimension, and let y be a vector of targets.

The Elastic Net problem is to solve

min_beta 0.5 || X * beta - y ||_2^2 + lambda_1 ||beta||_1 + 0.5 lambda_2 ||beta||_2^2

If lambda1 > 0 and lambda2 = 0, the problem is the LASSO. If lambda1 > 0 and lambda2 > 0, the problem is the Elastic Net. If lambda1 = 0 and lambda2 > 0, the problem is ridge regression. If lambda1 = 0 and lambda2 = 0, the problem is unregularized linear regression.

For efficiency reasons, it is not recommended to use this algorithm with --lambda1 (-l) = 0. In that case, use the ‘linear_regression’ program, which implements both unregularized linear regression and ridge regression.

To train a LARS/LASSO/Elastic Net model, the --input_file (-i) and --responses_file (-r) parameters must be given. The --lambda1 (-l), --lambda2 (-L), and --use_cholesky (-c) parameters control the training options. A trained model can be saved with the --output_model_file (-M). If no training is desired at all, a model can be passed via the --input_model_file (-m) parameter.

The program can also provide predictions for test data using either the trained model or the given input model. Test points can be specified with the --test_file (-t) parameter. Predicted responses to the test points can be saved with the --output_predictions_file (-o) output parameter.

Example

For example, the following command trains a model on the data 'data.csv' and responses 'responses.csv' with lambda1 set to 0.4 and lambda2 set to 0 (so, LASSO is being solved), and then the model is saved to 'lasso_model.bin':

$ mlpack_lars --input_file data.csv --responses_file responses.csv --lambda1
  0.4 --lambda2 0 --output_model_file lasso_model.bin

The following command uses the 'lasso_model.bin' to provide predicted responses for the data 'test.csv' and save those responses to 'test_predictions.csv':

$ mlpack_lars --input_model_file lasso_model.bin --test_file test.csv
  --output_predictions_file test_predictions.csv

See also

mlpack_linear_svm

Linear SVM is an L2-regularized support vector machine.

$ mlpack_linear_svm [--delta 1] [--epochs 50] [--help] [--info <string>]
        [--input_model_file <string>] [--labels_file <string>] [--lambda 0.0001]
        [--max_iterations 10000] [--no_intercept] [--num_classes 0] [--optimizer
        'lbfgs'] [--seed 0] [--shuffle] [--step_size 0.01] [--test_file
        <string>] [--test_labels_file <string>] [--tolerance 1e-10]
        [--training_file <string>] [--verbose] [--version] [--output_model_file
        <string>] [--predictions_file <string>] [--probabilities_file <string>]

An implementation of linear SVM for multiclass classification. Given labeled data, a model can be trained and saved for future use; or, a pre-trained model can be used to classify new points. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--delta (-d) double Margin of difference between correct class and other classes. 1
--epochs (-E) int Maximum number of full epochs over dataset for psgd 50
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) LinearSVMModel file Existing model (parameters). ''
--labels_file (-l) 1-d index matrix file A matrix containing labels (0 or 1) for the points in the training set (y). ''
--lambda (-r) double L2-regularization parameter for training. 0.0001
--max_iterations (-n) int Maximum iterations for optimizer (0 indicates no limit). 10000
--no_intercept (-N) flag Do not add the intercept term to the model.  
--num_classes (-c) int Number of classes for classification; if unspecified (or 0), the number of classes found in the labels will be used. 0
--optimizer (-O) string Optimizer to use for training (‘lbfgs’ or ‘psgd’). 'lbfgs'
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--shuffle (-S) flag Don’t shuffle the order in which data points are visited for parallel SGD.  
--step_size (-a) double Step size for parallel SGD optimizer. 0.01
--test_file (-T) 2-d matrix file Matrix containing test dataset. ''
--test_labels_file (-L) 1-d index matrix file Matrix containing test labels. ''
--tolerance (-e) double Convergence tolerance for optimizer. 1e-10
--training_file (-t) 2-d matrix file A matrix containing the training set (the matrix of predictors, X). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) LinearSVMModel file Output for trained linear svm model.
--predictions_file (-P) 1-d index matrix file If test data is specified, this matrix is where the predictions for the test set will be saved.
--probabilities_file (-p) 2-d matrix file If test data is specified, this matrix is where the class probabilities for the test set will be saved.

Detailed documentation

An implementation of linear SVMs that uses either L-BFGS or parallel SGD (stochastic gradient descent) to train the model.

This program allows loading a linear SVM model (via the --input_model_file (-m) parameter) or training a linear SVM model given training data (specified with the --training_file (-t) parameter), or both those things at once. In addition, this program allows classification on a test dataset (specified with the --test_file (-T) parameter) and the classification results may be saved with the --predictions_file (-P) output parameter. The trained linear SVM model may be saved using the --output_model_file (-M) output parameter.

The training data, if specified, may have class labels as its last dimension. Alternately, the --labels_file (-l) parameter may be used to specify a separate vector of labels.

When a model is being trained, there are many options. L2 regularization (to prevent overfitting) can be specified with the --lambda (-r) option, and the number of classes can be manually specified with the --num_classes (-c)and if an intercept term is not desired in the model, the --no_intercept (-N) parameter can be specified.Margin of difference between correct class and other classes can be specified with the --delta (-d) option.The optimizer used to train the model can be specified with the --optimizer (-O) parameter. Available options are ‘psgd’ (parallel stochastic gradient descent) and ‘lbfgs’ (the L-BFGS optimizer). There are also various parameters for the optimizer; the --max_iterations (-n) parameter specifies the maximum number of allowed iterations, and the --tolerance (-e) parameter specifies the tolerance for convergence. For the parallel SGD optimizer, the --step_size (-a) parameter controls the step size taken at each iteration by the optimizer and the maximum number of epochs (specified with --epochs (-E)). If the objective function for your data is oscillating between Inf and 0, the step size is probably too large. There are more parameters for the optimizers, but the C++ interface must be used to access these.

Optionally, the model can be used to predict the labels for another matrix of data points, if --test_file (-T) is specified. The --test_file (-T) parameter can be specified without the --training_file (-t) parameter, so long as an existing linear SVM model is given with the --input_model_file (-m) parameter. The output predictions from the linear SVM model may be saved with the --predictions_file (-P) parameter.

Example

As an example, to train a LinaerSVM on the data ‘'data.csv'’ with labels ‘'labels.csv'’ with L2 regularization of 0.1, saving the model to ‘'lsvm_model.bin'’, the following command may be used:

$ mlpack_linear_svm --training_file data.csv --labels_file labels.csv --lambda
  0.1 --delta 1 --num_classes 0 --output_model_file lsvm_model.bin

Then, to use that model to predict classes for the dataset ‘'test.csv'’, storing the output predictions in ‘'predictions.csv'’, the following command may be used:

$ mlpack_linear_svm --input_model_file lsvm_model.bin --test_file test.csv
  --predictions_file predictions.csv

See also

mlpack_lmnn

Large Margin Nearest Neighbors (LMNN)

$ mlpack_lmnn [--batch_size 50] [--center] [--distance_file <string>]
        [--help] [--info <string>] --input_file <string> [--k 1] [--labels_file
        <string>] [--linear_scan] [--max_iterations 100000] [--normalize]
        [--optimizer 'amsgrad'] [--passes 50] [--print_accuracy] [--range 1]
        [--rank 0] [--regularization 0.5] [--seed 0] [--step_size 0.01]
        [--tolerance 1e-07] [--verbose] [--version] [--centered_data_file
        <string>] [--output_file <string>] [--transformed_data_file <string>]

An implementation of Large Margin Nearest Neighbors (LMNN), a distance learning technique. Given a labeled dataset, this learns a transformation of the data that improves k-nearest-neighbor performance; this can be useful as a preprocessing step. Detailed documentation.

Input options

name type description default
--batch_size (-b) int Batch size for mini-batch SGD. 50
--center (-C) flag Perform mean-centering on the dataset. It is useful when the centroid of the data is far from the origin.  
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--distance_file (-d) 2-d matrix file Initial distance matrix to be used as starting point ''
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Input dataset to run LMNN on. required
--k (-k) int Number of target neighbors to use for each datapoint. 1
--labels_file (-l) 1-d index matrix file Labels for input dataset. ''
--linear_scan (-L) flag Don’t shuffle the order in which data points are visited for SGD or mini-batch SGD.  
--max_iterations (-n) int Maximum number of iterations for L-BFGS (0 indicates no limit). 100000
--normalize (-N) flag Use a normalized starting point for optimization. Itis useful for when points are far apart, or when SGD is returning NaN.  
--optimizer (-O) string Optimizer to use; ‘amsgrad’, ‘bbsgd’, ‘sgd’, or ‘lbfgs’. 'amsgrad'
--passes (-p) int Maximum number of full passes over dataset for AMSGrad, BB_SGD and SGD. 50
--print_accuracy (-P) flag Print accuracies on initial and transformed dataset  
--range (-R) int Number of iterations after which impostors needs to be recalculated 1
--rank (-A) int Rank of distance matrix to be optimized. 0
--regularization (-r) double Regularization for LMNN objective function 0.5
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--step_size (-a) double Step size for AMSGrad, BB_SGD and SGD (alpha). 0.01
--tolerance (-t) double Maximum tolerance for termination of AMSGrad, BB_SGD, SGD or L-BFGS. 1e-07
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--centered_data_file (-c) 2-d matrix file Output matrix for mean-centered dataset.
--output_file (-o) 2-d matrix file Output matrix for learned distance matrix.
--transformed_data_file (-D) 2-d matrix file Output matrix for transformed dataset.

Detailed documentation

This program implements Large Margin Nearest Neighbors, a distance learning technique. The method seeks to improve k-nearest-neighbor classification on a dataset. The method employes the strategy of reducing distance between similar labeled data points (a.k.a target neighbors) and increasing distance between differently labeled points (a.k.a impostors) using standard optimization techniques over the gradient of the distance between data points.

To work, this algorithm needs labeled data. It can be given as the last row of the input dataset (specified with --input_file (-i)), or alternatively as a separate matrix (specified with --labels_file (-l)). Additionally, a starting point for optimization (specified with --distance_file (-d)can be given, having (r x d) dimensionality. Here r should satisfy 1 <= r <= d, Consequently a Low-Rank matrix will be optimized. Alternatively, Low-Rank distance can be learned by specifying the --rank (-A)parameter (A Low-Rank matrix with uniformly distributed values will be used as initial learning point).

The program also requires number of targets neighbors to work with ( specified with --k (-k)), A regularization parameter can also be passed, It acts as a trade of between the pulling and pushing terms (specified with --regularization (-r)), In addition, this implementation of LMNN includes a parameter to decide the interval after which impostors must be re-calculated (specified with --range (-R)).

Output can either be the learned distance matrix (specified with --output_file (-o)), or the transformed dataset (specified with --transformed_data_file (-D)), or both. Additionally mean-centered dataset (specified with --centered_data_file (-c)) can be accessed given mean-centering (specified with --center (-C)) is performed on the dataset. Accuracy on initial dataset and final transformed dataset can be printed by specifying the --print_accuracy (-P)parameter.

This implementation of LMNN uses AdaGrad, BigBatch_SGD, stochastic gradient descent, mini-batch stochastic gradient descent, or the L_BFGS optimizer.

AdaGrad, specified by the value ‘adagrad’ for the parameter --optimizer (-O), uses maximum of past squared gradients. It primarily on six parameters: the step size (specified with --step_size (-a)), the batch size (specified with --batch_size (-b)), the maximum number of passes (specified with --passes (-p)). Inaddition, a normalized starting point can be used by specifying the --normalize (-N) parameter.

BigBatch_SGD, specified by the value ‘bbsgd’ for the parameter --optimizer (-O), depends primarily on four parameters: the step size (specified with --step_size (-a)), the batch size (specified with --batch_size (-b)), the maximum number of passes (specified with --passes (-p)). In addition, a normalized starting point can be used by specifying the --normalize (-N) parameter.

Stochastic gradient descent, specified by the value ‘sgd’ for the parameter --optimizer (-O), depends primarily on three parameters: the step size (specified with --step_size (-a)), the batch size (specified with --batch_size (-b)), and the maximum number of passes (specified with --passes (-p)). In addition, a normalized starting point can be used by specifying the --normalize (-N) parameter. Furthermore, mean-centering can be performed on the dataset by specifying the --center (-C)parameter.

The L-BFGS optimizer, specified by the value ‘lbfgs’ for the parameter --optimizer (-O), uses a back-tracking line search algorithm to minimize a function. The following parameters are used by L-BFGS: --max_iterations (-n), --tolerance (-t)(the optimization is terminated when the gradient norm is below this value). For more details on the L-BFGS optimizer, consult either the mlpack L-BFGS documentation (in lbfgs.hpp) or the vast set of published literature on L-BFGS. In addition, a normalized starting point can be used by specifying the --normalize (-N) parameter.

By default, the AMSGrad optimizer is used.

Example

Example - Let’s say we want to learn distance on iris dataset with number of targets as 3 using BigBatch_SGD optimizer. A simple call for the same will look like:

$ mlpack_lmnn --input_file iris.csv --labels_file iris_labels.csv --k 3
  --optimizer bbsgd --output_file output.csv

An another program call making use of range & regularization parameter with dataset having labels as last column can be made as:

$ mlpack_lmnn --input_file letter_recognition.csv --k 5 --range 10
  --regularization 0.4 --output_file output.csv

See also

mlpack_local_coordinate_coding

Local Coordinate Coding

$ mlpack_local_coordinate_coding [--atoms 0] [--help] [--info <string>]
        [--initial_dictionary_file <string>] [--input_model_file <string>]
        [--lambda 0] [--max_iterations 0] [--normalize] [--seed 0] [--test_file
        <string>] [--tolerance 0.01] [--training_file <string>] [--verbose]
        [--version] [--codes_file <string>] [--dictionary_file <string>]
        [--output_model_file <string>]

An implementation of Local Coordinate Coding (LCC), a data transformation technique. Given input data, this transforms each point to be expressed as a linear combination of a few points in the dataset; once an LCC model is trained, it can be used to transform points later also. Detailed documentation.

Input options

name type description default
--atoms (-k) int Number of atoms in the dictionary. 0
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--initial_dictionary_file (-i) 2-d matrix file Optional initial dictionary. ''
--input_model_file (-m) LocalCoordinateCoding file Input LCC model. ''
--lambda (-l) double Weighted l1-norm regularization parameter. 0
--max_iterations (-n) int Maximum number of iterations for LCC (0 indicates no limit). 0
--normalize (-N) flag If set, the input data matrix will be normalized before coding.  
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--test_file (-T) 2-d matrix file Test points to encode. ''
--tolerance (-o) double Tolerance for objective function. 0.01
--training_file (-t) 2-d matrix file Matrix of training data (X). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--codes_file (-c) 2-d matrix file Output codes matrix.
--dictionary_file (-d) 2-d matrix file Output dictionary matrix.
--output_model_file (-M) LocalCoordinateCoding file Output for trained LCC model.

Detailed documentation

An implementation of Local Coordinate Coding (LCC), which codes data that approximately lives on a manifold using a variation of l1-norm regularized sparse coding. Given a dense data matrix X with n points and d dimensions, LCC seeks to find a dense dictionary matrix D with k atoms in d dimensions, and a coding matrix Z with n points in k dimensions. Because of the regularization method used, the atoms in D should lie close to the manifold on which the data points lie.

The original data matrix X can then be reconstructed as D * Z. Therefore, this program finds a representation of each point in X as a sparse linear combination of atoms in the dictionary D.

The coding is found with an algorithm which alternates between a dictionary step, which updates the dictionary D, and a coding step, which updates the coding matrix Z.

To run this program, the input matrix X must be specified (with -i), along with the number of atoms in the dictionary (-k). An initial dictionary may also be specified with the --initial_dictionary_file (-i) parameter. The l1-norm regularization parameter is specified with the --lambda (-l) parameter.

Example

For example, to run LCC on the dataset 'data.csv' using 200 atoms and an l1-regularization parameter of 0.1, saving the dictionary --dictionary_file (-d) and the codes into --codes_file (-c), use

$ mlpack_local_coordinate_coding --training_file data.csv --atoms 200 --lambda
  0.1 --dictionary_file dict.csv --codes_file codes.csv

The maximum number of iterations may be specified with the --max_iterations (-n) parameter. Optionally, the input data matrix X can be normalized before coding with the --normalize (-N) parameter.

An LCC model may be saved using the --output_model_file (-M) output parameter. Then, to encode new points from the dataset 'points.csv' with the previously saved model 'lcc_model.bin', saving the new codes to 'new_codes.csv', the following command can be used:

$ mlpack_local_coordinate_coding --input_model_file lcc_model.bin --test_file
  points.csv --codes_file new_codes.csv

See also

mlpack_logistic_regression

L2-regularized Logistic Regression and Prediction

$ mlpack_logistic_regression [--batch_size 64] [--decision_boundary 0.5]
        [--help] [--info <string>] [--input_model_file <string>] [--labels_file
        <string>] [--lambda 0] [--max_iterations 10000] [--optimizer 'lbfgs']
        [--step_size 0.01] [--test_file <string>] [--tolerance 1e-10]
        [--training_file <string>] [--verbose] [--version] [--output_model_file
        <string>] [--predictions_file <string>] [--probabilities_file <string>]

An implementation of L2-regularized logistic regression for two-class classification. Given labeled data, a model can be trained and saved for future use; or, a pre-trained model can be used to classify new points. Detailed documentation.

Input options

name type description default
--batch_size (-b) int Batch size for SGD. 64
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--decision_boundary (-d) double Decision boundary for prediction; if the logistic function for a point is less than the boundary, the class is taken to be 0; otherwise, the class is 1. 0.5
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) LogisticRegression<> file Existing model (parameters). ''
--labels_file (-l) 1-d index matrix file A matrix containing labels (0 or 1) for the points in the training set (y). ''
--lambda (-L) double L2-regularization parameter for training. 0
--max_iterations (-n) int Maximum iterations for optimizer (0 indicates no limit). 10000
--optimizer (-O) string Optimizer to use for training (‘lbfgs’ or ‘sgd’). 'lbfgs'
--step_size (-s) double Step size for SGD optimizer. 0.01
--test_file (-T) 2-d matrix file Matrix containing test dataset. ''
--tolerance (-e) double Convergence tolerance for optimizer. 1e-10
--training_file (-t) 2-d matrix file A matrix containing the training set (the matrix of predictors, X). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) LogisticRegression<> file Output for trained logistic regression model.
--predictions_file (-P) 1-d index matrix file If test data is specified, this matrix is where the predictions for the test set will be saved.
--probabilities_file (-p) 2-d matrix file If test data is specified, this matrix is where the class probabilities for the test set will be saved.

Detailed documentation

An implementation of L2-regularized logistic regression using either the L-BFGS optimizer or SGD (stochastic gradient descent). This solves the regression problem

y = (1 / 1 + e^-(X * b)).

In this setting, y corresponds to class labels and X corresponds to data.

This program allows loading a logistic regression model (via the --input_model_file (-m) parameter) or training a logistic regression model given training data (specified with the --training_file (-t) parameter), or both those things at once. In addition, this program allows classification on a test dataset (specified with the --test_file (-T) parameter) and the classification results may be saved with the --predictions_file (-P) output parameter. The trained logistic regression model may be saved using the --output_model_file (-M) output parameter.

The training data, if specified, may have class labels as its last dimension. Alternately, the --labels_file (-l) parameter may be used to specify a separate matrix of labels.

When a model is being trained, there are many options. L2 regularization (to prevent overfitting) can be specified with the --lambda (-L) option, and the optimizer used to train the model can be specified with the --optimizer (-O) parameter. Available options are ‘sgd’ (stochastic gradient descent) and ‘lbfgs’ (the L-BFGS optimizer). There are also various parameters for the optimizer; the --max_iterations (-n) parameter specifies the maximum number of allowed iterations, and the --tolerance (-e) parameter specifies the tolerance for convergence. For the SGD optimizer, the --step_size (-s) parameter controls the step size taken at each iteration by the optimizer. The batch size for SGD is controlled with the --batch_size (-b) parameter. If the objective function for your data is oscillating between Inf and 0, the step size is probably too large. There are more parameters for the optimizers, but the C++ interface must be used to access these.

For SGD, an iteration refers to a single point. So to take a single pass over the dataset with SGD, --max_iterations (-n) should be set to the number of points in the dataset.

Optionally, the model can be used to predict the responses for another matrix of data points, if --test_file (-T) is specified. The --test_file (-T) parameter can be specified without the --training_file (-t) parameter, so long as an existing logistic regression model is given with the --input_model_file (-m) parameter. The output predictions from the logistic regression model may be saved with the --predictions_file (-P) parameter.

This implementation of logistic regression does not support the general multi-class case but instead only the two-class case. Any labels must be either 0 or 1. For more classes, see the softmax regression implementation.

Example

As an example, to train a logistic regression model on the data ‘'data.csv'’ with labels ‘'labels.csv'’ with L2 regularization of 0.1, saving the model to ‘'lr_model.bin'’, the following command may be used:

$ mlpack_logistic_regression --training_file data.csv --labels_file labels.csv
  --lambda 0.1 --output_model_file lr_model.bin

Then, to use that model to predict classes for the dataset ‘'test.csv'’, storing the output predictions in ‘'predictions.csv'’, the following command may be used:

$ mlpack_logistic_regression --input_model_file lr_model.bin --test_file
  test.csv --predictions_file predictions.csv

See also

mlpack_lsh

K-Approximate-Nearest-Neighbor Search with LSH

$ mlpack_lsh [--bucket_size 500] [--hash_width 0] [--help] [--info
        <string>] [--input_model_file <string>] [--k 0] [--num_probes 0]
        [--projections 10] [--query_file <string>] [--reference_file <string>]
        [--second_hash_size 99901] [--seed 0] [--tables 30]
        [--true_neighbors_file <string>] [--verbose] [--version]
        [--distances_file <string>] [--neighbors_file <string>]
        [--output_model_file <string>]

An implementation of approximate k-nearest-neighbor search with locality-sensitive hashing (LSH). Given a set of reference points and a set of query points, this will compute the k approximate nearest neighbors of each query point in the reference set; models can be saved for future use. Detailed documentation.

Input options

name type description default
--bucket_size (-B) int The size of a bucket in the second level hash. 500
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--hash_width (-H) double The hash width for the first-level hashing in the LSH preprocessing. By default, the LSH class automatically estimates a hash width for its use. 0
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) LSHSearch<> file Input LSH model. ''
--k (-k) int Number of nearest neighbors to find. 0
--num_probes (-T) int Number of additional probes for multiprobe LSH; if 0, traditional LSH is used. 0
--projections (-K) int The number of hash functions for each table 10
--query_file (-q) 2-d matrix file Matrix containing query points (optional). ''
--reference_file (-r) 2-d matrix file Matrix containing the reference dataset. ''
--second_hash_size (-S) int The size of the second level hash table. 99901
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--tables (-L) int The number of hash tables to be used. 30
--true_neighbors_file (-t) 2-d index matrix file Matrix of true neighbors to compute recall with (the recall is printed when -v is specified). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--distances_file (-d) 2-d matrix file Matrix to output distances into.
--neighbors_file (-n) 2-d index matrix file Matrix to output neighbors into.
--output_model_file (-M) LSHSearch<> file Output for trained LSH model.

Detailed documentation

This program will calculate the k approximate-nearest-neighbors of a set of points using locality-sensitive hashing. You may specify a separate set of reference points and query points, or just a reference set which will be used as both the reference and query set.

Example

For example, the following will return 5 neighbors from the data for each point in 'input.csv' and store the distances in 'distances.csv' and the neighbors in 'neighbors.csv':

$ mlpack_lsh --k 5 --reference_file input.csv --distances_file distances.csv
  --neighbors_file neighbors.csv

The output is organized such that row i and column j in the neighbors output corresponds to the index of the point in the reference set which is the j’th nearest neighbor from the point in the query set with index i. Row j and column i in the distances output file corresponds to the distance between those two points.

Because this is approximate-nearest-neighbors search, results may be different from run to run. Thus, the --seed (-s) parameter can be specified to set the random seed.

This program also has many other parameters to control its functionality; see the parameter-specific documentation for more information.

See also

mlpack_mean_shift

Mean Shift Clustering

$ mlpack_mean_shift [--force_convergence] [--help] [--in_place] [--info
        <string>] --input_file <string> [--labels_only] [--max_iterations 1000]
        [--radius 0] [--verbose] [--version] [--centroid_file <string>]
        [--output_file <string>]

A fast implementation of mean-shift clustering using dual-tree range search. Given a dataset, this uses the mean shift algorithm to produce and return a clustering of the data. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--force_convergence (-f) flag If specified, the mean shift algorithm will continue running regardless of max_iterations until the clusters converge.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--in_place (-P) flag If specified, a column containing the learned cluster assignments will be added to the input dataset file. In this case, –output_file is overridden. (Do not use with Python.)  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Input dataset to perform clustering on. required
--labels_only (-l) flag If specified, only the output labels will be written to the file specified by –output_file.  
--max_iterations (-m) int Maximum number of iterations before mean shift terminates. 1000
--radius (-r) double If the distance between two centroids is less than the given radius, one will be removed. A radius of 0 or less means an estimate will be calculated and used for the radius. 0
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--centroid_file (-C) 2-d matrix file If specified, the centroids of each cluster will be written to the given matrix.
--output_file (-o) 2-d matrix file Matrix to write output labels or labeled data to.

Detailed documentation

This program performs mean shift clustering on the given dataset, storing the learned cluster assignments either as a column of labels in the input dataset or separately.

The input dataset should be specified with the --input_file (-i) parameter, and the radius used for search can be specified with the --radius (-r) parameter. The maximum number of iterations before algorithm termination is controlled with the --max_iterations (-m) parameter.

The output labels may be saved with the --output_file (-o) output parameter and the centroids of each cluster may be saved with the --centroid_file (-C) output parameter.

Example

For example, to run mean shift clustering on the dataset 'data.csv' and store the centroids to 'centroids.csv', the following command may be used:

$ mlpack_mean_shift --input_file data.csv --centroid_file centroids.csv

See also

mlpack_nbc

Parametric Naive Bayes Classifier

$ mlpack_nbc [--help] [--incremental_variance] [--info <string>]
        [--input_model_file <string>] [--labels_file <string>] [--test_file
        <string>] [--training_file <string>] [--verbose] [--version]
        [--output_file <string>] [--output_model_file <string>]
        [--output_probs_file <string>] [--predictions_file <string>]
        [--probabilities_file <string>]

An implementation of the Naive Bayes Classifier, used for classification. Given labeled data, an NBC model can be trained and saved, or, a pre-trained model can be used for classification. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--incremental_variance (-I) flag The variance of each class will be calculated incrementally.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) NBCModel file Input Naive Bayes model. ''
--labels_file (-l) 1-d index matrix file A file containing labels for the training set. ''
--test_file (-T) 2-d matrix file A matrix containing the test set. ''
--training_file (-t) 2-d matrix file A matrix containing the training set. ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 1-d index matrix file The matrix in which the predicted labels for the test set will be written (deprecated).
--output_model_file (-M) NBCModel file File to save trained Naive Bayes model to.
--output_probs_file 2-d matrix file The matrix in which the predicted probability of labels for the test set will be written (deprecated).
--predictions_file (-a) 1-d index matrix file The matrix in which the predicted labels for the test set will be written.
--probabilities_file (-p) 2-d matrix file The matrix in which the predicted probability of labels for the test set will be written.

Detailed documentation

This program trains the Naive Bayes classifier on the given labeled training set, or loads a model from the given model file, and then may use that trained model to classify the points in a given test set.

The training set is specified with the --training_file (-t) parameter. Labels may be either the last row of the training set, or alternately the --labels_file (-l) parameter may be specified to pass a separate matrix of labels.

If training is not desired, a pre-existing model may be loaded with the --input_model_file (-m) parameter.

The --incremental_variance (-I) parameter can be used to force the training to use an incremental algorithm for calculating variance. This is slower, but can help avoid loss of precision in some cases.

If classifying a test set is desired, the test set may be specified with the --test_file (-T) parameter, and the classifications may be saved with the --predictions_file (-a)predictions parameter. If saving the trained model is desired, this may be done with the --output_model_file (-M) output parameter.

Note: the --output_file (-o) and --output_probs_file parameters are deprecated and will be removed in mlpack 4.0.0. Use --predictions_file (-a) and --probabilities_file (-p) instead.

Example

For example, to train a Naive Bayes classifier on the dataset 'data.csv' with labels 'labels.csv' and save the model to 'nbc_model.bin', the following command may be used:

$ mlpack_nbc --training_file data.csv --labels_file labels.csv
  --output_model_file nbc_model.bin

Then, to use 'nbc_model.bin' to predict the classes of the dataset 'test_set.csv' and save the predicted classes to 'predictions.csv', the following command may be used:

$ mlpack_nbc --input_model_file nbc_model.bin --test_file test_set.csv
  --output_file predictions.csv

See also

mlpack_nca

Neighborhood Components Analysis (NCA)

$ mlpack_nca [--armijo_constant 0.0001] [--batch_size 50] [--help]
        [--info <string>] --input_file <string> [--labels_file <string>]
        [--linear_scan] [--max_iterations 500000] [--max_line_search_trials 50]
        [--max_step 1e+20] [--min_step 1e-20] [--normalize] [--num_basis 5]
        [--optimizer 'sgd'] [--seed 0] [--step_size 0.01] [--tolerance 1e-07]
        [--verbose] [--version] [--wolfe 0.9] [--output_file <string>]

An implementation of neighborhood components analysis, a distance learning technique that can be used for preprocessing. Given a labeled dataset, this uses NCA, which seeks to improve the k-nearest-neighbor classification, and returns the learned distance metric. Detailed documentation.

Input options

name type description default
--armijo_constant (-A) double Armijo constant for L-BFGS. 0.0001
--batch_size (-b) int Batch size for mini-batch SGD. 50
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Input dataset to run NCA on. required
--labels_file (-l) 1-d index matrix file Labels for input dataset. ''
--linear_scan (-L) flag Don’t shuffle the order in which data points are visited for SGD or mini-batch SGD.  
--max_iterations (-n) int Maximum number of iterations for SGD or L-BFGS (0 indicates no limit). 500000
--max_line_search_trials (-T) int Maximum number of line search trials for L-BFGS. 50
--max_step (-M) double Maximum step of line search for L-BFGS. 1e+20
--min_step (-m) double Minimum step of line search for L-BFGS. 1e-20
--normalize (-N) flag Use a normalized starting point for optimization. This is useful for when points are far apart, or when SGD is returning NaN.  
--num_basis (-B) int Number of memory points to be stored for L-BFGS. 5
--optimizer (-O) string Optimizer to use; ‘sgd’ or ‘lbfgs’. 'sgd'
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--step_size (-a) double Step size for stochastic gradient descent (alpha). 0.01
--tolerance (-t) double Maximum tolerance for termination of SGD or L-BFGS. 1e-07
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  
--wolfe (-w) double Wolfe condition parameter for L-BFGS. 0.9

Output options

name type description
--output_file (-o) 2-d matrix file Output matrix for learned distance matrix.

Detailed documentation

This program implements Neighborhood Components Analysis, both a linear dimensionality reduction technique and a distance learning technique. The method seeks to improve k-nearest-neighbor classification on a dataset by scaling the dimensions. The method is nonparametric, and does not require a value of k. It works by using stochastic (“soft”) neighbor assignments and using optimization techniques over the gradient of the accuracy of the neighbor assignments.

To work, this algorithm needs labeled data. It can be given as the last row of the input dataset (specified with --input_file (-i)), or alternatively as a separate matrix (specified with --labels_file (-l)).

This implementation of NCA uses stochastic gradient descent, mini-batch stochastic gradient descent, or the L_BFGS optimizer. These optimizers do not guarantee global convergence for a nonconvex objective function (NCA’s objective function is nonconvex), so the final results could depend on the random seed or other optimizer parameters.

Stochastic gradient descent, specified by the value ‘sgd’ for the parameter --optimizer (-O), depends primarily on three parameters: the step size (specified with --step_size (-a)), the batch size (specified with --batch_size (-b)), and the maximum number of iterations (specified with --max_iterations (-n)). In addition, a normalized starting point can be used by specifying the --normalize (-N) parameter, which is necessary if many warnings of the form ‘Denominator of p_i is 0!’ are given. Tuning the step size can be a tedious affair. In general, the step size is too large if the objective is not mostly uniformly decreasing, or if zero-valued denominator warnings are being issued. The step size is too small if the objective is changing very slowly. Setting the termination condition can be done easily once a good step size parameter is found; either increase the maximum iterations to a large number and allow SGD to find a minimum, or set the maximum iterations to 0 (allowing infinite iterations) and set the tolerance (specified by --tolerance (-t)) to define the maximum allowed difference between objectives for SGD to terminate. Be careful—setting the tolerance instead of the maximum iterations can take a very long time and may actually never converge due to the properties of the SGD optimizer. Note that a single iteration of SGD refers to a single point, so to take a single pass over the dataset, set the value of the --max_iterations (-n) parameter equal to the number of points in the dataset.

The L-BFGS optimizer, specified by the value ‘lbfgs’ for the parameter --optimizer (-O), uses a back-tracking line search algorithm to minimize a function. The following parameters are used by L-BFGS: --num_basis (-B) (specifies the number of memory points used by L-BFGS), --max_iterations (-n), --armijo_constant (-A), --wolfe (-w), --tolerance (-t) (the optimization is terminated when the gradient norm is below this value), --max_line_search_trials (-T), --min_step (-m), and --max_step (-M) (which both refer to the line search routine). For more details on the L-BFGS optimizer, consult either the mlpack L-BFGS documentation (in lbfgs.hpp) or the vast set of published literature on L-BFGS.

By default, the SGD optimizer is used.

See also

mlpack_knn

$ mlpack_knn [--algorithm 'dual_tree'] [--epsilon 0] [--help] [--info
        <string>] [--input_model_file <string>] [--k 0] [--leaf_size 20]
        [--query_file <string>] [--random_basis] [--reference_file <string>]
        [--rho 0.7] [--seed 0] [--tau 0] [--tree_type 'kd']
        [--true_distances_file <string>] [--true_neighbors_file <string>]
        [--verbose] [--version] [--distances_file <string>] [--neighbors_file
        <string>] [--output_model_file <string>]

An implementation of k-nearest-neighbor search using single-tree and dual-tree algorithms. Given a set of reference points and query points, this can find the k nearest neighbors in the reference set of each query point using trees; trees that are built can be saved for future use. Detailed documentation.

Input options

name type description default
--algorithm (-a) string Type of neighbor search: ‘naive’, ‘single_tree’, ‘dual_tree’, ‘greedy’. 'dual_tree'
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--epsilon (-e) double If specified, will do approximate nearest neighbor search with given relative error. 0
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) KNNModel file Pre-trained kNN model. ''
--k (-k) int Number of nearest neighbors to find. 0
--leaf_size (-l) int Leaf size for tree building (used for kd-trees, vp trees, random projection trees, UB trees, R trees, R* trees, X trees, Hilbert R trees, R+ trees, R++ trees, spill trees, and octrees). 20
--query_file (-q) 2-d matrix file Matrix containing query points (optional). ''
--random_basis (-R) flag Before tree-building, project the data onto a random orthogonal basis.  
--reference_file (-r) 2-d matrix file Matrix containing the reference dataset. ''
--rho (-b) double Balance threshold (only valid for spill trees). 0.7
--seed (-s) int Random seed (if 0, std::time(NULL) is used). 0
--tau (-u) double Overlapping size (only valid for spill trees). 0
--tree_type (-t) string Type of tree to use: ‘kd’, ‘vp’, ‘rp’, ‘max-rp’, ‘ub’, ‘cover’, ‘r’, ‘r-star’, ‘x’, ‘ball’, ‘hilbert-r’, ‘r-plus’, ‘r-plus-plus’, ‘spill’, ‘oct’. 'kd'
--true_distances_file (-D) 2-d matrix file Matrix of true distances to compute the effective error (average relative error) (it is printed when -v is specified). ''
--true_neighbors_file (-T) 2-d index matrix file Matrix of true neighbors to compute the recall (it is printed when -v is specified). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--distances_file (-d) 2-d matrix file Matrix to output distances into.
--neighbors_file (-n) 2-d index matrix file Matrix to output neighbors into.
--output_model_file (-M) KNNModel file If specified, the kNN model will be output here.

Detailed documentation

This program will calculate the k-nearest-neighbors of a set of points using kd-trees or cover trees (cover tree support is experimental and may be slow). You may specify a separate set of reference points and query points, or just a reference set which will be used as both the reference and query set.

Example

For example, the following command will calculate the 5 nearest neighbors of each point in 'input.csv' and store the distances in 'distances.csv' and the neighbors in 'neighbors.csv':

$ mlpack_knn --k 5 --reference_file input.csv --neighbors_file neighbors.csv
  --distances_file distances.csv

The output is organized such that row i and column j in the neighbors output matrix corresponds to the index of the point in the reference set which is the j’th nearest neighbor from the point in the query set with index i. Row j and column i in the distances output matrix corresponds to the distance between those two points.

See also

mlpack_kfn

$ mlpack_kfn [--algorithm 'dual_tree'] [--epsilon 0] [--help] [--info
        <string>] [--input_model_file <string>] [--k 0] [--leaf_size 20]
        [--percentage 1] [--query_file <string>] [--random_basis]
        [--reference_file <string>] [--seed 0] [--tree_type 'kd']
        [--true_distances_file <string>] [--true_neighbors_file <string>]
        [--verbose] [--version] [--distances_file <string>] [--neighbors_file
        <string>] [--output_model_file <string>]

An implementation of k-furthest-neighbor search using single-tree and dual-tree algorithms. Given a set of reference points and query points, this can find the k furthest neighbors in the reference set of each query point using trees; trees that are built can be saved for future use. Detailed documentation.

Input options

name type description default
--algorithm (-a) string Type of neighbor search: ‘naive’, ‘single_tree’, ‘dual_tree’, ‘greedy’. 'dual_tree'
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--epsilon (-e) double If specified, will do approximate furthest neighbor search with given relative error. Must be in the range [0,1). 0
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) KFNModel file Pre-trained kFN model. ''
--k (-k) int Number of furthest neighbors to find. 0
--leaf_size (-l) int Leaf size for tree building (used for kd-trees, vp trees, random projection trees, UB trees, R trees, R* trees, X trees, Hilbert R trees, R+ trees, R++ trees, and octrees). 20
--percentage (-p) double If specified, will do approximate furthest neighbor search. Must be in the range (0,1] (decimal form). Resultant neighbors will be at least (p*100) % of the distance as the true furthest neighbor. 1
--query_file (-q) 2-d matrix file Matrix containing query points (optional). ''
--random_basis (-R) flag Before tree-building, project the data onto a random orthogonal basis.  
--reference_file (-r) 2-d matrix file Matrix containing the reference dataset. ''
--seed (-s) int Random seed (if 0, std::time(NULL) is used). 0
--tree_type (-t) string Type of tree to use: ‘kd’, ‘vp’, ‘rp’, ‘max-rp’, ‘ub’, ‘cover’, ‘r’, ‘r-star’, ‘x’, ‘ball’, ‘hilbert-r’, ‘r-plus’, ‘r-plus-plus’, ‘oct’. 'kd'
--true_distances_file (-D) 2-d matrix file Matrix of true distances to compute the effective error (average relative error) (it is printed when -v is specified). ''
--true_neighbors_file (-T) 2-d index matrix file Matrix of true neighbors to compute the recall (it is printed when -v is specified). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--distances_file (-d) 2-d matrix file Matrix to output distances into.
--neighbors_file (-n) 2-d index matrix file Matrix to output neighbors into.
--output_model_file (-M) KFNModel file If specified, the kFN model will be output here.

Detailed documentation

This program will calculate the k-furthest-neighbors of a set of points. You may specify a separate set of reference points and query points, or just a reference set which will be used as both the reference and query set.

Example

For example, the following will calculate the 5 furthest neighbors of eachpoint in 'input.csv' and store the distances in 'distances.csv' and the neighbors in 'neighbors.csv':

$ mlpack_kfn --k 5 --reference_file input.csv --distances_file distances.csv
  --neighbors_file neighbors.csv

The output files are organized such that row i and column j in the neighbors output matrix corresponds to the index of the point in the reference set which is the j’th furthest neighbor from the point in the query set with index i. Row i and column j in the distances output file corresponds to the distance between those two points.

See also

mlpack_nmf

Non-negative Matrix Factorization

$ mlpack_nmf [--help] [--info <string>] [--initial_h_file <string>]
        [--initial_w_file <string>] --input_file <string> [--max_iterations
        10000] [--min_residue 1e-05] --rank 0 [--seed 0] [--update_rules
        'multdist'] [--verbose] [--version] [--h_file <string>] [--w_file
        <string>]

An implementation of non-negative matrix factorization. This can be used to decompose an input dataset into two low-rank non-negative components. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--initial_h_file (-q) 2-d matrix file Initial H matrix. ''
--initial_w_file (-p) 2-d matrix file Initial W matrix. ''
--input_file (-i) 2-d matrix file Input dataset to perform NMF on. required
--max_iterations (-m) int Number of iterations before NMF terminates (0 runs until convergence. 10000
--min_residue (-e) double The minimum root mean square residue allowed for each iteration, below which the program terminates. 1e-05
--rank (-r) int Rank of the factorization. required
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--update_rules (-u) string Update rules for each iteration; ( multdist | multdiv | als ). 'multdist'
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--h_file (-H) 2-d matrix file Matrix to save the calculated H to.
--w_file (-W) 2-d matrix file Matrix to save the calculated W to.

Detailed documentation

This program performs non-negative matrix factorization on the given dataset, storing the resulting decomposed matrices in the specified files. For an input dataset V, NMF decomposes V into two matrices W and H such that

V = W * H

where all elements in W and H are non-negative. If V is of size (n x m), then W will be of size (n x r) and H will be of size (r x m), where r is the rank of the factorization (specified by the --rank (-r) parameter).

Optionally, the desired update rules for each NMF iteration can be chosen from the following list:

  • multdist: multiplicative distance-based update rules (Lee and Seung 1999)
  • multdiv: multiplicative divergence-based update rules (Lee and Seung 1999)
  • als: alternating least squares update rules (Paatero and Tapper 1994)

The maximum number of iterations is specified with --max_iterations (-m), and the minimum residue required for algorithm termination is specified with the --min_residue (-e) parameter.

Example

For example, to run NMF on the input matrix 'V.csv' using the ‘multdist’ update rules with a rank-10 decomposition and storing the decomposed matrices into 'W.csv' and 'H.csv', the following command could be used:

$ mlpack_nmf --input_file V.csv --w_file W.csv --h_file H.csv --rank 10
  --update_rules multdist

See also

mlpack_pca

Principal Components Analysis

$ mlpack_pca [--decomposition_method 'exact'] [--help] [--info <string>]
        --input_file <string> [--new_dimensionality 0] [--scale]
        [--var_to_retain 0] [--verbose] [--version] [--output_file <string>]

An implementation of several strategies for principal components analysis (PCA), a common preprocessing step. Given a dataset and a desired new dimensionality, this can reduce the dimensionality of the data using the linear transformation determined by PCA. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--decomposition_method (-c) string Method used for the principal components analysis: ‘exact’, ‘randomized’, ‘randomized-block-krylov’, ‘quic’. 'exact'
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Input dataset to perform PCA on. required
--new_dimensionality (-d) int Desired dimensionality of output dataset. If 0, no dimensionality reduction is performed. 0
--scale (-s) flag If set, the data will be scaled before running PCA, such that the variance of each feature is 1.  
--var_to_retain (-r) double Amount of variance to retain; should be between 0 and 1. If 1, all variance is retained. Overrides -d. 0
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d matrix file Matrix to save modified dataset to.

Detailed documentation

This program performs principal components analysis on the given dataset using the exact, randomized, randomized block Krylov, or QUIC SVD method. It will transform the data onto its principal components, optionally performing dimensionality reduction by ignoring the principal components with the smallest eigenvalues.

Use the --input_file (-i) parameter to specify the dataset to perform PCA on. A desired new dimensionality can be specified with the --new_dimensionality (-d) parameter, or the desired variance to retain can be specified with the --var_to_retain (-r) parameter. If desired, the dataset can be scaled before running PCA with the --scale (-s) parameter.

Multiple different decomposition techniques can be used. The method to use can be specified with the --decomposition_method (-c) parameter, and it may take the values ‘exact’, ‘randomized’, or ‘quic’.

Example

For example, to reduce the dimensionality of the matrix 'data.csv' to 5 dimensions using randomized SVD for the decomposition, storing the output matrix to 'data_mod.csv', the following command can be used:

$ mlpack_pca --input_file data.csv --new_dimensionality 5
  --decomposition_method randomized --output_file data_mod.csv

See also

mlpack_perceptron

Perceptron

$ mlpack_perceptron [--help] [--info <string>] [--input_model_file
        <string>] [--labels_file <string>] [--max_iterations 1000] [--test_file
        <string>] [--training_file <string>] [--verbose] [--version]
        [--output_file <string>] [--output_model_file <string>]
        [--predictions_file <string>]

An implementation of a perceptron—a single level neural network–=for classification. Given labeled data, a perceptron can be trained and saved for future use; or, a pre-trained perceptron can be used for classification on new points. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) PerceptronModel file Input perceptron model. ''
--labels_file (-l) 1-d index matrix file A matrix containing labels for the training set. ''
--max_iterations (-n) int The maximum number of iterations the perceptron is to be run 1000
--test_file (-T) 2-d matrix file A matrix containing the test set. ''
--training_file (-t) 2-d matrix file A matrix containing the training set. ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 1-d index matrix file The matrix in which the predicted labels for the test set will be written.
--output_model_file (-M) PerceptronModel file Output for trained perceptron model.
--predictions_file (-P) 1-d index matrix file The matrix in which the predicted labels for the test set will be written.

Detailed documentation

This program implements a perceptron, which is a single level neural network. The perceptron makes its predictions based on a linear predictor function combining a set of weights with the feature vector. The perceptron learning rule is able to converge, given enough iterations (specified using the --max_iterations (-n) parameter), if the data supplied is linearly separable. The perceptron is parameterized by a matrix of weight vectors that denote the numerical weights of the neural network.

This program allows loading a perceptron from a model (via the --input_model_file (-m) parameter) or training a perceptron given training data (via the --training_file (-t) parameter), or both those things at once. In addition, this program allows classification on a test dataset (via the --test_file (-T) parameter) and the classification results on the test set may be saved with the --predictions_file (-P) output parameter. The perceptron model may be saved with the --output_model_file (-M) output parameter.

Note: the following parameter is deprecated and will be removed in mlpack 4.0.0: --output_file (-o). Use --predictions_file (-P) instead of --output_file (-o).

Example

The training data given with the --training_file (-t) option may have class labels as its last dimension (so, if the training data is in CSV format, labels should be the last column). Alternately, the --labels_file (-l) parameter may be used to specify a separate matrix of labels.

All these options make it easy to train a perceptron, and then re-use that perceptron for later classification. The invocation below trains a perceptron on 'training_data.csv' with labels 'training_labels.csv', and saves the model to 'perceptron_model.bin'.

$ mlpack_perceptron --training_file training_data.csv --labels_file
  training_labels.csv --output_model_file perceptron_model.bin

Then, this model can be re-used for classification on the test data 'test_data.csv'. The example below does precisely that, saving the predicted classes to 'predictions.csv'.

$ mlpack_perceptron --input_model_file perceptron_model.bin --test_file
  test_data.csv --predictions_file predictions.csv

Note that all of the options may be specified at once: predictions may be calculated right after training a model, and model training can occur even if an existing perceptron model is passed with the --input_model_file (-m) parameter. However, note that the number of classes and the dimensionality of all data must match. So you cannot pass a perceptron model trained on 2 classes and then re-train with a 4-class dataset. Similarly, attempting classification on a 3-dimensional dataset with a perceptron that has been trained on 8 dimensions will cause an error.

See also

mlpack_preprocess_split

Split Data

$ mlpack_preprocess_split [--help] [--info <string>] --input_file
        <string> [--input_labels_file <string>] [--no_shuffle] [--seed 0]
        [--stratify_data] [--test_ratio 0.2] [--verbose] [--version]
        [--test_file <string>] [--test_labels_file <string>] [--training_file
        <string>] [--training_labels_file <string>]

A utility to split data into a training and testing dataset. This can also split labels according to the same split. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Matrix containing data. required
--input_labels_file (-I) 2-d index matrix file Matrix containing labels. ''
--no_shuffle (-S) flag Avoid shuffling the data before splitting.  
--seed (-s) int Random seed (0 for std::time(NULL)). 0
--stratify_data (-z) flag Stratify the data according to labels  
--test_ratio (-r) double Ratio of test set; if not set,the ratio defaults to 0.2 0.2
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--test_file (-T) 2-d matrix file Matrix to save test data to.
--test_labels_file (-L) 2-d index matrix file Matrix to save test labels to.
--training_file (-t) 2-d matrix file Matrix to save training data to.
--training_labels_file (-l) 2-d index matrix file Matrix to save train labels to.

Detailed documentation

This utility takes a dataset and optionally labels and splits them into a training set and a test set. Before the split, the points in the dataset are randomly reordered. The percentage of the dataset to be used as the test set can be specified with the --test_ratio (-r) parameter; the default is 0.2 (20%).

The output training and test matrices may be saved with the --training_file (-t) and --test_file (-T) output parameters.

Optionally, labels can also be split along with the data by specifying the --input_labels_file (-I) parameter. Splitting labels works the same way as splitting the data. The output training and test labels may be saved with the --training_labels_file (-l) and --test_labels_file (-L) output parameters, respectively.

Example

So, a simple example where we want to split the dataset 'X.csv' into 'X_train.csv' and 'X_test.csv' with 60% of the data in the training set and 40% of the dataset in the test set, we could run

$ mlpack_preprocess_split --input_file X.csv --training_file X_train.csv
  --test_file X_test.csv --test_ratio 0.4

Also by default the dataset is shuffled and split; you can provide the --no_shuffle (-S) option to avoid shuffling the data; an example to avoid shuffling of data is:

$ mlpack_preprocess_split --input_file X.csv --training_file X_train.csv
  --test_file X_test.csv --test_ratio 0.4 --no_shuffle

If we had a dataset 'X.csv' and associated labels 'y.csv', and we wanted to split these into 'X_train.csv', 'y_train.csv', 'X_test.csv', and 'y_test.csv', with 30% of the data in the test set, we could run

$ mlpack_preprocess_split --input_file X.csv --input_labels_file y.csv
  --test_ratio 0.3 --training_file X_train.csv --training_labels_file
  y_train.csv --test_file X_test.csv --test_labels_file y_test.csv

To maintain the ratio of each class in the train and test sets, the--stratify_data (-z) option can be used.

$ mlpack_preprocess_split --input_file X.csv --training_file X_train.csv
  --test_file X_test.csv --test_ratio 0.4 --stratify_data

See also

mlpack_preprocess_binarize

Binarize Data

$ mlpack_preprocess_binarize [--dimension 0] [--help] [--info <string>]
        --input_file <string> [--threshold 0] [--verbose] [--version]
        [--output_file <string>]

A utility to binarize a dataset. Given a dataset, this utility converts each value in the desired dimension(s) to 0 or 1; this can be a useful preprocessing step. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--dimension (-d) int Dimension to apply the binarization. If not set, the program will binarize every dimension by default. 0
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Input data matrix. required
--threshold (-t) double Threshold to be applied for binarization. If not set, the threshold defaults to 0.0. 0
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d matrix file Matrix in which to save the output.

Detailed documentation

This utility takes a dataset and binarizes the variables into either 0 or 1 given threshold. User can apply binarization on a dimension or the whole dataset. The dimension to apply binarization to can be specified using the --dimension (-d) parameter; if left unspecified, every dimension will be binarized. The threshold for binarization can also be specified with the --threshold (-t) parameter; the default threshold is 0.0.

The binarized matrix may be saved with the --output_file (-o) output parameter.

Example

For example, if we want to set all variables greater than 5 in the dataset 'X.csv' to 1 and variables less than or equal to 5.0 to 0, and save the result to 'Y.csv', we could run

$ mlpack_preprocess_binarize --input_file X.csv --threshold 5 --output_file
  Y.csv

But if we want to apply this to only the first (0th) dimension of 'X.csv', we could instead run

$ mlpack_preprocess_binarize --input_file X.csv --threshold 5 --dimension 0
  --output_file Y.csv

See also

mlpack_preprocess_describe

Descriptive Statistics

$ mlpack_preprocess_describe [--dimension 0] [--help] [--info <string>]
        --input_file <string> [--population] [--precision 4] [--row_major]
        [--verbose] [--version] [--width 8]

A utility for printing descriptive statistics about a dataset. This prints a number of details about a dataset in a tabular format. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--dimension (-d) int Dimension of the data. Use this to specify a dimension 0
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Matrix containing data, required
--population (-P) flag If specified, the program will calculate statistics assuming the dataset is the population. By default, the program will assume the dataset as a sample.  
--precision (-p) int Precision of the output statistics. 4
--row_major (-r) flag If specified, the program will calculate statistics across rows, not across columns. (Remember that in mlpack, a column represents a point, so this option is generally not necessary.)  
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  
--width (-w) int Width of the output table. 8

Detailed documentation

This utility takes a dataset and prints out the descriptive statistics of the data. Descriptive statistics is the discipline of quantitatively describing the main features of a collection of information, or the quantitative description itself. The program does not modify the original file, but instead prints out the statistics to the console. The printed result will look like a table.

Optionally, width and precision of the output can be adjusted by a user using the --width (-w) and --precision (-p) parameters. A user can also select a specific dimension to analyze if there are too many dimensions. The --population (-P) parameter can be specified when the dataset should be considered as a population. Otherwise, the dataset will be considered as a sample.

Example

So, a simple example where we want to print out statistical facts about the dataset 'X.csv' using the default settings, we could run

$ mlpack_preprocess_describe --input_file X.csv --verbose

If we want to customize the width to 10 and precision to 5 and consider the dataset as a population, we could run

$ mlpack_preprocess_describe --input_file X.csv --width 10 --precision 5
  --verbose

See also

mlpack_preprocess_scale

Scale Data

$ mlpack_preprocess_scale [--epsilon 1e-06] [--help] [--info <string>]
        --input_file <string> [--input_model_file <string>] [--inverse_scaling]
        [--max_value 1] [--min_value 0] [--scaler_method 'standard_scaler']
        [--seed 0] [--verbose] [--version] [--output_file <string>]
        [--output_model_file <string>]

A utility to perform feature scaling on datasets using one of sixtechniques. Both scaling and inverse scaling are supported, andscalers can be saved and then applied to other datasets. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--epsilon (-r) double regularization Parameter for pcawhitening, or zcawhitening, should be between -1 to 1. 1e-06
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Matrix containing data. required
--input_model_file (-m) ScalingModel file Input Scaling model. ''
--inverse_scaling (-f) flag Inverse Scaling to get original dataset  
--max_value (-e) int Ending value of range for min_max_scaler. 1
--min_value (-b) int Starting value of range for min_max_scaler. 0
--scaler_method (-a) string method to use for scaling, the default is standard_scaler. 'standard_scaler'
--seed (-s) int Random seed (0 for std::time(NULL)). 0
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d matrix file Matrix to save scaled data to.
--output_model_file (-M) ScalingModel file Output scaling model.

Detailed documentation

This utility takes a dataset and performs feature scaling using one of the six scaler methods namely: ‘max_abs_scaler’, ‘mean_normalization’, ‘min_max_scaler’ ,’standard_scaler’, ‘pca_whitening’ and ‘zca_whitening’. The function takes a matrix as --input_file (-i) and a scaling method type which you can specify using --scaler_method (-a) parameter; the default is standard scaler, and outputs a matrix with scaled feature.

The output scaled feature matrix may be saved with the --output_file (-o) output parameters.

The model to scale features can be saved using --output_model_file (-M) and later can be loaded back using--input_model_file (-m).

Example

So, a simple example where we want to scale the dataset 'X.csv' into 'X_scaled.csv' with standard_scaler as scaler_method, we could run

$ mlpack_preprocess_scale --input_file X.csv --output_file X_scaled.csv
  --scaler_method standard_scaler

A simple example where we want to whiten the dataset 'X.csv' into 'X_whitened.csv' with PCA as whitening_method and use 0.01 as regularization parameter, we could run

$ mlpack_preprocess_scale --input_file X.csv --output_file X_scaled.csv
  --scaler_method pca_whitening --epsilon 0.01

You can also retransform the scaled dataset back using--inverse_scaling (-f). An example to rescale : 'X_scaled.csv' into 'X.csv'using the saved model --input_model_file (-m) is:

$ mlpack_preprocess_scale --input_file X_scaled.csv --output_file X.csv
  --inverse_scaling --input_model_file saved.bin

Another simple example where we want to scale the dataset 'X.csv' into 'X_scaled.csv' with min_max_scaler as scaler method, where scaling range is 1 to 3 instead of default 0 to 1. We could run

$ mlpack_preprocess_scale --input_file X.csv --output_file X_scaled.csv
  --scaler_method min_max_scaler --min_value 1 --max_value 3

See also

mlpack_preprocess_one_hot_encoding

One Hot Encoding

$ mlpack_preprocess_one_hot_encoding [--dimensions []] [--help] [--info
        <string>] --input_file <string> [--verbose] [--version] [--output_file
        <string>]

A utility to do one-hot encoding on features of dataset. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--dimensions (-d) int vector Index of dimensions that need to be one-hot encoded (if unspecified, all categorical dimensions are one-hot encoded). []
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d categorical matrix file Matrix containing data. required
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) 2-d matrix file Matrix to save one-hot encoded features data to.

Detailed documentation

This utility takes a dataset and a vector of indices and does one-hot encoding of the respective features at those indices. Indices represent the IDs of the dimensions to be one-hot encoded.

If no dimensions are specified with --dimensions (-d), then all categorical-type dimensions will be one-hot encoded. Otherwise, only the dimensions given in --dimensions (-d) will be one-hot encoded.

The output matrix with encoded features may be saved with the --output_file (-o) parameters.

Example

So, a simple example where we want to encode 1st and 3rd feature from dataset 'X.csv' into 'X_output.csv' would be

$ mlpack_preprocess_one_hot_encoding --input_file X.arff --output_file
  X_ouput.csv --dimensions 1 --dimensions 3

See also

mlpack_radical

RADICAL

$ mlpack_radical [--angles 150] [--help] [--info <string>] --input_file
        <string> [--noise_std_dev 0.175] [--objective] [--replicates 30] [--seed
        0] [--sweeps 0] [--verbose] [--version] [--output_ic_file <string>]
        [--output_unmixing_file <string>]

An implementation of RADICAL, a method for independent component analysis (ICA). Given a dataset, this can decompose the dataset into an unmixing matrix and an independent component matrix; this can be useful for preprocessing. Detailed documentation.

Input options

name type description default
--angles (-a) int Number of angles to consider in brute-force search during Radical2D. 150
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) 2-d matrix file Input dataset for ICA. required
--noise_std_dev (-n) double Standard deviation of Gaussian noise. 0.175
--objective (-O) flag If set, an estimate of the final objective function is printed.  
--replicates (-r) int Number of Gaussian-perturbed replicates to use (per point) in Radical2D. 30
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--sweeps (-S) int Number of sweeps; each sweep calls Radical2D once for each pair of dimensions. 0
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_ic_file (-o) 2-d matrix file Matrix to save independent components to.
--output_unmixing_file (-u) 2-d matrix file Matrix to save unmixing matrix to.

Detailed documentation

An implementation of RADICAL, a method for independent component analysis (ICA). Assuming that we have an input matrix X, the goal is to find a square unmixing matrix W such that Y = W * X and the dimensions of Y are independent components. If the algorithm is running particularly slowly, try reducing the number of replicates.

The input matrix to perform ICA on should be specified with the --input_file (-i) parameter. The output matrix Y may be saved with the --output_ic_file (-o) output parameter, and the output unmixing matrix W may be saved with the --output_unmixing_file (-u) output parameter.

Example

For example, to perform ICA on the matrix 'X.csv' with 40 replicates, saving the independent components to 'ic.csv', the following command may be used:

$ mlpack_radical --input_file X.csv --replicates 40 --output_ic_file ic.csv

See also

mlpack_random_forest

Random forests

$ mlpack_random_forest [--help] [--info <string>] [--input_model_file
        <string>] [--labels_file <string>] [--maximum_depth 0]
        [--minimum_gain_split 0] [--minimum_leaf_size 1] [--num_trees 10]
        [--print_training_accuracy] [--seed 0] [--subspace_dim 0] [--test_file
        <string>] [--test_labels_file <string>] [--training_file <string>]
        [--verbose] [--version] [--warm_start] [--output_model_file <string>]
        [--predictions_file <string>] [--probabilities_file <string>]

An implementation of the standard random forest algorithm by Leo Breiman for classification. Given labeled data, a random forest can be trained and saved for future use; or, a pre-trained random forest can be used for classification. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) RandomForestModel file Pre-trained random forest to use for classification. ''
--labels_file (-l) 1-d index matrix file Labels for training dataset. ''
--maximum_depth (-D) int Maximum depth of the tree (0 means no limit). 0
--minimum_gain_split (-g) double Minimum gain needed to make a split when building a tree. 0
--minimum_leaf_size (-n) int Minimum number of points in each leaf node. 1
--num_trees (-N) int Number of trees in the random forest. 10
--print_training_accuracy (-a) flag If set, then the accuracy of the model on the training set will be predicted (verbose must also be specified).  
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--subspace_dim (-d) int Dimensionality of random subspace to use for each split. ‘0’ will autoselect the square root of data dimensionality. 0
--test_file (-T) 2-d matrix file Test dataset to produce predictions for. ''
--test_labels_file (-L) 1-d index matrix file Test dataset labels, if accuracy calculation is desired. ''
--training_file (-t) 2-d matrix file Training dataset. ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  
--warm_start (-w) flag If true and passed along with training and input_model then trains more trees on top of existing model.  

Output options

name type description
--output_model_file (-M) RandomForestModel file Model to save trained random forest to.
--predictions_file (-p) 1-d index matrix file Predicted classes for each point in the test set.
--probabilities_file (-P) 2-d matrix file Predicted class probabilities for each point in the test set.

Detailed documentation

This program is an implementation of the standard random forest classification algorithm by Leo Breiman. A random forest can be trained and saved for later use, or a random forest may be loaded and predictions or class probabilities for points may be generated.

The training set and associated labels are specified with the --training_file (-t) and --labels_file (-l) parameters, respectively. The labels should be in the range [0, num_classes - 1]. Optionally, if --labels_file (-l) is not specified, the labels are assumed to be the last dimension of the training dataset.

When a model is trained, the --output_model_file (-M) output parameter may be used to save the trained model. A model may be loaded for predictions with the --input_model_file (-m)parameter. The --input_model_file (-m) parameter may not be specified when the --training_file (-t) parameter is specified. The --minimum_leaf_size (-n) parameter specifies the minimum number of training points that must fall into each leaf for it to be split. The --num_trees (-N) controls the number of trees in the random forest. The --minimum_gain_split (-g) parameter controls the minimum required gain for a decision tree node to split. Larger values will force higher-confidence splits. The --maximum_depth (-D) parameter specifies the maximum depth of the tree. The --subspace_dim (-d) parameter is used to control the number of random dimensions chosen for an individual node’s split. If --print_training_accuracy (-a) is specified, the calculated accuracy on the training set will be printed.

Test data may be specified with the --test_file (-T) parameter, and if performance measures are desired for that test set, labels for the test points may be specified with the --test_labels_file (-L) parameter. Predictions for each test point may be saved via the --predictions_file (-p)output parameter. Class probabilities for each prediction may be saved with the --probabilities_file (-P) output parameter.

Example

For example, to train a random forest with a minimum leaf size of 20 using 10 trees on the dataset contained in 'data.csv'with labels 'labels.csv', saving the output random forest to 'rf_model.bin' and printing the training error, one could call

$ mlpack_random_forest --training_file data.csv --labels_file labels.csv
  --minimum_leaf_size 20 --num_trees 10 --output_model_file rf_model.bin
  --print_training_accuracy

Then, to use that model to classify points in 'test_set.csv' and print the test error given the labels 'test_labels.csv' using that model, while saving the predictions for each point to 'predictions.csv', one could call

$ mlpack_random_forest --input_model_file rf_model.bin --test_file
  test_set.csv --test_labels_file test_labels.csv --predictions_file
  predictions.csv

See also

mlpack_krann

K-Rank-Approximate-Nearest-Neighbors (kRANN)

$ mlpack_krann [--alpha 0.95] [--first_leaf_exact] [--help] [--info
        <string>] [--input_model_file <string>] [--k 0] [--leaf_size 20]
        [--naive] [--query_file <string>] [--random_basis] [--reference_file
        <string>] [--sample_at_leaves] [--seed 0] [--single_mode]
        [--single_sample_limit 20] [--tau 5] [--tree_type 'kd'] [--verbose]
        [--version] [--distances_file <string>] [--neighbors_file <string>]
        [--output_model_file <string>]

An implementation of rank-approximate k-nearest-neighbor search (kRANN) using single-tree and dual-tree algorithms. Given a set of reference points and query points, this can find the k nearest neighbors in the reference set of each query point using trees; trees that are built can be saved for future use. Detailed documentation.

Input options

name type description default
--alpha (-a) double The desired success probability. 0.95
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--first_leaf_exact (-X) flag The flag to trigger sampling only after exactly exploring the first leaf.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) RAModel file Pre-trained kNN model. ''
--k (-k) int Number of nearest neighbors to find. 0
--leaf_size (-l) int Leaf size for tree building (used for kd-trees, UB trees, R trees, R* trees, X trees, Hilbert R trees, R+ trees, R++ trees, and octrees). 20
--naive (-N) flag If true, sampling will be done without using a tree.  
--query_file (-q) 2-d matrix file Matrix containing query points (optional). ''
--random_basis (-R) flag Before tree-building, project the data onto a random orthogonal basis.  
--reference_file (-r) 2-d matrix file Matrix containing the reference dataset. ''
--sample_at_leaves (-L) flag The flag to trigger sampling at leaves.  
--seed (-s) int Random seed (if 0, std::time(NULL) is used). 0
--single_mode (-S) flag If true, single-tree search is used (as opposed to dual-tree search.  
--single_sample_limit (-z) int The limit on the maximum number of samples (and hence the largest node you can approximate). 20
--tau (-T) double The allowed rank-error in terms of the percentile of the data. 5
--tree_type (-t) string Type of tree to use: ‘kd’, ‘ub’, ‘cover’, ‘r’, ‘x’, ‘r-star’, ‘hilbert-r’, ‘r-plus’, ‘r-plus-plus’, ‘oct’. 'kd'
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--distances_file (-d) 2-d matrix file Matrix to output distances into.
--neighbors_file (-n) 2-d index matrix file Matrix to output neighbors into.
--output_model_file (-M) RAModel file If specified, the kNN model will be output here.

Detailed documentation

This program will calculate the k rank-approximate-nearest-neighbors of a set of points. You may specify a separate set of reference points and query points, or just a reference set which will be used as both the reference and query set. You must specify the rank approximation (in %) (and optionally the success probability).

Example

For example, the following will return 5 neighbors from the top 0.1% of the data (with probability 0.95) for each point in 'input.csv' and store the distances in 'distances.csv' and the neighbors in 'neighbors.csv.csv':

$ mlpack_krann --reference_file input.csv --k 5 --distances_file distances.csv
  --neighbors_file neighbors.csv --tau 0.1

Note that tau must be set such that the number of points in the corresponding percentile of the data is greater than k. Thus, if we choose tau = 0.1 with a dataset of 1000 points and k = 5, then we are attempting to choose 5 nearest neighbors out of the closest 1 point – this is invalid and the program will terminate with an error message.

The output matrices are organized such that row i and column j in the neighbors output file corresponds to the index of the point in the reference set which is the i’th nearest neighbor from the point in the query set with index j. Row i and column j in the distances output file corresponds to the distance between those two points.

See also

mlpack_softmax_regression

Softmax Regression

$ mlpack_softmax_regression [--help] [--info <string>]
        [--input_model_file <string>] [--labels_file <string>] [--lambda 0.0001]
        [--max_iterations 400] [--no_intercept] [--number_of_classes 0]
        [--test_file <string>] [--test_labels_file <string>] [--training_file
        <string>] [--verbose] [--version] [--output_model_file <string>]
        [--predictions_file <string>] [--probabilities_file <string>]

An implementation of softmax regression for classification, which is a multiclass generalization of logistic regression. Given labeled data, a softmax regression model can be trained and saved for future use, or, a pre-trained softmax regression model can be used for classification of new points. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) SoftmaxRegression file File containing existing model (parameters). ''
--labels_file (-l) 1-d index matrix file A matrix containing labels (0 or 1) for the points in the training set (y). The labels must order as a row. ''
--lambda (-r) double L2-regularization constant 0.0001
--max_iterations (-n) int Maximum number of iterations before termination. 400
--no_intercept (-N) flag Do not add the intercept term to the model.  
--number_of_classes (-c) int Number of classes for classification; if unspecified (or 0), the number of classes found in the labels will be used. 0
--test_file (-T) 2-d matrix file Matrix containing test dataset. ''
--test_labels_file (-L) 1-d index matrix file Matrix containing test labels. ''
--training_file (-t) 2-d matrix file A matrix containing the training set (the matrix of predictors, X). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) SoftmaxRegression file File to save trained softmax regression model to.
--predictions_file (-p) 1-d index matrix file Matrix to save predictions for test dataset into.
--probabilities_file (-P) 2-d matrix file Matrix to save class probabilities for test dataset into.

Detailed documentation

This program performs softmax regression, a generalization of logistic regression to the multiclass case, and has support for L2 regularization. The program is able to train a model, load an existing model, and give predictions (and optionally their accuracy) for test data.

Training a softmax regression model is done by giving a file of training points with the --training_file (-t) parameter and their corresponding labels with the --labels_file (-l) parameter. The number of classes can be manually specified with the --number_of_classes (-c) parameter, and the maximum number of iterations of the L-BFGS optimizer can be specified with the --max_iterations (-n) parameter. The L2 regularization constant can be specified with the --lambda (-r) parameter and if an intercept term is not desired in the model, the --no_intercept (-N) parameter can be specified.

The trained model can be saved with the --output_model_file (-M) output parameter. If training is not desired, but only testing is, a model can be loaded with the --input_model_file (-m) parameter. At the current time, a loaded model cannot be trained further, so specifying both --input_model_file (-m) and --training_file (-t) is not allowed.

The program is also able to evaluate a model on test data. A test dataset can be specified with the --test_file (-T) parameter. Class predictions can be saved with the --predictions_file (-p) output parameter. If labels are specified for the test data with the --test_labels_file (-L) parameter, then the program will print the accuracy of the predictions on the given test set and its corresponding labels.

Example

For example, to train a softmax regression model on the data 'dataset.csv' with labels 'labels.csv' with a maximum of 1000 iterations for training, saving the trained model to 'sr_model.bin', the following command can be used:

$ mlpack_softmax_regression --training_file dataset.csv --labels_file
  labels.csv --output_model_file sr_model.bin

Then, to use 'sr_model.bin' to classify the test points in 'test_points.csv', saving the output predictions to 'predictions.csv', the following command can be used:

$ mlpack_softmax_regression --input_model_file sr_model.bin --test_file
  test_points.csv --predictions_file predictions.csv

See also

mlpack_sparse_coding

Sparse Coding

$ mlpack_sparse_coding [--atoms 15] [--help] [--info <string>]
        [--initial_dictionary_file <string>] [--input_model_file <string>]
        [--lambda1 0] [--lambda2 0] [--max_iterations 0] [--newton_tolerance
        1e-06] [--normalize] [--objective_tolerance 0.01] [--seed 0]
        [--test_file <string>] [--training_file <string>] [--verbose]
        [--version] [--codes_file <string>] [--dictionary_file <string>]
        [--output_model_file <string>]

An implementation of Sparse Coding with Dictionary Learning. Given a dataset, this will decompose the dataset into a sparse combination of a few dictionary elements, where the dictionary is learned during computation; a dictionary can be reused for future sparse coding of new points. Detailed documentation.

Input options

name type description default
--atoms (-k) int Number of atoms in the dictionary. 15
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--initial_dictionary_file (-i) 2-d matrix file Optional initial dictionary matrix. ''
--input_model_file (-m) SparseCoding file File containing input sparse coding model. ''
--lambda1 (-l) double Sparse coding l1-norm regularization parameter. 0
--lambda2 (-L) double Sparse coding l2-norm regularization parameter. 0
--max_iterations (-n) int Maximum number of iterations for sparse coding (0 indicates no limit). 0
--newton_tolerance (-w) double Tolerance for convergence of Newton method. 1e-06
--normalize (-N) flag If set, the input data matrix will be normalized before coding.  
--objective_tolerance (-o) double Tolerance for convergence of the objective function. 0.01
--seed (-s) int Random seed. If 0, ‘std::time(NULL)’ is used. 0
--test_file (-T) 2-d matrix file Optional matrix to be encoded by trained model. ''
--training_file (-t) 2-d matrix file Matrix of training data (X). ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--codes_file (-c) 2-d matrix file Matrix to save the output sparse codes of the test matrix (–test_file) to.
--dictionary_file (-d) 2-d matrix file Matrix to save the output dictionary to.
--output_model_file (-M) SparseCoding file File to save trained sparse coding model to.

Detailed documentation

An implementation of Sparse Coding with Dictionary Learning, which achieves sparsity via an l1-norm regularizer on the codes (LASSO) or an (l1+l2)-norm regularizer on the codes (the Elastic Net). Given a dense data matrix X with d dimensions and n points, sparse coding seeks to find a dense dictionary matrix D with k atoms in d dimensions, and a sparse coding matrix Z with n points in k dimensions.

The original data matrix X can then be reconstructed as Z * D. Therefore, this program finds a representation of each point in X as a sparse linear combination of atoms in the dictionary D.

The sparse coding is found with an algorithm which alternates between a dictionary step, which updates the dictionary D, and a sparse coding step, which updates the sparse coding matrix.

Once a dictionary D is found, the sparse coding model may be used to encode other matrices, and saved for future usage.

To run this program, either an input matrix or an already-saved sparse coding model must be specified. An input matrix may be specified with the --training_file (-t) option, along with the number of atoms in the dictionary (specified with the --atoms (-k) parameter). It is also possible to specify an initial dictionary for the optimization, with the --initial_dictionary_file (-i) parameter. An input model may be specified with the --input_model_file (-m) parameter.

Example

As an example, to build a sparse coding model on the dataset 'data.csv' using 200 atoms and an l1-regularization parameter of 0.1, saving the model into 'model.bin', use

$ mlpack_sparse_coding --training_file data.csv --atoms 200 --lambda1 0.1
  --output_model_file model.bin

Then, this model could be used to encode a new matrix, 'otherdata.csv', and save the output codes to 'codes.csv':

$ mlpack_sparse_coding --input_model_file model.bin --test_file otherdata.csv
  --codes_file codes.csv

See also

mlpack_adaboost

AdaBoost

$ mlpack_adaboost [--help] [--info <string>] [--input_model_file
        <string>] [--iterations 1000] [--labels_file <string>] [--test_file
        <string>] [--tolerance 1e-10] [--training_file <string>] [--verbose]
        [--version] [--weak_learner 'decision_stump'] [--output_file <string>]
        [--output_model_file <string>] [--predictions_file <string>]
        [--probabilities_file <string>]

An implementation of the AdaBoost.MH (Adaptive Boosting) algorithm for classification. This can be used to train an AdaBoost model on labeled data or use an existing AdaBoost model to predict the classes of new points. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) AdaBoostModel file Input AdaBoost model. ''
--iterations (-i) int The maximum number of boosting iterations to be run (0 will run until convergence.) 1000
--labels_file (-l) 1-d index matrix file Labels for the training set. ''
--test_file (-T) 2-d matrix file Test dataset. ''
--tolerance (-e) double The tolerance for change in values of the weighted error during training. 1e-10
--training_file (-t) 2-d matrix file Dataset for training AdaBoost. ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  
--weak_learner (-w) string The type of weak learner to use: ‘decision_stump’, or ‘perceptron’. 'decision_stump'

Output options

name type description
--output_file (-o) 1-d index matrix file Predicted labels for the test set.
--output_model_file (-M) AdaBoostModel file Output trained AdaBoost model.
--predictions_file (-P) 1-d index matrix file Predicted labels for the test set.
--probabilities_file (-p) 2-d matrix file Predicted class probabilities for each point in the test set.

Detailed documentation

This program implements the AdaBoost (or Adaptive Boosting) algorithm. The variant of AdaBoost implemented here is AdaBoost.MH. It uses a weak learner, either decision stumps or perceptrons, and over many iterations, creates a strong learner that is a weighted ensemble of weak learners. It runs these iterations until a tolerance value is crossed for change in the value of the weighted training error.

For more information about the algorithm, see the paper “Improved Boosting Algorithms Using Confidence-Rated Predictions”, by R.E. Schapire and Y. Singer.

This program allows training of an AdaBoost model, and then application of that model to a test dataset. To train a model, a dataset must be passed with the --training_file (-t) option. Labels can be given with the --labels_file (-l) option; if no labels are specified, the labels will be assumed to be the last column of the input dataset. Alternately, an AdaBoost model may be loaded with the --input_model_file (-m) option.

Once a model is trained or loaded, it may be used to provide class predictions for a given test dataset. A test dataset may be specified with the --test_file (-T) parameter. The predicted classes for each point in the test dataset are output to the --predictions_file (-P) output parameter. The AdaBoost model itself is output to the --output_model_file (-M) output parameter.

Note: the following parameter is deprecated and will be removed in mlpack 4.0.0: --output_file (-o). Use --predictions_file (-P) instead of --output_file (-o).

Example

For example, to run AdaBoost on an input dataset 'data.csv' with labels 'labels.csv'and perceptrons as the weak learner type, storing the trained model in 'model.bin', one could use the following command:

$ mlpack_adaboost --training_file data.csv --labels_file labels.csv
  --output_model_file model.bin --weak_learner perceptron

Similarly, an already-trained model in 'model.bin' can be used to provide class predictions from test data 'test_data.csv' and store the output in 'predictions.csv' with the following command:

$ mlpack_adaboost --input_model_file model.bin --test_file test_data.csv
  --predictions_file predictions.csv

See also

mlpack_linear_regression

Simple Linear Regression and Prediction

$ mlpack_linear_regression [--help] [--info <string>]
        [--input_model_file <string>] [--lambda 0] [--test_file <string>]
        [--training_file <string>] [--training_responses_file <string>]
        [--verbose] [--version] [--output_model_file <string>]
        [--output_predictions_file <string>]

An implementation of simple linear regression and ridge regression using ordinary least squares. Given a dataset and responses, a model can be trained and saved for later use, or a pre-trained model can be used to output regression predictions for a test set. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) LinearRegression file Existing LinearRegression model to use. ''
--lambda (-l) double Tikhonov regularization for ridge regression. If 0, the method reduces to linear regression. 0
--test_file (-T) 2-d matrix file Matrix containing X’ (test regressors). ''
--training_file (-t) 2-d matrix file Matrix containing training set X (regressors). ''
--training_responses_file (-r) 1-d matrix file Optional vector containing y (responses). If not given, the responses are assumed to be the last row of the input file. ''
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_model_file (-M) LinearRegression file Output LinearRegression model.
--output_predictions_file (-o) 1-d matrix file If –test_file is specified, this matrix is where the predicted responses will be saved.

Detailed documentation

An implementation of simple linear regression and simple ridge regression using ordinary least squares. This solves the problem

y = X * b + e

where X (specified by --training_file (-t)) and y (specified either as the last column of the input matrix --training_file (-t) or via the --training_responses_file (-r) parameter) are known and b is the desired variable. If the covariance matrix (X’X) is not invertible, or if the solution is overdetermined, then specify a Tikhonov regularization constant (with --lambda (-l)) greater than 0, which will regularize the covariance matrix to make it invertible. The calculated b may be saved with the --output_predictions_file (-o) output parameter.

Optionally, the calculated value of b is used to predict the responses for another matrix X’ (specified by the --test_file (-T) parameter):

y’ = X’ * b

and the predicted responses y’ may be saved with the --output_predictions_file (-o) output parameter. This type of regression is related to least-angle regression, which mlpack implements as the ‘lars’ program.

Example

For example, to run a linear regression on the dataset 'X.csv' with responses 'y.csv', saving the trained model to 'lr_model.bin', the following command could be used:

$ mlpack_linear_regression --training_file X.csv --training_responses_file
  y.csv --output_model_file lr_model.bin

Then, to use 'lr_model.bin' to predict responses for a test set 'X_test.csv', saving the predictions to 'X_test_responses.csv', the following command could be used:

$ mlpack_linear_regression --input_model_file lr_model.bin --test_file
  X_test.csv --output_predictions_file X_test_responses.csv

See also

mlpack_preprocess_imputer

Impute Data

$ mlpack_preprocess_imputer [--custom_value 0] [--dimension 0] [--help]
        [--info <string>] --input_file <string> --missing_value <string>
        --strategy <string> [--verbose] [--version] [--output_file <string>]

This utility provides several imputation strategies for missing data. Given a dataset with missing values, this can impute according to several strategies, including user-defined values. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--custom_value (-c) double User-defined custom imputation value. 0
--dimension (-d) int The dimension to apply imputation to. 0
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_file (-i) string File containing data. required
--missing_value (-m) string User defined missing value. required
--strategy (-s) string imputation strategy to be applied. Strategies should be one of ‘custom’, ‘mean’, ‘median’, and ‘listwise_deletion’. required
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--output_file (-o) string File to save output into.

Detailed documentation

This utility takes a dataset and converts a user-defined missing variable to another to provide more meaningful analysis.

The program does not modify the original file, but instead makes a separate file to save the output data; You can save the output by specifying the file name with--output_file (-o).

Example

For example, if we consider ‘NULL’ in dimension 0 to be a missing variable and want to delete whole row containing the NULL in the column-wise'dataset.csv', and save the result to 'result.csv', we could run :

$ mlpack_preprocess_imputer --input_file dataset --output_file result
  --missing_value NULL --dimension 0 --strategy listwise_deletion

See also

mlpack_image_converter

Image Converter

$ mlpack_image_converter [--channels 0] [--dataset_file <string>]
        [--height 0] [--help] [--info <string>] --input [] [--quality 90]
        [--save] [--verbose] [--version] [--width 0] [--output_file <string>]

A utility to load an image or set of images into a single dataset that can then be used by other mlpack methods and utilities. This can also unpack an image dataset into individual files, for instance after mlpack methods have been used. Detailed documentation.

Input options

name type description default
--channels (-c) int Number of channels in the image. 0
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--dataset_file (-I) 2-d matrix file Input matrix to save as images. ''
--height (-H) int Height of the images. 0
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input (-i) string vector Image filenames which have to be loaded/saved. required
--quality (-q) int Compression of the image if saved as jpg (0-100). 90
--save (-s) flag Save a dataset as images.  
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  
--width (-w) int Width of the image. 0

Output options

name type description
--output_file (-o) 2-d matrix file Matrix to save images data to, Onlyneeded if you are specifying ‘save’ option.

Detailed documentation

This utility takes an image or an array of images and loads them to a matrix. You can optionally specify the height --height (-H) width --width (-w) and channel --channels (-c) of the images that needs to be loaded; otherwise, these parameters will be automatically detected from the image. There are other options too, that can be specified such as --quality (-q).

You can also provide a dataset and save them as images using --dataset_file (-I) and --save (-s) as an parameter.

Example

An example to load an image :

$ mlpack_image_converter --input X --height 256 --width 256 --channels 3
  --output_file Y.csv

An example to save an image is :

$ mlpack_image_converter --input X --height 256 --width 256 --channels 3
  --dataset_file Y.csv --save

See also

$ mlpack_range_search [--help] [--info <string>] [--input_model_file
        <string>] [--leaf_size 20] [--max 0] [--min 0] [--naive] [--query_file
        <string>] [--random_basis] [--reference_file <string>] [--seed 0]
        [--single_mode] [--tree_type 'kd'] [--verbose] [--version]
        [--distances_file <string>] [--neighbors_file <string>]
        [--output_model_file <string>]

An implementation of range search with single-tree and dual-tree algorithms. Given a set of reference points and a set of query points and a range, this can find the set of reference points within the desired range for each query point, and any trees built during the computation can be saved for reuse with future range searches. Detailed documentation.

Input options

name type description default
--check_input_matrices flag If specified, the input matrix is checked for NaN and inf values; an exception is thrown if any are found.  
--help (-h) flag Default help info. Only exists in CLI binding.  
--info string Print help on a specific option. Only exists in CLI binding. ''
--input_model_file (-m) RSModel file File containing pre-trained range search model. ''
--leaf_size (-l) int Leaf size for tree building (used for kd-trees, vp trees, random projection trees, UB trees, R trees, R* trees, X trees, Hilbert R trees, R+ trees, R++ trees, and octrees). 20
--max (-U) double Upper bound in range (if not specified, +inf will be used. 0
--min (-L) double Lower bound in range. 0
--naive (-N) flag If true, O(n^2) naive mode is used for computation.  
--query_file (-q) 2-d matrix file File containing query points (optional). ''
--random_basis (-R) flag Before tree-building, project the data onto a random orthogonal basis.  
--reference_file (-r) 2-d matrix file Matrix containing the reference dataset. ''
--seed (-s) int Random seed (if 0, std::time(NULL) is used). 0
--single_mode (-S) flag If true, single-tree search is used (as opposed to dual-tree search).  
--tree_type (-t) string Type of tree to use: ‘kd’, ‘vp’, ‘rp’, ‘max-rp’, ‘ub’, ‘cover’, ‘r’, ‘r-star’, ‘x’, ‘ball’, ‘hilbert-r’, ‘r-plus’, ‘r-plus-plus’, ‘oct’. 'kd'
--verbose (-v) flag Display informational messages and the full list of parameters and timers at the end of execution.  
--version (-V) flag Display the version of mlpack. Only exists in CLI binding.  

Output options

name type description
--distances_file (-d) string File to output distances into.
--neighbors_file (-n) string File to output neighbors into.
--output_model_file (-M) RSModel file If specified, the range search model will be saved to the given file.

Detailed documentation

This program implements range search with a Euclidean distance metric. For a given query point, a given range, and a given set of reference points, the program will return all of the reference points with distance to the query point in the given range. This is performed for an entire set of query points. You may specify a separate set of reference and query points, or only a reference set – which is then used as both the reference and query set. The given range is taken to be inclusive (that is, points with a distance exactly equal to the minimum and maximum of the range are included in the results).

Example

For example, the following will calculate the points within the range [2, 5] of each point in 'input.csv' and store the distances in'distances.csv' and the neighbors in 'neighbors.csv'

$ mlpack_range_search --min 2 --max 5 --distances_file input --distances_file
  distances --neighbors_file neighbors

The output files are organized such that line i corresponds to the points found for query point i. Because sometimes 0 points may be found in the given range, lines of the output files may be empty. The points are not ordered in any specific manner.

Because the number of points returned for each query point may differ, the resultant CSV-like files may not be loadable by many programs. However, at this time a better way to store this non-square result is not known. As a result, any output files will be written as CSVs in this manner, regardless of the given extension.

See also

changelog/history

mlpack 4.3.0

2023-11-22
  • Fix include ordering issue for LinearRegression #3541.

  • Fix L1 regularization in case where weight is zero #3545.

  • Use HTTPS for all auto-downloaded dependencies #3550.

  • More robust detection of C++17 mode in the MSVC “compiler” (#3555, #3557).

  • Fix setting number of classes correctly in SoftmaxRegression::Train() #3553.

  • Adapt MultiheadAttention and LayerNorm ANN layers to new Layer interface #3547.

  • Fix inconsistent use of the “input” parameter to the Backward method in ANNs #3551.

mlpack 4.2.1

2023-09-05
  • Reinforcement Learning: Gaussian noise #3515.

  • Reinforcement Learning: Twin Delayed Deep Deterministic Policy Gradient #3512.

  • Reinforcement Learning: Ornstein-Uhlenbeck noise #3499.

  • Reinforcement Learning: Deep Deterministic Policy Gradient #3494.

  • Add ClassProbabilities() member to DecisionTree so that the internal details of trees can be more easily inspected #3511.

  • Bipolar sigmoid activation function added and invertible functions fixed #3506.

  • Add auto-configured mlpack/config.hpp to contain configuration details of mlpack that are required at compile time. STB detection is now done in this file with the MLPACK_HAS_STB macro #3519.

  • Fix CRAN package alias for R bindings #3543.

mlpack 4.2.0

2023-06-14
  • Adapt C_ReLU, ReLU6, FlexibleReLU layer for new neural network API #3445.

  • Fix PReLU, add integration test to it #3473.

  • Fix bug in LogSoftMax derivative #3469.

  • Add serialize method to GaussianInitialization, LecunNormalInitialization, KathirvalavakumarSubavathiInitialization, NguyenWidrowInitialization, and OrthogonalInitialization #3483.

  • Allow categorical features to preprocess_one_hot_encode #3487.

  • Install mlpack and cereal headers as part of R package #3488.

  • Add intercept and normalization support to LARS #3493.

  • Allow adding two features simultaneously to LARS models #3493.

  • Adapt FTSwish activation function #3485.

  • Adapt Hyper-Sinh activation function #3491.

mlpack 4.1.0

2023-04-26
  • Adapt HardTanH layer #3454.

  • Adapt Softmin layer for new neural network API #3437.

  • Adapt PReLU layer for new neural network API #3420.

  • Add CF decomposition methods: QUIC_SVDPolicy and BlockKrylovSVDPolicy (#3413, #3404).

  • Update outdated code in tutorials (#3398, #3401).

  • Bugfix for non-square convolution kernels #3376.

  • Fix a few missing includes in <mlpack.hpp> #3374.

  • Fix DBSCAN handling of non-core points #3346.

  • Avoid deprecation warnings in Armadillo 11.4.4+ #3405.

  • Issue runtime error when serialization of neural networks is attempted but MLPACK_ENABLE_ANN_SERIALIZATION is not defined #3451.

mlpack 4.0.1

2022-12-23
  • Fix mapping of categorical data for Julia bindings #3305.

  • Bugfix: catch all exceptions when running bindings from Julia, instead of crashing #3304.

  • Various Python configuration fixes for Windows and OS X (#3312, #3313, #3311, #3309, #3308, #3297, #3302).

  • Optimize and strip compiled Python bindings when possible, resulting in significant size minimization #3310.

  • The /std:c++17 and /Zc:__cplusplus options are now required when using Visual Studio #3318. Documentation and compile-time checks added.

  • Set BUILD_TESTS to OFF by default. If you want to build tests, like mlpack_test, manually set BUILD_TESTS to ON in your CMake configuration step #3316.

  • Fix handling of transposed matrix parameters in Python, Julia, R, and Go bindings #3327.

  • Comment out definition of ARMA_NO DEBUG. This allows various Armadillo run-time checks such as non-conforming matrices and out-of-bounds element access. In turn this helps tracking down bugs and incorrect usage #3322.

mlpack 4.0.0

2022-10-23
  • Bump C++ standard requirement to C++14 #3233.

  • Fix Perceptron to work with cross-validation framework #3190.

  • Migrate from boost tests to Catch2 framework #2523, #2584.

  • Bump minimum armadillo version from 8.400 to 9.800 #3043, #3048.

  • Adding a copy constructor in the Convolution layer #3067.

  • Replace boost::spirit parser by a local efficient implementation #2942.

  • Disable correctly the autodownloader + fix tests stability #3076.

  • Replace boost::any with core::v2::any or std::any if available #3006.

  • Remove old non used Boost headers #3005.

  • Replace boost::enable_if with std::enable_if #2998.

  • Replace boost::is_same with std::is_same #2993.

  • Remove invalid option for emsmallen and STB #2960.

  • Check for armadillo dependencies before downloading armadillo #2954.

  • Disable the usage of autodownloader by default #2953.

  • Install dependencies downloaded with the autodownloader #2952.

  • Download older Boost if the compiler is old #2940.

  • Add support for embedded systems #2531.

  • Build mlpack executable statically if the library is statically linked #2931.

  • Fix cover tree loop bug on embedded arm systems #2869.

  • Fix a LAPACK bug in FindArmadillo.cmake #2929.

  • Add an autodownloader to get mlpack dependencies #2927.

  • Remove Coverage files and configurations from CMakeLists #2866.

  • Added Multi Label Soft Margin Loss loss function for neural networks #2345.

  • Added Decision Tree Regressor #2905. It can be used using the class mlpack::tree::DecisionTreeRegressor. It is accessible only though C++.

  • Added dict-style inspection of mlpack models in python bindings #2868.

  • Added Extra Trees Algorithm #2883. Currently, it can be used using the class mlpack::tree::ExtraTrees, but only through C++.

  • Add Flatten T Swish activation function (flatten-t-swish.hpp)

  • Added warm start feature to Random Forest #2881; this feature is accessible from mlpack’s bindings to different languages.

  • Added Pixel Shuffle layer #2563.

  • Add “check_input_matrices” option to python bindings that checks for NaN and inf values in all the input matrices #2787.

  • Add Adjusted R squared functionality to R2Score::Evaluate #2624.

  • Disabled all the bindings by default in CMake #2782.

  • Added an implementation to Stratify Data #2671.

  • Add BUILD_DOCS CMake option to control whether Doxygen documentation is built (default ON) #2730.

  • Add Triplet Margin Loss function #2762.

  • Add finalizers to Julia binding model types to fix memory handling #2756.

  • HMM: add functions to calculate likelihood for data stream with/without pre-calculated emission probability #2142.

  • Replace Boost serialization library with Cereal #2458.

  • Add PYTHON_INSTALL_PREFIX CMake option to specify installation root for Python bindings #2797.

  • Removed boost::visitor from model classes for knn, kfn, cf, range_search, krann, and kde bindings #2803.

  • Add k-means++ initialization strategy #2813.

  • NegativeLogLikelihood<> now expects classes in the range 0 to numClasses - 1 #2534.

  • Add Lambda1(), Lambda2(), UseCholesky(), and Tolerance() members to LARS so parameters for training can be modified #2861.

  • Remove unused ElemType template parameter from DecisionTree and RandomForest #2874.

  • Fix Python binding build when the CMake variable USE_OPENMP is set to OFF #2884.

  • The mlpack_test target is no longer built as part of make all. Use make mlpack_test to build the tests.

  • Fixes to HoeffdingTree: ensure that training still works when empty constructor is used #2964.

  • Fix Julia model serialization bug #2970.

  • Fix LoadCSV() to use pre-populated DatasetInfo objects #2980.

  • Add probabilities option to softmax regression binding, to get class probabilities for test points #3001.

  • Fix thread safety issues in mlpack bindings to other languages #2995.

  • Fix double-free of model pointers in R bindings #3034.

  • Fix Julia, Python, R, and Go handling of categorical data for decision_tree() and hoeffding_tree() #2971.

  • Depend on pkgbuild for R bindings #3081.

  • Replaced Numpy deprecated code in Python bindings #3126.

mlpack 3.4.2

2020-10-26
  • Added Mean Absolute Percentage Error.

  • Added Softmin activation function as layer in ann/layer.

  • Fix spurious ARMA_64BIT_WORD compilation warnings on 32-bit systems #2665.

mlpack 3.4.1

2020-09-07
  • Fix incorrect parsing of required matrix/model parameters for command-line bindings #2600.

  • Add manual type specification support to data::Load() and data::Save() (#2084, #2135, #2602).

  • Remove use of internal Armadillo functionality (#2596, #2601, #2602).

mlpack 3.4.0

2020-09-01
  • Issue warnings when metrics produce NaNs in KFoldCV #2595.

  • Added bindings for R during Google Summer of Code #2556.

  • Added common striptype function for all bindings #2556.

  • Refactored common utility function of bindings to bindings/util #2556.

  • Renamed InformationGain to HoeffdingInformationGain in methods/hoeffding_trees/information_gain.hpp #2556.

  • Added macro for changing stream of printing and warnings/errors #2556.

  • Added Spatial Dropout layer #2564.

  • Force CMake to show error when it didn’t find Python/modules #2568.

  • Refactor ProgramInfo() to separate out all the different information #2558.

  • Add bindings for one-hot encoding #2325.

  • Added Soft Actor-Critic to RL methods #2487.

  • Added Categorical DQN to q_networks #2454.

  • Added N-step DQN to q_networks #2461.

  • Add Silhoutte Score metric and Pairwise Distances #2406.

  • Add Go bindings for some missed models #2460.

  • Replace boost program_options dependency with CLI11 #2459.

  • Additional functionality for the ARFF loader #2486; use case sensitive categories #2516.

  • Add bayesian_linear_regression binding for the command-line, Python, Julia, and Go. Also called “Bayesian Ridge”, this is equivalent to a version of linear regression where the regularization parameter is automatically tuned #2030.

  • Fix defeatist search for spill tree traversals (#2566, #1269).

  • Fix incremental training of logistic regression models #2560.

  • Change default configuration of BUILD_PYTHON_BINDINGS to OFF #2575.

mlpack 3.3.2

2020-06-18
  • Added Noisy DQN to q_networks #2446.

  • Add Go bindings #1884.

  • Added Dueling DQN to q_networks, Noisy linear layer to ann/layer and Empty loss to ann/loss_functions #2414.

  • Storing and adding accessor method for action in q_learning #2413.

  • Added accessor methods for ANN layers #2321.

  • Addition of Elliot activation function #2268.

  • Add adaptive max pooling and adaptive mean pooling layers #2195.

  • Add parameter to avoid shuffling of data in preprocess_split #2293.

  • Add MatType parameter to LSHSearch, allowing sparse matrices to be used for search #2395.

  • Documentation fixes to resolve Doxygen warnings and issues #2400.

  • Add Load and Save of Sparse Matrix #2344.

  • Add Intersection over Union (IoU) metric for bounding boxes #2402.

  • Add Non Maximal Supression (NMS) metric for bounding boxes #2410.

  • Fix no_intercept and probability computation for linear SVM bindings #2419.

  • Fix incorrect neighbors for k > 1 searches in approx_kfn binding, for the QDAFN algorithm #2448.

  • Fix serialization of kernels with state for FastMKS #2452.

  • Add RBF layer in ann module to make RBFN architecture #2261.

mlpack 3.3.1

2020-04-29
  • Minor Julia and Python documentation fixes #2373.

  • Updated terminal state and fixed bugs for Pendulum environment (#2354, #2369).

  • Added EliSH activation function #2323.

  • Add L1 Loss function #2203.

  • Pass CMAKE_CXX_FLAGS (compilation options) correctly to Python build #2367.

  • Expose ensmallen Callbacks for sparseautoencoder #2198.

  • Bugfix for LARS class causing invalid read #2374.

  • Add serialization support from Julia; use mlpack.serialize() and mlpack.deserialize() to save and load from IOBuffers.

mlpack 3.3.0

2020-04-07
  • Added Normal Distribution to ann/dists #2382.

  • Templated return type of Forward function of loss functions #2339.

  • Added R2 Score regression metric #2323.

  • Added poisson negative log likelihood loss function #2196.

  • Added huber loss function #2199.

  • Added mean squared logarithmic error loss function for neural networks #2210.

  • Added mean bias loss function for neural networks #2210.

  • The DecisionStump class has been marked deprecated; use the DecisionTree class with NoRecursion=true or use ID3DecisionStump instead #2099.

  • Added probabilities_file parameter to get the probabilities matrix of AdaBoost classifier #2050.

  • Fix STB header search paths #2104.

  • Add DISABLE_DOWNLOADS CMake configuration option #2104.

  • Add padding layer in TransposedConvolutionLayer #2082.

  • Fix pkgconfig generation on non-Linux systems #2101.

  • Use log-space to represent HMM initial state and transition probabilities #2081.

  • Add functions to access parameters of Convolution and AtrousConvolution layers #1985.

  • Add Compute Error function in lars regression and changing Train function to return computed error #2139.

  • Add Julia bindings #1949. Build settings can be controlled with the BUILD_JULIA_BINDINGS=(ON/OFF) and JULIA_EXECUTABLE=/path/to/julia CMake parameters.

  • CMake fix for finding STB include directory #2145.

  • Add bindings for loading and saving images #2019; mlpack_image_converter from the command-line, mlpack.image_converter() from Python.

  • Add normalization support for CF binding #2136.

  • Add Mish activation function #2158.

  • Update init_rules in AMF to allow users to merge two initialization rules #2151.

  • Add GELU activation function #2183.

  • Better error handling of eigendecompositions and Cholesky decompositions (#2088, #1840).

  • Add LiSHT activation function #2182.

  • Add Valid and Same Padding for Transposed Convolution layer #2163.

  • Add CELU activation function #2191

  • Add Log-Hyperbolic-Cosine Loss function #2207.

  • Change neural network types to avoid unnecessary use of rvalue references #2259.

  • Bump minimum Boost version to 1.58 #2305.

  • Refactor STB support so HAS_STB macro is not needed when compiling against mlpack #2312.

  • Add Hard Shrink Activation Function #2186.

  • Add Soft Shrink Activation Function #2174.

  • Add Hinge Embedding Loss Function #2229.

  • Add Cosine Embedding Loss Function #2209.

  • Add Margin Ranking Loss Function #2264.

  • Bugfix for incorrect parameter vector sizes in logistic regression and softmax regression #2359.

mlpack 3.2.2

2019-11-26
  • Add valid and same padding option in Convolution and Atrous Convolution layer #1988.

  • Add Model() to the FFN class to access individual layers #2043.

  • Update documentation for pip and conda installation packages #2044.

  • Add bindings for linear SVM #1935; mlpack_linear_svm from the command-line, linear_svm() from Python.

  • Add support to return the layer name as std::string #1987.

  • Speed and memory improvements for the Transposed Convolution layer #1493.

  • Fix Windows Python build configuration #1885.

  • Validate md5 of STB library after download #2087.

  • Add __version__ to __init__.py #2092.

  • Correctly handle RNN sequences that are shorter than the value of rho #2102.

mlpack 3.2.1

2019-10-01
  • Enforce CMake version check for ensmallen #2032.

  • Fix CMake check for Armadillo version #2029.

  • Better handling of when STB is not installed #2033.

  • Fix Naive Bayes classifier computations in high dimensions #2022.

mlpack 3.2.0

2019-09-25
  • Fix some potential infinity errors in Naive Bayes Classifier #2022.

  • Fix occasionally-failing RADICAL test #1924.

  • Fix gcc 9 OpenMP compilation issue #1970.

  • Added support for loading and saving of images #1903.

  • Add Multiple Pole Balancing Environment (#1901, #1951).

  • Added functionality for scaling of data #1876; see the command-line binding mlpack_preprocess_scale or Python binding preprocess_scale().

  • Add new parameter maximum_depth to decision tree and random forest bindings #1916.

  • Fix prediction output of softmax regression when test set accuracy is calculated #1922.

  • Pendulum environment now checks for termination. All RL environments now have an option to terminate after a set number of time steps (no limit by default) #1941.

  • Add support for probabilistic KDE (kernel density estimation) error bounds when using the Gaussian kernel #1934.

  • Fix negative distances for cover tree computation #1979.

  • Fix cover tree building when all pairwise distances are 0 #1986.

  • Improve KDE pruning by reclaiming not used error tolerance (#1954, #1984).

  • Optimizations for sparse matrix accesses in z-score normalization for CF #1989.

  • Add kmeans_max_iterations option to GMM training binding gmm_train_main.

  • Bump minimum Armadillo version to 8.400.0 due to ensmallen dependency requirement #2015.

mlpack 3.1.1

2019-05-26
  • Fix random forest bug for numerical-only data #1887.

  • Significant speedups for random forest #1887.

  • Random forest now has minimum_gain_split and subspace_dim parameters #1887.

  • Decision tree parameter print_training_error deprecated in favor of print_training_accuracy.

  • output option changed to predictions for adaboost and perceptron binding. Old options are now deprecated and will be preserved until mlpack 4.0.0 #1882.

  • Concatenated ReLU layer #1843.

  • Accelerate NormalizeLabels function using hashing instead of linear search (see src/mlpack/core/data/normalize_labels_impl.hpp) #1780.

  • Add ConfusionMatrix() function for checking performance of classifiers #1798.

  • Install ensmallen headers when it is downloaded during build #1900.

mlpack 3.1.0

2019-04-25
  • Add DiagonalGaussianDistribution and DiagonalGMM classes to speed up the diagonal covariance computation and deprecate DiagonalConstraint #1666.

  • Add kernel density estimation (KDE) implementation with bindings to other languages #1301.

  • Where relevant, all models with a Train() method now return a double value representing the goodness of fit (i.e. final objective value, error, etc.) #1678.

  • Add implementation for linear support vector machine (see src/mlpack/methods/linear_svm).

  • Change DBSCAN to use PointSelectionPolicy and add OrderedPointSelection #1625.

  • Residual block support #1594.

  • Bidirectional RNN #1626.

  • Dice loss layer (#1674, #1714) and hard sigmoid layer #1776.

  • output option changed to predictions and output_probabilities to probabilities for Naive Bayes binding (mlpack_nbc/nbc()). Old options are now deprecated and will be preserved until mlpack 4.0.0 #1616.

  • Add support for Diagonal GMMs to HMM code (#1658, #1666). This can provide large speedup when a diagonal GMM is acceptable as an emission probability distribution.

  • Python binding improvements: check parameter type #1717, avoid copying Pandas dataframes #1711, handle Pandas Series objects #1700.

mlpack 3.0.4

2018-11-13
  • Bump minimum CMake version to 3.3.2.

  • CMake fixes for Ninja generator by Marc Espie.

mlpack 3.0.3

2018-07-27
  • Fix Visual Studio compilation issue #1443.

  • Allow running local_coordinate_coding binding with no initial_dictionary parameter when input_model is not specified #1457.

  • Make use of OpenMP optional via the CMake ‘USE_OPENMP’ configuration variable #1474.

  • Accelerate FNN training by 20-30% by avoiding redundant calculations #1467.

  • Fix math::RandomSeed() usage in tests (#1462, #1440).

  • Generate better Python setup.py with documentation #1460.

mlpack 3.0.2

2018-06-08
  • Documentation generation fixes for Python bindings #1421.

  • Fix build error for man pages if command-line bindings are not being built #1424.

  • Add ‘shuffle’ parameter and Shuffle() method to KFoldCV #1412. This will shuffle the data when the object is constructed, or when Shuffle() is called.

  • Added neural network layers: AtrousConvolution #1390, Embedding #1401, and LayerNorm (layer normalization) #1389.

  • Add Pendulum environment for reinforcement learning #1388 and update Mountain Car environment #1394.

mlpack 3.0.1

2018-05-10
  • Fix intermittently failing tests #1387.

  • Add big-batch SGD (BBSGD) optimizer in src/mlpack/core/optimizers/bigbatch_sgd/ #1131.

  • Fix simple compiler warnings (#1380, #1373).

  • Simplify NeighborSearch constructor and Train() overloads #1378.

  • Add warning for OpenMP setting differences (#1358/#1382). When mlpack is compiled with OpenMP but another application is not (or vice versa), a compilation warning will now be issued.

  • Restructured loss functions in src/mlpack/methods/ann/ #1365.

  • Add environments for reinforcement learning tests (#1368, #1370, #1329).

  • Allow single outputs for multiple timestep inputs for recurrent neural networks #1348.

  • Add He and LeCun normal initializations for neural networks #1342. Neural networks: add He and LeCun normal initializations #1342, add FReLU and SELU activation functions (#1346, #1341), add alpha-dropout #1349.

mlpack 3.0.0

2018-03-30
  • Speed and memory improvements for DBSCAN. –single_mode can now be used for situations where previously RAM usage was too high.

  • Bump minimum required version of Armadillo to 6.500.0.

  • Add automatically generated Python bindings. These have the same interface as the command-line programs.

  • Add deep learning infrastructure in src/mlpack/methods/ann/.

  • Add reinforcement learning infrastructure in src/mlpack/methods/reinforcement_learning/.

  • Add optimizers: AdaGrad, CMAES, CNE, FrankeWolfe, GradientDescent, GridSearch, IQN, Katyusha, LineSearch, ParallelSGD, SARAH, SCD, SGDR, SMORMS3, SPALeRA, SVRG.

  • Add hyperparameter tuning infrastructure and cross-validation infrastructure in src/mlpack/core/cv/ and src/mlpack/core/hpt/.

  • Fix bug in mean shift.

  • Add random forests (see src/mlpack/methods/random_forest).

  • Numerous other bugfixes and testing improvements.

  • Add randomized Krylov SVD and Block Krylov SVD.

mlpack 2.2.5

2017-08-25
  • Compilation fix for some systems #1082.

  • Fix PARAM_INT_OUT() #1100.

mlpack 2.2.4

2017-07-18
  • Speed and memory improvements for DBSCAN. –single_mode can now be used for situations where previously RAM usage was too high.

  • Fix bug in CF causing incorrect recommendations.

mlpack 2.2.3

2017-05-24
  • Bug fix for –predictions_file in mlpack_decision_tree program.

mlpack 2.2.2

2017-05-04
  • Install backwards-compatibility mlpack_allknn and mlpack_allkfn programs; note they are deprecated and will be removed in mlpack 3.0.0 #992.

  • Fix RStarTree bug that surfaced on OS X only #964.

  • Small fixes for MiniBatchSGD and SGD and tests.

mlpack 2.2.1

2017-04-13
  • Compilation fix for mlpack_nca and mlpack_test on older Armadillo versions #984.

mlpack 2.2.0

2017-03-21
  • Bugfix for mlpack_knn program #816.

  • Add decision tree implementation in methods/decision_tree/. This is very similar to a C4.5 tree learner.

  • Add DBSCAN implementation in methods/dbscan/.

  • Add support for multidimensional discrete distributions (#810, #830).

  • Better output for Log::Debug/Log::Info/Log::Warn/Log::Fatal for Armadillo objects (#895, #928).

  • Refactor categorical CSV loading with boost::spirit for faster loading #681.

mlpack 2.1.1

2016-12-22
  • HMMs now use random initialization; this should fix some convergence issues #828.

  • HMMs now initialize emissions according to the distribution of observations #833.

  • Minor fix for formatted output #814.

  • Fix DecisionStump to properly work with any input type.

mlpack 2.1.0

2016-10-31
  • Fixed CoverTree to properly handle single-point datasets.

  • Fixed a bug in CosineTree (and thus QUIC-SVD) that caused split failures for some datasets #717.

  • Added mlpack_preprocess_describe program, which can be used to print statistics on a given dataset #742.

  • Fix prioritized recursion for k-furthest-neighbor search (mlpack_kfn and the KFN class), leading to orders-of-magnitude speedups in some cases.

  • Bump minimum required version of Armadillo to 4.200.0.

  • Added simple Gradient Descent optimizer, found in src/mlpack/core/optimizers/gradient_descent/ #792.

  • Added approximate furthest neighbor search algorithms QDAFN and DrusillaSelect in src/mlpack/methods/approx_kfn/, with command-line program mlpack_approx_kfn.

mlpack 2.0.3

2016-07-21
  • Added multiprobe LSH #691. The parameter ‘T’ to LSHSearch::Search() can now be used to control the number of extra bins that are probed, as can the -T (–num_probes) option to mlpack_lsh.

  • Added the Hilbert R tree to src/mlpack/core/tree/rectangle_tree/ #664. It can be used as the typedef HilbertRTree, and it is now an option in the mlpack_knn, mlpack_kfn, mlpack_range_search, and mlpack_krann command-line programs.

  • Added the mlpack_preprocess_split and mlpack_preprocess_binarize programs, which can be used for preprocessing code (#650, #666).

  • Added OpenMP support to LSHSearch and mlpack_lsh #700.

mlpack 2.0.2

2016-06-20
  • Added the function LSHSearch::Projections(), which returns an arma::cube with each projection table in a slice #663. Instead of Projection(i), you should now use Projections().slice(i).

  • A new constructor has been added to LSHSearch that creates objects using projection tables provided in an arma::cube #663.

  • Handle zero-variance dimensions in DET #515.

  • Add MiniBatchSGD optimizer (src/mlpack/core/optimizers/minibatch_sgd/) and allow its use in mlpack_logistic_regression and mlpack_nca programs.

  • Add better backtrace support from Grzegorz Krajewski for Log::Fatal messages when compiled with debugging and profiling symbols. This requires libbfd and libdl to be present during compilation.

  • CosineTree test fix from Mikhail Lozhnikov #358.

  • Fixed HMM initial state estimation #600.

  • Changed versioning macros __MLPACK_VERSION_MAJOR, __MLPACK_VERSION_MINOR, and __MLPACK_VERSION_PATCH to MLPACK_VERSION_MAJOR, MLPACK_VERSION_MINOR, and MLPACK_VERSION_PATCH. The old names will remain in place until mlpack 3.0.0.

  • Renamed mlpack_allknn, mlpack_allkfn, and mlpack_allkrann to mlpack_knn, mlpack_kfn, and mlpack_krann. The mlpack_allknn, mlpack_allkfn, and mlpack_allkrann programs will remain as copies until mlpack 3.0.0.

  • Add –random_initialization option to mlpack_hmm_train, for use when no labels are provided.

  • Add –kill_empty_clusters option to mlpack_kmeans and KillEmptyClusters policy for the KMeans class (#595, #596).

mlpack 2.0.1

2016-02-04
  • Fix CMake to properly detect when MKL is being used with Armadillo.

  • Minor parameter handling fixes to mlpack_logistic_regression (#504, #505).

  • Properly install arma_config.hpp.

  • Memory handling fixes for Hoeffding tree code.

  • Add functions that allow changing training-time parameters to HoeffdingTree class.

  • Fix infinite loop in sparse coding test.

  • Documentation spelling fixes #501.

  • Properly handle covariances for Gaussians with large condition number #496, preventing GMMs from filling with NaNs during training (and also HMMs that use GMMs).

  • CMake fixes for finding LAPACK and BLAS as Armadillo dependencies when ATLAS is used.

  • CMake fix for projects using mlpack’s CMake configuration from elsewhere #512.

mlpack 2.0.0

2015-12-24
  • Removed overclustering support from k-means because it is not well-tested, may be buggy, and is (I think) unused. If this was support you were using, open a bug or get in touch with us; it would not be hard for us to reimplement it.

  • Refactored KMeans to allow different types of Lloyd iterations.

  • Added implementations of k-means: Elkan’s algorithm, Hamerly’s algorithm, Pelleg-Moore’s algorithm, and the DTNN (dual-tree nearest neighbor) algorithm.

  • Significant acceleration of LRSDP via the use of accu(a % b) instead of trace(a * b).

  • Added MatrixCompletion class (matrix_completion), which performs nuclear norm minimization to fill unknown values of an input matrix.

  • No more dependence on Boost.Random; now we use C++11 STL random support.

  • Add softmax regression, contributed by Siddharth Agrawal and QiaoAn Chen.

  • Changed NeighborSearch, RangeSearch, FastMKS, LSH, and RASearch API; these classes now take the query sets in the Search() method, instead of in the constructor.

  • Use OpenMP, if available. For now OpenMP support is only available in the DET training code.

  • Add support for predicting new test point values to LARS and the command-line ‘lars’ program.

  • Add serialization support for Perceptron and LogisticRegression.

  • Refactor SoftmaxRegression to predict into an arma::Row object, and add a softmax_regression program.

  • Refactor LSH to allow loading and saving of models.

  • ToString() is removed entirely #487.

  • Add –input_model_file and –output_model_file options to appropriate machine learning algorithms.

  • Rename all executables to start with an “mlpack” prefix #229.

  • Add HoeffdingTree and mlpack_hoeffding_tree, an implementation of the streaming decision tree methodology from Domingos and Hulten in 2000.

mlpack 1.0.12

2015-01-07
  • Switch to 3-clause BSD license (from LGPL).

mlpack 1.0.11

2014-12-11
  • Proper handling of dimension calculation in PCA.

  • Load parameter vectors properly for LinearRegression models.

  • Linker fixes for AugLagrangian specializations under Visual Studio.

  • Add support for observation weights to LinearRegression.

  • MahalanobisDistance<> now takes the root of the distance by default and therefore satisfies the triangle inequality (TakeRoot now defaults to true).

  • Better handling of optional Armadillo HDF5 dependency.

  • Fixes for numerous intermittent test failures.

  • math::RandomSeed() now sets the random seed for recent (>=3.930) Armadillo versions.

  • Handle Newton method convergence better for SparseCoding::OptimizeDictionary() and make maximum iterations a parameter.

  • Known bug: CosineTree construction may fail in some cases on i386 systems #358.

mlpack 1.0.10

2014-08-29
  • Bugfix for NeighborSearch regression which caused very slow allknn/allkfn. Speeds are now restored to approximately 1.0.8 speeds, with significant improvement for the cover tree #347.

  • Detect dependencies correctly when ARMA_USE_WRAPPER is not being defined (i.e., libarmadillo.so does not exist).

  • Bugfix for compilation under Visual Studio #348.

mlpack 1.0.9

2014-07-28
  • GMM initialization is now safer and provides a working GMM when constructed with only the dimensionality and number of Gaussians #301.

  • Check for division by 0 in Forward-Backward Algorithm in HMMs #301.

  • Fix MaxVarianceNewCluster (used when re-initializing clusters for k-means) #301.

  • Fixed implementation of Viterbi algorithm in HMM::Predict() #303.

  • Significant speedups for dual-tree algorithms using the cover tree (#235, #314) including a faster implementation of FastMKS.

  • Fix for LRSDP optimizer so that it compiles and can be used #312.

  • CF (collaborative filtering) now expects users and items to be zero-indexed, not one-indexed #311.

  • CF::GetRecommendations() API change: now requires the number of recommendations as the first parameter. The number of users in the local neighborhood should be specified with CF::NumUsersForSimilarity().

  • Removed incorrect PeriodicHRectBound #58.

  • Refactor LRSDP into LRSDP class and standalone function to be optimized #305.

  • Fix for centering in kernel PCA #337.

  • Added simulated annealing (SA) optimizer, contributed by Zhihao Lou.

  • HMMs now support initial state probabilities; these can be set in the constructor, trained, or set manually with HMM::Initial() #302.

  • Added Nyström method for kernel matrix approximation by Marcus Edel.

  • Kernel PCA now supports using Nyström method for approximation.

  • Ball trees now work with dual-tree algorithms, via the BallBound<> bound structure #307; fixed by Yash Vadalia.

  • The NMF class is now AMF<>, and supports far more types of factorizations, by Sumedh Ghaisas.

  • A QUIC-SVD implementation has returned, written by Siddharth Agrawal and based on older code from Mudit Gupta.

  • Added perceptron and decision stump by Udit Saxena (these are weak learners for an eventual AdaBoost class).

  • Sparse autoencoder added by Siddharth Agrawal.

mlpack 1.0.8

2014-01-06
  • Memory leak in NeighborSearch index-mapping code fixed #298.

  • GMMs can be trained using the existing model as a starting point by specifying an additional boolean parameter to GMM::Estimate() #296.

  • Logistic regression implementation added in methods/logistic_regression (see also #293).

  • L-BFGS optimizer now returns its function via Function().

  • Version information is now obtainable via mlpack::util::GetVersion() or the __MLPACK_VERSION_MAJOR, __MLPACK_VERSION_MINOR, and __MLPACK_VERSION_PATCH macros #297.

  • Fix typos in allkfn and allkrann output.

mlpack 1.0.7

2013-10-04
  • Cover tree support for range search (range_search), rank-approximate nearest neighbors (allkrann), minimum spanning tree calculation (emst), and FastMKS (fastmks).

  • Dual-tree FastMKS implementation added and tested.

  • Added collaborative filtering package (cf) that can provide recommendations when given users and items.

  • Fix for correctness of Kernel PCA (kernel_pca) #270.

  • Speedups for PCA and Kernel PCA #198.

  • Fix for correctness of Neighborhood Components Analysis (NCA) #279.

  • Minor speedups for dual-tree algorithms.

  • Fix for Naive Bayes Classifier (nbc) #269.

  • Added a ridge regression option to LinearRegression (linear_regression) #286.

  • Gaussian Mixture Models (gmm::GMM<>) now support arbitrary covariance matrix constraints #283.

  • MVU (mvu) removed because it is known to not work #183.

  • Minor updates and fixes for kernels (in mlpack::kernel).

mlpack 1.0.6

2013-06-13
  • Minor bugfix so that FastMKS gets built.

mlpack 1.0.5

2013-05-01
  • Speedups of cover tree traversers #235.

  • Addition of rank-approximate nearest neighbors (RANN), found in src/mlpack/methods/rann/.

  • Addition of fast exact max-kernel search (FastMKS), found in src/mlpack/methods/fastmks/.

  • Fix for EM covariance estimation; this should improve GMM training time.

  • More parameters for GMM estimation.

  • Force GMM and GaussianDistribution covariance matrices to be positive definite, so that training converges much more often.

  • Add parameter for the tolerance of the Baum-Welch algorithm for HMM training.

  • Fix for compilation with clang compiler.

  • Fix for k-furthest-neighbor-search.

mlpack 1.0.4

2013-02-08
  • Force minimum Armadillo version to 2.4.2.

  • Better output of class types to streams; a class with a ToString() method implemented can be sent to a stream with operator«.

  • Change return type of GMM::Estimate() to double #257.

  • Style fixes for k-means and RADICAL.

  • Handle size_t support correctly with Armadillo 3.6.2 #258.

  • Add locality-sensitive hashing (LSH), found in src/mlpack/methods/lsh/.

  • Better tests for SGD (stochastic gradient descent) and NCA (neighborhood components analysis).

mlpack 1.0.3

2012-09-16
  • Remove internal sparse matrix support because Armadillo 3.4.0 now includes it. When using Armadillo versions older than 3.4.0, sparse matrix support is not available.

  • NCA (neighborhood components analysis) now support an arbitrary optimizer #245, including stochastic gradient descent #249.

mlpack 1.0.2

2012-08-15
  • Added density estimation trees, found in src/mlpack/methods/det/.

  • Added non-negative matrix factorization, found in src/mlpack/methods/nmf/.

  • Added experimental cover tree implementation, found in src/mlpack/core/tree/cover_tree/ #157.

  • Better reporting of boost::program_options errors #225.

  • Fix for timers on Windows (#212, #211).

  • Fix for allknn and allkfn output #204.

  • Sparse coding dictionary initialization is now a template parameter #220.

mlpack 1.0.1

2012-03-03
  • Added kernel principal components analysis (kernel PCA), found in src/mlpack/methods/kernel_pca/ #74.

  • Fix for Lovasz-Theta AugLagrangian tests #182.

  • Fixes for allknn output (#185, #186).

  • Added range search executable #192.

  • Adapted citations in documentation to BibTeX; no citations in -h output #195.

  • Stop use of ‘const char*’ and prefer ‘std::string’ #176.

  • Support seeds for random numbers #177.

mlpack 1.0.0

2011-12-17
  • Initial release. See any resolved tickets numbered less than #196 or execute this query: http://www.mlpack.org/trac/query?status=closed&milestone=mlpack+1.0.0