classification:
regression:
clustering:
preprocessing:
misc. / other:
transformations:
overview
Highquality documentation is a development goal of mlpack. mlpack’s documentation is split into two parts: documentation for the bindings, and documentation for the C++ library. Generally, working with the bindings is a good choice for simple machine learning and data science tasks, and writing C++ is a good idea when complex or custom functionality is desired.
All interfaces are heavily documented, and if you find a documentation issue, please report it.
quickstart
Just getting started with mlpack? Try these quickstart tutorials for the bindings to other languages.
Quickstart Tutorials
These tutorials give very quick “getting started” examples that you can use to get started with mlpack in different languages.
 mlpack in Python quickstart guide
 mlpack commandline quickstart guide
 mlpack in Julia quickstart guide
 mlpack in Go quickstart guide
 mlpack in R quickstart guide
Once you’re comfortable with the quickstart guides for the language of your choice, full documentation for every binding can be found below. Quick links are in the left sidebar.
The C++ interfaces of mlpack are carefully documented and doxygen is used to provide automaticallygenerated searchable documentation.
tutorials
A number of tutorials are available covering individual algorithms and functionality inside of mlpack, both for bindings to other languages and for the C++ interface.
Introductory Tutorials
These tutorials introduce the basic concepts of working with mlpack, aimed at developers who want to use and contribute to mlpack but are not sure where to start.
 Building mlpack From Source
 Building mlpack From Source on Windows
 File formats and loading data in mlpack
 Matrices in mlpack
 Writing an mlpack binding
 mlpack Timers
 Simple Sample mlpack Programs
 Sample C++ ML App for Windows
Methodspecific Tutorials
These tutorials introduce the various methods mlpack offers, aimed at users who want to get started quickly. These tutorials start with simple examples and progress to complex, extensible uses.
 NeighborSearch tutorial (knearestneighbors)
 Linear/ridge regression tutorial (mlpack_linear_regression)
 RangeSearch tutorial (mlpack_range_search)
 Density Estimation Tree (DET) tutorial
 KMeans tutorial (kmeans)
 Fast maxkernel search tutorial (fastmks)
 EMST Tutorial
 Alternating Matrix Factorization tutorial
 Collaborative filtering tutorial
 Approximate furthest neighbor search (mlpack_approx_kfn) tutorial
 Neural Network tutorial
 Reinforcement Learning Tutorial
Advanced Tutorials
These tutorials discuss some of the more advanced functionality contained in mlpack.
Policy Class Documentation
mlpack uses templates to achieve its genericity and flexibility. Some of the template types used by mlpack are common across multiple machine learning algorithms. The links below provide documentation for some of these common types.
mlpack 3.4.2 binding documentation
data formats
mlpack bindings for Go take and return a restricted set of types, for simplicity. These include primitive types, matrix/vector types, categorical matrix types, and model types. Each type is detailed below.
int
: An integer (i.e.,1
).float64
: A floatingpoint number (i.e.,0.5
).bool
: A boolean flag option (true
orfalse
).string
: A character string (i.e.,"hello"
).array of ints
: An array of integers; i.e.,[]int{0, 1, 2}
.array of strings
: An array of strings; i.e.,[]string{"hello", "goodbye"}
.*mat.Dense
: A 2d gonum Matrix. If the type is not alreadyfloat64
, it will be converted.*mat.Dense (with ints)
: A 2d gonum Matrix. If the type is not alreadyint64
, it will be converted.*mat.Dense (1d)
: A 1d gonum Matrix (that is, a Matrix where either the number of rows or number of columns is 1).*mat.Dense (1d with ints)
: A 1d gonum Matrix (that is, a Matrix where either the number of rows or number of columns is 1).matrixWithInfo
: A Tuple(matrixWithInfo) containingfloat64
data (Data) along with a boolean array (Categoricals) indicating which dimensions are categorical (represented bytrue
) and which are numeric (represented byfalse
). The number of elements in the boolean array should be the same as the dimensionality of the data matrix. It is expected that each row of the matrix corresponds to a single data point when calling mlpack bindings.mlpackModel
: An mlpack model pointer. This type holds a pointer to C++ memory containing the mlpack model. Note that this means the mlpack model itself cannot be easily inspected in Go. However, the pointer can be passed to subsequent calls to mlpack functions.
Adaboost()
AdaBoost
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Adaboost().
param := mlpack.AdaboostOptions()
param.InputModel = nil
param.Iterations = 1000
param.Labels = mat.NewDense(1, 1, nil)
param.Test = mat.NewDense(1, 1, nil)
param.Tolerance = 1e10
param.Training = mat.NewDense(1, 1, nil)
param.WeakLearner = "decision_stump"
output, output_model, predictions, probabilities := mlpack.Adaboost(param)
An implementation of the AdaBoost.MH (Adaptive Boosting) algorithm for classification. This can be used to train an AdaBoost model on labeled data or use an existing AdaBoost model to predict the classes of new points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

InputModel 
adaBoostModel 
Input AdaBoost model.  nil 
Iterations 
int 
The maximum number of boosting iterations to be run (0 will run until convergence.)  1000 
Labels 
*mat.Dense (1d with ints) 
Labels for the training set.  mat.NewDense(1, 1, nil) 
Test 
*mat.Dense 
Test dataset.  mat.NewDense(1, 1, nil) 
Tolerance 
float64 
The tolerance for change in values of the weighted error during training.  1e10 
Training 
*mat.Dense 
Dataset for training AdaBoost.  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
WeakLearner 
string 
The type of weak learner to use: ‘decision_stump’, or ‘perceptron’.  "decision_stump" 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense (1d with ints) 
Predicted labels for the test set. 
outputModel 
adaBoostModel 
Output trained AdaBoost model. 
predictions 
*mat.Dense (1d with ints) 
Predicted labels for the test set. 
probabilities 
*mat.Dense 
Predicted class probabilities for each point in the test set. 
Detailed documentation
This program implements the AdaBoost (or Adaptive Boosting) algorithm. The variant of AdaBoost implemented here is AdaBoost.MH. It uses a weak learner, either decision stumps or perceptrons, and over many iterations, creates a strong learner that is a weighted ensemble of weak learners. It runs these iterations until a tolerance value is crossed for change in the value of the weighted training error.
For more information about the algorithm, see the paper “Improved Boosting Algorithms Using ConfidenceRated Predictions”, by R.E. Schapire and Y. Singer.
This program allows training of an AdaBoost model, and then application of that model to a test dataset. To train a model, a dataset must be passed with the Training
option. Labels can be given with the Labels
option; if no labels are specified, the labels will be assumed to be the last column of the input dataset. Alternately, an AdaBoost model may be loaded with the InputModel
option.
Once a model is trained or loaded, it may be used to provide class predictions for a given test dataset. A test dataset may be specified with the Test
parameter. The predicted classes for each point in the test dataset are output to the Predictions
output parameter. The AdaBoost model itself is output to the OutputModel
output parameter.
Note: the following parameter is deprecated and will be removed in mlpack 4.0.0: Output
.
Use Predictions
instead of Output
.
Example
For example, to run AdaBoost on an input dataset data
with labels labels
and perceptrons as the weak learner type, storing the trained model in model
, one could use the following command:
// Initialize optional parameters for Adaboost().
param := mlpack.AdaboostOptions()
param.Training = data
param.Labels = labels
param.WeakLearner = "perceptron"
_, model, _, _ := mlpack.Adaboost(param)
Similarly, an alreadytrained model in model
can be used to provide class predictions from test data test_data
and store the output in predictions
with the following command:
// Initialize optional parameters for Adaboost().
param := mlpack.AdaboostOptions()
param.InputModel = &model
param.Test = test_data
_, _, predictions, _ := mlpack.Adaboost(param)
See also
 AdaBoost on Wikipedia
 Improved boosting algorithms using confidencerated predictions (pdf)
 Perceptron
 Decision Stump
 mlpack::adaboost::AdaBoost C++ class documentation
ApproxKfn()
Approximate furthest neighbor search
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for ApproxKfn().
param := mlpack.ApproxKfnOptions()
param.Algorithm = "ds"
param.CalculateError = false
param.ExactDistances = mat.NewDense(1, 1, nil)
param.InputModel = nil
param.K = 0
param.NumProjections = 5
param.NumTables = 5
param.Query = mat.NewDense(1, 1, nil)
param.Reference = mat.NewDense(1, 1, nil)
distances, neighbors, output_model := mlpack.ApproxKfn(param)
An implementation of two strategies for furthest neighbor search. This can be used to compute the furthest neighbor of query point(s) from a set of points; furthest neighbor models can be saved and reused with future query point(s). Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Algorithm 
string 
Algorithm to use: ‘ds’ or ‘qdafn’.  "ds" 
CalculateError 
bool 
If set, calculate the average distance error for the first furthest neighbor only.  false 
ExactDistances 
*mat.Dense 
Matrix containing exact distances to furthest neighbors; this can be used to avoid explicit calculation when –calculate_error is set.  mat.NewDense(1, 1, nil) 
InputModel 
approxkfnModel 
File containing input model.  nil 
K 
int 
Number of furthest neighbors to search for.  0 
NumProjections 
int 
Number of projections to use in each hash table.  5 
NumTables 
int 
Number of hash tables to use.  5 
Query 
*mat.Dense 
Matrix containing query points.  mat.NewDense(1, 1, nil) 
Reference 
*mat.Dense 
Matrix containing the reference dataset.  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

distances 
*mat.Dense 
Matrix to save furthest neighbor distances to. 
neighbors 
*mat.Dense (with ints) 
Matrix to save neighbor indices to. 
outputModel 
approxkfnModel 
File to save output model to. 
Detailed documentation
This program implements two strategies for furthest neighbor search. These strategies are:
 The ‘qdafn’ algorithm from “Approximate Furthest Neighbor in High Dimensions” by R. Pagh, F. Silvestri, J. Sivertsen, and M. Skala, in Similarity Search and Applications 2015 (SISAP).
 The ‘DrusillaSelect’ algorithm from “Fast approximate furthest neighbors with datadependent candidate selection”, by R.R. Curtin and A.B. Gardner, in Similarity Search and Applications 2016 (SISAP).
These two strategies give approximate results for the furthest neighbor search problem and can be used as fast replacements for other furthest neighbor techniques such as those found in the mlpack_kfn program. Note that typically, the ‘ds’ algorithm requires far fewer tables and projections than the ‘qdafn’ algorithm.
Specify a reference set (set to search in) with Reference
, specify a query set with Query
, and specify algorithm parameters with NumTables
and NumProjections
(or don’t and defaults will be used). The algorithm to be used (either ‘ds’—the default—or ‘qdafn’) may be specified with Algorithm
. Also specify the number of neighbors to search for with K
.
Note that for ‘qdafn’ in lower dimensions, NumProjections
may need to be set to a high value in order to return results for each query point.
If no query set is specified, the reference set will be used as the query set. The OutputModel
output parameter may be used to store the built model, and an input model may be loaded instead of specifying a reference set with the InputModel
option.
Results for each query point can be stored with the Neighbors
and Distances
output parameters. Each row of these output matrices holds the k distances or neighbor indices for each query point.
Example
For example, to find the 5 approximate furthest neighbors with reference_set
as the reference set and query_set
as the query set using DrusillaSelect, storing the furthest neighbor indices to neighbors
and the furthest neighbor distances to distances
, one could call
// Initialize optional parameters for ApproxKfn().
param := mlpack.ApproxKfnOptions()
param.Query = query_set
param.Reference = reference_set
param.K = 5
param.Algorithm = "ds"
distances, neighbors, _ := mlpack.ApproxKfn(param)
and to perform approximate allfurthestneighbors search with k=1 on the set data
storing only the furthest neighbor distances to distances
, one could call
// Initialize optional parameters for ApproxKfn().
param := mlpack.ApproxKfnOptions()
param.Reference = reference_set
param.K = 1
distances, _, _ := mlpack.ApproxKfn(param)
A trained model can be reused. If a model has been previously saved to model
, then we may find 3 approximate furthest neighbors on a query set new_query_set
using that model and store the furthest neighbor indices into neighbors
by calling
// Initialize optional parameters for ApproxKfn().
param := mlpack.ApproxKfnOptions()
param.InputModel = &model
param.Query = new_query_set
param.K = 3
_, neighbors, _ := mlpack.ApproxKfn(param)
See also
 kfurthestneighbor search
 knearestneighbor search
 Fast approximate furthest neighbors with datadependent candidate selection (pdf)
 Approximate furthest neighbor in high dimensions (pdf)
 mlpack::neighbor::QDAFN class documentation
 mlpack::neighbor::DrusillaSelect class documentation
BayesianLinearRegression()
BayesianLinearRegression
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for BayesianLinearRegression().
param := mlpack.BayesianLinearRegressionOptions()
param.Center = false
param.Input = mat.NewDense(1, 1, nil)
param.InputModel = nil
param.Responses = mat.NewDense(1, 1, nil)
param.Scale = false
param.Test = mat.NewDense(1, 1, nil)
output_model, predictions, stds := mlpack.BayesianLinearRegression(param)
An implementation of the bayesian linear regression. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Center 
bool 
Center the data and fit the intercept if enabled.  false 
Input 
*mat.Dense 
Matrix of covariates (X).  mat.NewDense(1, 1, nil) 
InputModel 
bayesianLinearRegression 
Trained BayesianLinearRegression model to use.  nil 
Responses 
*mat.Dense (1d) 
Matrix of responses/observations (y).  mat.NewDense(1, 1, nil) 
Scale 
bool 
Scale each feature by their standard deviations if enabled.  false 
Test 
*mat.Dense 
Matrix containing points to regress on (test points).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
bayesianLinearRegression 
Output BayesianLinearRegression model. 
predictions 
*mat.Dense 
If –test_file is specified, this file is where the predicted responses will be saved. 
stds 
*mat.Dense 
If specified, this is where the standard deviations of the predictive distribution will be saved. 
Detailed documentation
An implementation of the bayesian linear regression. This model is a probabilistic view and implementation of the linear regression. The final solution is obtained by computing a posterior distribution from gaussian likelihood and a zero mean gaussian isotropic prior distribution on the solution. Optimization is AUTOMATIC and does not require cross validation. The optimization is performed by maximization of the evidence function. Parameters are tuned during the maximization of the marginal likelihood. This procedure includes the Ockham’s razor that penalizes over complex solutions.
This program is able to train a Bayesian linear regression model or load a model from file, output regression predictions for a test set, and save the trained model to a file.
To train a BayesianLinearRegression model, the Input
and Responses
parameters must be given. The Center
and Scale
parameters control the centering and the normalizing options. A trained model can be saved with the OutputModel
. If no training is desired at all, a model can be passed via the InputModel
parameter.
The program can also provide predictions for test data using either the trained model or the given input model. Test points can be specified with the Test
parameter. Predicted responses to the test points can be saved with the Predictions
output parameter. The corresponding standard deviation can be save by precising the Stds
parameter.
Example
For example, the following command trains a model on the data data
and responses responses
with center set to true and scale set to false (so, Bayesian linear regression is being solved, and then the model is saved to blr_model
:
// Initialize optional parameters for BayesianLinearRegression().
param := mlpack.BayesianLinearRegressionOptions()
param.Input = data
param.Responses = responses
param.Center = 1
param.Scale = 0
blr_model, _, _ := mlpack.BayesianLinearRegression(param)
The following command uses the blr_model
to provide predicted responses for the data test
and save those responses to test_predictions
:
// Initialize optional parameters for BayesianLinearRegression().
param := mlpack.BayesianLinearRegressionOptions()
param.InputModel = &blr_model
param.Test = test
_, test_predictions, _ := mlpack.BayesianLinearRegression(param)
Because the estimator computes a predictive distribution instead of a simple point estimate, the Stds
parameter allows one to save the prediction uncertainties:
// Initialize optional parameters for BayesianLinearRegression().
param := mlpack.BayesianLinearRegressionOptions()
param.InputModel = &blr_model
param.Test = test
_, test_predictions, stds := mlpack.BayesianLinearRegression(param)
See also
 Bayesian Interpolation
 Bayesian Linear Regression, Section 3.3
 mlpack::regression::BayesianLinearRegression C++ class documentation
Cf()
Collaborative Filtering
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Cf().
param := mlpack.CfOptions()
param.Algorithm = "NMF"
param.AllUserRecommendations = false
param.InputModel = nil
param.Interpolation = "average"
param.IterationOnlyTermination = false
param.MaxIterations = 1000
param.MinResidue = 1e05
param.NeighborSearch = "euclidean"
param.Neighborhood = 5
param.Normalization = "none"
param.Query = mat.NewDense(1, 1, nil)
param.Rank = 0
param.Recommendations = 5
param.Seed = 0
param.Test = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
output, output_model := mlpack.Cf(param)
An implementation of several collaborative filtering (CF) techniques for recommender systems. This can be used to train a new CF model, or use an existing CF model to compute recommendations. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Algorithm 
string 
Algorithm used for matrix factorization.  "NMF" 
AllUserRecommendations 
bool 
Generate recommendations for all users.  false 
InputModel 
cfModel 
Trained CF model to load.  nil 
Interpolation 
string 
Algorithm used for weight interpolation.  "average" 
IterationOnlyTermination 
bool 
Terminate only when the maximum number of iterations is reached.  false 
MaxIterations 
int 
Maximum number of iterations. If set to zero, there is no limit on the number of iterations.  1000 
MinResidue 
float64 
Residue required to terminate the factorization (lower values generally mean better fits).  1e05 
NeighborSearch 
string 
Algorithm used for neighbor search.  "euclidean" 
Neighborhood 
int 
Size of the neighborhood of similar users to consider for each query user.  5 
Normalization 
string 
Normalization performed on the ratings.  "none" 
Query 
*mat.Dense (with ints) 
List of query users for which recommendations should be generated.  mat.NewDense(1, 1, nil) 
Rank 
int 
Rank of decomposed matrices (if 0, a heuristic is used to estimate the rank).  0 
Recommendations 
int 
Number of recommendations to generate for each query user.  5 
Seed 
int 
Set the random seed (0 uses std::time(NULL)).  0 
Test 
*mat.Dense 
Test set to calculate RMSE on.  mat.NewDense(1, 1, nil) 
Training 
*mat.Dense 
Input dataset to perform CF on.  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense (with ints) 
Matrix that will store output recommendations. 
outputModel 
cfModel 
Output for trained CF model. 
Detailed documentation
This program performs collaborative filtering (CF) on the given dataset. Given a list of user, item and preferences (the Training
parameter), the program will perform a matrix decomposition and then can perform a series of actions related to collaborative filtering. Alternately, the program can load an existing saved CF model with the InputModel
parameter and then use that model to provide recommendations or predict values.
The input matrix should be a 3dimensional matrix of ratings, where the first dimension is the user, the second dimension is the item, and the third dimension is that user’s rating of that item. Both the users and items should be numeric indices, not names. The indices are assumed to start from 0.
A set of query users for which recommendations can be generated may be specified with the Query
parameter; alternately, recommendations may be generated for every user in the dataset by specifying the AllUserRecommendations
parameter. In addition, the number of recommendations per user to generate can be specified with the Recommendations
parameter, and the number of similar users (the size of the neighborhood) to be considered when generating recommendations can be specified with the Neighborhood
parameter.
For performing the matrix decomposition, the following optimization algorithms can be specified via the Algorithm
parameter:
 ‘RegSVD’ – Regularized SVD using a SGD optimizer
 ‘NMF’ – Nonnegative matrix factorization with alternating least squares update rules
 ‘BatchSVD’ – SVD batch learning
 ‘SVDIncompleteIncremental’ – SVD incomplete incremental learning
 ‘SVDCompleteIncremental’ – SVD complete incremental learning
 ‘BiasSVD’ – Bias SVD using a SGD optimizer
 ‘SVDPP’ – SVD++ using a SGD optimizer
The following neighbor search algorithms can be specified via the NeighborSearch
parameter:
 ‘cosine’ – Cosine Search Algorithm
 ‘euclidean’ – Euclidean Search Algorithm
 ‘pearson’ – Pearson Search Algorithm
The following weight interpolation algorithms can be specified via the Interpolation
parameter:
 ‘average’ – Average Interpolation Algorithm
 ‘regression’ – Regression Interpolation Algorithm
 ‘similarity’ – Similarity Interpolation Algorithm
The following ranking normalization algorithms can be specified via the Normalization
parameter:
 ‘none’ – No Normalization
 ‘item_mean’ – Item Mean Normalization
 ‘overall_mean’ – Overall Mean Normalization
 ‘user_mean’ – User Mean Normalization
 ‘z_score’ – ZScore Normalization
A trained model may be saved to with the OutputModel
output parameter.
Example
To train a CF model on a dataset training_set
using NMF for decomposition and saving the trained model to model
, one could call:
// Initialize optional parameters for Cf().
param := mlpack.CfOptions()
param.Training = training_set
param.Algorithm = "NMF"
_, model := mlpack.Cf(param)
Then, to use this model to generate recommendations for the list of users in the query set users
, storing 5 recommendations in recommendations
, one could call
// Initialize optional parameters for Cf().
param := mlpack.CfOptions()
param.InputModel = &model
param.Query = users
param.Recommendations = 5
recommendations, _ := mlpack.Cf(param)
See also
 Collaborative filtering tutorial
 Alternating Matrix Factorization tutorial
 Collaborative Filtering on Wikipedia
 Matrix factorization on Wikipedia
 Matrix factorization techniques for recommender systems (pdf)
 mlpack::cf::CFType class documentation
Dbscan()
DBSCAN clustering
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Dbscan().
param := mlpack.DbscanOptions()
param.Epsilon = 1
param.MinSize = 5
param.Naive = false
param.SelectionType = "ordered"
param.SingleMode = false
param.TreeType = "kd"
assignments, centroids := mlpack.Dbscan(input, param)
An implementation of DBSCAN clustering. Given a dataset, this can compute and return a clustering of that dataset. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Epsilon 
float64 
Radius of each range search.  1 
input 
*mat.Dense 
Input dataset to cluster.  required 
MinSize 
int 
Minimum number of points for a cluster.  5 
Naive 
bool 
If set, bruteforce range search (not treebased) will be used.  false 
SelectionType 
string 
If using point selection policy, the type of selection to use (‘ordered’, ‘random’).  "ordered" 
SingleMode 
bool 
If set, singletree range search (not dualtree) will be used.  false 
TreeType 
string 
If using singletree or dualtree search, the type of tree to use (‘kd’, ‘r’, ‘rstar’, ‘x’, ‘hilbertr’, ‘rplus’, ‘rplusplus’, ‘cover’, ‘ball’).  "kd" 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

assignments 
*mat.Dense (1d with ints) 
Output matrix for assignments of each point. 
centroids 
*mat.Dense 
Matrix to save output centroids to. 
Detailed documentation
This program implements the DBSCAN algorithm for clustering using accelerated treebased range search. The type of tree that is used may be parameterized, or bruteforce range search may also be used.
The input dataset to be clustered may be specified with the Input
parameter; the radius of each range search may be specified with the Epsilon
parameters, and the minimum number of points in a cluster may be specified with the MinSize
parameter.
The Assignments
and Centroids
output parameters may be used to save the output of the clustering. Assignments
contains the cluster assignments of each point, and Centroids
contains the centroids of each cluster.
The range search may be controlled with the TreeType
, SingleMode
, and Naive
parameters. TreeType
can control the type of tree used for range search; this can take a variety of values: ‘kd’, ‘r’, ‘rstar’, ‘x’, ‘hilbertr’, ‘rplus’, ‘rplusplus’, ‘cover’, ‘ball’. The SingleMode
parameter will force singletree search (as opposed to the default dualtree search), and ‘Naive
will force bruteforce range search.
Example
An example usage to run DBSCAN on the dataset in input
with a radius of 0.5 and a minimum cluster size of 5 is given below:
// Initialize optional parameters for Dbscan().
param := mlpack.DbscanOptions()
param.Epsilon = 0.5
param.MinSize = 5
_, _ := mlpack.Dbscan(input, param)
See also
 DBSCAN on Wikipedia
 A densitybased algorithm for discovering clusters in large spatial databases with noise (pdf)
 mlpack::dbscan::DBSCAN class documentation
DecisionStump()
Decision Stump
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for DecisionStump().
param := mlpack.DecisionStumpOptions()
param.BucketSize = 6
param.InputModel = nil
param.Labels = mat.NewDense(1, 1, nil)
param.Test = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
output_model, predictions := mlpack.DecisionStump(param)
An implementation of a decision stump, which is a singlelevel decision tree. Given labeled data, a new decision stump can be trained; or, an existing decision stump can be used to classify points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

BucketSize 
int 
The minimum number of training points in each decision stump bucket.  6 
InputModel 
dsModel 
Decision stump model to load.  nil 
Labels 
*mat.Dense (1d with ints) 
Labels for the training set. If not specified, the labels are assumed to be the last row of the training data.  mat.NewDense(1, 1, nil) 
Test 
*mat.Dense 
A dataset to calculate predictions for.  mat.NewDense(1, 1, nil) 
Training 
*mat.Dense 
The dataset to train on.  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
dsModel 
Output decision stump model to save. 
predictions 
*mat.Dense (1d with ints) 
The output matrix that will hold the predicted labels for the test set. 
Detailed documentation
This program implements a decision stump, which is a singlelevel decision tree. The decision stump will split on one dimension of the input data, and will split into multiple buckets. The dimension and bins are selected by maximizing the information gain of the split. Optionally, the minimum number of training points in each bin can be specified with the BucketSize
parameter.
The decision stump is parameterized by a splitting dimension and a vector of values that denote the splitting values of each bin.
This program enables several applications: a decision tree may be trained or loaded, and then that decision tree may be used to classify a given set of test points. The decision tree may also be saved to a file for later usage.
To train a decision stump, training data should be passed with the Training
parameter, and their corresponding labels should be passed with the Labels
option. Optionally, if Labels
is not specified, the labels are assumed to be the last dimension of the training dataset. The BucketSize
parameter controls the minimum number of training points in each decision stump bucket.
For classifying a test set, a decision stump may be loaded with the InputModel
parameter (useful for the situation where a stump has already been trained), and a test set may be specified with the Test
parameter. The predicted labels can be saved with the Predictions
output parameter.
Because decision stumps are trained in batch, retraining does not make sense and thus it is not possible to pass both Training
and InputModel
; instead, simply build a new decision stump with the training data.
After training, a decision stump can be saved with the OutputModel
output parameter. That stump may later be reused in subsequent calls to this program (or others).
See also
 Decision tree
 Decision stumps on Wikipedia
 mlpack::decision_stump::DecisionStump class documentation
DecisionTree()
Decision tree
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for DecisionTree().
param := mlpack.DecisionTreeOptions()
param.InputModel = nil
param.Labels = mat.NewDense(1, 1, nil)
param.MaximumDepth = 0
param.MinimumGainSplit = 1e07
param.MinimumLeafSize = 20
param.PrintTrainingAccuracy = false
param.PrintTrainingError = false
param.Test = mat.NewDense(1, 1, nil)
param.TestLabels = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
param.Weights = mat.NewDense(1, 1, nil)
output_model, predictions, probabilities := mlpack.DecisionTree(param)
An implementation of an ID3style decision tree for classification, which supports categorical data. Given labeled data with numeric or categorical features, a decision tree can be trained and saved; or, an existing decision tree can be used for classification on new points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

InputModel 
decisionTreeModel 
Pretrained decision tree, to be used with test points.  nil 
Labels 
*mat.Dense (1d with ints) 
Training labels.  mat.NewDense(1, 1, nil) 
MaximumDepth 
int 
Maximum depth of the tree (0 means no limit).  0 
MinimumGainSplit 
float64 
Minimum gain for node splitting.  1e07 
MinimumLeafSize 
int 
Minimum number of points in a leaf.  20 
PrintTrainingAccuracy 
bool 
Print the training accuracy.  false 
PrintTrainingError 
bool 
Print the training error (deprecated; will be removed in mlpack 4.0.0).  false 
Test 
matrixWithInfo 
Testing dataset (may be categorical).  mat.NewDense(1, 1, nil) 
TestLabels 
*mat.Dense (1d with ints) 
Test point labels, if accuracy calculation is desired.  mat.NewDense(1, 1, nil) 
Training 
matrixWithInfo 
Training dataset (may be categorical).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Weights 
*mat.Dense 
The weight of labels  mat.NewDense(1, 1, nil) 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
decisionTreeModel 
Output for trained decision tree. 
predictions 
*mat.Dense (1d with ints) 
Class predictions for each test point. 
probabilities 
*mat.Dense 
Class probabilities for each test point. 
Detailed documentation
Train and evaluate using a decision tree. Given a dataset containing numeric or categorical features, and associated labels for each point in the dataset, this program can train a decision tree on that data.
The training set and associated labels are specified with the Training
and Labels
parameters, respectively. The labels should be in the range [0, num_classes  1]. Optionally, if Labels
is not specified, the labels are assumed to be the last dimension of the training dataset.
When a model is trained, the OutputModel
output parameter may be used to save the trained model. A model may be loaded for predictions with the InputModel
parameter. The InputModel
parameter may not be specified when the Training
parameter is specified. The MinimumLeafSize
parameter specifies the minimum number of training points that must fall into each leaf for it to be split. The MinimumGainSplit
parameter specifies the minimum gain that is needed for the node to split. The MaximumDepth
parameter specifies the maximum depth of the tree. If PrintTrainingError
is specified, the training error will be printed.
Test data may be specified with the Test
parameter, and if performance numbers are desired for that test set, labels may be specified with the TestLabels
parameter. Predictions for each test point may be saved via the Predictions
output parameter. Class probabilities for each prediction may be saved with the Probabilities
output parameter.
Example
For example, to train a decision tree with a minimum leaf size of 20 on the dataset contained in data
with labels labels
, saving the output model to tree
and printing the training error, one could call
// Initialize optional parameters for DecisionTree().
param := mlpack.DecisionTreeOptions()
param.Training = data
param.Labels = labels
param.MinimumLeafSize = 20
param.MinimumGainSplit = 0.001
param.PrintTrainingAccuracy = true
tree, _, _ := mlpack.DecisionTree(param)
Then, to use that model to classify points in test_set
and print the test error given the labels test_labels
using that model, while saving the predictions for each point to predictions
, one could call
// Initialize optional parameters for DecisionTree().
param := mlpack.DecisionTreeOptions()
param.InputModel = &tree
param.Test = test_set
param.TestLabels = test_labels
_, predictions, _ := mlpack.DecisionTree(param)
See also
 Decision stump
 Random forest
 Decision trees on Wikipedia
 Induction of Decision Trees (pdf)
 mlpack::tree::DecisionTree class documentation
Det()
Density Estimation With Density Estimation Trees
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Det().
param := mlpack.DetOptions()
param.Folds = 10
param.InputModel = nil
param.MaxLeafSize = 10
param.MinLeafSize = 5
param.PathFormat = "lr"
param.SkipPruning = false
param.Test = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
output_model, tag_counters_file, tag_file, test_set_estimates,
training_set_estimates, vi := mlpack.Det(param)
An implementation of density estimation trees for the density estimation task. Density estimation trees can be trained or used to predict the density at locations given by query points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Folds 
int 
The number of folds of crossvalidation to perform for the estimation (0 is LOOCV)  10 
InputModel 
dTree 
Trained density estimation tree to load.  nil 
MaxLeafSize 
int 
The maximum size of a leaf in the unpruned, fully grown DET.  10 
MinLeafSize 
int 
The minimum size of a leaf in the unpruned, fully grown DET.  5 
PathFormat 
string 
The format of path printing: ‘lr’, ‘idlr’, or ‘lrid’.  "lr" 
SkipPruning 
bool 
Whether to bypass the pruning process and output the unpruned tree only.  false 
Test 
*mat.Dense 
A set of test points to estimate the density of.  mat.NewDense(1, 1, nil) 
Training 
*mat.Dense 
The data set on which to build a density estimation tree.  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
dTree 
Output to save trained density estimation tree to. 
tagCountersFile 
string 
The file to output the number of points that went to each leaf. 
tagFile 
string 
The file to output the tags (and possibly paths) for each sample in the test set. 
testSetEstimates 
*mat.Dense 
The output estimates on the test set from the final optimally pruned tree. 
trainingSetEstimates 
*mat.Dense 
The output density estimates on the training set from the final optimally pruned tree. 
vi 
*mat.Dense 
The output variable importance values for each feature. 
Detailed documentation
This program performs a number of functions related to Density Estimation Trees. The optimal Density Estimation Tree (DET) can be trained on a set of data (specified by Training
) using crossvalidation (with number of folds specified with the Folds
parameter). This trained density estimation tree may then be saved with the OutputModel
output parameter.
The variable importances (that is, the feature importance values for each dimension) may be saved with the Vi
output parameter, and the density estimates for each training point may be saved with the TrainingSetEstimates
output parameter.
Enabling path printing for each node outputs the path from the root node to a leaf for each entry in the test set, or training set (if a test set is not provided). Strings like ‘LRLRLR’ (indicating that traversal went to the left child, then the right child, then the left child, and so forth) will be output. If ‘lrid’ or ‘idlr’ are given as the PathFormat
parameter, then the ID (tag) of every node along the path will be printed after or before the L or R character indicating the direction of traversal, respectively.
This program also can provide density estimates for a set of test points, specified in the Test
parameter. The density estimation tree used for this task will be the tree that was trained on the given training points, or a tree given as the parameter InputModel
. The density estimates for the test points may be saved using the TestSetEstimates
output parameter.
See also
 Density estimation tree (DET) tutorial
 Density estimation on Wikipedia
 Density estimation trees (pdf)
 mlpack::tree::DTree class documentation
Emst()
Fast Euclidean Minimum Spanning Tree
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Emst().
param := mlpack.EmstOptions()
param.LeafSize = 1
param.Naive = false
output := mlpack.Emst(input, param)
An implementation of the DualTree Boruvka algorithm for computing the Euclidean minimum spanning tree of a set of input points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

input 
*mat.Dense 
Input data matrix.  required 
LeafSize 
int 
Leaf size in the kdtree. Oneelement leaves give the empirically best performance, but at the cost of greater memory requirements.  1 
Naive 
bool 
Compute the MST using O(n^2) naive algorithm.  false 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Output data. Stored as an edge list. 
Detailed documentation
This program can compute the Euclidean minimum spanning tree of a set of input points using the dualtree Boruvka algorithm.
The set to calculate the minimum spanning tree of is specified with the Input
parameter, and the output may be saved with the Output
output parameter.
The LeafSize
parameter controls the leaf size of the kdtree that is used to calculate the minimum spanning tree, and if the Naive
option is given, then bruteforce search is used (this is typically much slower in low dimensions). The leaf size does not affect the results, but it may have some effect on the runtime of the algorithm.
Example
For example, the minimum spanning tree of the input dataset data
can be calculated with a leaf size of 20 and stored as spanning_tree
using the following command:
// Initialize optional parameters for Emst().
param := mlpack.EmstOptions()
param.LeafSize = 20
spanning_tree := mlpack.Emst(data, param)
The output matrix is a threedimensional matrix, where each row indicates an edge. The first dimension corresponds to the lesser index of the edge; the second dimension corresponds to the greater index of the edge; and the third column corresponds to the distance between the two points.
See also
 EMST Tutorial
 Minimum spanning tree on Wikipedia
 Fast Euclidean Minimum Spanning Tree: Algorithm, Analysis, and Applications (pdf)
 mlpack::emst::DualTreeBoruvka class documentation
Fastmks()
FastMKS (Fast MaxKernel Search)
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Fastmks().
param := mlpack.FastmksOptions()
param.Bandwidth = 1
param.Base = 2
param.Degree = 2
param.InputModel = nil
param.K = 0
param.Kernel = "linear"
param.Naive = false
param.Offset = 0
param.Query = mat.NewDense(1, 1, nil)
param.Reference = mat.NewDense(1, 1, nil)
param.Scale = 1
param.Single = false
indices, kernels, output_model := mlpack.Fastmks(param)
An implementation of the singletree and dualtree fast maxkernel search (FastMKS) algorithm. Given a set of reference points and a set of query points, this can find the reference point with maximum kernel value for each query point; trained models can be reused for future queries. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Bandwidth 
float64 
Bandwidth (for Gaussian, Epanechnikov, and triangular kernels).  1 
Base 
float64 
Base to use during cover tree construction.  2 
Degree 
float64 
Degree of polynomial kernel.  2 
InputModel 
fastmksModel 
Input FastMKS model to use.  nil 
K 
int 
Number of maximum kernels to find.  0 
Kernel 
string 
Kernel type to use: ‘linear’, ‘polynomial’, ‘cosine’, ‘gaussian’, ‘epanechnikov’, ‘triangular’, ‘hyptan’.  "linear" 
Naive 
bool 
If true, O(n^2) naive mode is used for computation.  false 
Offset 
float64 
Offset of kernel (for polynomial and hyptan kernels).  0 
Query 
*mat.Dense 
The query dataset.  mat.NewDense(1, 1, nil) 
Reference 
*mat.Dense 
The reference dataset.  mat.NewDense(1, 1, nil) 
Scale 
float64 
Scale of kernel (for hyptan kernel).  1 
Single 
bool 
If true, singletree search is used (as opposed to dualtree search.  false 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

indices 
*mat.Dense (with ints) 
Output matrix of indices. 
kernels 
*mat.Dense 
Output matrix of kernels. 
outputModel 
fastmksModel 
Output for FastMKS model. 
Detailed documentation
This program will find the k maximum kernels of a set of points, using a query set and a reference set (which can optionally be the same set). More specifically, for each point in the query set, the k points in the reference set with maximum kernel evaluations are found. The kernel function used is specified with the Kernel
parameter.
Example
For example, the following command will calculate, for each point in the query set query
, the five points in the reference set reference
with maximum kernel evaluation using the linear kernel. The kernel evaluations may be saved with the kernels
output parameter and the indices may be saved with the indices
output parameter.
// Initialize optional parameters for Fastmks().
param := mlpack.FastmksOptions()
param.K = 5
param.Reference = reference
param.Query = query
param.Kernel = "linear"
indices, kernels, _ := mlpack.Fastmks(param)
The output matrices are organized such that row i and column j in the indices matrix corresponds to the index of the point in the reference set that has j’th largest kernel evaluation with the point in the query set with index i. Row i and column j in the kernels matrix corresponds to the kernel evaluation between those two points.
This program performs FastMKS using a cover tree. The base used to build the cover tree can be specified with the Base
parameter.
See also
 Fast maxkernel search tutorial (fastmks)
 knearestneighbor search
 Dualtree Fast Exact MaxKernel Search (pdf)
 mlpack::fastmks::FastMKS class documentation
GmmTrain()
Gaussian Mixture Model (GMM) Training
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for GmmTrain().
param := mlpack.GmmTrainOptions()
param.DiagonalCovariance = false
param.InputModel = nil
param.KmeansMaxIterations = 1000
param.MaxIterations = 250
param.NoForcePositive = false
param.Noise = 0
param.Percentage = 0.02
param.RefinedStart = false
param.Samplings = 100
param.Seed = 0
param.Tolerance = 1e10
param.Trials = 1
output_model := mlpack.GmmTrain(gaussians, input, param)
An implementation of the EM algorithm for training Gaussian mixture models (GMMs). Given a dataset, this can train a GMM for future use with other tools. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

DiagonalCovariance 
bool 
Force the covariance of the Gaussians to be diagonal. This can accelerate training time significantly.  false 
gaussians 
int 
Number of Gaussians in the GMM.  required 
input 
*mat.Dense 
The training data on which the model will be fit.  required 
InputModel 
gmm 
Initial input GMM model to start training with.  nil 
KmeansMaxIterations 
int 
Maximum number of iterations for the kmeans algorithm (used to initialize EM).  1000 
MaxIterations 
int 
Maximum number of iterations of EM algorithm (passing 0 will run until convergence).  250 
NoForcePositive 
bool 
Do not force the covariance matrices to be positive definite.  false 
Noise 
float64 
Variance of zeromean Gaussian noise to add to data.  0 
Percentage 
float64 
If using –refined_start, specify the percentage of the dataset used for each sampling (should be between 0.0 and 1.0).  0.02 
RefinedStart 
bool 
During the initialization, use refined initial positions for kmeans clustering (Bradley and Fayyad, 1998).  false 
Samplings 
int 
If using –refined_start, specify the number of samplings used for initial points.  100 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
Tolerance 
float64 
Tolerance for convergence of EM.  1e10 
Trials 
int 
Number of trials to perform in training GMM.  1 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
gmm 
Output for trained GMM model. 
Detailed documentation
This program takes a parametric estimate of a Gaussian mixture model (GMM) using the EM algorithm to find the maximum likelihood estimate. The model may be saved and reused by other mlpack GMM tools.
The input data to train on must be specified with the Input
parameter, and the number of Gaussians in the model must be specified with the Gaussians
parameter. Optionally, many trials with different random initializations may be run, and the result with highest loglikelihood on the training data will be taken. The number of trials to run is specified with the Trials
parameter. By default, only one trial is run.
The tolerance for convergence and maximum number of iterations of the EM algorithm are specified with the Tolerance
and MaxIterations
parameters, respectively. The GMM may be initialized for training with another model, specified with the InputModel
parameter. Otherwise, the model is initialized by running kmeans on the data. The kmeans clustering initialization can be controlled with the KmeansMaxIterations
, RefinedStart
, Samplings
, and Percentage
parameters. If RefinedStart
is specified, then the BradleyFayyad refined start initialization will be used. This can often lead to better clustering results.
The ‘diagonal_covariance’ flag will cause the learned covariances to be diagonal matrices. This significantly simplifies the model itself and causes training to be faster, but restricts the ability to fit more complex GMMs.
If GMM training fails with an error indicating that a covariance matrix could not be inverted, make sure that the NoForcePositive
parameter is not specified. Alternately, adding a small amount of Gaussian noise (using the Noise
parameter) to the entire dataset may help prevent Gaussians with zero variance in a particular dimension, which is usually the cause of noninvertible covariance matrices.
The NoForcePositive
parameter, if set, will avoid the checks after each iteration of the EM algorithm which ensure that the covariance matrices are positive definite. Specifying the flag can cause faster runtime, but may also cause nonpositive definite covariance matrices, which will cause the program to crash.
Example
As an example, to train a 6Gaussian GMM on the data in data
with a maximum of 100 iterations of EM and 3 trials, saving the trained GMM to gmm
, the following command can be used:
// Initialize optional parameters for GmmTrain().
param := mlpack.GmmTrainOptions()
param.Trials = 3
gmm := mlpack.GmmTrain(data, 6, param)
To retrain that GMM on another set of data data2
, the following command may be used:
// Initialize optional parameters for GmmTrain().
param := mlpack.GmmTrainOptions()
param.InputModel = &gmm
new_gmm := mlpack.GmmTrain(data2, 6, param)
See also
 GmmGenerate()
 GmmProbability()
 Gaussian Mixture Models on Wikipedia
 mlpack::gmm::GMM class documentation
GmmGenerate()
GMM Sample Generator
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for GmmGenerate().
param := mlpack.GmmGenerateOptions()
param.Seed = 0
output := mlpack.GmmGenerate(inputModel, samples, param)
A sample generator for pretrained GMMs. Given a pretrained GMM, this can sample new points randomly from that distribution. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

inputModel 
gmm 
Input GMM model to generate samples from.  required 
samples 
int 
Number of samples to generate.  required 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Matrix to save output samples in. 
Detailed documentation
This program is able to generate samples from a pretrained GMM (use gmm_train to train a GMM). The pretrained GMM must be specified with the InputModel
parameter. The number of samples to generate is specified by the Samples
parameter. Output samples may be saved with the Output
output parameter.
Example
The following command can be used to generate 100 samples from the pretrained GMM gmm
and store those generated samples in samples
:
// Initialize optional parameters for GmmGenerate().
param := mlpack.GmmGenerateOptions()
samples := mlpack.GmmGenerate(&gmm, 100, param)
See also
 GmmTrain()
 GmmProbability()
 Gaussian Mixture Models on Wikipedia
 mlpack::gmm::GMM class documentation
GmmProbability()
GMM Probability Calculator
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for GmmProbability().
param := mlpack.GmmProbabilityOptions()
output := mlpack.GmmProbability(input, inputModel, )
A probability calculator for GMMs. Given a pretrained GMM and a set of points, this can compute the probability that each point is from the given GMM. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

input 
*mat.Dense 
Input matrix to calculate probabilities of.  required 
inputModel 
gmm 
Input GMM to use as model.  required 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Matrix to store calculated probabilities in. 
Detailed documentation
This program calculates the probability that given points came from a given GMM (that is, P(X  gmm)). The GMM is specified with the InputModel
parameter, and the points are specified with the Input
parameter. The output probabilities may be saved via the Output
output parameter.
Example
So, for example, to calculate the probabilities of each point in points
coming from the pretrained GMM gmm
, while storing those probabilities in probs
, the following command could be used:
// Initialize optional parameters for GmmProbability().
param := mlpack.GmmProbabilityOptions()
probs := mlpack.GmmProbability(&gmm, points, param)
See also
HmmTrain()
Hidden Markov Model (HMM) Training
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for HmmTrain().
param := mlpack.HmmTrainOptions()
param.Batch = false
param.Gaussians = 0
param.InputModel = nil
param.LabelsFile = ""
param.Seed = 0
param.States = 0
param.Tolerance = 1e05
param.Type = "gaussian"
output_model := mlpack.HmmTrain(inputFile, param)
An implementation of training algorithms for Hidden Markov Models (HMMs). Given labeled or unlabeled data, an HMM can be trained for further use with other mlpack HMM tools. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Batch 
bool 
If true, input_file (and if passed, labels_file) are expected to contain a list of files to use as input observation sequences (and label sequences).  false 
Gaussians 
int 
Number of gaussians in each GMM (necessary when type is ‘gmm’).  0 
inputFile 
string 
File containing input observations.  required 
InputModel 
hmmModel 
Preexisting HMM model to initialize training with.  nil 
LabelsFile 
string 
Optional file of hidden states, used for labeled training.  "" 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
States 
int 
Number of hidden states in HMM (necessary, unless model_file is specified).  0 
Tolerance 
float64 
Tolerance of the BaumWelch algorithm.  1e05 
Type 
string 
Type of HMM: discrete  gaussian  diag_gmm  gmm.  "gaussian" 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
hmmModel 
Output for trained HMM. 
Detailed documentation
This program allows a Hidden Markov Model to be trained on labeled or unlabeled data. It supports four types of HMMs: Discrete HMMs, Gaussian HMMs, GMM HMMs, or Diagonal GMM HMMs
Either one input sequence can be specified (with InputFile
), or, a file containing files in which input sequences can be found (when InputFile
andBatch
are used together). In addition, labels can be provided in the file specified by LabelsFile
, and if Batch
is used, the file given to LabelsFile
should contain a list of files of labels corresponding to the sequences in the file given to InputFile
.
The HMM is trained with the BaumWelch algorithm if no labels are provided. The tolerance of the BaumWelch algorithm can be set with the Tolerance
option. By default, the transition matrix is randomly initialized and the emission distributions are initialized to fit the extent of the data.
Optionally, a precreated HMM model can be used as a guess for the transition matrix and emission probabilities; this is specifiable with OutputModel
.
See also
 HmmGenerate()
 HmmLoglik()
 HmmViterbi()
 Hidden Mixture Models on Wikipedia
 mlpack::hmm::HMM class documentation
HmmLoglik()
Hidden Markov Model (HMM) Sequence LogLikelihood
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for HmmLoglik().
param := mlpack.HmmLoglikOptions()
log_likelihood := mlpack.HmmLoglik(input, inputModel, )
A utility for computing the loglikelihood of a sequence for Hidden Markov Models (HMMs). Given a pretrained HMM and an observation sequence, this computes and returns the loglikelihood of that sequence being observed from that HMM. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

input 
*mat.Dense 
File containing observations,  required 
inputModel 
hmmModel 
File containing HMM.  required 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

logLikelihood 
float64 
Loglikelihood of the sequence. 
Detailed documentation
This utility takes an alreadytrained HMM, specified with the InputModel
parameter, and evaluates the loglikelihood of a sequence of observations, given with the Input
parameter. The computed loglikelihood is given as output.
Example
For example, to compute the loglikelihood of the sequence seq
with the pretrained HMM hmm
, the following command may be used:
// Initialize optional parameters for HmmLoglik().
param := mlpack.HmmLoglikOptions()
_ := mlpack.HmmLoglik(seq, &hmm, param)
See also
 HmmTrain()
 HmmGenerate()
 HmmViterbi()
 Hidden Mixture Models on Wikipedia
 mlpack::hmm::HMM class documentation
HmmViterbi()
Hidden Markov Model (HMM) Viterbi State Prediction
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for HmmViterbi().
param := mlpack.HmmViterbiOptions()
output := mlpack.HmmViterbi(input, inputModel, )
A utility for computing the most probable hidden state sequence for Hidden Markov Models (HMMs). Given a pretrained HMM and an observed sequence, this uses the Viterbi algorithm to compute and return the most probable hidden state sequence. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

input 
*mat.Dense 
Matrix containing observations,  required 
inputModel 
hmmModel 
Trained HMM to use.  required 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense (with ints) 
File to save predicted state sequence to. 
Detailed documentation
This utility takes an alreadytrained HMM, specified as InputModel
, and evaluates the most probable hidden state sequence of a given sequence of observations (specified as ‘Input
, using the Viterbi algorithm. The computed state sequence may be saved using the Output
output parameter.
Example
For example, to predict the state sequence of the observations obs
using the HMM hmm
, storing the predicted state sequence to states
, the following command could be used:
// Initialize optional parameters for HmmViterbi().
param := mlpack.HmmViterbiOptions()
states := mlpack.HmmViterbi(obs, &hmm, param)
See also
 HmmTrain()
 HmmGenerate()
 HmmLoglik()
 Hidden Mixture Models on Wikipedia
 mlpack::hmm::HMM class documentation
HmmGenerate()
Hidden Markov Model (HMM) Sequence Generator
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for HmmGenerate().
param := mlpack.HmmGenerateOptions()
param.Seed = 0
param.StartState = 0
output, state := mlpack.HmmGenerate(length, model, param)
A utility to generate random sequences from a pretrained Hidden Markov Model (HMM). The length of the desired sequence can be specified, and a random sequence of observations is returned. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

length 
int 
Length of sequence to generate.  required 
model 
hmmModel 
Trained HMM to generate sequences with.  required 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
StartState 
int 
Starting state of sequence.  0 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Matrix to save observation sequence to. 
state 
*mat.Dense (with ints) 
Matrix to save hidden state sequence to. 
Detailed documentation
This utility takes an alreadytrained HMM, specified as the Model
parameter, and generates a random observation sequence and hidden state sequence based on its parameters. The observation sequence may be saved with the Output
output parameter, and the internal state sequence may be saved with the State
output parameter.
The state to start the sequence in may be specified with the StartState
parameter.
Example
For example, to generate a sequence of length 150 from the HMM hmm
and save the observation sequence to observations
and the hidden state sequence to states
, the following command may be used:
// Initialize optional parameters for HmmGenerate().
param := mlpack.HmmGenerateOptions()
observations, states := mlpack.HmmGenerate(&hmm, 150, param)
See also
 HmmTrain()
 HmmLoglik()
 HmmViterbi()
 Hidden Mixture Models on Wikipedia
 mlpack::hmm::HMM class documentation
HoeffdingTree()
Hoeffding trees
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for HoeffdingTree().
param := mlpack.HoeffdingTreeOptions()
param.BatchMode = false
param.Bins = 10
param.Confidence = 0.95
param.InfoGain = false
param.InputModel = nil
param.Labels = mat.NewDense(1, 1, nil)
param.MaxSamples = 5000
param.MinSamples = 100
param.NumericSplitStrategy = "binary"
param.ObservationsBeforeBinning = 100
param.Passes = 1
param.Test = mat.NewDense(1, 1, nil)
param.TestLabels = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
output_model, predictions, probabilities := mlpack.HoeffdingTree(param)
An implementation of Hoeffding trees, a form of streaming decision tree for classification. Given labeled data, a Hoeffding tree can be trained and saved for later use, or a pretrained Hoeffding tree can be used for predicting the classifications of new points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

BatchMode 
bool 
If true, samples will be considered in batch instead of as a stream. This generally results in better trees but at the cost of memory usage and runtime.  false 
Bins 
int 
If the ‘domingos’ split strategy is used, this specifies the number of bins for each numeric split.  10 
Confidence 
float64 
Confidence before splitting (between 0 and 1).  0.95 
InfoGain 
bool 
If set, information gain is used instead of Gini impurity for calculating Hoeffding bounds.  false 
InputModel 
hoeffdingTreeModel 
Input trained Hoeffding tree model.  nil 
Labels 
*mat.Dense (1d with ints) 
Labels for training dataset.  mat.NewDense(1, 1, nil) 
MaxSamples 
int 
Maximum number of samples before splitting.  5000 
MinSamples 
int 
Minimum number of samples before splitting.  100 
NumericSplitStrategy 
string 
The splitting strategy to use for numeric features: ‘domingos’ or ‘binary’.  "binary" 
ObservationsBeforeBinning 
int 
If the ‘domingos’ split strategy is used, this specifies the number of samples observed before binning is performed.  100 
Passes 
int 
Number of passes to take over the dataset.  1 
Test 
matrixWithInfo 
Testing dataset (may be categorical).  mat.NewDense(1, 1, nil) 
TestLabels 
*mat.Dense (1d with ints) 
Labels of test data.  mat.NewDense(1, 1, nil) 
Training 
matrixWithInfo 
Training dataset (may be categorical).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
hoeffdingTreeModel 
Output for trained Hoeffding tree model. 
predictions 
*mat.Dense (1d with ints) 
Matrix to output label predictions for test data into. 
probabilities 
*mat.Dense 
In addition to predicting labels, provide rediction probabilities in this matrix. 
Detailed documentation
This program implements Hoeffding trees, a form of streaming decision tree suited best for large (or streaming) datasets. This program supports both categorical and numeric data. Given an input dataset, this program is able to train the tree with numerous training options, and save the model to a file. The program is also able to use a trained model or a model from file in order to predict classes for a given test set.
The training file and associated labels are specified with the Training
and Labels
parameters, respectively. Optionally, if Labels
is not specified, the labels are assumed to be the last dimension of the training dataset.
The training may be performed in batch mode (like a typical decision tree algorithm) by specifying the BatchMode
option, but this may not be the best option for large datasets.
When a model is trained, it may be saved via the OutputModel
output parameter. A model may be loaded from file for further training or testing with the InputModel
parameter.
Test data may be specified with the Test
parameter, and if performance statistics are desired for that test set, labels may be specified with the TestLabels
parameter. Predictions for each test point may be saved with the Predictions
output parameter, and class probabilities for each prediction may be saved with the Probabilities
output parameter.
Example
For example, to train a Hoeffding tree with confidence 0.99 with data dataset
, saving the trained tree to tree
, the following command may be used:
// Initialize optional parameters for HoeffdingTree().
param := mlpack.HoeffdingTreeOptions()
param.Training = dataset
param.Confidence = 0.99
tree, _, _ := mlpack.HoeffdingTree(param)
Then, this tree may be used to make predictions on the test set test_set
, saving the predictions into predictions
and the class probabilities into class_probs
with the following command:
// Initialize optional parameters for HoeffdingTree().
param := mlpack.HoeffdingTreeOptions()
param.InputModel = &tree
param.Test = test_set
_, predictions, class_probs := mlpack.HoeffdingTree(param)
See also
 DecisionTree()
 RandomForest()
 Mining HighSpeed Data Streams (pdf)
 mlpack::tree::HoeffdingTree class documentation
Kde()
Kernel Density Estimation
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Kde().
param := mlpack.KdeOptions()
param.AbsError = 0
param.Algorithm = "dualtree"
param.Bandwidth = 1
param.InitialSampleSize = 100
param.InputModel = nil
param.Kernel = "gaussian"
param.McBreakCoef = 0.4
param.McEntryCoef = 3
param.McProbability = 0.95
param.MonteCarlo = false
param.Query = mat.NewDense(1, 1, nil)
param.Reference = mat.NewDense(1, 1, nil)
param.RelError = 0.05
param.Tree = "kdtree"
output_model, predictions := mlpack.Kde(param)
An implementation of kernel density estimation with dualtree algorithms. Given a set of reference points and query points and a kernel function, this can estimate the density function at the location of each query point using trees; trees that are built can be saved for later use. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

AbsError 
float64 
Relative error tolerance for the prediction.  0 
Algorithm 
string 
Algorithm to use for the prediction.(‘dualtree’, ‘singletree’).  "dualtree" 
Bandwidth 
float64 
Bandwidth of the kernel.  1 
InitialSampleSize 
int 
Initial sample size for Monte Carlo estimations.  100 
InputModel 
kdeModel 
Contains pretrained KDE model.  nil 
Kernel 
string 
Kernel to use for the prediction.(‘gaussian’, ‘epanechnikov’, ‘laplacian’, ‘spherical’, ‘triangular’).  "gaussian" 
McBreakCoef 
float64 
Controls what fraction of the amount of node’s descendants is the limit for the sample size before it recurses.  0.4 
McEntryCoef 
float64 
Controls how much larger does the amount of node descendants has to be compared to the initial sample size in order to be a candidate for Monte Carlo estimations.  3 
McProbability 
float64 
Probability of the estimation being bounded by relative error when using Monte Carlo estimations.  0.95 
MonteCarlo 
bool 
Whether to use Monte Carlo estimations when possible.  false 
Query 
*mat.Dense 
Query dataset to KDE on.  mat.NewDense(1, 1, nil) 
Reference 
*mat.Dense 
Input reference dataset use for KDE.  mat.NewDense(1, 1, nil) 
RelError 
float64 
Relative error tolerance for the prediction.  0.05 
Tree 
string 
Tree to use for the prediction.(‘kdtree’, ‘balltree’, ‘covertree’, ‘octree’, ‘rtree’).  "kdtree" 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
kdeModel 
If specified, the KDE model will be saved here. 
predictions 
*mat.Dense (1d) 
Vector to store density predictions. 
Detailed documentation
This program performs a Kernel Density Estimation. KDE is a nonparametric way of estimating probability density function. For each query point the program will estimate its probability density by applying a kernel function to each reference point. The computational complexity of this is O(N^2) where there are N query points and N reference points, but this implementation will typically see better performance as it uses an approximate dual or single tree algorithm for acceleration.
Dual or single tree optimization avoids many barely relevant calculations (as kernel function values decrease with distance), so it is an approximate computation. You can specify the maximum relative error tolerance for each query value with RelError
as well as the maximum absolute error tolerance with the parameter AbsError
. This program runs using an Euclidean metric. Kernel function can be selected using the Kernel
option. You can also choose what which type of tree to use for the dualtree algorithm with Tree
. It is also possible to select whether to use dualtree algorithm or singletree algorithm using the Algorithm
option.
Monte Carlo estimations can be used to accelerate the KDE estimate when the Gaussian Kernel is used. This provides a probabilistic guarantee on the the error of the resulting KDE instead of an absolute guarantee.To enable Monte Carlo estimations, the MonteCarlo
flag can be used, and success probability can be set with the McProbability
option. It is possible to set the initial sample size for the Monte Carlo estimation using InitialSampleSize
. This implementation will only consider a node, as a candidate for the Monte Carlo estimation, if its number of descendant nodes is bigger than the initial sample size. This can be controlled using a coefficient that will multiply the initial sample size and can be set using McEntryCoef
. To avoid using the same amount of computations an exact approach would take, this program recurses the tree whenever a fraction of the amount of the node’s descendant points have already been computed. This fraction is set using McBreakCoef
.
Example
For example, the following will run KDE using the data in ref_data
for training and the data in qu_data
as query data. It will apply an Epanechnikov kernel with a 0.2 bandwidth to each reference point and use a KDTree for the dualtree optimization. The returned predictions will be within 5% of the real KDE value for each query point.
// Initialize optional parameters for Kde().
param := mlpack.KdeOptions()
param.Reference = ref_data
param.Query = qu_data
param.Bandwidth = 0.2
param.Kernel = "epanechnikov"
param.Tree = "kdtree"
param.RelError = 0.05
_, out_data := mlpack.Kde(param)
the predicted density estimations will be stored in out_data
.
If no Query
is provided, then KDE will be computed on the Reference
dataset.
It is possible to select either a reference dataset or an input model but not both at the same time. If an input model is selected and parameter values are not set (e.g. Bandwidth
) then default parameter values will be used.
In addition to the last program call, it is also possible to activate Monte Carlo estimations if a Gaussian kernel is used. This can provide faster results, but the KDE will only have a probabilistic guarantee of meeting the desired error bound (instead of an absolute guarantee). The following example will run KDE using a Monte Carlo estimation when possible. The results will be within a 5% of the real KDE value with a 95% probability. Initial sample size for the Monte Carlo estimation will be 200 points and a node will be a candidate for the estimation only when it contains 700 (i.e. 3.5200) points. If a node contains 700 points and 420 (i.e. 0.6700) have already been sampled, then the algorithm will recurse instead of keep sampling.
// Initialize optional parameters for Kde().
param := mlpack.KdeOptions()
param.Reference = ref_data
param.Query = qu_data
param.Bandwidth = 0.2
param.Kernel = "gaussian"
param.Tree = "kdtree"
param.RelError = 0.05
param.MonteCarlo =
param.McProbability = 0.95
param.InitialSampleSize = 200
param.McEntryCoef = 3.5
param.McBreakCoef = 0.6
_, out_data := mlpack.Kde(param)
See also
 Knn()
 Kernel density estimation on Wikipedia
 TreeIndependent DualTree Algorithms
 Fast Highdimensional Kernel Summations Using the Monte Carlo Multipole Method
 mlpack::kde::KDE C++ class documentation
KernelPca()
Kernel Principal Components Analysis
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for KernelPca().
param := mlpack.KernelPcaOptions()
param.Bandwidth = 1
param.Center = false
param.Degree = 1
param.KernelScale = 1
param.NewDimensionality = 0
param.NystroemMethod = false
param.Offset = 0
param.Sampling = "kmeans"
output := mlpack.KernelPca(input, kernel, param)
An implementation of Kernel Principal Components Analysis (KPCA). This can be used to perform nonlinear dimensionality reduction or preprocessing on a given dataset. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Bandwidth 
float64 
Bandwidth, for ‘gaussian’ and ‘laplacian’ kernels.  1 
Center 
bool 
If set, the transformed data will be centered about the origin.  false 
Degree 
float64 
Degree of polynomial, for ‘polynomial’ kernel.  1 
input 
*mat.Dense 
Input dataset to perform KPCA on.  required 
kernel 
string 
The kernel to use; see the above documentation for the list of usable kernels.  required 
KernelScale 
float64 
Scale, for ‘hyptan’ kernel.  1 
NewDimensionality 
int 
If not 0, reduce the dimensionality of the output dataset by ignoring the dimensions with the smallest eigenvalues.  0 
NystroemMethod 
bool 
If set, the Nystroem method will be used.  false 
Offset 
float64 
Offset, for ‘hyptan’ and ‘polynomial’ kernels.  0 
Sampling 
string 
Sampling scheme to use for the Nystroem method: ‘kmeans’, ‘random’, ‘ordered’  "kmeans" 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Matrix to save modified dataset to. 
Detailed documentation
This program performs Kernel Principal Components Analysis (KPCA) on the specified dataset with the specified kernel. This will transform the data onto the kernel principal components, and optionally reduce the dimensionality by ignoring the kernel principal components with the smallest eigenvalues.
For the case where a linear kernel is used, this reduces to regular PCA.
The kernels that are supported are listed below:

‘linear’: the standard linear dot product (same as normal PCA): K(x, y) = x^T y

‘gaussian’: a Gaussian kernel; requires bandwidth: K(x, y) = exp(( x  y  ^ 2) / (2 * (bandwidth ^ 2)))

‘polynomial’: polynomial kernel; requires offset and degree: K(x, y) = (x^T y + offset) ^ degree

‘hyptan’: hyperbolic tangent kernel; requires scale and offset: K(x, y) = tanh(scale * (x^T y) + offset)

‘laplacian’: Laplacian kernel; requires bandwidth: K(x, y) = exp(( x  y ) / bandwidth)

‘epanechnikov’: Epanechnikov kernel; requires bandwidth: K(x, y) = max(0, 1   x  y ^2 / bandwidth^2)

‘cosine’: cosine distance: K(x, y) = 1  (x^T y) / ( x  *  y )
The parameters for each of the kernels should be specified with the options Bandwidth
, KernelScale
, Offset
, or Degree
(or a combination of those parameters).
Optionally, the Nystroem method (“Using the Nystroem method to speed up kernel machines”, 2001) can be used to calculate the kernel matrix by specifying the NystroemMethod
parameter. This approach works by using a subset of the data as basis to reconstruct the kernel matrix; to specify the sampling scheme, the Sampling
parameter is used. The sampling scheme for the Nystroem method can be chosen from the following list: ‘kmeans’, ‘random’, ‘ordered’.
Example
For example, the following command will perform KPCA on the dataset input
using the Gaussian kernel, and saving the transformed data to transformed
:
// Initialize optional parameters for KernelPca().
param := mlpack.KernelPcaOptions()
transformed := mlpack.KernelPca(input, "gaussian", param)
See also
 Kernel principal component analysis on Wikipedia
 Kernel Principal Component Analysis (pdf)
 mlpack::kpca::KernelPCA class documentation
Kmeans()
KMeans Clustering
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Kmeans().
param := mlpack.KmeansOptions()
param.Algorithm = "naive"
param.AllowEmptyClusters = false
param.InPlace = false
param.InitialCentroids = mat.NewDense(1, 1, nil)
param.KillEmptyClusters = false
param.LabelsOnly = false
param.MaxIterations = 1000
param.Percentage = 0.02
param.RefinedStart = false
param.Samplings = 100
param.Seed = 0
centroid, output := mlpack.Kmeans(clusters, input, param)
An implementation of several strategies for efficient kmeans clustering. Given a dataset and a value of k, this computes and returns a kmeans clustering on that data. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Algorithm 
string 
Algorithm to use for the Lloyd iteration (‘naive’, ‘pellegmoore’, ‘elkan’, ‘hamerly’, ‘dualtree’, or ‘dualtreecovertree’).  "naive" 
AllowEmptyClusters 
bool 
Allow empty clusters to be persist.  false 
clusters 
int 
Number of clusters to find (0 autodetects from initial centroids).  required 
InPlace 
bool 
If specified, a column containing the learned cluster assignments will be added to the input dataset file. In this case, –output_file is overridden. (Do not use in Python.)  false 
InitialCentroids 
*mat.Dense 
Start with the specified initial centroids.  mat.NewDense(1, 1, nil) 
input 
*mat.Dense 
Input dataset to perform clustering on.  required 
KillEmptyClusters 
bool 
Remove empty clusters when they occur.  false 
LabelsOnly 
bool 
Only output labels into output file.  false 
MaxIterations 
int 
Maximum number of iterations before kmeans terminates.  1000 
Percentage 
float64 
Percentage of dataset to use for each refined start sampling (use when –refined_start is specified).  0.02 
RefinedStart 
bool 
Use the refined initial point strategy by Bradley and Fayyad to choose initial points.  false 
Samplings 
int 
Number of samplings to perform for refined start (use when –refined_start is specified).  100 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

centroid 
*mat.Dense 
If specified, the centroids of each cluster will be written to the given file. 
output 
*mat.Dense 
Matrix to store output labels or labeled data to. 
Detailed documentation
This program performs KMeans clustering on the given dataset. It can return the learned cluster assignments, and the centroids of the clusters. Empty clusters are not allowed by default; when a cluster becomes empty, the point furthest from the centroid of the cluster with maximum variance is taken to fill that cluster.
Optionally, the Bradley and Fayyad approach (“Refining initial points for kmeans clustering”, 1998) can be used to select initial points by specifying the RefinedStart
parameter. This approach works by taking random samplings of the dataset; to specify the number of samplings, the Samplings
parameter is used, and to specify the percentage of the dataset to be used in each sample, the Percentage
parameter is used (it should be a value between 0.0 and 1.0).
There are several options available for the algorithm used for each Lloyd iteration, specified with the Algorithm
option. The standard O(kN) approach can be used (‘naive’). Other options include the PellegMoore treebased algorithm (‘pellegmoore’), Elkan’s triangleinequality based algorithm (‘elkan’), Hamerly’s modification to Elkan’s algorithm (‘hamerly’), the dualtree kmeans algorithm (‘dualtree’), and the dualtree kmeans algorithm using the cover tree (‘dualtreecovertree’).
The behavior for when an empty cluster is encountered can be modified with the AllowEmptyClusters
option. When this option is specified and there is a cluster owning no points at the end of an iteration, that cluster’s centroid will simply remain in its position from the previous iteration. If the KillEmptyClusters
option is specified, then when a cluster owns no points at the end of an iteration, the cluster centroid is simply filled with DBL_MAX, killing it and effectively reducing k for the rest of the computation. Note that the default option when neither empty cluster option is specified can be timeconsuming to calculate; therefore, specifying either of these parameters will often accelerate runtime.
Initial clustering assignments may be specified using the InitialCentroids
parameter, and the maximum number of iterations may be specified with the MaxIterations
parameter.
Example
As an example, to use Hamerly’s algorithm to perform kmeans clustering with k=10 on the dataset data
, saving the centroids to centroids
and the assignments for each point to assignments
, the following command could be used:
// Initialize optional parameters for Kmeans().
param := mlpack.KmeansOptions()
centroids, assignments := mlpack.Kmeans(data, 10, param)
To run kmeans on that same dataset with initial centroids specified in initial
with a maximum of 500 iterations, storing the output centroids in final
the following command may be used:
// Initialize optional parameters for Kmeans().
param := mlpack.KmeansOptions()
param.InitialCentroids = initial
param.MaxIterations = 500
final, _ := mlpack.Kmeans(data, 10, param)
See also
 KMeans tutorial
 Dbscan()
 Using the triangle inequality to accelerate kmeans (pdf)
 Making kmeans even faster (pdf)
 Accelerating exact kmeans algorithms with geometric reasoning (pdf)
 A dualtree algorithm for fast kmeans clustering with large k (pdf)
 mlpack::kmeans::KMeans class documentation
Lars()
LARS
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Lars().
param := mlpack.LarsOptions()
param.Input = mat.NewDense(1, 1, nil)
param.InputModel = nil
param.Lambda1 = 0
param.Lambda2 = 0
param.Responses = mat.NewDense(1, 1, nil)
param.Test = mat.NewDense(1, 1, nil)
param.UseCholesky = false
output_model, output_predictions := mlpack.Lars(param)
An implementation of Least Angle Regression (Stagewise/laSso), also known as LARS. This can train a LARS/LASSO/Elastic Net model and use that model or a pretrained model to output regression predictions for a test set. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Input 
*mat.Dense 
Matrix of covariates (X).  mat.NewDense(1, 1, nil) 
InputModel 
lars 
Trained LARS model to use.  nil 
Lambda1 
float64 
Regularization parameter for l1norm penalty.  0 
Lambda2 
float64 
Regularization parameter for l2norm penalty.  0 
Responses 
*mat.Dense 
Matrix of responses/observations (y).  mat.NewDense(1, 1, nil) 
Test 
*mat.Dense 
Matrix containing points to regress on (test points).  mat.NewDense(1, 1, nil) 
UseCholesky 
bool 
Use Cholesky decomposition during computation rather than explicitly computing the full Gram matrix.  false 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
lars 
Output LARS model. 
outputPredictions 
*mat.Dense 
If –test_file is specified, this file is where the predicted responses will be saved. 
Detailed documentation
An implementation of LARS: Least Angle Regression (Stagewise/laSso). This is a stagewise homotopybased algorithm for L1regularized linear regression (LASSO) and L1+L2regularized linear regression (Elastic Net).
This program is able to train a LARS/LASSO/Elastic Net model or load a model from file, output regression predictions for a test set, and save the trained model to a file. The LARS algorithm is described in more detail below:
Let X be a matrix where each row is a point and each column is a dimension, and let y be a vector of targets.
The Elastic Net problem is to solve
min_beta 0.5  X * beta  y _2^2 + lambda_1 beta_1 + 0.5 lambda_2 beta_2^2
If lambda1 > 0 and lambda2 = 0, the problem is the LASSO. If lambda1 > 0 and lambda2 > 0, the problem is the Elastic Net. If lambda1 = 0 and lambda2 > 0, the problem is ridge regression. If lambda1 = 0 and lambda2 = 0, the problem is unregularized linear regression.
For efficiency reasons, it is not recommended to use this algorithm with Lambda1
= 0. In that case, use the ‘linear_regression’ program, which implements both unregularized linear regression and ridge regression.
To train a LARS/LASSO/Elastic Net model, the Input
and Responses
parameters must be given. The Lambda1
, Lambda2
, and UseCholesky
parameters control the training options. A trained model can be saved with the OutputModel
. If no training is desired at all, a model can be passed via the InputModel
parameter.
The program can also provide predictions for test data using either the trained model or the given input model. Test points can be specified with the Test
parameter. Predicted responses to the test points can be saved with the OutputPredictions
output parameter.
Example
For example, the following command trains a model on the data data
and responses responses
with lambda1 set to 0.4 and lambda2 set to 0 (so, LASSO is being solved), and then the model is saved to lasso_model
:
// Initialize optional parameters for Lars().
param := mlpack.LarsOptions()
param.Input = data
param.Responses = responses
param.Lambda1 = 0.4
param.Lambda2 = 0
lasso_model, _ := mlpack.Lars(param)
The following command uses the lasso_model
to provide predicted responses for the data test
and save those responses to test_predictions
:
// Initialize optional parameters for Lars().
param := mlpack.LarsOptions()
param.InputModel = &lasso_model
param.Test = test
_, test_predictions := mlpack.Lars(param)
See also
LinearRegression()
Simple Linear Regression and Prediction
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for LinearRegression().
param := mlpack.LinearRegressionOptions()
param.InputModel = nil
param.Lambda = 0
param.Test = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
param.TrainingResponses = mat.NewDense(1, 1, nil)
output_model, output_predictions := mlpack.LinearRegression(param)
An implementation of simple linear regression and ridge regression using ordinary least squares. Given a dataset and responses, a model can be trained and saved for later use, or a pretrained model can be used to output regression predictions for a test set. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

InputModel 
linearRegression 
Existing LinearRegression model to use.  nil 
Lambda 
float64 
Tikhonov regularization for ridge regression. If 0, the method reduces to linear regression.  0 
Test 
*mat.Dense 
Matrix containing X’ (test regressors).  mat.NewDense(1, 1, nil) 
Training 
*mat.Dense 
Matrix containing training set X (regressors).  mat.NewDense(1, 1, nil) 
TrainingResponses 
*mat.Dense (1d) 
Optional vector containing y (responses). If not given, the responses are assumed to be the last row of the input file.  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
linearRegression 
Output LinearRegression model. 
outputPredictions 
*mat.Dense (1d) 
If –test_file is specified, this matrix is where the predicted responses will be saved. 
Detailed documentation
An implementation of simple linear regression and simple ridge regression using ordinary least squares. This solves the problem
y = X * b + e
where X (specified by Training
) and y (specified either as the last column of the input matrix Training
or via the TrainingResponses
parameter) are known and b is the desired variable. If the covariance matrix (X’X) is not invertible, or if the solution is overdetermined, then specify a Tikhonov regularization constant (with Lambda
) greater than 0, which will regularize the covariance matrix to make it invertible. The calculated b may be saved with the OutputPredictions
output parameter.
Optionally, the calculated value of b is used to predict the responses for another matrix X’ (specified by the Test
parameter):
y’ = X’ * b
and the predicted responses y’ may be saved with the OutputPredictions
output parameter. This type of regression is related to leastangle regression, which mlpack implements as the ‘lars’ program.
Example
For example, to run a linear regression on the dataset X
with responses y
, saving the trained model to lr_model
, the following command could be used:
// Initialize optional parameters for LinearRegression().
param := mlpack.LinearRegressionOptions()
param.Training = X
param.TrainingResponses = y
lr_model, _ := mlpack.LinearRegression(param)
Then, to use lr_model
to predict responses for a test set X_test
, saving the predictions to X_test_responses
, the following command could be used:
// Initialize optional parameters for LinearRegression().
param := mlpack.LinearRegressionOptions()
param.InputModel = &lr_model
param.Test = X_test
_, X_test_responses := mlpack.LinearRegression(param)
See also
 Linear/ridge regression tutorial
 Lars()
 Linear regression on Wikipedia
 mlpack::regression::LinearRegression C++ class documentation
LinearSvm()
Linear SVM is an L2regularized support vector machine.
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for LinearSvm().
param := mlpack.LinearSvmOptions()
param.Delta = 1
param.Epochs = 50
param.InputModel = nil
param.Labels = mat.NewDense(1, 1, nil)
param.Lambda = 0.0001
param.MaxIterations = 10000
param.NoIntercept = false
param.NumClasses = 0
param.Optimizer = "lbfgs"
param.Seed = 0
param.Shuffle = false
param.StepSize = 0.01
param.Test = mat.NewDense(1, 1, nil)
param.TestLabels = mat.NewDense(1, 1, nil)
param.Tolerance = 1e10
param.Training = mat.NewDense(1, 1, nil)
output_model, predictions, probabilities := mlpack.LinearSvm(param)
An implementation of linear SVM for multiclass classification. Given labeled data, a model can be trained and saved for future use; or, a pretrained model can be used to classify new points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Delta 
float64 
Margin of difference between correct class and other classes.  1 
Epochs 
int 
Maximum number of full epochs over dataset for psgd  50 
InputModel 
linearsvmModel 
Existing model (parameters).  nil 
Labels 
*mat.Dense (1d with ints) 
A matrix containing labels (0 or 1) for the points in the training set (y).  mat.NewDense(1, 1, nil) 
Lambda 
float64 
L2regularization parameter for training.  0.0001 
MaxIterations 
int 
Maximum iterations for optimizer (0 indicates no limit).  10000 
NoIntercept 
bool 
Do not add the intercept term to the model.  false 
NumClasses 
int 
Number of classes for classification; if unspecified (or 0), the number of classes found in the labels will be used.  0 
Optimizer 
string 
Optimizer to use for training (‘lbfgs’ or ‘psgd’).  "lbfgs" 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
Shuffle 
bool 
Don’t shuffle the order in which data points are visited for parallel SGD.  false 
StepSize 
float64 
Step size for parallel SGD optimizer.  0.01 
Test 
*mat.Dense 
Matrix containing test dataset.  mat.NewDense(1, 1, nil) 
TestLabels 
*mat.Dense (1d with ints) 
Matrix containing test labels.  mat.NewDense(1, 1, nil) 
Tolerance 
float64 
Convergence tolerance for optimizer.  1e10 
Training 
*mat.Dense 
A matrix containing the training set (the matrix of predictors, X).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
linearsvmModel 
Output for trained linear svm model. 
predictions 
*mat.Dense (1d with ints) 
If test data is specified, this matrix is where the predictions for the test set will be saved. 
probabilities 
*mat.Dense 
If test data is specified, this matrix is where the class probabilities for the test set will be saved. 
Detailed documentation
An implementation of linear SVMs that uses either LBFGS or parallel SGD (stochastic gradient descent) to train the model.
This program allows loading a linear SVM model (via the InputModel
parameter) or training a linear SVM model given training data (specified with the Training
parameter), or both those things at once. In addition, this program allows classification on a test dataset (specified with the Test
parameter) and the classification results may be saved with the Predictions
output parameter. The trained linear SVM model may be saved using the OutputModel
output parameter.
The training data, if specified, may have class labels as its last dimension. Alternately, the Labels
parameter may be used to specify a separate vector of labels.
When a model is being trained, there are many options. L2 regularization (to prevent overfitting) can be specified with the Lambda
option, and the number of classes can be manually specified with the NumClasses
and if an intercept term is not desired in the model, the NoIntercept
parameter can be specified.Margin of difference between correct class and other classes can be specified with the Delta
option.The optimizer used to train the model can be specified with the Optimizer
parameter. Available options are ‘psgd’ (parallel stochastic gradient descent) and ‘lbfgs’ (the LBFGS optimizer). There are also various parameters for the optimizer; the MaxIterations
parameter specifies the maximum number of allowed iterations, and the Tolerance
parameter specifies the tolerance for convergence. For the parallel SGD optimizer, the StepSize
parameter controls the step size taken at each iteration by the optimizer and the maximum number of epochs (specified with Epochs
). If the objective function for your data is oscillating between Inf and 0, the step size is probably too large. There are more parameters for the optimizers, but the C++ interface must be used to access these.
Optionally, the model can be used to predict the labels for another matrix of data points, if Test
is specified. The Test
parameter can be specified without the Training
parameter, so long as an existing linear SVM model is given with the InputModel
parameter. The output predictions from the linear SVM model may be saved with the Predictions
parameter.
Example
As an example, to train a LinaerSVM on the data ‘data
’ with labels ‘labels
’ with L2 regularization of 0.1, saving the model to ‘lsvm_model
’, the following command may be used:
// Initialize optional parameters for LinearSvm().
param := mlpack.LinearSvmOptions()
param.Training = data
param.Labels = labels
param.Lambda = 0.1
param.Delta = 1
param.NumClasses = 0
lsvm_model, _, _ := mlpack.LinearSvm(param)
Then, to use that model to predict classes for the dataset ‘test
’, storing the output predictions in ‘predictions
’, the following command may be used:
// Initialize optional parameters for LinearSvm().
param := mlpack.LinearSvmOptions()
param.InputModel = &lsvm_model
param.Test = test
_, predictions, _ := mlpack.LinearSvm(param)
See also
 RandomForest()
 LogisticRegression()
 LinearSVM on Wikipedia
 mlpack::svm::LinearSVM C++ class documentation
Lmnn()
Large Margin Nearest Neighbors (LMNN)
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Lmnn().
param := mlpack.LmnnOptions()
param.BatchSize = 50
param.Center = false
param.Distance = mat.NewDense(1, 1, nil)
param.K = 1
param.Labels = mat.NewDense(1, 1, nil)
param.LinearScan = false
param.MaxIterations = 100000
param.Normalize = false
param.Optimizer = "amsgrad"
param.Passes = 50
param.PrintAccuracy = false
param.Range = 1
param.Rank = 0
param.Regularization = 0.5
param.Seed = 0
param.StepSize = 0.01
param.Tolerance = 1e07
centered_data, output, transformed_data := mlpack.Lmnn(input, param)
An implementation of Large Margin Nearest Neighbors (LMNN), a distance learning technique. Given a labeled dataset, this learns a transformation of the data that improves knearestneighbor performance; this can be useful as a preprocessing step. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

BatchSize 
int 
Batch size for minibatch SGD.  50 
Center 
bool 
Perform meancentering on the dataset. It is useful when the centroid of the data is far from the origin.  false 
Distance 
*mat.Dense 
Initial distance matrix to be used as starting point  mat.NewDense(1, 1, nil) 
input 
*mat.Dense 
Input dataset to run LMNN on.  required 
K 
int 
Number of target neighbors to use for each datapoint.  1 
Labels 
*mat.Dense (1d with ints) 
Labels for input dataset.  mat.NewDense(1, 1, nil) 
LinearScan 
bool 
Don’t shuffle the order in which data points are visited for SGD or minibatch SGD.  false 
MaxIterations 
int 
Maximum number of iterations for LBFGS (0 indicates no limit).  100000 
Normalize 
bool 
Use a normalized starting point for optimization. Itis useful for when points are far apart, or when SGD is returning NaN.  false 
Optimizer 
string 
Optimizer to use; ‘amsgrad’, ‘bbsgd’, ‘sgd’, or ‘lbfgs’.  "amsgrad" 
Passes 
int 
Maximum number of full passes over dataset for AMSGrad, BB_SGD and SGD.  50 
PrintAccuracy 
bool 
Print accuracies on initial and transformed dataset  false 
Range 
int 
Number of iterations after which impostors needs to be recalculated  1 
Rank 
int 
Rank of distance matrix to be optimized.  0 
Regularization 
float64 
Regularization for LMNN objective function  0.5 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
StepSize 
float64 
Step size for AMSGrad, BB_SGD and SGD (alpha).  0.01 
Tolerance 
float64 
Maximum tolerance for termination of AMSGrad, BB_SGD, SGD or LBFGS.  1e07 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

centeredData 
*mat.Dense 
Output matrix for meancentered dataset. 
output 
*mat.Dense 
Output matrix for learned distance matrix. 
transformedData 
*mat.Dense 
Output matrix for transformed dataset. 
Detailed documentation
This program implements Large Margin Nearest Neighbors, a distance learning technique. The method seeks to improve knearestneighbor classification on a dataset. The method employes the strategy of reducing distance between similar labeled data points (a.k.a target neighbors) and increasing distance between differently labeled points (a.k.a impostors) using standard optimization techniques over the gradient of the distance between data points.
To work, this algorithm needs labeled data. It can be given as the last row of the input dataset (specified with Input
), or alternatively as a separate matrix (specified with Labels
). Additionally, a starting point for optimization (specified with Distance
can be given, having (r x d) dimensionality. Here r should satisfy 1 <= r <= d, Consequently a LowRank matrix will be optimized. Alternatively, LowRank distance can be learned by specifying the Rank
parameter (A LowRank matrix with uniformly distributed values will be used as initial learning point).
The program also requires number of targets neighbors to work with ( specified with K
), A regularization parameter can also be passed, It acts as a trade of between the pulling and pushing terms (specified with Regularization
), In addition, this implementation of LMNN includes a parameter to decide the interval after which impostors must be recalculated (specified with Range
).
Output can either be the learned distance matrix (specified with Output
), or the transformed dataset (specified with TransformedData
), or both. Additionally meancentered dataset (specified with CenteredData
) can be accessed given meancentering (specified with Center
) is performed on the dataset. Accuracy on initial dataset and final transformed dataset can be printed by specifying the PrintAccuracy
parameter.
This implementation of LMNN uses AdaGrad, BigBatch_SGD, stochastic gradient descent, minibatch stochastic gradient descent, or the L_BFGS optimizer.
AdaGrad, specified by the value ‘adagrad’ for the parameter Optimizer
, uses maximum of past squared gradients. It primarily on six parameters: the step size (specified with StepSize
), the batch size (specified with BatchSize
), the maximum number of passes (specified with Passes
). Inaddition, a normalized starting point can be used by specifying the Normalize
parameter.
BigBatch_SGD, specified by the value ‘bbsgd’ for the parameter Optimizer
, depends primarily on four parameters: the step size (specified with StepSize
), the batch size (specified with BatchSize
), the maximum number of passes (specified with Passes
). In addition, a normalized starting point can be used by specifying the Normalize
parameter.
Stochastic gradient descent, specified by the value ‘sgd’ for the parameter Optimizer
, depends primarily on three parameters: the step size (specified with StepSize
), the batch size (specified with BatchSize
), and the maximum number of passes (specified with Passes
). In addition, a normalized starting point can be used by specifying the Normalize
parameter. Furthermore, meancentering can be performed on the dataset by specifying the Center
parameter.
The LBFGS optimizer, specified by the value ‘lbfgs’ for the parameter Optimizer
, uses a backtracking line search algorithm to minimize a function. The following parameters are used by LBFGS: MaxIterations
, Tolerance
(the optimization is terminated when the gradient norm is below this value). For more details on the LBFGS optimizer, consult either the mlpack LBFGS documentation (in lbfgs.hpp) or the vast set of published literature on LBFGS. In addition, a normalized starting point can be used by specifying the Normalize
parameter.
By default, the AMSGrad optimizer is used.
Example
Example  Let’s say we want to learn distance on iris dataset with number of targets as 3 using BigBatch_SGD optimizer. A simple call for the same will look like:
// Initialize optional parameters for MlpackLmnn().
param := mlpack.MlpackLmnnOptions()
param.Labels = iris_labels
param.K = 3
param.Optimizer = "bbsgd"
_, output, _ := mlpack.MlpackLmnn(iris, param)
An another program call making use of range & regularization parameter with dataset having labels as last column can be made as:
// Initialize optional parameters for MlpackLmnn().
param := mlpack.MlpackLmnnOptions()
param.K = 5
param.Range = 10
param.Regularization = 0.4
_, output, _ := mlpack.MlpackLmnn(letter_recognition, param)
See also
 Nca()
 Large margin nearest neighbor on Wikipedia
 Distance metric learning for large margin nearest neighbor classification (pdf)
 mlpack::lmnn::LMNN C++ class documentation
LocalCoordinateCoding()
Local Coordinate Coding
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for LocalCoordinateCoding().
param := mlpack.LocalCoordinateCodingOptions()
param.Atoms = 0
param.InitialDictionary = mat.NewDense(1, 1, nil)
param.InputModel = nil
param.Lambda = 0
param.MaxIterations = 0
param.Normalize = false
param.Seed = 0
param.Test = mat.NewDense(1, 1, nil)
param.Tolerance = 0.01
param.Training = mat.NewDense(1, 1, nil)
codes, dictionary, output_model := mlpack.LocalCoordinateCoding(param)
An implementation of Local Coordinate Coding (LCC), a data transformation technique. Given input data, this transforms each point to be expressed as a linear combination of a few points in the dataset; once an LCC model is trained, it can be used to transform points later also. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Atoms 
int 
Number of atoms in the dictionary.  0 
InitialDictionary 
*mat.Dense 
Optional initial dictionary.  mat.NewDense(1, 1, nil) 
InputModel 
localCoordinateCoding 
Input LCC model.  nil 
Lambda 
float64 
Weighted l1norm regularization parameter.  0 
MaxIterations 
int 
Maximum number of iterations for LCC (0 indicates no limit).  0 
Normalize 
bool 
If set, the input data matrix will be normalized before coding.  false 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
Test 
*mat.Dense 
Test points to encode.  mat.NewDense(1, 1, nil) 
Tolerance 
float64 
Tolerance for objective function.  0.01 
Training 
*mat.Dense 
Matrix of training data (X).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

codes 
*mat.Dense 
Output codes matrix. 
dictionary 
*mat.Dense 
Output dictionary matrix. 
outputModel 
localCoordinateCoding 
Output for trained LCC model. 
Detailed documentation
An implementation of Local Coordinate Coding (LCC), which codes data that approximately lives on a manifold using a variation of l1norm regularized sparse coding. Given a dense data matrix X with n points and d dimensions, LCC seeks to find a dense dictionary matrix D with k atoms in d dimensions, and a coding matrix Z with n points in k dimensions. Because of the regularization method used, the atoms in D should lie close to the manifold on which the data points lie.
The original data matrix X can then be reconstructed as D * Z. Therefore, this program finds a representation of each point in X as a sparse linear combination of atoms in the dictionary D.
The coding is found with an algorithm which alternates between a dictionary step, which updates the dictionary D, and a coding step, which updates the coding matrix Z.
To run this program, the input matrix X must be specified (with i), along with the number of atoms in the dictionary (k). An initial dictionary may also be specified with the InitialDictionary
parameter. The l1norm regularization parameter is specified with the Lambda
parameter.
Example
For example, to run LCC on the dataset data
using 200 atoms and an l1regularization parameter of 0.1, saving the dictionary Dictionary
and the codes into Codes
, use
// Initialize optional parameters for LocalCoordinateCoding().
param := mlpack.LocalCoordinateCodingOptions()
param.Training = data
param.Atoms = 200
param.Lambda = 0.1
codes, dict, _ := mlpack.LocalCoordinateCoding(param)
The maximum number of iterations may be specified with the MaxIterations
parameter. Optionally, the input data matrix X can be normalized before coding with the Normalize
parameter.
An LCC model may be saved using the OutputModel
output parameter. Then, to encode new points from the dataset points
with the previously saved model lcc_model
, saving the new codes to new_codes
, the following command can be used:
// Initialize optional parameters for LocalCoordinateCoding().
param := mlpack.LocalCoordinateCodingOptions()
param.InputModel = &lcc_model
param.Test = points
new_codes, _, _ := mlpack.LocalCoordinateCoding(param)
See also
 SparseCoding()
 Nonlinear learning using local coordinate coding (pdf)
 mlpack::lcc::LocalCoordinateCoding C++ class documentation
LogisticRegression()
L2regularized Logistic Regression and Prediction
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for LogisticRegression().
param := mlpack.LogisticRegressionOptions()
param.BatchSize = 64
param.DecisionBoundary = 0.5
param.InputModel = nil
param.Labels = mat.NewDense(1, 1, nil)
param.Lambda = 0
param.MaxIterations = 10000
param.Optimizer = "lbfgs"
param.StepSize = 0.01
param.Test = mat.NewDense(1, 1, nil)
param.Tolerance = 1e10
param.Training = mat.NewDense(1, 1, nil)
output, output_model, output_probabilities, predictions, probabilities :=
mlpack.LogisticRegression(param)
An implementation of L2regularized logistic regression for twoclass classification. Given labeled data, a model can be trained and saved for future use; or, a pretrained model can be used to classify new points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

BatchSize 
int 
Batch size for SGD.  64 
DecisionBoundary 
float64 
Decision boundary for prediction; if the logistic function for a point is less than the boundary, the class is taken to be 0; otherwise, the class is 1.  0.5 
InputModel 
logisticRegression 
Existing model (parameters).  nil 
Labels 
*mat.Dense (1d with ints) 
A matrix containing labels (0 or 1) for the points in the training set (y).  mat.NewDense(1, 1, nil) 
Lambda 
float64 
L2regularization parameter for training.  0 
MaxIterations 
int 
Maximum iterations for optimizer (0 indicates no limit).  10000 
Optimizer 
string 
Optimizer to use for training (‘lbfgs’ or ‘sgd’).  "lbfgs" 
StepSize 
float64 
Step size for SGD optimizer.  0.01 
Test 
*mat.Dense 
Matrix containing test dataset.  mat.NewDense(1, 1, nil) 
Tolerance 
float64 
Convergence tolerance for optimizer.  1e10 
Training 
*mat.Dense 
A matrix containing the training set (the matrix of predictors, X).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense (1d with ints) 
If test data is specified, this matrix is where the predictions for the test set will be saved. 
outputModel 
logisticRegression 
Output for trained logistic regression model. 
outputProbabilities 
*mat.Dense 
If test data is specified, this matrix is where the class probabilities for the test set will be saved. 
predictions 
*mat.Dense (1d with ints) 
If test data is specified, this matrix is where the predictions for the test set will be saved. 
probabilities 
*mat.Dense 
If test data is specified, this matrix is where the class probabilities for the test set will be saved. 
Detailed documentation
An implementation of L2regularized logistic regression using either the LBFGS optimizer or SGD (stochastic gradient descent). This solves the regression problem
y = (1 / 1 + e^(X * b))
where y takes values 0 or 1.
This program allows loading a logistic regression model (via the InputModel
parameter) or training a logistic regression model given training data (specified with the Training
parameter), or both those things at once. In addition, this program allows classification on a test dataset (specified with the Test
parameter) and the classification results may be saved with the Predictions
output parameter. The trained logistic regression model may be saved using the OutputModel
output parameter.
The training data, if specified, may have class labels as its last dimension. Alternately, the Labels
parameter may be used to specify a separate matrix of labels.
When a model is being trained, there are many options. L2 regularization (to prevent overfitting) can be specified with the Lambda
option, and the optimizer used to train the model can be specified with the Optimizer
parameter. Available options are ‘sgd’ (stochastic gradient descent) and ‘lbfgs’ (the LBFGS optimizer). There are also various parameters for the optimizer; the MaxIterations
parameter specifies the maximum number of allowed iterations, and the Tolerance
parameter specifies the tolerance for convergence. For the SGD optimizer, the StepSize
parameter controls the step size taken at each iteration by the optimizer. The batch size for SGD is controlled with the BatchSize
parameter. If the objective function for your data is oscillating between Inf and 0, the step size is probably too large. There are more parameters for the optimizers, but the C++ interface must be used to access these.
For SGD, an iteration refers to a single point. So to take a single pass over the dataset with SGD, MaxIterations
should be set to the number of points in the dataset.
Optionally, the model can be used to predict the responses for another matrix of data points, if Test
is specified. The Test
parameter can be specified without the Training
parameter, so long as an existing logistic regression model is given with the InputModel
parameter. The output predictions from the logistic regression model may be saved with the Predictions
parameter.
Note : The following parameters are deprecated and will be removed in mlpack 4: Output
, OutputProbabilities
Use Predictions
instead of Output
Use Probabilities
instead of OutputProbabilities
This implementation of logistic regression does not support the general multiclass case but instead only the twoclass case. Any labels must be either 0 or 1. For more classes, see the softmax_regression program.
Example
As an example, to train a logistic regression model on the data ‘data
’ with labels ‘labels
’ with L2 regularization of 0.1, saving the model to ‘lr_model
’, the following command may be used:
// Initialize optional parameters for LogisticRegression().
param := mlpack.LogisticRegressionOptions()
param.Training = data
param.Labels = labels
param.Lambda = 0.1
_, lr_model, _, _, _ := mlpack.LogisticRegression(param)
Then, to use that model to predict classes for the dataset ‘test
’, storing the output predictions in ‘predictions
’, the following command may be used:
// Initialize optional parameters for LogisticRegression().
param := mlpack.LogisticRegressionOptions()
param.InputModel = &lr_model
param.Test = test
predictions, _, _, _, _ := mlpack.LogisticRegression(param)
See also
 SoftmaxRegression()
 RandomForest()
 Logistic regression on Wikipedia
 mlpack::regression::LogisticRegression C++ class documentation
Lsh()
KApproximateNearestNeighbor Search with LSH
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Lsh().
param := mlpack.LshOptions()
param.BucketSize = 500
param.HashWidth = 0
param.InputModel = nil
param.K = 0
param.NumProbes = 0
param.Projections = 10
param.Query = mat.NewDense(1, 1, nil)
param.Reference = mat.NewDense(1, 1, nil)
param.SecondHashSize = 99901
param.Seed = 0
param.Tables = 30
param.TrueNeighbors = mat.NewDense(1, 1, nil)
distances, neighbors, output_model := mlpack.Lsh(param)
An implementation of approximate knearestneighbor search with localitysensitive hashing (LSH). Given a set of reference points and a set of query points, this will compute the k approximate nearest neighbors of each query point in the reference set; models can be saved for future use. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

BucketSize 
int 
The size of a bucket in the second level hash.  500 
HashWidth 
float64 
The hash width for the firstlevel hashing in the LSH preprocessing. By default, the LSH class automatically estimates a hash width for its use.  0 
InputModel 
lshSearch 
Input LSH model.  nil 
K 
int 
Number of nearest neighbors to find.  0 
NumProbes 
int 
Number of additional probes for multiprobe LSH; if 0, traditional LSH is used.  0 
Projections 
int 
The number of hash functions for each table  10 
Query 
*mat.Dense 
Matrix containing query points (optional).  mat.NewDense(1, 1, nil) 
Reference 
*mat.Dense 
Matrix containing the reference dataset.  mat.NewDense(1, 1, nil) 
SecondHashSize 
int 
The size of the second level hash table.  99901 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
Tables 
int 
The number of hash tables to be used.  30 
TrueNeighbors 
*mat.Dense (with ints) 
Matrix of true neighbors to compute recall with (the recall is printed when v is specified).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

distances 
*mat.Dense 
Matrix to output distances into. 
neighbors 
*mat.Dense (with ints) 
Matrix to output neighbors into. 
outputModel 
lshSearch 
Output for trained LSH model. 
Detailed documentation
This program will calculate the k approximatenearestneighbors of a set of points using localitysensitive hashing. You may specify a separate set of reference points and query points, or just a reference set which will be used as both the reference and query set.
Example
For example, the following will return 5 neighbors from the data for each point in input
and store the distances in distances
and the neighbors in neighbors
:
// Initialize optional parameters for Lsh().
param := mlpack.LshOptions()
param.K = 5
param.Reference = input
distances, neighbors, _ := mlpack.Lsh(param)
The output is organized such that row i and column j in the neighbors output corresponds to the index of the point in the reference set which is the j’th nearest neighbor from the point in the query set with index i. Row j and column i in the distances output file corresponds to the distance between those two points.
Because this is approximatenearestneighbors search, results may be different from run to run. Thus, the Seed
parameter can be specified to set the random seed.
This program also has many other parameters to control its functionality; see the parameterspecific documentation for more information.
See also
 Knn()
 Krann()
 Localitysensitive hashing on Wikipedia
 Localitysensitive hashing scheme based on pstable distributions(pdf)
 mlpack::neighbor::LSHSearch C++ class documentation
MeanShift()
Mean Shift Clustering
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for MeanShift().
param := mlpack.MeanShiftOptions()
param.ForceConvergence = false
param.InPlace = false
param.LabelsOnly = false
param.MaxIterations = 1000
param.Radius = 0
centroid, output := mlpack.MeanShift(input, param)
A fast implementation of meanshift clustering using dualtree range search. Given a dataset, this uses the mean shift algorithm to produce and return a clustering of the data. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

ForceConvergence 
bool 
If specified, the mean shift algorithm will continue running regardless of max_iterations until the clusters converge.  false 
InPlace 
bool 
If specified, a column containing the learned cluster assignments will be added to the input dataset file. In this case, –output_file is overridden. (Do not use with Python.)  false 
input 
*mat.Dense 
Input dataset to perform clustering on.  required 
LabelsOnly 
bool 
If specified, only the output labels will be written to the file specified by –output_file.  false 
MaxIterations 
int 
Maximum number of iterations before mean shift terminates.  1000 
Radius 
float64 
If the distance between two centroids is less than the given radius, one will be removed. A radius of 0 or less means an estimate will be calculated and used for the radius.  0 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

centroid 
*mat.Dense 
If specified, the centroids of each cluster will be written to the given matrix. 
output 
*mat.Dense 
Matrix to write output labels or labeled data to. 
Detailed documentation
This program performs mean shift clustering on the given dataset, storing the learned cluster assignments either as a column of labels in the input dataset or separately.
The input dataset should be specified with the Input
parameter, and the radius used for search can be specified with the Radius
parameter. The maximum number of iterations before algorithm termination is controlled with the MaxIterations
parameter.
The output labels may be saved with the Output
output parameter and the centroids of each cluster may be saved with the Centroid
output parameter.
Example
For example, to run mean shift clustering on the dataset data
and store the centroids to centroids
, the following command may be used:
// Initialize optional parameters for MeanShift().
param := mlpack.MeanShiftOptions()
centroids, _ := mlpack.MeanShift(data, param)
See also
 Kmeans()
 Dbscan()
 Mean shift on Wikipedia
 Mean Shift, Mode Seeking, and Clustering (pdf)
 mlpack::mean_shift::MeanShift C++ class documentation
Nbc()
Parametric Naive Bayes Classifier
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Nbc().
param := mlpack.NbcOptions()
param.IncrementalVariance = false
param.InputModel = nil
param.Labels = mat.NewDense(1, 1, nil)
param.Test = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
output, output_model, output_probs, predictions, probabilities :=
mlpack.Nbc(param)
An implementation of the Naive Bayes Classifier, used for classification. Given labeled data, an NBC model can be trained and saved, or, a pretrained model can be used for classification. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

IncrementalVariance 
bool 
The variance of each class will be calculated incrementally.  false 
InputModel 
nbcModel 
Input Naive Bayes model.  nil 
Labels 
*mat.Dense (1d with ints) 
A file containing labels for the training set.  mat.NewDense(1, 1, nil) 
Test 
*mat.Dense 
A matrix containing the test set.  mat.NewDense(1, 1, nil) 
Training 
*mat.Dense 
A matrix containing the training set.  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense (1d with ints) 
The matrix in which the predicted labels for the test set will be written (deprecated). 
outputModel 
nbcModel 
File to save trained Naive Bayes model to. 
outputProbs 
*mat.Dense 
The matrix in which the predicted probability of labels for the test set will be written (deprecated). 
predictions 
*mat.Dense (1d with ints) 
The matrix in which the predicted labels for the test set will be written. 
probabilities 
*mat.Dense 
The matrix in which the predicted probability of labels for the test set will be written. 
Detailed documentation
This program trains the Naive Bayes classifier on the given labeled training set, or loads a model from the given model file, and then may use that trained model to classify the points in a given test set.
The training set is specified with the Training
parameter. Labels may be either the last row of the training set, or alternately the Labels
parameter may be specified to pass a separate matrix of labels.
If training is not desired, a preexisting model may be loaded with the InputModel
parameter.
The IncrementalVariance
parameter can be used to force the training to use an incremental algorithm for calculating variance. This is slower, but can help avoid loss of precision in some cases.
If classifying a test set is desired, the test set may be specified with the Test
parameter, and the classifications may be saved with the Predictions
predictions parameter. If saving the trained model is desired, this may be done with the OutputModel
output parameter.
Note: the Output
and OutputProbs
parameters are deprecated and will be removed in mlpack 4.0.0. Use Predictions
and Probabilities
instead.
Example
For example, to train a Naive Bayes classifier on the dataset data
with labels labels
and save the model to nbc_model
, the following command may be used:
// Initialize optional parameters for Nbc().
param := mlpack.NbcOptions()
param.Training = data
param.Labels = labels
_, nbc_model, _, _, _ := mlpack.Nbc(param)
Then, to use nbc_model
to predict the classes of the dataset test_set
and save the predicted classes to predictions
, the following command may be used:
// Initialize optional parameters for Nbc().
param := mlpack.NbcOptions()
param.InputModel = &nbc_model
param.Test = test_set
predictions, _, _, _, _ := mlpack.Nbc(param)
See also
 SoftmaxRegression()
 RandomForest()
 Naive Bayes classifier on Wikipedia
 mlpack::naive_bayes::NaiveBayesClassifier C++ class documentation
Nca()
Neighborhood Components Analysis (NCA)
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Nca().
param := mlpack.NcaOptions()
param.ArmijoConstant = 0.0001
param.BatchSize = 50
param.Labels = mat.NewDense(1, 1, nil)
param.LinearScan = false
param.MaxIterations = 500000
param.MaxLineSearchTrials = 50
param.MaxStep = 1e+20
param.MinStep = 1e20
param.Normalize = false
param.NumBasis = 5
param.Optimizer = "sgd"
param.Seed = 0
param.StepSize = 0.01
param.Tolerance = 1e07
param.Wolfe = 0.9
output := mlpack.Nca(input, param)
An implementation of neighborhood components analysis, a distance learning technique that can be used for preprocessing. Given a labeled dataset, this uses NCA, which seeks to improve the knearestneighbor classification, and returns the learned distance metric. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

ArmijoConstant 
float64 
Armijo constant for LBFGS.  0.0001 
BatchSize 
int 
Batch size for minibatch SGD.  50 
input 
*mat.Dense 
Input dataset to run NCA on.  required 
Labels 
*mat.Dense (1d with ints) 
Labels for input dataset.  mat.NewDense(1, 1, nil) 
LinearScan 
bool 
Don’t shuffle the order in which data points are visited for SGD or minibatch SGD.  false 
MaxIterations 
int 
Maximum number of iterations for SGD or LBFGS (0 indicates no limit).  500000 
MaxLineSearchTrials 
int 
Maximum number of line search trials for LBFGS.  50 
MaxStep 
float64 
Maximum step of line search for LBFGS.  1e+20 
MinStep 
float64 
Minimum step of line search for LBFGS.  1e20 
Normalize 
bool 
Use a normalized starting point for optimization. This is useful for when points are far apart, or when SGD is returning NaN.  false 
NumBasis 
int 
Number of memory points to be stored for LBFGS.  5 
Optimizer 
string 
Optimizer to use; ‘sgd’ or ‘lbfgs’.  "sgd" 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
StepSize 
float64 
Step size for stochastic gradient descent (alpha).  0.01 
Tolerance 
float64 
Maximum tolerance for termination of SGD or LBFGS.  1e07 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Wolfe 
float64 
Wolfe condition parameter for LBFGS.  0.9 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Output matrix for learned distance matrix. 
Detailed documentation
This program implements Neighborhood Components Analysis, both a linear dimensionality reduction technique and a distance learning technique. The method seeks to improve knearestneighbor classification on a dataset by scaling the dimensions. The method is nonparametric, and does not require a value of k. It works by using stochastic (“soft”) neighbor assignments and using optimization techniques over the gradient of the accuracy of the neighbor assignments.
To work, this algorithm needs labeled data. It can be given as the last row of the input dataset (specified with Input
), or alternatively as a separate matrix (specified with Labels
).
This implementation of NCA uses stochastic gradient descent, minibatch stochastic gradient descent, or the L_BFGS optimizer. These optimizers do not guarantee global convergence for a nonconvex objective function (NCA’s objective function is nonconvex), so the final results could depend on the random seed or other optimizer parameters.
Stochastic gradient descent, specified by the value ‘sgd’ for the parameter Optimizer
, depends primarily on three parameters: the step size (specified with StepSize
), the batch size (specified with BatchSize
), and the maximum number of iterations (specified with MaxIterations
). In addition, a normalized starting point can be used by specifying the Normalize
parameter, which is necessary if many warnings of the form ‘Denominator of p_i is 0!’ are given. Tuning the step size can be a tedious affair. In general, the step size is too large if the objective is not mostly uniformly decreasing, or if zerovalued denominator warnings are being issued. The step size is too small if the objective is changing very slowly. Setting the termination condition can be done easily once a good step size parameter is found; either increase the maximum iterations to a large number and allow SGD to find a minimum, or set the maximum iterations to 0 (allowing infinite iterations) and set the tolerance (specified by Tolerance
) to define the maximum allowed difference between objectives for SGD to terminate. Be careful—setting the tolerance instead of the maximum iterations can take a very long time and may actually never converge due to the properties of the SGD optimizer. Note that a single iteration of SGD refers to a single point, so to take a single pass over the dataset, set the value of the MaxIterations
parameter equal to the number of points in the dataset.
The LBFGS optimizer, specified by the value ‘lbfgs’ for the parameter Optimizer
, uses a backtracking line search algorithm to minimize a function. The following parameters are used by LBFGS: NumBasis
(specifies the number of memory points used by LBFGS), MaxIterations
, ArmijoConstant
, Wolfe
, Tolerance
(the optimization is terminated when the gradient norm is below this value), MaxLineSearchTrials
, MinStep
, and MaxStep
(which both refer to the line search routine). For more details on the LBFGS optimizer, consult either the mlpack LBFGS documentation (in lbfgs.hpp) or the vast set of published literature on LBFGS.
By default, the SGD optimizer is used.
See also
 Lmnn()
 Neighbourhood components analysis on Wikipedia
 Neighbourhood components analysis (pdf)
 mlpack::nca::NCA C++ class documentation
Knn()
kNearestNeighbors Search
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Knn().
param := mlpack.KnnOptions()
param.Algorithm = "dual_tree"
param.Epsilon = 0
param.InputModel = nil
param.K = 0
param.LeafSize = 20
param.Query = mat.NewDense(1, 1, nil)
param.RandomBasis = false
param.Reference = mat.NewDense(1, 1, nil)
param.Rho = 0.7
param.Seed = 0
param.Tau = 0
param.TreeType = "kd"
param.TrueDistances = mat.NewDense(1, 1, nil)
param.TrueNeighbors = mat.NewDense(1, 1, nil)
distances, neighbors, output_model := mlpack.Knn(param)
An implementation of knearestneighbor search using singletree and dualtree algorithms. Given a set of reference points and query points, this can find the k nearest neighbors in the reference set of each query point using trees; trees that are built can be saved for future use. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Algorithm 
string 
Type of neighbor search: ‘naive’, ‘single_tree’, ‘dual_tree’, ‘greedy’.  "dual_tree" 
Epsilon 
float64 
If specified, will do approximate nearest neighbor search with given relative error.  0 
InputModel 
knnModel 
Pretrained kNN model.  nil 
K 
int 
Number of nearest neighbors to find.  0 
LeafSize 
int 
Leaf size for tree building (used for kdtrees, vp trees, random projection trees, UB trees, R trees, R* trees, X trees, Hilbert R trees, R+ trees, R++ trees, spill trees, and octrees).  20 
Query 
*mat.Dense 
Matrix containing query points (optional).  mat.NewDense(1, 1, nil) 
RandomBasis 
bool 
Before treebuilding, project the data onto a random orthogonal basis.  false 
Reference 
*mat.Dense 
Matrix containing the reference dataset.  mat.NewDense(1, 1, nil) 
Rho 
float64 
Balance threshold (only valid for spill trees).  0.7 
Seed 
int 
Random seed (if 0, std::time(NULL) is used).  0 
Tau 
float64 
Overlapping size (only valid for spill trees).  0 
TreeType 
string 
Type of tree to use: ‘kd’, ‘vp’, ‘rp’, ‘maxrp’, ‘ub’, ‘cover’, ‘r’, ‘rstar’, ‘x’, ‘ball’, ‘hilbertr’, ‘rplus’, ‘rplusplus’, ‘spill’, ‘oct’.  "kd" 
TrueDistances 
*mat.Dense 
Matrix of true distances to compute the effective error (average relative error) (it is printed when v is specified).  mat.NewDense(1, 1, nil) 
TrueNeighbors 
*mat.Dense (with ints) 
Matrix of true neighbors to compute the recall (it is printed when v is specified).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

distances 
*mat.Dense 
Matrix to output distances into. 
neighbors 
*mat.Dense (with ints) 
Matrix to output neighbors into. 
outputModel 
knnModel 
If specified, the kNN model will be output here. 
Detailed documentation
This program will calculate the knearestneighbors of a set of points using kdtrees or cover trees (cover tree support is experimental and may be slow). You may specify a separate set of reference points and query points, or just a reference set which will be used as both the reference and query set.
Example
For example, the following command will calculate the 5 nearest neighbors of each point in input
and store the distances in distances
and the neighbors in neighbors
:
// Initialize optional parameters for Knn().
param := mlpack.KnnOptions()
param.K = 5
param.Reference = input
distances, neighbors, _ := mlpack.Knn(param)
The output is organized such that row i and column j in the neighbors output matrix corresponds to the index of the point in the reference set which is the j’th nearest neighbor from the point in the query set with index i. Row j and column i in the distances output matrix corresponds to the distance between those two points.
See also
 Lsh()
 Krann()
 Kfn()
 NeighborSearch tutorial (knearestneighbors)
 Treeindependent dualtree algorithms (pdf)
 mlpack::neighbor::NeighborSearch C++ class documentation
Kfn()
kFurthestNeighbors Search
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Kfn().
param := mlpack.KfnOptions()
param.Algorithm = "dual_tree"
param.Epsilon = 0
param.InputModel = nil
param.K = 0
param.LeafSize = 20
param.Percentage = 1
param.Query = mat.NewDense(1, 1, nil)
param.RandomBasis = false
param.Reference = mat.NewDense(1, 1, nil)
param.Seed = 0
param.TreeType = "kd"
param.TrueDistances = mat.NewDense(1, 1, nil)
param.TrueNeighbors = mat.NewDense(1, 1, nil)
distances, neighbors, output_model := mlpack.Kfn(param)
An implementation of kfurthestneighbor search using singletree and dualtree algorithms. Given a set of reference points and query points, this can find the k furthest neighbors in the reference set of each query point using trees; trees that are built can be saved for future use. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Algorithm 
string 
Type of neighbor search: ‘naive’, ‘single_tree’, ‘dual_tree’, ‘greedy’.  "dual_tree" 
Epsilon 
float64 
If specified, will do approximate furthest neighbor search with given relative error. Must be in the range [0,1).  0 
InputModel 
kfnModel 
Pretrained kFN model.  nil 
K 
int 
Number of furthest neighbors to find.  0 
LeafSize 
int 
Leaf size for tree building (used for kdtrees, vp trees, random projection trees, UB trees, R trees, R* trees, X trees, Hilbert R trees, R+ trees, R++ trees, and octrees).  20 
Percentage 
float64 
If specified, will do approximate furthest neighbor search. Must be in the range (0,1] (decimal form). Resultant neighbors will be at least (p*100) % of the distance as the true furthest neighbor.  1 
Query 
*mat.Dense 
Matrix containing query points (optional).  mat.NewDense(1, 1, nil) 
RandomBasis 
bool 
Before treebuilding, project the data onto a random orthogonal basis.  false 
Reference 
*mat.Dense 
Matrix containing the reference dataset.  mat.NewDense(1, 1, nil) 
Seed 
int 
Random seed (if 0, std::time(NULL) is used).  0 
TreeType 
string 
Type of tree to use: ‘kd’, ‘vp’, ‘rp’, ‘maxrp’, ‘ub’, ‘cover’, ‘r’, ‘rstar’, ‘x’, ‘ball’, ‘hilbertr’, ‘rplus’, ‘rplusplus’, ‘oct’.  "kd" 
TrueDistances 
*mat.Dense 
Matrix of true distances to compute the effective error (average relative error) (it is printed when v is specified).  mat.NewDense(1, 1, nil) 
TrueNeighbors 
*mat.Dense (with ints) 
Matrix of true neighbors to compute the recall (it is printed when v is specified).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

distances 
*mat.Dense 
Matrix to output distances into. 
neighbors 
*mat.Dense (with ints) 
Matrix to output neighbors into. 
outputModel 
kfnModel 
If specified, the kFN model will be output here. 
Detailed documentation
This program will calculate the kfurthestneighbors of a set of points. You may specify a separate set of reference points and query points, or just a reference set which will be used as both the reference and query set.
Example
For example, the following will calculate the 5 furthest neighbors of eachpoint in input
and store the distances in distances
and the neighbors in neighbors
:
// Initialize optional parameters for Kfn().
param := mlpack.KfnOptions()
param.K = 5
param.Reference = input
distances, neighbors, _ := mlpack.Kfn(param)
The output files are organized such that row i and column j in the neighbors output matrix corresponds to the index of the point in the reference set which is the j’th furthest neighbor from the point in the query set with index i. Row i and column j in the distances output file corresponds to the distance between those two points.
See also
 ApproxKfn()
 Knn()
 Treeindependent dualtree algorithms (pdf)
 mlpack::neighbor::NeighborSearch C++ class documentation
Nmf()
Nonnegative Matrix Factorization
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Nmf().
param := mlpack.NmfOptions()
param.InitialH = mat.NewDense(1, 1, nil)
param.InitialW = mat.NewDense(1, 1, nil)
param.MaxIterations = 10000
param.MinResidue = 1e05
param.Seed = 0
param.UpdateRules = "multdist"
h, w := mlpack.Nmf(input, rank, param)
An implementation of nonnegative matrix factorization. This can be used to decompose an input dataset into two lowrank nonnegative components. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

InitialH 
*mat.Dense 
Initial H matrix.  mat.NewDense(1, 1, nil) 
InitialW 
*mat.Dense 
Initial W matrix.  mat.NewDense(1, 1, nil) 
input 
*mat.Dense 
Input dataset to perform NMF on.  required 
MaxIterations 
int 
Number of iterations before NMF terminates (0 runs until convergence.  10000 
MinResidue 
float64 
The minimum root mean square residue allowed for each iteration, below which the program terminates.  1e05 
rank 
int 
Rank of the factorization.  required 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
UpdateRules 
string 
Update rules for each iteration; ( multdist  multdiv  als ).  "multdist" 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

h 
*mat.Dense 
Matrix to save the calculated H to. 
w 
*mat.Dense 
Matrix to save the calculated W to. 
Detailed documentation
This program performs nonnegative matrix factorization on the given dataset, storing the resulting decomposed matrices in the specified files. For an input dataset V, NMF decomposes V into two matrices W and H such that
V = W * H
where all elements in W and H are nonnegative. If V is of size (n x m), then W will be of size (n x r) and H will be of size (r x m), where r is the rank of the factorization (specified by the Rank
parameter).
Optionally, the desired update rules for each NMF iteration can be chosen from the following list:
 multdist: multiplicative distancebased update rules (Lee and Seung 1999)
 multdiv: multiplicative divergencebased update rules (Lee and Seung 1999)
 als: alternating least squares update rules (Paatero and Tapper 1994)
The maximum number of iterations is specified with MaxIterations
, and the minimum residue required for algorithm termination is specified with the MinResidue
parameter.
Example
For example, to run NMF on the input matrix V
using the ‘multdist’ update rules with a rank10 decomposition and storing the decomposed matrices into W
and H
, the following command could be used:
// Initialize optional parameters for Nmf().
param := mlpack.NmfOptions()
param.UpdateRules = "multdist"
H, W := mlpack.Nmf(V, 10, param)
See also
 Cf()
 Alternating matrix factorization tutorial
 Nonnegative matrix factorization on Wikipedia
 Algorithms for nonnegative matrix factorization (pdf)
 mlpack::amf::AMF C++ class documentation
Pca()
Principal Components Analysis
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Pca().
param := mlpack.PcaOptions()
param.DecompositionMethod = "exact"
param.NewDimensionality = 0
param.Scale = false
param.VarToRetain = 0
output := mlpack.Pca(input, param)
An implementation of several strategies for principal components analysis (PCA), a common preprocessing step. Given a dataset and a desired new dimensionality, this can reduce the dimensionality of the data using the linear transformation determined by PCA. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

DecompositionMethod 
string 
Method used for the principal components analysis: ‘exact’, ‘randomized’, ‘randomizedblockkrylov’, ‘quic’.  "exact" 
input 
*mat.Dense 
Input dataset to perform PCA on.  required 
NewDimensionality 
int 
Desired dimensionality of output dataset. If 0, no dimensionality reduction is performed.  0 
Scale 
bool 
If set, the data will be scaled before running PCA, such that the variance of each feature is 1.  false 
VarToRetain 
float64 
Amount of variance to retain; should be between 0 and 1. If 1, all variance is retained. Overrides d.  0 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Matrix to save modified dataset to. 
Detailed documentation
This program performs principal components analysis on the given dataset using the exact, randomized, randomized block Krylov, or QUIC SVD method. It will transform the data onto its principal components, optionally performing dimensionality reduction by ignoring the principal components with the smallest eigenvalues.
Use the Input
parameter to specify the dataset to perform PCA on. A desired new dimensionality can be specified with the NewDimensionality
parameter, or the desired variance to retain can be specified with the VarToRetain
parameter. If desired, the dataset can be scaled before running PCA with the Scale
parameter.
Multiple different decomposition techniques can be used. The method to use can be specified with the DecompositionMethod
parameter, and it may take the values ‘exact’, ‘randomized’, or ‘quic’.
Example
For example, to reduce the dimensionality of the matrix data
to 5 dimensions using randomized SVD for the decomposition, storing the output matrix to data_mod
, the following command can be used:
// Initialize optional parameters for Pca().
param := mlpack.PcaOptions()
param.NewDimensionality = 5
param.DecompositionMethod = "randomized"
data_mod := mlpack.Pca(data, param)
See also
Perceptron()
Perceptron
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Perceptron().
param := mlpack.PerceptronOptions()
param.InputModel = nil
param.Labels = mat.NewDense(1, 1, nil)
param.MaxIterations = 1000
param.Test = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
output, output_model, predictions := mlpack.Perceptron(param)
An implementation of a perceptron—a single level neural network–=for classification. Given labeled data, a perceptron can be trained and saved for future use; or, a pretrained perceptron can be used for classification on new points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

InputModel 
perceptronModel 
Input perceptron model.  nil 
Labels 
*mat.Dense (1d with ints) 
A matrix containing labels for the training set.  mat.NewDense(1, 1, nil) 
MaxIterations 
int 
The maximum number of iterations the perceptron is to be run  1000 
Test 
*mat.Dense 
A matrix containing the test set.  mat.NewDense(1, 1, nil) 
Training 
*mat.Dense 
A matrix containing the training set.  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense (1d with ints) 
The matrix in which the predicted labels for the test set will be written. 
outputModel 
perceptronModel 
Output for trained perceptron model. 
predictions 
*mat.Dense (1d with ints) 
The matrix in which the predicted labels for the test set will be written. 
Detailed documentation
This program implements a perceptron, which is a single level neural network. The perceptron makes its predictions based on a linear predictor function combining a set of weights with the feature vector. The perceptron learning rule is able to converge, given enough iterations (specified using the MaxIterations
parameter), if the data supplied is linearly separable. The perceptron is parameterized by a matrix of weight vectors that denote the numerical weights of the neural network.
This program allows loading a perceptron from a model (via the InputModel
parameter) or training a perceptron given training data (via the Training
parameter), or both those things at once. In addition, this program allows classification on a test dataset (via the Test
parameter) and the classification results on the test set may be saved with the Predictions
output parameter. The perceptron model may be saved with the OutputModel
output parameter.
Note: the following parameter is deprecated and will be removed in mlpack 4.0.0: Output
.
Use Predictions
instead of Output
.
Example
The training data given with the Training
option may have class labels as its last dimension (so, if the training data is in CSV format, labels should be the last column). Alternately, the Labels
parameter may be used to specify a separate matrix of labels.
All these options make it easy to train a perceptron, and then reuse that perceptron for later classification. The invocation below trains a perceptron on training_data
with labels training_labels
, and saves the model to perceptron_model
.
// Initialize optional parameters for Perceptron().
param := mlpack.PerceptronOptions()
param.Training = training_data
param.Labels = training_labels
_, perceptron_model, _ := mlpack.Perceptron(param)
Then, this model can be reused for classification on the test data test_data
. The example below does precisely that, saving the predicted classes to predictions
.
// Initialize optional parameters for Perceptron().
param := mlpack.PerceptronOptions()
param.InputModel = &perceptron_model
param.Test = test_data
_, _, predictions := mlpack.Perceptron(param)
Note that all of the options may be specified at once: predictions may be calculated right after training a model, and model training can occur even if an existing perceptron model is passed with the InputModel
parameter. However, note that the number of classes and the dimensionality of all data must match. So you cannot pass a perceptron model trained on 2 classes and then retrain with a 4class dataset. Similarly, attempting classification on a 3dimensional dataset with a perceptron that has been trained on 8 dimensions will cause an error.
See also
PreprocessSplit()
Split Data
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for PreprocessSplit().
param := mlpack.PreprocessSplitOptions()
param.InputLabels = mat.NewDense(1, 1, nil)
param.NoShuffle = false
param.Seed = 0
param.TestRatio = 0.2
test, test_labels, training, training_labels :=
mlpack.PreprocessSplit(input, param)
A utility to split data into a training and testing dataset. This can also split labels according to the same split. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

input 
*mat.Dense 
Matrix containing data.  required 
InputLabels 
*mat.Dense (with ints) 
Matrix containing labels.  mat.NewDense(1, 1, nil) 
NoShuffle 
bool 
Avoid shuffling and splitting the data.  false 
Seed 
int 
Random seed (0 for std::time(NULL)).  0 
TestRatio 
float64 
Ratio of test set; if not set,the ratio defaults to 0.2  0.2 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

test 
*mat.Dense 
Matrix to save test data to. 
testLabels 
*mat.Dense (with ints) 
Matrix to save test labels to. 
training 
*mat.Dense 
Matrix to save training data to. 
trainingLabels 
*mat.Dense (with ints) 
Matrix to save train labels to. 
Detailed documentation
This utility takes a dataset and optionally labels and splits them into a training set and a test set. Before the split, the points in the dataset are randomly reordered. The percentage of the dataset to be used as the test set can be specified with the TestRatio
parameter; the default is 0.2 (20%).
The output training and test matrices may be saved with the Training
and Test
output parameters.
Optionally, labels can be also be split along with the data by specifying the InputLabels
parameter. Splitting labels works the same way as splitting the data. The output training and test labels may be saved with the TrainingLabels
and TestLabels
output parameters, respectively.
Example
So, a simple example where we want to split the dataset X
into X_train
and X_test
with 60% of the data in the training set and 40% of the dataset in the test set, we could run
// Initialize optional parameters for PreprocessSplit().
param := mlpack.PreprocessSplitOptions()
param.TestRatio = 0.4
X_test, _, X_train, _ := mlpack.PreprocessSplit(X, param)
Also by default the dataset is shuffled and split; you can provide the NoShuffle
option to avoid shuffling the data; an example to avoid shuffling of data is:
// Initialize optional parameters for PreprocessSplit().
param := mlpack.PreprocessSplitOptions()
param.TestRatio = 0.4
param.NoShuffle = true
X_test, _, X_train, _ := mlpack.PreprocessSplit(X, param)
If we had a dataset X
and associated labels y
, and we wanted to split these into X_train
, y_train
, X_test
, and y_test
, with 30% of the data in the test set, we could run
// Initialize optional parameters for PreprocessSplit().
param := mlpack.PreprocessSplitOptions()
param.InputLabels = y
param.TestRatio = 0.3
X_test, y_test, X_train, y_train := mlpack.PreprocessSplit(X, param)
See also
PreprocessBinarize()
Binarize Data
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for PreprocessBinarize().
param := mlpack.PreprocessBinarizeOptions()
param.Dimension = 0
param.Threshold = 0
output := mlpack.PreprocessBinarize(input, param)
A utility to binarize a dataset. Given a dataset, this utility converts each value in the desired dimension(s) to 0 or 1; this can be a useful preprocessing step. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Dimension 
int 
Dimension to apply the binarization. If not set, the program will binarize every dimension by default.  0 
input 
*mat.Dense 
Input data matrix.  required 
Threshold 
float64 
Threshold to be applied for binarization. If not set, the threshold defaults to 0.0.  0 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Matrix in which to save the output. 
Detailed documentation
This utility takes a dataset and binarizes the variables into either 0 or 1 given threshold. User can apply binarization on a dimension or the whole dataset. The dimension to apply binarization to can be specified using the Dimension
parameter; if left unspecified, every dimension will be binarized. The threshold for binarization can also be specified with the Threshold
parameter; the default threshold is 0.0.
The binarized matrix may be saved with the Output
output parameter.
Example
For example, if we want to set all variables greater than 5 in the dataset X
to 1 and variables less than or equal to 5.0 to 0, and save the result to Y
, we could run
// Initialize optional parameters for PreprocessBinarize().
param := mlpack.PreprocessBinarizeOptions()
param.Threshold = 5
Y := mlpack.PreprocessBinarize(X, param)
But if we want to apply this to only the first (0th) dimension of X
, we could instead run
// Initialize optional parameters for PreprocessBinarize().
param := mlpack.PreprocessBinarizeOptions()
param.Threshold = 5
param.Dimension = 0
Y := mlpack.PreprocessBinarize(X, param)
See also
PreprocessDescribe()
Descriptive Statistics
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for PreprocessDescribe().
param := mlpack.PreprocessDescribeOptions()
param.Dimension = 0
param.Population = false
param.Precision = 4
param.RowMajor = false
param.Width = 8
:= mlpack.PreprocessDescribe(input, param)
A utility for printing descriptive statistics about a dataset. This prints a number of details about a dataset in a tabular format. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Dimension 
int 
Dimension of the data. Use this to specify a dimension  0 
input 
*mat.Dense 
Matrix containing data,  required 
Population 
bool 
If specified, the program will calculate statistics assuming the dataset is the population. By default, the program will assume the dataset as a sample.  false 
Precision 
int 
Precision of the output statistics.  4 
RowMajor 
bool 
If specified, the program will calculate statistics across rows, not across columns. (Remember that in mlpack, a column represents a point, so this option is generally not necessary.)  false 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Width 
int 
Width of the output table.  8 
Detailed documentation
This utility takes a dataset and prints out the descriptive statistics of the data. Descriptive statistics is the discipline of quantitatively describing the main features of a collection of information, or the quantitative description itself. The program does not modify the original file, but instead prints out the statistics to the console. The printed result will look like a table.
Optionally, width and precision of the output can be adjusted by a user using the Width
and Precision
parameters. A user can also select a specific dimension to analyze if there are too many dimensions. The Population
parameter can be specified when the dataset should be considered as a population. Otherwise, the dataset will be considered as a sample.
Example
So, a simple example where we want to print out statistical facts about the dataset X
using the default settings, we could run
// Initialize optional parameters for PreprocessDescribe().
param := mlpack.PreprocessDescribeOptions()
param.Verbose = true
:= mlpack.PreprocessDescribe(X, param)
If we want to customize the width to 10 and precision to 5 and consider the dataset as a population, we could run
// Initialize optional parameters for PreprocessDescribe().
param := mlpack.PreprocessDescribeOptions()
param.Width = 10
param.Precision = 5
param.Verbose = true
:= mlpack.PreprocessDescribe(X, param)
See also
Impute Data
This utility provides several imputation strategies for missing data. Given a dataset with missing values, this can impute according to several strategies, including userdefined values. .
PreprocessScale()
Scale Data
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for PreprocessScale().
param := mlpack.PreprocessScaleOptions()
param.Epsilon = 1e06
param.InputModel = nil
param.InverseScaling = false
param.MaxValue = 1
param.MinValue = 0
param.ScalerMethod = "standard_scaler"
param.Seed = 0
output, output_model := mlpack.PreprocessScale(input, param)
A utility to perform feature scaling on datasets using one of sixtechniques. Both scaling and inverse scaling are supported, andscalers can be saved and then applied to other datasets. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Epsilon 
float64 
regularization Parameter for pcawhitening, or zcawhitening, should be between 1 to 1.  1e06 
input 
*mat.Dense 
Matrix containing data.  required 
InputModel 
scalingModel 
Input Scaling model.  nil 
InverseScaling 
bool 
Inverse Scaling to get original dataset  false 
MaxValue 
int 
Ending value of range for min_max_scaler.  1 
MinValue 
int 
Starting value of range for min_max_scaler.  0 
ScalerMethod 
string 
method to use for scaling, the default is standard_scaler.  "standard_scaler" 
Seed 
int 
Random seed (0 for std::time(NULL)).  0 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Matrix to save scaled data to. 
outputModel 
scalingModel 
Output scaling model. 
Detailed documentation
This utility takes a dataset and performs feature scaling using one of the six scaler methods namely: ‘max_abs_scaler’, ‘mean_normalization’, ‘min_max_scaler’ ,’standard_scaler’, ‘pca_whitening’ and ‘zca_whitening’. The function takes a matrix as Input
and a scaling method type which you can specify using ScalerMethod
parameter; the default is standard scaler, and outputs a matrix with scaled feature.
The output scaled feature matrix may be saved with the Output
output parameters.
The model to scale features can be saved using OutputModel
and later can be loaded back usingInputModel
.
Example
So, a simple example where we want to scale the dataset X
into X_scaled
with standard_scaler as scaler_method, we could run
// Initialize optional parameters for PreprocessScale().
param := mlpack.PreprocessScaleOptions()
param.ScalerMethod = "standard_scaler"
X_scaled, _ := mlpack.PreprocessScale(X, param)
A simple example where we want to whiten the dataset X
into X_whitened
with PCA as whitening_method and use 0.01 as regularization parameter, we could run
// Initialize optional parameters for PreprocessScale().
param := mlpack.PreprocessScaleOptions()
param.ScalerMethod = "pca_whitening"
param.Epsilon = 0.01
X_scaled, _ := mlpack.PreprocessScale(X, param)
You can also retransform the scaled dataset back usingInverseScaling
. An example to rescale : X_scaled
into X
using the saved model InputModel
is:
// Initialize optional parameters for PreprocessScale().
param := mlpack.PreprocessScaleOptions()
param.InverseScaling = true
param.InputModel = &saved
X, _ := mlpack.PreprocessScale(X_scaled, param)
Another simple example where we want to scale the dataset X
into X_scaled
with min_max_scaler as scaler method, where scaling range is 1 to 3 instead of default 0 to 1. We could run
// Initialize optional parameters for PreprocessScale().
param := mlpack.PreprocessScaleOptions()
param.ScalerMethod = "min_max_scaler"
param.MinValue = 1
param.MaxValue = 3
X_scaled, _ := mlpack.PreprocessScale(X, param)
See also
PreprocessOneHotEncoding()
One Hot Encoding
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for PreprocessOneHotEncoding().
param := mlpack.PreprocessOneHotEncodingOptions()
output := mlpack.PreprocessOneHotEncoding(dimensions, input, )
A utility to do onehot encoding on features of dataset. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

dimensions 
array of ints 
Index of dimensions thatneed to be onehot encoded.  required 
input 
*mat.Dense 
Matrix containing data.  required 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Matrix to save onehot encoded features data to. 
Detailed documentation
This utility takes a dataset and a vector of indices and does onehot encoding of the respective features at those indices. Indices represent the IDs of the dimensions to be onehot encoded.
The output matrix with encoded features may be saved with the Output
parameters.
Example
So, a simple example where we want to encode 1st and 3rd feature from dataset X
into X_output
would be
// Initialize optional parameters for PreprocessOneHotEncoding().
param := mlpack.PreprocessOneHotEncodingOptions()
X_ouput := mlpack.PreprocessOneHotEncoding(X, 1, 3, param)
See also
ImageConverter()
Image Converter
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for ImageConverter().
param := mlpack.ImageConverterOptions()
param.Channels = 0
param.Dataset = mat.NewDense(1, 1, nil)
param.Height = 0
param.Quality = 90
param.Save = false
param.Width = 0
output := mlpack.ImageConverter(input, param)
A utility to load an image or set of images into a single dataset that can then be used by other mlpack methods and utilities. This can also unpack an image dataset into individual files, for instance after mlpack methods have been used. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Channels 
int 
Number of channels in the image.  0 
Dataset 
*mat.Dense 
Input matrix to save as images.  mat.NewDense(1, 1, nil) 
Height 
int 
Height of the images.  0 
input 
array of strings 
Image filenames which have to be loaded/saved.  required 
Quality 
int 
Compression of the image if saved as jpg (0100).  90 
Save 
bool 
Save a dataset as images.  false 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Width 
int 
Width of the image.  0 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

output 
*mat.Dense 
Matrix to save images data to, Onlyneeded if you are specifying ‘save’ option. 
Detailed documentation
This utility takes an image or an array of images and loads them to a matrix. You can optionally specify the height Height
width Width
and channel Channels
of the images that needs to be loaded; otherwise, these parameters will be automatically detected from the image.
There are other options too, that can be specified such as Quality
.
You can also provide a dataset and save them as images using Dataset
and Save
as an parameter.
Example
An example to load an image :
// Initialize optional parameters for ImageConverter().
param := mlpack.ImageConverterOptions()
param.Height = 256
param.Width = 256
param.Channels = 3
Y := mlpack.ImageConverter(X, param)
An example to save an image is :
// Initialize optional parameters for ImageConverter().
param := mlpack.ImageConverterOptions()
param.Height = 256
param.Width = 256
param.Channels = 3
param.Dataset = Y
param.Save = true
_ := mlpack.ImageConverter(X, param)
See also
Radical()
RADICAL
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Radical().
param := mlpack.RadicalOptions()
param.Angles = 150
param.NoiseStdDev = 0.175
param.Objective = false
param.Replicates = 30
param.Seed = 0
param.Sweeps = 0
output_ic, output_unmixing := mlpack.Radical(input, param)
An implementation of RADICAL, a method for independent component analysis (ICA). Given a dataset, this can decompose the dataset into an unmixing matrix and an independent component matrix; this can be useful for preprocessing. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Angles 
int 
Number of angles to consider in bruteforce search during Radical2D.  150 
input 
*mat.Dense 
Input dataset for ICA.  required 
NoiseStdDev 
float64 
Standard deviation of Gaussian noise.  0.175 
Objective 
bool 
If set, an estimate of the final objective function is printed.  false 
Replicates 
int 
Number of Gaussianperturbed replicates to use (per point) in Radical2D.  30 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
Sweeps 
int 
Number of sweeps; each sweep calls Radical2D once for each pair of dimensions.  0 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputIc 
*mat.Dense 
Matrix to save independent components to. 
outputUnmixing 
*mat.Dense 
Matrix to save unmixing matrix to. 
Detailed documentation
An implementation of RADICAL, a method for independent component analysis (ICA). Assuming that we have an input matrix X, the goal is to find a square unmixing matrix W such that Y = W * X and the dimensions of Y are independent components. If the algorithm is running particularly slowly, try reducing the number of replicates.
The input matrix to perform ICA on should be specified with the Input
parameter. The output matrix Y may be saved with the OutputIc
output parameter, and the output unmixing matrix W may be saved with the OutputUnmixing
output parameter.
Example
For example, to perform ICA on the matrix X
with 40 replicates, saving the independent components to ic
, the following command may be used:
// Initialize optional parameters for Radical().
param := mlpack.RadicalOptions()
param.Replicates = 40
ic, _ := mlpack.Radical(X, param)
See also
 Independent component analysis on Wikipedia
 ICA using spacings estimates of entropy (pdf)
 mlpack::radical::Radical C++ class documentation
RandomForest()
Random forests
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for RandomForest().
param := mlpack.RandomForestOptions()
param.InputModel = nil
param.Labels = mat.NewDense(1, 1, nil)
param.MaximumDepth = 0
param.MinimumGainSplit = 0
param.MinimumLeafSize = 1
param.NumTrees = 10
param.PrintTrainingAccuracy = false
param.Seed = 0
param.SubspaceDim = 0
param.Test = mat.NewDense(1, 1, nil)
param.TestLabels = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
output_model, predictions, probabilities := mlpack.RandomForest(param)
An implementation of the standard random forest algorithm by Leo Breiman for classification. Given labeled data, a random forest can be trained and saved for future use; or, a pretrained random forest can be used for classification. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

InputModel 
randomForestModel 
Pretrained random forest to use for classification.  nil 
Labels 
*mat.Dense (1d with ints) 
Labels for training dataset.  mat.NewDense(1, 1, nil) 
MaximumDepth 
int 
Maximum depth of the tree (0 means no limit).  0 
MinimumGainSplit 
float64 
Minimum gain needed to make a split when building a tree.  0 
MinimumLeafSize 
int 
Minimum number of points in each leaf node.  1 
NumTrees 
int 
Number of trees in the random forest.  10 
PrintTrainingAccuracy 
bool 
If set, then the accuracy of the model on the training set will be predicted (verbose must also be specified).  false 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
SubspaceDim 
int 
Dimensionality of random subspace to use for each split. ‘0’ will autoselect the square root of data dimensionality.  0 
Test 
*mat.Dense 
Test dataset to produce predictions for.  mat.NewDense(1, 1, nil) 
TestLabels 
*mat.Dense (1d with ints) 
Test dataset labels, if accuracy calculation is desired.  mat.NewDense(1, 1, nil) 
Training 
*mat.Dense 
Training dataset.  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
randomForestModel 
Model to save trained random forest to. 
predictions 
*mat.Dense (1d with ints) 
Predicted classes for each point in the test set. 
probabilities 
*mat.Dense 
Predicted class probabilities for each point in the test set. 
Detailed documentation
This program is an implementation of the standard random forest classification algorithm by Leo Breiman. A random forest can be trained and saved for later use, or a random forest may be loaded and predictions or class probabilities for points may be generated.
The training set and associated labels are specified with the Training
and Labels
parameters, respectively. The labels should be in the range [0, num_classes  1]. Optionally, if Labels
is not specified, the labels are assumed to be the last dimension of the training dataset.
When a model is trained, the OutputModel
output parameter may be used to save the trained model. A model may be loaded for predictions with the InputModel
parameter. The InputModel
parameter may not be specified when the Training
parameter is specified. The MinimumLeafSize
parameter specifies the minimum number of training points that must fall into each leaf for it to be split. The NumTrees
controls the number of trees in the random forest. The MinimumGainSplit
parameter controls the minimum required gain for a decision tree node to split. Larger values will force higherconfidence splits. The MaximumDepth
parameter specifies the maximum depth of the tree. The SubspaceDim
parameter is used to control the number of random dimensions chosen for an individual node’s split. If PrintTrainingAccuracy
is specified, the calculated accuracy on the training set will be printed.
Test data may be specified with the Test
parameter, and if performance measures are desired for that test set, labels for the test points may be specified with the TestLabels
parameter. Predictions for each test point may be saved via the Predictions
output parameter. Class probabilities for each prediction may be saved with the Probabilities
output parameter.
Example
For example, to train a random forest with a minimum leaf size of 20 using 10 trees on the dataset contained in data
with labels labels
, saving the output random forest to rf_model
and printing the training error, one could call
// Initialize optional parameters for RandomForest().
param := mlpack.RandomForestOptions()
param.Training = data
param.Labels = labels
param.MinimumLeafSize = 20
param.NumTrees = 10
param.PrintTrainingAccuracy = true
rf_model, _, _ := mlpack.RandomForest(param)
Then, to use that model to classify points in test_set
and print the test error given the labels test_labels
using that model, while saving the predictions for each point to predictions
, one could call
// Initialize optional parameters for RandomForest().
param := mlpack.RandomForestOptions()
param.InputModel = &rf_model
param.Test = test_set
param.TestLabels = test_labels
_, predictions, _ := mlpack.RandomForest(param)
See also
 DecisionTree()
 HoeffdingTree()
 SoftmaxRegression()
 Random forest on Wikipedia
 Random forests (pdf)
 mlpack::tree::RandomForest C++ class documentation
Range Search
An implementation of range search with singletree and dualtree algorithms. Given a set of reference points and a set of query points and a range, this can find the set of reference points within the desired range for each query point, and any trees built during the computation can be saved for reuse with future range searches. .
Krann()
KRankApproximateNearestNeighbors (kRANN)
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for Krann().
param := mlpack.KrannOptions()
param.Alpha = 0.95
param.FirstLeafExact = false
param.InputModel = nil
param.K = 0
param.LeafSize = 20
param.Naive = false
param.Query = mat.NewDense(1, 1, nil)
param.RandomBasis = false
param.Reference = mat.NewDense(1, 1, nil)
param.SampleAtLeaves = false
param.Seed = 0
param.SingleMode = false
param.SingleSampleLimit = 20
param.Tau = 5
param.TreeType = "kd"
distances, neighbors, output_model := mlpack.Krann(param)
An implementation of rankapproximate knearestneighbor search (kRANN) using singletree and dualtree algorithms. Given a set of reference points and query points, this can find the k nearest neighbors in the reference set of each query point using trees; trees that are built can be saved for future use. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Alpha 
float64 
The desired success probability.  0.95 
FirstLeafExact 
bool 
The flag to trigger sampling only after exactly exploring the first leaf.  false 
InputModel 
rannModel 
Pretrained kNN model.  nil 
K 
int 
Number of nearest neighbors to find.  0 
LeafSize 
int 
Leaf size for tree building (used for kdtrees, UB trees, R trees, R* trees, X trees, Hilbert R trees, R+ trees, R++ trees, and octrees).  20 
Naive 
bool 
If true, sampling will be done without using a tree.  false 
Query 
*mat.Dense 
Matrix containing query points (optional).  mat.NewDense(1, 1, nil) 
RandomBasis 
bool 
Before treebuilding, project the data onto a random orthogonal basis.  false 
Reference 
*mat.Dense 
Matrix containing the reference dataset.  mat.NewDense(1, 1, nil) 
SampleAtLeaves 
bool 
The flag to trigger sampling at leaves.  false 
Seed 
int 
Random seed (if 0, std::time(NULL) is used).  0 
SingleMode 
bool 
If true, singletree search is used (as opposed to dualtree search.  false 
SingleSampleLimit 
int 
The limit on the maximum number of samples (and hence the largest node you can approximate).  20 
Tau 
float64 
The allowed rankerror in terms of the percentile of the data.  5 
TreeType 
string 
Type of tree to use: ‘kd’, ‘ub’, ‘cover’, ‘r’, ‘x’, ‘rstar’, ‘hilbertr’, ‘rplus’, ‘rplusplus’, ‘oct’.  "kd" 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

distances 
*mat.Dense 
Matrix to output distances into. 
neighbors 
*mat.Dense (with ints) 
Matrix to output neighbors into. 
outputModel 
rannModel 
If specified, the kNN model will be output here. 
Detailed documentation
This program will calculate the k rankapproximatenearestneighbors of a set of points. You may specify a separate set of reference points and query points, or just a reference set which will be used as both the reference and query set. You must specify the rank approximation (in %) (and optionally the success probability).
Example
For example, the following will return 5 neighbors from the top 0.1% of the data (with probability 0.95) for each point in input
and store the distances in distances
and the neighbors in neighbors.csv
:
// Initialize optional parameters for Krann().
param := mlpack.KrannOptions()
param.Reference = input
param.K = 5
param.Tau = 0.1
distances, neighbors, _ := mlpack.Krann(param)
Note that tau must be set such that the number of points in the corresponding percentile of the data is greater than k. Thus, if we choose tau = 0.1 with a dataset of 1000 points and k = 5, then we are attempting to choose 5 nearest neighbors out of the closest 1 point – this is invalid and the program will terminate with an error message.
The output matrices are organized such that row i and column j in the neighbors output file corresponds to the index of the point in the reference set which is the i’th nearest neighbor from the point in the query set with index j. Row i and column j in the distances output file corresponds to the distance between those two points.
See also
 Knn()
 Lsh()
 Rankapproximate nearest neighbor search: Retaining meaning and speed in high dimensions (pdf)
 mlpack::neighbor::RASearch C++ class documentation
SoftmaxRegression()
Softmax Regression
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for SoftmaxRegression().
param := mlpack.SoftmaxRegressionOptions()
param.InputModel = nil
param.Labels = mat.NewDense(1, 1, nil)
param.Lambda = 0.0001
param.MaxIterations = 400
param.NoIntercept = false
param.NumberOfClasses = 0
param.Test = mat.NewDense(1, 1, nil)
param.TestLabels = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
output_model, predictions := mlpack.SoftmaxRegression(param)
An implementation of softmax regression for classification, which is a multiclass generalization of logistic regression. Given labeled data, a softmax regression model can be trained and saved for future use, or, a pretrained softmax regression model can be used for classification of new points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

InputModel 
softmaxRegression 
File containing existing model (parameters).  nil 
Labels 
*mat.Dense (1d with ints) 
A matrix containing labels (0 or 1) for the points in the training set (y). The labels must order as a row.  mat.NewDense(1, 1, nil) 
Lambda 
float64 
L2regularization constant  0.0001 
MaxIterations 
int 
Maximum number of iterations before termination.  400 
NoIntercept 
bool 
Do not add the intercept term to the model.  false 
NumberOfClasses 
int 
Number of classes for classification; if unspecified (or 0), the number of classes found in the labels will be used.  0 
Test 
*mat.Dense 
Matrix containing test dataset.  mat.NewDense(1, 1, nil) 
TestLabels 
*mat.Dense (1d with ints) 
Matrix containing test labels.  mat.NewDense(1, 1, nil) 
Training 
*mat.Dense 
A matrix containing the training set (the matrix of predictors, X).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

outputModel 
softmaxRegression 
File to save trained softmax regression model to. 
predictions 
*mat.Dense (1d with ints) 
Matrix to save predictions for test dataset into. 
Detailed documentation
This program performs softmax regression, a generalization of logistic regression to the multiclass case, and has support for L2 regularization. The program is able to train a model, load an existing model, and give predictions (and optionally their accuracy) for test data.
Training a softmax regression model is done by giving a file of training points with the Training
parameter and their corresponding labels with the Labels
parameter. The number of classes can be manually specified with the NumberOfClasses
parameter, and the maximum number of iterations of the LBFGS optimizer can be specified with the MaxIterations
parameter. The L2 regularization constant can be specified with the Lambda
parameter and if an intercept term is not desired in the model, the NoIntercept
parameter can be specified.
The trained model can be saved with the OutputModel
output parameter. If training is not desired, but only testing is, a model can be loaded with the InputModel
parameter. At the current time, a loaded model cannot be trained further, so specifying both InputModel
and Training
is not allowed.
The program is also able to evaluate a model on test data. A test dataset can be specified with the Test
parameter. Class predictions can be saved with the Predictions
output parameter. If labels are specified for the test data with the TestLabels
parameter, then the program will print the accuracy of the predictions on the given test set and its corresponding labels.
Example
For example, to train a softmax regression model on the data dataset
with labels labels
with a maximum of 1000 iterations for training, saving the trained model to sr_model
, the following command can be used:
// Initialize optional parameters for SoftmaxRegression().
param := mlpack.SoftmaxRegressionOptions()
param.Training = dataset
param.Labels = labels
sr_model, _ := mlpack.SoftmaxRegression(param)
Then, to use sr_model
to classify the test points in test_points
, saving the output predictions to predictions
, the following command can be used:
// Initialize optional parameters for SoftmaxRegression().
param := mlpack.SoftmaxRegressionOptions()
param.InputModel = &sr_model
param.Test = test_points
_, predictions := mlpack.SoftmaxRegression(param)
See also
 LogisticRegression()
 RandomForest()
 Multinomial logistic regression (softmax regression) on Wikipedia
 mlpack::regression::SoftmaxRegression C++ class documentation
SparseCoding()
Sparse Coding
import (
"mlpack.org/v1/mlpack"
"gonum.org/v1/gonum/mat"
)
// Initialize optional parameters for SparseCoding().
param := mlpack.SparseCodingOptions()
param.Atoms = 15
param.InitialDictionary = mat.NewDense(1, 1, nil)
param.InputModel = nil
param.Lambda1 = 0
param.Lambda2 = 0
param.MaxIterations = 0
param.NewtonTolerance = 1e06
param.Normalize = false
param.ObjectiveTolerance = 0.01
param.Seed = 0
param.Test = mat.NewDense(1, 1, nil)
param.Training = mat.NewDense(1, 1, nil)
codes, dictionary, output_model := mlpack.SparseCoding(param)
An implementation of Sparse Coding with Dictionary Learning. Given a dataset, this will decompose the dataset into a sparse combination of a few dictionary elements, where the dictionary is learned during computation; a dictionary can be reused for future sparse coding of new points. Detailed documentation.
Input options
There are two types of input options: required options, which are passed directly to the function call, and optional options, which are passed via an initialized struct, which allows keyword access to each of the options.
name  type  description  default 

Atoms 
int 
Number of atoms in the dictionary.  15 
InitialDictionary 
*mat.Dense 
Optional initial dictionary matrix.  mat.NewDense(1, 1, nil) 
InputModel 
sparseCoding 
File containing input sparse coding model.  nil 
Lambda1 
float64 
Sparse coding l1norm regularization parameter.  0 
Lambda2 
float64 
Sparse coding l2norm regularization parameter.  0 
MaxIterations 
int 
Maximum number of iterations for sparse coding (0 indicates no limit).  0 
NewtonTolerance 
float64 
Tolerance for convergence of Newton method.  1e06 
Normalize 
bool 
If set, the input data matrix will be normalized before coding.  false 
ObjectiveTolerance 
float64 
Tolerance for convergence of the objective function.  0.01 
Seed 
int 
Random seed. If 0, ‘std::time(NULL)’ is used.  0 
Test 
*mat.Dense 
Optional matrix to be encoded by trained model.  mat.NewDense(1, 1, nil) 
Training 
*mat.Dense 
Matrix of training data (X).  mat.NewDense(1, 1, nil) 
Verbose 
bool 
Display informational messages and the full list of parameters and timers at the end of execution.  false 
Output options
Output options are returned via Go’s support for multiple return values, in the order listed below.
name  type  description 

codes 
*mat.Dense 
Matrix to save the output sparse codes of the test matrix (–test_file) to. 
dictionary 
*mat.Dense 
Matrix to save the output dictionary to. 
outputModel 
sparseCoding 
File to save trained sparse coding model to. 
Detailed documentation
An implementation of Sparse Coding with Dictionary Learning, which achieves sparsity via an l1norm regularizer on the codes (LASSO) or an (l1+l2)norm regularizer on the codes (the Elastic Net). Given a dense data matrix X with d dimensions and n points, sparse coding seeks to find a dense dictionary matrix D with k atoms in d dimensions, and a sparse coding matrix Z with n points in k dimensions.
The original data matrix X can then be reconstructed as Z * D. Therefore, this program finds a representation of each point in X as a sparse linear combination of atoms in the dictionary D.
The sparse coding is found with an algorithm which alternates between a dictionary step, which updates the dictionary D, and a sparse coding step, which updates the sparse coding matrix.
Once a dictionary D is found, the sparse coding model may be used to encode other matrices, and saved for future usage.
To run this program, either an input matrix or an alreadysaved sparse coding model must be specified. An input matrix may be specified with the Training
option, along with the number of atoms in the dictionary (specified with the Atoms
parameter). It is also possible to specify an initial dictionary for the optimization, with the InitialDictionary
parameter. An input model may be specified with the InputModel
parameter.
Example
As an example, to build a sparse coding model on the dataset data
using 200 atoms and an l1regularization parameter of 0.1, saving the model into model
, use
// Initialize optional parameters for SparseCoding().
param := mlpack.SparseCodingOptions()
param.Training = data
param.Atoms = 200
param.Lambda1 = 0.1
_, _, model := mlpack.SparseCoding(param)
Then, this model could be used to encode a new matrix, otherdata
, and save the output codes to codes
:
// Initialize optional parameters for SparseCoding().
param := mlpack.SparseCodingOptions()
param.InputModel = &model
param.Test = otherdata
codes, _, _ := mlpack.SparseCoding(param)
See also
 LocalCoordinateCoding()
 Sparse dictionary learning on Wikipedia
 Efficient sparse coding algorithms (pdf)
 Regularization and variable selection via the elastic net
 mlpack::sparse_coding::SparseCoding C++ class documentation
changelog/history
mlpack 3.4.2
20201026

Added Mean Absolute Percentage Error.

Added Softmin activation function as layer in ann/layer.

Fix spurious ARMA_64BIT_WORD compilation warnings on 32bit systems #2665.
mlpack 3.4.1
20200907

Fix incorrect parsing of required matrix/model parameters for commandline bindings #2600.

Add manual type specification support to
data::Load()
anddata::Save()
(#2084, #2135, #2602). 
Remove use of internal Armadillo functionality (#2596, #2601, #2602).
mlpack 3.4.0
20200901

Issue warnings when metrics produce NaNs in KFoldCV #2595.

Added bindings for R during Google Summer of Code #2556.

Added common striptype function for all bindings #2556.

Refactored common utility function of bindings to bindings/util #2556.

Renamed InformationGain to HoeffdingInformationGain in methods/hoeffding_trees/information_gain.hpp #2556.

Added macro for changing stream of printing and warnings/errors #2556.

Added Spatial Dropout layer #2564.

Force CMake to show error when it didn’t find Python/modules #2568.

Refactor
ProgramInfo()
to separate out all the different information #2558. 
Add bindings for onehot encoding #2325.

Added Soft ActorCritic to RL methods #2487.

Added Categorical DQN to q_networks #2454.

Added Nstep DQN to q_networks #2461.

Add Silhoutte Score metric and Pairwise Distances #2406.

Add Go bindings for some missed models #2460.

Replace boost program_options dependency with CLI11 #2459.

Additional functionality for the ARFF loader #2486; use case sensitive categories #2516.

Add
bayesian_linear_regression
binding for the commandline, Python, Julia, and Go. Also called “Bayesian Ridge”, this is equivalent to a version of linear regression where the regularization parameter is automatically tuned #2030. 
Fix defeatist search for spill tree traversals (#2566, #1269).

Fix incremental training of logistic regression models #2560.

Change default configuration of
BUILD_PYTHON_BINDINGS
toOFF
#2575.
mlpack 3.3.2
20200618

Added Noisy DQN to q_networks #2446.

Add Go bindings #1884.

Added Dueling DQN to q_networks, Noisy linear layer to ann/layer and Empty loss to ann/loss_functions #2414.

Storing and adding accessor method for action in q_learning #2413.

Added accessor methods for ANN layers #2321.

Addition of
Elliot
activation function #2268. 
Add adaptive max pooling and adaptive mean pooling layers #2195.

Add parameter to avoid shuffling of data in preprocess_split #2293.

Add
MatType
parameter toLSHSearch
, allowing sparse matrices to be used for search #2395. 
Documentation fixes to resolve Doxygen warnings and issues #2400.

Add Load and Save of Sparse Matrix #2344.

Add Intersection over Union (IoU) metric for bounding boxes #2402.

Add Non Maximal Supression (NMS) metric for bounding boxes #2410.

Fix
no_intercept
and probability computation for linear SVM bindings #2419. 
Fix incorrect neighbors for
k > 1
searches inapprox_kfn
binding, for theQDAFN
algorithm #2448. 
Fix serialization of kernels with state for FastMKS #2452.

Add
RBF
layer in ann module to makeRBFN
architecture #2261.
mlpack 3.3.1
20200429

Minor Julia and Python documentation fixes #2373.

Updated terminal state and fixed bugs for Pendulum environment (#2354, #2369).

Added
EliSH
activation function #2323. 
Add L1 Loss function #2203.

Pass CMAKE_CXX_FLAGS (compilation options) correctly to Python build #2367.

Expose ensmallen Callbacks for sparseautoencoder #2198.

Bugfix for LARS class causing invalid read #2374.

Add serialization support from Julia; use
mlpack.serialize()
andmlpack.deserialize()
to save and load fromIOBuffer
s.
mlpack 3.3.0
20200407

Added
Normal Distribution
toann/dists
#2382. 
Templated return type of
Forward function
of loss functions #2339. 
Added
R2 Score
regression metric #2323. 
Added
poisson negative log likelihood
loss function #2196. 
Added
huber
loss function #2199. 
Added
mean squared logarithmic error
loss function for neural networks #2210. 
Added
mean bias loss function
for neural networks #2210. 
The DecisionStump class has been marked deprecated; use the
DecisionTree
class withNoRecursion=true
or useID3DecisionStump
instead #2099. 
Added
probabilities_file
parameter to get the probabilities matrix of AdaBoost classifier #2050. 
Fix STB header search paths #2104.

Add
DISABLE_DOWNLOADS
CMake configuration option #2104. 
Add padding layer in TransposedConvolutionLayer #2082.

Fix pkgconfig generation on nonLinux systems #2101.

Use logspace to represent HMM initial state and transition probabilities #2081.

Add functions to access parameters of
Convolution
andAtrousConvolution
layers #1985. 
Add Compute Error function in lars regression and changing Train function to return computed error #2139.

Add Julia bindings #1949. Build settings can be controlled with the
BUILD_JULIA_BINDINGS=(ON/OFF)
andJULIA_EXECUTABLE=/path/to/julia
CMake parameters. 
CMake fix for finding STB include directory #2145.

Add bindings for loading and saving images #2019;
mlpack_image_converter
from the commandline,mlpack.image_converter()
from Python. 
Add normalization support for CF binding #2136.

Add Mish activation function #2158.

Update
init_rules
in AMF to allow users to merge two initialization rules #2151. 
Add GELU activation function #2183.

Better error handling of eigendecompositions and Cholesky decompositions (#2088, #1840).

Add LiSHT activation function #2182.

Add Valid and Same Padding for Transposed Convolution layer #2163.

Add CELU activation function #2191

Add LogHyperbolicCosine Loss function #2207.

Change neural network types to avoid unnecessary use of rvalue references #2259.

Bump minimum Boost version to 1.58 #2305.

Refactor STB support so
HAS_STB
macro is not needed when compiling against mlpack #2312. 
Add Hard Shrink Activation Function #2186.

Add Soft Shrink Activation Function #2174.

Add Hinge Embedding Loss Function #2229.

Add Cosine Embedding Loss Function #2209.

Add Margin Ranking Loss Function #2264.

Bugfix for incorrect parameter vector sizes in logistic regression and softmax regression #2359.
mlpack 3.2.2
20191126

Add
valid
andsame
padding option inConvolution
andAtrous Convolution
layer #1988. 
Add Model() to the FFN class to access individual layers #2043.

Update documentation for pip and conda installation packages #2044.

Add bindings for linear SVM #1935;
mlpack_linear_svm
from the commandline,linear_svm()
from Python. 
Add support to return the layer name as
std::string
#1987. 
Speed and memory improvements for the Transposed Convolution layer #1493.

Fix Windows Python build configuration #1885.

Validate md5 of STB library after download #2087.

Add
__version__
to__init__.py
#2092. 
Correctly handle RNN sequences that are shorter than the value of rho #2102.
mlpack 3.2.1
20191001

Enforce CMake version check for ensmallen #2032.

Fix CMake check for Armadillo version #2029.

Better handling of when STB is not installed #2033.

Fix Naive Bayes classifier computations in high dimensions #2022.
mlpack 3.2.0
20190925

Fix some potential infinity errors in Naive Bayes Classifier #2022.

Fix occasionallyfailing RADICAL test #1924.

Fix gcc 9 OpenMP compilation issue #1970.

Added support for loading and saving of images #1903.

Add Multiple Pole Balancing Environment (#1901, #1951).

Added functionality for scaling of data #1876; see the commandline binding
mlpack_preprocess_scale
or Python bindingpreprocess_scale()
. 
Add new parameter
maximum_depth
to decision tree and random forest bindings #1916. 
Fix prediction output of softmax regression when test set accuracy is calculated #1922.

Pendulum environment now checks for termination. All RL environments now have an option to terminate after a set number of time steps (no limit by default) #1941.

Add support for probabilistic KDE (kernel density estimation) error bounds when using the Gaussian kernel #1934.

Fix negative distances for cover tree computation #1979.

Fix cover tree building when all pairwise distances are 0 #1986.

Improve KDE pruning by reclaiming not used error tolerance (#1954, #1984).

Optimizations for sparse matrix accesses in zscore normalization for CF #1989.

Add
kmeans_max_iterations
option to GMM training bindinggmm_train_main
. 
Bump minimum Armadillo version to 8.400.0 due to ensmallen dependency requirement #2015.
mlpack 3.1.1
20190526

Fix random forest bug for numericalonly data #1887.

Significant speedups for random forest #1887.

Random forest now has
minimum_gain_split
andsubspace_dim
parameters #1887. 
Decision tree parameter
print_training_error
deprecated in favor ofprint_training_accuracy
. 
output
option changed topredictions
for adaboost and perceptron binding. Old options are now deprecated and will be preserved until mlpack 4.0.0 #1882. 
Concatenated ReLU layer #1843.

Accelerate NormalizeLabels function using hashing instead of linear search (see
src/mlpack/core/data/normalize_labels_impl.hpp
) #1780. 
Add
ConfusionMatrix()
function for checking performance of classifiers #1798. 
Install ensmallen headers when it is downloaded during build #1900.
mlpack 3.1.0
20190425

Add DiagonalGaussianDistribution and DiagonalGMM classes to speed up the diagonal covariance computation and deprecate DiagonalConstraint #1666.

Add kernel density estimation (KDE) implementation with bindings to other languages #1301.

Where relevant, all models with a
Train()
method now return adouble
value representing the goodness of fit (i.e. final objective value, error, etc.) #1678. 
Add implementation for linear support vector machine (see
src/mlpack/methods/linear_svm
). 
Change DBSCAN to use PointSelectionPolicy and add OrderedPointSelection #1625.

Residual block support #1594.

Bidirectional RNN #1626.

Dice loss layer (#1674, #1714) and hard sigmoid layer #1776.

output
option changed topredictions
andoutput_probabilities
toprobabilities
for Naive Bayes binding (mlpack_nbc
/nbc()
). Old options are now deprecated and will be preserved until mlpack 4.0.0 #1616. 
Add support for Diagonal GMMs to HMM code (#1658, #1666). This can provide large speedup when a diagonal GMM is acceptable as an emission probability distribution.

Python binding improvements: check parameter type #1717, avoid copying Pandas dataframes #1711, handle Pandas Series objects #1700.
mlpack 3.0.4
20181113

Bump minimum CMake version to 3.3.2.

CMake fixes for Ninja generator by Marc Espie.
mlpack 3.0.3
20180727

Fix Visual Studio compilation issue #1443.

Allow running local_coordinate_coding binding with no initial_dictionary parameter when input_model is not specified #1457.

Make use of OpenMP optional via the CMake ‘USE_OPENMP’ configuration variable #1474.

Accelerate FNN training by 2030% by avoiding redundant calculations #1467.

Fix math::RandomSeed() usage in tests (#1462, #1440).

Generate better Python setup.py with documentation #1460.
mlpack 3.0.2
20180608

Documentation generation fixes for Python bindings #1421.

Fix build error for man pages if commandline bindings are not being built #1424.

Add ‘shuffle’ parameter and Shuffle() method to KFoldCV #1412. This will shuffle the data when the object is constructed, or when Shuffle() is called.

Added neural network layers: AtrousConvolution #1390, Embedding #1401, and LayerNorm (layer normalization) #1389.

Add Pendulum environment for reinforcement learning #1388 and update Mountain Car environment #1394.
mlpack 3.0.1
20180510

Fix intermittently failing tests #1387.

Add bigbatch SGD (BBSGD) optimizer in src/mlpack/core/optimizers/bigbatch_sgd/ #1131.

Fix simple compiler warnings (#1380, #1373).

Simplify NeighborSearch constructor and Train() overloads #1378.

Add warning for OpenMP setting differences (#1358/#1382). When mlpack is compiled with OpenMP but another application is not (or vice versa), a compilation warning will now be issued.

Restructured loss functions in src/mlpack/methods/ann/ #1365.

Add environments for reinforcement learning tests (#1368, #1370, #1329).

Allow single outputs for multiple timestep inputs for recurrent neural networks #1348.

Add He and LeCun normal initializations for neural networks #1342. Neural networks: add He and LeCun normal initializations #1342, add FReLU and SELU activation functions (#1346, #1341), add alphadropout #1349.
mlpack 3.0.0
20180330

Speed and memory improvements for DBSCAN. –single_mode can now be used for situations where previously RAM usage was too high.

Bump minimum required version of Armadillo to 6.500.0.

Add automatically generated Python bindings. These have the same interface as the commandline programs.

Add deep learning infrastructure in src/mlpack/methods/ann/.

Add reinforcement learning infrastructure in src/mlpack/methods/reinforcement_learning/.

Add optimizers: AdaGrad, CMAES, CNE, FrankeWolfe, GradientDescent, GridSearch, IQN, Katyusha, LineSearch, ParallelSGD, SARAH, SCD, SGDR, SMORMS3, SPALeRA, SVRG.

Add hyperparameter tuning infrastructure and crossvalidation infrastructure in src/mlpack/core/cv/ and src/mlpack/core/hpt/.

Fix bug in mean shift.

Add random forests (see src/mlpack/methods/random_forest).

Numerous other bugfixes and testing improvements.

Add randomized Krylov SVD and Block Krylov SVD.
mlpack 2.2.5
20170825
mlpack 2.2.4
20170718

Speed and memory improvements for DBSCAN. –single_mode can now be used for situations where previously RAM usage was too high.

Fix bug in CF causing incorrect recommendations.
mlpack 2.2.3
20170524
 Bug fix for –predictions_file in mlpack_decision_tree program.
mlpack 2.2.2
20170504

Install backwardscompatibility mlpack_allknn and mlpack_allkfn programs; note they are deprecated and will be removed in mlpack 3.0.0 #992.

Fix RStarTree bug that surfaced on OS X only #964.

Small fixes for MiniBatchSGD and SGD and tests.
mlpack 2.2.1
20170413
 Compilation fix for mlpack_nca and mlpack_test on older Armadillo versions #984.
mlpack 2.2.0
20170321

Bugfix for mlpack_knn program #816.

Add decision tree implementation in methods/decision_tree/. This is very similar to a C4.5 tree learner.

Add DBSCAN implementation in methods/dbscan/.

Add support for multidimensional discrete distributions (#810, #830).

Better output for Log::Debug/Log::Info/Log::Warn/Log::Fatal for Armadillo objects (#895, #928).

Refactor categorical CSV loading with boost::spirit for faster loading #681.
mlpack 2.1.1
20161222

HMMs now use random initialization; this should fix some convergence issues #828.

HMMs now initialize emissions according to the distribution of observations #833.

Minor fix for formatted output #814.

Fix DecisionStump to properly work with any input type.
mlpack 2.1.0
20161031

Fixed CoverTree to properly handle singlepoint datasets.

Fixed a bug in CosineTree (and thus QUICSVD) that caused split failures for some datasets #717.

Added mlpack_preprocess_describe program, which can be used to print statistics on a given dataset #742.

Fix prioritized recursion for kfurthestneighbor search (mlpack_kfn and the KFN class), leading to ordersofmagnitude speedups in some cases.

Bump minimum required version of Armadillo to 4.200.0.

Added simple Gradient Descent optimizer, found in src/mlpack/core/optimizers/gradient_descent/ #792.

Added approximate furthest neighbor search algorithms QDAFN and DrusillaSelect in src/mlpack/methods/approx_kfn/, with commandline program mlpack_approx_kfn.
mlpack 2.0.3
20160721

Added multiprobe LSH #691. The parameter ‘T’ to LSHSearch::Search() can now be used to control the number of extra bins that are probed, as can the T (–num_probes) option to mlpack_lsh.

Added the Hilbert R tree to src/mlpack/core/tree/rectangle_tree/ #664. It can be used as the typedef HilbertRTree, and it is now an option in the mlpack_knn, mlpack_kfn, mlpack_range_search, and mlpack_krann commandline programs.

Added the mlpack_preprocess_split and mlpack_preprocess_binarize programs, which can be used for preprocessing code (#650, #666).

Added OpenMP support to LSHSearch and mlpack_lsh #700.
mlpack 2.0.2
20160620

Added the function LSHSearch::Projections(), which returns an arma::cube with each projection table in a slice #663. Instead of Projection(i), you should now use Projections().slice(i).

A new constructor has been added to LSHSearch that creates objects using projection tables provided in an arma::cube #663.

Handle zerovariance dimensions in DET #515.

Add MiniBatchSGD optimizer (src/mlpack/core/optimizers/minibatch_sgd/) and allow its use in mlpack_logistic_regression and mlpack_nca programs.

Add better backtrace support from Grzegorz Krajewski for Log::Fatal messages when compiled with debugging and profiling symbols. This requires libbfd and libdl to be present during compilation.

CosineTree test fix from Mikhail Lozhnikov #358.

Fixed HMM initial state estimation #600.

Changed versioning macros __MLPACK_VERSION_MAJOR, __MLPACK_VERSION_MINOR, and __MLPACK_VERSION_PATCH to MLPACK_VERSION_MAJOR, MLPACK_VERSION_MINOR, and MLPACK_VERSION_PATCH. The old names will remain in place until mlpack 3.0.0.

Renamed mlpack_allknn, mlpack_allkfn, and mlpack_allkrann to mlpack_knn, mlpack_kfn, and mlpack_krann. The mlpack_allknn, mlpack_allkfn, and mlpack_allkrann programs will remain as copies until mlpack 3.0.0.

Add –random_initialization option to mlpack_hmm_train, for use when no labels are provided.

Add –kill_empty_clusters option to mlpack_kmeans and KillEmptyClusters policy for the KMeans class (#595, #596).
mlpack 2.0.1
20160204

Fix CMake to properly detect when MKL is being used with Armadillo.

Minor parameter handling fixes to mlpack_logistic_regression (#504, #505).

Properly install arma_config.hpp.

Memory handling fixes for Hoeffding tree code.

Add functions that allow changing trainingtime parameters to HoeffdingTree class.

Fix infinite loop in sparse coding test.

Documentation spelling fixes #501.

Properly handle covariances for Gaussians with large condition number #496, preventing GMMs from filling with NaNs during training (and also HMMs that use GMMs).

CMake fixes for finding LAPACK and BLAS as Armadillo dependencies when ATLAS is used.

CMake fix for projects using mlpack’s CMake configuration from elsewhere #512.
mlpack 2.0.0
20151224

Removed overclustering support from kmeans because it is not welltested, may be buggy, and is (I think) unused. If this was support you were using, open a bug or get in touch with us; it would not be hard for us to reimplement it.

Refactored KMeans to allow different types of Lloyd iterations.

Added implementations of kmeans: Elkan’s algorithm, Hamerly’s algorithm, PellegMoore’s algorithm, and the DTNN (dualtree nearest neighbor) algorithm.

Significant acceleration of LRSDP via the use of accu(a % b) instead of trace(a * b).

Added MatrixCompletion class (matrix_completion), which performs nuclear norm minimization to fill unknown values of an input matrix.

No more dependence on Boost.Random; now we use C++11 STL random support.

Add softmax regression, contributed by Siddharth Agrawal and QiaoAn Chen.

Changed NeighborSearch, RangeSearch, FastMKS, LSH, and RASearch API; these classes now take the query sets in the Search() method, instead of in the constructor.

Use OpenMP, if available. For now OpenMP support is only available in the DET training code.

Add support for predicting new test point values to LARS and the commandline ‘lars’ program.

Add serialization support for Perceptron and LogisticRegression.

Refactor SoftmaxRegression to predict into an arma::Row
object, and add a softmax_regression program. 
Refactor LSH to allow loading and saving of models.

ToString() is removed entirely #487.

Add –input_model_file and –output_model_file options to appropriate machine learning algorithms.

Rename all executables to start with an “mlpack” prefix #229.

Add HoeffdingTree and mlpack_hoeffding_tree, an implementation of the streaming decision tree methodology from Domingos and Hulten in 2000.
mlpack 1.0.12
20150107
 Switch to 3clause BSD license (from LGPL).
mlpack 1.0.11
20141211

Proper handling of dimension calculation in PCA.

Load parameter vectors properly for LinearRegression models.

Linker fixes for AugLagrangian specializations under Visual Studio.

Add support for observation weights to LinearRegression.

MahalanobisDistance<> now takes the root of the distance by default and therefore satisfies the triangle inequality (TakeRoot now defaults to true).

Better handling of optional Armadillo HDF5 dependency.

Fixes for numerous intermittent test failures.

math::RandomSeed() now sets the random seed for recent (>=3.930) Armadillo versions.

Handle Newton method convergence better for SparseCoding::OptimizeDictionary() and make maximum iterations a parameter.

Known bug: CosineTree construction may fail in some cases on i386 systems #358.
mlpack 1.0.10
20140829

Bugfix for NeighborSearch regression which caused very slow allknn/allkfn. Speeds are now restored to approximately 1.0.8 speeds, with significant improvement for the cover tree #347.

Detect dependencies correctly when ARMA_USE_WRAPPER is not being defined (i.e., libarmadillo.so does not exist).

Bugfix for compilation under Visual Studio #348.
mlpack 1.0.9
20140728

GMM initialization is now safer and provides a working GMM when constructed with only the dimensionality and number of Gaussians #301.

Check for division by 0 in ForwardBackward Algorithm in HMMs #301.

Fix MaxVarianceNewCluster (used when reinitializing clusters for kmeans) #301.

Fixed implementation of Viterbi algorithm in HMM::Predict() #303.

Significant speedups for dualtree algorithms using the cover tree (#235, #314) including a faster implementation of FastMKS.

Fix for LRSDP optimizer so that it compiles and can be used #312.

CF (collaborative filtering) now expects users and items to be zeroindexed, not oneindexed #311.

CF::GetRecommendations() API change: now requires the number of recommendations as the first parameter. The number of users in the local neighborhood should be specified with CF::NumUsersForSimilarity().

Removed incorrect PeriodicHRectBound #58.

Refactor LRSDP into LRSDP class and standalone function to be optimized #305.

Fix for centering in kernel PCA #337.

Added simulated annealing (SA) optimizer, contributed by Zhihao Lou.

HMMs now support initial state probabilities; these can be set in the constructor, trained, or set manually with HMM::Initial() #302.

Added Nyström method for kernel matrix approximation by Marcus Edel.

Kernel PCA now supports using Nyström method for approximation.

Ball trees now work with dualtree algorithms, via the BallBound<> bound structure #307; fixed by Yash Vadalia.

The NMF class is now AMF<>, and supports far more types of factorizations, by Sumedh Ghaisas.

A QUICSVD implementation has returned, written by Siddharth Agrawal and based on older code from Mudit Gupta.

Added perceptron and decision stump by Udit Saxena (these are weak learners for an eventual AdaBoost class).

Sparse autoencoder added by Siddharth Agrawal.
mlpack 1.0.8
20140106

Memory leak in NeighborSearch indexmapping code fixed #298.

GMMs can be trained using the existing model as a starting point by specifying an additional boolean parameter to GMM::Estimate() #296.

Logistic regression implementation added in methods/logistic_regression (see also #293).

LBFGS optimizer now returns its function via Function().

Version information is now obtainable via mlpack::util::GetVersion() or the __MLPACK_VERSION_MAJOR, __MLPACK_VERSION_MINOR, and __MLPACK_VERSION_PATCH macros #297.

Fix typos in allkfn and allkrann output.
mlpack 1.0.7
20131004

Cover tree support for range search (range_search), rankapproximate nearest neighbors (allkrann), minimum spanning tree calculation (emst), and FastMKS (fastmks).

Dualtree FastMKS implementation added and tested.

Added collaborative filtering package (cf) that can provide recommendations when given users and items.

Fix for correctness of Kernel PCA (kernel_pca) #270.

Speedups for PCA and Kernel PCA #198.

Fix for correctness of Neighborhood Components Analysis (NCA) #279.

Minor speedups for dualtree algorithms.

Fix for Naive Bayes Classifier (nbc) #269.

Added a ridge regression option to LinearRegression (linear_regression) #286.

Gaussian Mixture Models (gmm::GMM<>) now support arbitrary covariance matrix constraints #283.

MVU (mvu) removed because it is known to not work #183.

Minor updates and fixes for kernels (in mlpack::kernel).
mlpack 1.0.6
20130613
 Minor bugfix so that FastMKS gets built.
mlpack 1.0.5
20130501

Speedups of cover tree traversers #235.

Addition of rankapproximate nearest neighbors (RANN), found in src/mlpack/methods/rann/.

Addition of fast exact maxkernel search (FastMKS), found in src/mlpack/methods/fastmks/.

Fix for EM covariance estimation; this should improve GMM training time.

More parameters for GMM estimation.

Force GMM and GaussianDistribution covariance matrices to be positive definite, so that training converges much more often.

Add parameter for the tolerance of the BaumWelch algorithm for HMM training.

Fix for compilation with clang compiler.

Fix for kfurthestneighborsearch.
mlpack 1.0.4
20130208

Force minimum Armadillo version to 2.4.2.

Better output of class types to streams; a class with a ToString() method implemented can be sent to a stream with operator«.

Change return type of GMM::Estimate() to double #257.

Style fixes for kmeans and RADICAL.

Handle size_t support correctly with Armadillo 3.6.2 #258.

Add localitysensitive hashing (LSH), found in src/mlpack/methods/lsh/.

Better tests for SGD (stochastic gradient descent) and NCA (neighborhood components analysis).
mlpack 1.0.3
20120916

Remove internal sparse matrix support because Armadillo 3.4.0 now includes it. When using Armadillo versions older than 3.4.0, sparse matrix support is not available.

NCA (neighborhood components analysis) now support an arbitrary optimizer #245, including stochastic gradient descent #249.
mlpack 1.0.2
20120815

Added density estimation trees, found in src/mlpack/methods/det/.

Added nonnegative matrix factorization, found in src/mlpack/methods/nmf/.

Added experimental cover tree implementation, found in src/mlpack/core/tree/cover_tree/ #157.

Better reporting of boost::program_options errors #225.

Fix for timers on Windows (#212, #211).

Fix for allknn and allkfn output #204.

Sparse coding dictionary initialization is now a template parameter #220.
mlpack 1.0.1
20120303

Added kernel principal components analysis (kernel PCA), found in src/mlpack/methods/kernel_pca/ #74.

Fix for LovaszTheta AugLagrangian tests #182.

Fixes for allknn output (#185, #186).

Added range search executable #192.

Adapted citations in documentation to BibTeX; no citations in h output #195.

Stop use of ‘const char*’ and prefer ‘std::string’ #176.

Support seeds for random numbers #177.
mlpack 1.0.0
20111217
 Initial release. See any resolved tickets numbered less than #196 or execute this query: http://www.mlpack.org/trac/query?status=closed&milestone=mlpack+1.0.0