mlpack has a system to automatically generate bindings to other languages, such as Python and command-line programs, and it is extensible to other languages with some amount of ease. The maintenance burden of this system is low, and it is designed in such a way that the bindings produced are always up to date across languages and up to date with the mlpack library itself.
This document describes the full functioning of the system, and is a good place to start for someone who wishes to understand the system so that they can contribute a new binding language, or someone who wants to understand so they can adapt the system for use in their own project, or someone who is simply curious enough to see how the sausage is made.
The document is split into several sections:
C++ is not the most popular language on the planet, and it (unfortunately) can scare many away with its ultra-verbose error messages, confusing template rules, and complex metaprogramming techniques. Most practitioners of machine learning tend to avoid writing native C++ and instead prefer other languages—probably most notably Python.
In the case of Python, many projects will use tools like SWIG (http://www.swig.org/) to automatically generate bindings, or they might hand-write Cython. The same types of strategies may be used for other languages; hand-written MEX files may be used for MATLAB, hand-written RCpp bindings might be used for R bindings, and so forth.
However, these approaches have a fundamental flaw: the hand-written bindings must be maintained, and risk going out of date as the rest of the library changes or new functionality is added. This incurs a maintenance burden: each major change to the library means that someone must update the bindings and test that they are still working. mlpack is not prepared to handle this maintenance workload; therefore an alternate solution is needed.
At the time of the design of this system, mlpack shipped headers for a C++ library as well as many (~40) hand-written command-line programs that used the mlpack::IO object to manage command-line arguments. These programs all had similar structure, and could be logically split into three sections:
The user might interface with this command-line program like the following:
That is, they would pass a number of input options—some were numeric values (like
3 ); some were filenames (like
reference.csv ); and a few other types also. Therefore, the first stage of the program—parsing input options—would be handled by reading the command line and loading any input matrices. Preparing the output, which usually consists of data matrices (i.e.
d.csv ) involves saving the matrix returned by the algorithm to the user's desired file.
Ideally, any binding to any language would have this same structure, and the actual "run the machine learning algorithm" code could be identical. For MATLAB, for instance, we would not need to read the file
reference.csv but instead the user would simply pass their data matrix as an argument. So each input and output parameter would need to be handled differently, but the algorithm could be run identically across all bindings.
Therefore, design of an automatically-generated binding system would simply involve generating the boilerplate code necessary to parse input options for a given language, and to return output options to a user.
This section details what a binding file might actually look like. It is good to have this API in mind when reading the following sections.
Each mlpack binding is typically contained in the
src/mlpack/methods/ folder corresponding to a given machine learning algorithm, with the suffix
_main.cpp ; so an example is
These files have roughly two parts:
mlpackMain(), which is the actual machine learning code
Here is a simple example file:
We can see that we have defined the basic program information in the
BINDING_SEE_ALSO() macros. This is, for instance, what is displayed to describe the binding if the user passed the
--help option for a command-line program.
Then, we define five parameters, three input and two output, that define the data and options that the mean shift clustering will function on. These parameters are defined with the
PARAM macros, of which there are many. The names of these macros specify the type, whether the parameter is required, and whether the parameter is input or output. Some examples:
PARAM_STRING_IN()– a string-type input parameter
PARAM_MATRIX_OUT()– a matrix-type output parameter
PARAM_DOUBLE_IN_REQ()– a required double-type input parameter
PARAM_UMATRIX_IN()– an unsigned matrix-type input parameter
PARAM_MODEL_IN()– a serializable model-type input parameter
Note that each of these macros may have slightly different syntax. See the links above for further documentation.
In order to write a new binding, then, you simply must write
BINDING_SEE_ALSO() definitions of the program with some docuentation, define the input and output parameters as
PARAM macros, and then write an
mlpackMain() function that actually performs the functionality of the binding. Inside of
Then, assuming that your program is saved in the file
program_name_main.cpp, generating bindings for other languages is a simple addition to the
In this example,
add_markdown_docs() will generate documentation that is typically used to build the website. The "category" parameter should be one of the categories in
This section describes the general structure of the
IO code and how one might write a new binding for mlpack. After reading this section it should be relatively clear how one could use the
IO functionality along with CMake to add a binding for a new mlpack machine learning method. If it is not clear, then the examples in the following sections should clarify.
Any mlpack program should be documented with the
BINDING_SEE_ALSO() macros, which is available from the
<mlpack/core/util/mlpack_main.hpp> header. The macros are of the form
The short documentation should be two sentences indicating what the program implements and does, and a quick overview of how it can be used and what it should be used for. When writing new short documentation, it is a good idea to take a look at the existing documentation to get an idea of the general format.
For the "see also" section, you can specify as many
SEE_ALSO() calls as you see fit. These are links used at the "see also" section of the website documentation for each binding, and it's very important that relevant links are provided (also to other bindings). See the
SEE_ALSO() documentation for more details.
Although it is possible to provide very short documentation, it is certainly better to provide a long description including
Examples are very important, and are probably what most users are going to immediately search for, instead of taking a long time to read and carefully consider all of the written documentation.
However, it is difficult to write language-agnostic documentation. For instance, in a command-line program, an output parameter '--output_file' would be specified on the command line as an input parameter, but in Python, the output parameter 'output' would actually simply be returned from the call to the Python function. Therefore, we must be careful how our documentation refers to input and output parameters. The following general guidelines can help:
PRINT_PARAM_STRING()macro to print the names of parameters. For instance,
'--shuffle'for a command line program and
'shuffle'for a Python binding. The
PRINT_PARAM_STRING()macro also takes into account the type of the parameter.
PRINT_MODEL()macro to introduce example datasets or models, which can be useful when introducing an example usage of the program. So you could write
'"to @c run @c with @c a @c dataset @c "
PRINT_CALL()macro to print example invocations of the program. The first argument is the name of the program, and then the following arguments should be the name of a parameter followed by the value of that parameter.
'the results may be saved using the PRINT_PARAM_STRING("output") parameter', and not
'the results are returned through the PRINT_PARAM_STRING("output") parameter'.
Each of these macros (
PRINT_CALL() ) provides different output depending on the language. Below are some example of documentation strings and their outputs for different languages. Note that the output might not be exactly as written or formatted here, but the general gist should be the same.
There exist several macros that can be used after a
BINDING_EXAMPLE() definition to define the parameters that can be specified for a given mlpack program. These macros all have the same general definition: the name of the macro specifies the type of the parameter, whether or not the parameter is required, and whether the parameter is an input or output parameter. Then as arguments to the macros, the name, description, and sometimes the single-character alias and the default value of the parameter.
To give a flavor of how these definitions look, the definition
will define a string input parameter
algorithm (referenced as
'--algorithm' from the command-line or
'algorithm' from Python) with the description
The algorithm to use: 'svd' or 'blah'. The single-character alias
'-a' can be used from a command-line program (but means nothing in Python).
There are numerous different macros that can be used:
PARAM_FLAG()- boolean flag parameter
PARAM_INT_IN()- integer input parameter
PARAM_INT_OUT()- integer output parameter
PARAM_DOUBLE_IN()- double input parameter
PARAM_DOUBLE_OUT()- double output parameter
PARAM_STRING_IN()- string input parameter
PARAM_STRING_OUT()- string output parameter
PARAM_MATRIX_IN()- double-valued matrix (
arma::mat) input parameter
PARAM_MATRIX_OUT()- double-valued matrix (
arma::mat) output parameter
PARAM_UMATRIX_IN()- size_t-valued matrix (
arma::Mat<size_t>) input parameter
PARAM_UMATRIX_OUT()- size_t-valued matrix (
arma::Mat<size_t>) output parameter
PARAM_TMATRIX_IN()- transposed double-valued matrix (
arma::mat) input parameter
PARAM_TMATRIX_OUT()- transposed double-valued matrix (
arma::mat) output parameter
PARAM_MATRIX_AND_INFO_IN()- matrix with categoricals input parameter (
PARAM_COL_IN()- double-valued column vector (
arma::vec) input parameter
PARAM_COL_OUT()- double-valued column vector (
arma::vec) output parameter
PARAM_UCOL_IN()- size_t-valued column vector (
arma::Col<size_t>) input parameter
PARAM_UCOL_OUT()- size_t-valued column vector (
arma::Col<size_t>) output parameter
PARAM_ROW_IN()- double-valued row vector (
arma::rowvec) input parameter
PARAM_ROW_OUT()- double-valued row vector (
arma::rowvec) output parameter
PARAM_MODEL_IN()- serializable model input parameter
PARAM_MODEL_OUT()- serializable model output parameter
And for input parameters, the parameter may also be required:
Click the links for each macro to read further documentation. Note also that each possible combination of
REQ is not available—output options cannot be required, and some combinations simply have not been added because they have not been needed.
PARAM_MODEL_OUT() macros are used to serialize mlpack models. These could be used, for instance, to allow the user to save a trained model (like a linear regression model) or load an input model. The first parameter to the
PARAM_MODEL_OUT() macro should be the C++ type of the model to be serialized; this type must have a function
(i.e. the type must be serializable via mlpack's boost::serialization shim). For example, to allow a user to specify an input model of type
LinearRegression, the follow definition could be used:
Then, the user will be able to specify their model from the command-line as
--input_model_file and from Python using the
input_model option to the generated binding.
From the command line, matrix-type and model-type options (both input and output) are loaded from or saved to the specified file. This means that
_file is appended to the name of the parameter; so if the parameter name is
data and it is of a matrix or model type, then the name that the user will specify on the command line will be
--data_file. This displayed parameter name change only occurs with matrix and model type parameters for command-line programs.
PARAM_MATRIX_AND_INFO() macro defines a categorical matrix parameter (more specifically, a matrix type that can support categorical columns). From the C++ program side, this means that the parameter type is
std::tuple<data::DatasetInfo, arma::mat>. From the user side, for a command-line program, this means that the user will pass the filename of a dataset that can have categorical features, such as an ARFF dataset. For a Python program, the user may pass a Pandas matrix with categorical columns. When the program is run, the input that the user gives will be processed and the
data::DatasetInfo object will be filled with the dimension types and the
arma::mat object will be filled with the data itself.
To give some examples, the parameter definitions from the example "random_numbers" program in the previous section are shown below.
Note that even the parameter documentation strings must be a little be agnostic to the binding type, because the command-line interface is so different than the Python interface to the user.
IO module provides a unified abstract interface for getting input from and providing output to users without needing to consider the language (command-line, Python, MATLAB, etc.) that the user is running the program from. This means that after the
BINDING_EXAMPLE() macros and the
PARAM_*() macros have been defined, a language-agnostic
mlpackMain() function can be written. This function then can perform the actual computation that the entire program is meant to.
Inside of an
mlpackMain() function, the
mlpack::IO module can be used to access input parameters and set output parameters. There are two main functions for this, plus a utility printing function:
IO::GetParam<T>()- get a reference to a parameter
IO::HasParam()- returns true if the user specified the parameter
IO::GetPrintableParam<T>()- returns a string representing the value of the parameter
So, to print "hello" if the user specified the
print_hello parameter, the following code could be used:
To access a string that a user passed in to the
string parameter, the following code could be used:
Matrix types are accessed in the same way:
Similarly, model types can be accessed. If a
LinearRegression model was specified by the user as the parameter
model, the following code can access the model:
Matrices with categoricals are a little trickier to access since the C++ parameter type is
std::tuple<data::DatasetInfo, arma::mat>. The example below creates references to both the
DatasetInfo and matrix objects, assuming the user has passed a matrix with categoricals as the
These two functions can be used to write an entire program. The third function,
GetPrintableParam(), can be used to help provide useful output in a program. Typically, this function should be used if you want to provide some kind of error message about a matrix or model parameter, but want to avoid printing the matrix itself. For instance, printing a matrix parameter with
GetPrintableParam() will print the filename for a command-line binding or the size of a matrix for a Python binding.
GetPrintableParam() for a model parameter will print the filename for the model for a command-line binding or a simple string representing the type of the model for a Python binding.
Putting all of these ideas together, here is the
mlpackMain() function that could be created for the "random_numbers" program from earlier sections.
More documentation for the IO module can either be found on the mlpack::IO documentation page, or by reading the existing mlpack bindings. These can be found in the
src/mlpack/methods/ folders, by finding the
_main.cpp files. For instance,
src/mlpack/methods/neighbor_search/knn_main.cpp is the k-nearest-neighbor search program definition.
This section describes the internal functionality of the IO module and the associated macros. If you are only interested in writing mlpack programs, this section is probably not worth reading.
There are eight main components involved with mlpack bindings:
The mlpack::IO module is a singleton class that stores, at runtime, the binding name, the documentation, and the parameter information and values. In order to do this, each parameter and the program documentation must make themselves known to the IO singleton. This is accomplished by having the
PARAM_*() macros declare global variables that, in their constructors, register themselves with the IO singleton.
BINDING_NAME() macro declares an object of type mlpack::util::ProgramName. The
BINDING_SHORT_DESC() macro declares an object of type mlpack::util::ShortDescription. The
BINDING_LONG_DESC() macro declares an object of type mlpack::util::LongDescription. The
BINDING_EXAMPLE() macro declares an object of type mlpack::util::Example. The
BINDING_SEE_ALSO() macro declares an object of type mlpack::util::SeeAlso. The
ProgramName class constructor calls IO::RegisterProgramName() in order to register the given program name. The
ShortDescription class constructor calls IO::RegisterShortDescription() in order to register the given short description. The
LongDescription class constructor calls IO::RegisterLongDescription() in order to register the given long description. The
Example class constructor calls IO::RegisterExample() in order to register the given example. The
SeeAlso class constructor calls IO::RegisterSeeAlso() in order to register the given see-also link.
PARAM_*() macros declare an object that will, in its constructor, call IO::Add() to register that parameter with the IO singleton. The specific type of that object will depend on the binding type being used.
The IO::Add() function takes an mlpack::util::ParamData object as its input. This
ParamData object has a number of fields that must be set to properly describe the parameter. Each of the fields is documented and probably self-explanatory, but three fields deserve further explanation:
std::string tnamemember is used to encode the true type of the parameter—which is not known by the IO singleton at runtime. This should be set to
Tis the type of the parameter.
boost::any valuemember is used to hold the actual value of the parameter. Typically this will simply be the parameter held by a
boost::anyobject, but for some types it may be more complex. For instance, for a command-line matrix option, the
valueparameter will actually hold a tuple containing both the filename and the matrix itself.
std::string cppTypeshould be a string containing the type as seen in C++ code. Typically this can be encoded by stringifying a
PARAM_*() macro argument.
Thus, the global object defined by the
PARAM_*() macro must turn its arguments into a fully specified
ParamData object and then call IO::Add() with it.
With different binding types, different behavior is often required for the
GetPrintableParam<T>() functions. In order to handle this, the IO singleton also holds a function pointer map, so that a given type of option can call specific functionality for a certain task. This function map is accessible as
IO::functionMap and is not meant to be used by users, but instead by people writing binding types.
Each function in the map must have signature
The use of void pointers allows any type to be specified as input or output to the function without changing the signature for the map. The IO function map is of type
and the first map key is the typename (
tname) of the parameter, and the second map key is the string name of the function. For instance, calling
will call the
GetParam() function for the type of the
"param" parameter. Examples are probably easiest to understand how this functionality works; see the IO::GetParam<T>() source to see how this might be used.
The IO singleton expects the following functions to be defined in the function map for each type:
GetParam– return a pointer to the parameter in
GetPrintableParam– return a pointer to a string description of the parameter in
If these functions are properly defined, then the IO module will work correctly. Other functions may also be defined; these may be used by other parts of the binding infrastructure for different languages.
This section describes the internal functionality of the command-line program binding generator. If you are only interested in writing mlpack programs, this section probably is not worth reading. This section is worth reading only if you want to know the specifics of how the
mlpackMain() function and macros get turned into a fully working command-line program.
The code for the command-line bindings is found in
Any command-line program must be compiled with the
BINDING_TYPE macro set to the value
BINDING_TYPE_CLI. This is handled by the CMake macro
BINDING_TYPE is set to
BINDING_TYPE_CLI, the following is set in
src/mlpack/core/util/mlpack_main.hpp, which must be included by every mlpack binding:
PARAM_*() macros are of type mlpack::bindings::cli::CLIOption.
BINDING_LONG_DESC()and BINDING_EXAMPLE() are set: The
PRINT_PARAM_STRING()macro is defined as mlpack::bindings::cli::ParamString(). The
PRINT_DATASET()macro is defined as mlpack::bindings::cli::PrintDataset(). The
PRINT_MODEL()macro is defined as mlpack::bindings::cli::PrintModel(). The
PRINT_CALL()macro is defined as mlpack::bindings::cli::ProgramCall().
int main()is defined as:
Thus any mlpack command-line binding first processes the command-line arguments with mlpack::bindings::cli::ParseCommandLine(), then runs the binding with
mlpackMain(), then cleans up with mlpack::bindings::cli::EndProgram().
ParseCommandLine() function reads the input parameters and sets the values in IO. For matrix-type and model-type parameters, this reads the filenames from the command-line, but does not load the matrix or model. Instead the matrix or model is loaded the first time it is accessed with
--help parameter is handled by the mlpack::bindings::cli::PrintHelp() function.
At the end of program execution, the mlpack::bindings::cli::EndProgram() function is called. This writes any output matrix or model parameters to disk, and prints the program parameters and timers if
--verbose was given.
For command line bindings, the matrix, model, and matrix with categorical type parameters all require special handling, since it is not possible to pass a matrix of any reasonable size or a model on the command line directly. Therefore for a matrix or model parameter, the user specifies the file containing that matrix or model parameter. If the parameter is an input parameter, then the file is loaded when
GetParam<T>() is called. If the parameter is an output parameter, then the matrix or model is saved to the file when
EndProgram() is called.
The actual implementation of this is that the
boost::any value member of the
ParamData struct does not hold the model or the matrix, but instead a
std::tuple containing both the matrix or the model, and the filename associated with that matrix or model.
This means that functions like
GetPrintableParam<T>() (and all of the other associated functions in the IO function map) must have special handling for matrix or model types. See those implementatipns for more details—the special handling is enforced via SFINAE.
ParseCommandLine() function uses
CLI11 to read the values from the command line into the
ParamData structs held by the IO singleton.
In order to set up
CLI11—and to keep its headers from needing to be included by the rest of the library—the code loops over each parameter known by the IO singleton and calls the
"AddToPO" function from the function map. This in turn calls the necessary functions to register a given parameter with
CLI11, and once all parameters have been registered, the facilities provided by
CLI11 are used to parse the command line input properly.
This section describes the internal functionality of the mlpack Python binding generator. If you are only interested in writing new bindings or building the bindings, this section is probably not worth reading. But if you are interested in the internal working of the Python binding generator, then this section is for you.
The Python bindings are significantly more complex than the command line bindings because we cannot just compile directly to a finished product. Instead we need a multi-stage compilation:
This is done with a combination of C++ code to generate the .pyx bindings, CMake to run the actual compilation and generate the setup.py file, some utility Python functions, and tests written in both Python and C++. This code is primarily contained in
The standard Python matrix library is numpy, so mlpack bindings should accept numpy matrices as input. Fortunately, numpy Cython bindings already exist, which make it easy to convert from a numpy object to an Armadillo object without copying any data. This code can be found in
src/mlpack/bindings/python/mlpack/arma_numpy.pyx, and is used by the Python
mlpack also supports categorical matrices; in Python, the typical way of representing matrices with categorical features is with Pandas. Therefore, mlpack also accepts Pandas matrices, and if any of the Pandas matrix dimensions are categorical, these are properly encoded. The function
mlpack/bindings/python/mlpack/matrix_utils.py is used to perform this conversion.
We use (or abuse) Cython functionality in order to give the user a model object that they can use in their Python code. However, we do not want to (or have the infrastructure to) write bindings for every method that a serializable model class might support; therefore, we only desire to return a memory pointer to the model to the user.
In this way, a user that receives a model from an output parameter can then reuse the model as an input parameter to another binding (or the same binding).
To return a function pointer we have to define a Cython class in the following way (this example is taken from the perceptron binding):
This class definition is automatically generated when the .pyx file is automatically generated.
A boilerplate setup.py file can be found in
src/mlpack/bindings/python/setup.py.in. This will be configured by CMake to produce the final
setup.py file, but in order to do this, a list of the .pyx files to be compiled must be gathered.
add_python_binding() macro is defined in
src/mlpack/bindings/python/CMakeLists.txt. This adds the given binding to the
MLPACK_PYXS variable, which is then inserted into
setup.py as part of the
configure_file() step in
A binding named
program is built into a program called
generate_pyx_program (this a CMake target, so you can build these individually if you like). The file
src/mlpack/bindings/python/generate_pyx.cpp.in is configured by CMake to set the name of the program and the
*_main.cpp file to include correctly, then the
mlpack::bindings::python::PrintPYX() function is called by the program. The
PrintPYX() function uses the parameters that have been set in the IO singleton by the
PARAM_*() macros in order to actually print a fully-working .pyx file that can be compiled. The file has several sections:
Any output parameters for Python bindings are returned in a dict containing named elements.
After building the
generate_pyx_program target, the
build_pyx_program target is built as a dependency of the
python target. This simply takes the generated .pyx file and uses Python setuptools to compile this to a Python binding.
We cannot do our tests only from the Boost Unit Test Framework in C++ because we need to see that we are able to load parameters properly from Python and return output correctly.
The tests are in
src/mlpack/bindings/python/tests/ and test both the actual bindings and also the auxiliary Python code included in
Adding a new binding type to mlpack is fairly straightforward once the general structure of the IO singleton and the function map that IO uses is understood. For each different language that bindings are desired for, the route to a solution will be particularly different—so it is hard to provide any general guidance for how to make new bindings that will be applicable to each language.
In general, the first thing to handle will be how matrices are passed back and forth between the target language. Typically this might mean getting the memory address of an input matrix and wrapping an
arma::mat object around that memory address. This can be handled in the
GetParam() function that is part of the IO singleton function map; see
get_param.hpp for both the IO and Python bindings for an example (in
Serialization of models is also a tricky consideration; in some languages you will be able to pass a pointer to the model itself. This is generally best—users should not expect to be able to manipulate the model in the target language, but they should expect that they can pass a model back and forth without paying a runtime penalty. So, for example, serializing a model using a
boost::text_oarchive and then returning the string that represents the model is not acceptable, because that string can be extremely large and the time it takes to decode the model can be very large.
The strategy of generating a binding definition for the target language, like what is done with Python, can be a useful strategy that should be considered. If this is the route that is desired, a large amount of CMake boilerplate may be necessary. The Python CMake configuration can be referred to as an example, but probably a large amount of adaptation to other languages will be necessary.
Lastly, when adding a new language, be sure to make sure it works with the Markdown documentation generator. In order to make this happen, you will need to modify all of the
add_markdown_docs() calls in the different
CMakeLists.txt files to contain the name of the language you have written a binding for. You will also need to modify every function in
src/mlpack/bindings/markdown/print_doc_functions_impl.hpp to correctly call out to the corresponding function for the language that you have written bindings for.