Model Hierarchy
Model Classes¶
In DynaML all model implementations fit into a well defined class hierarchy. In fact every DynaML machine learning model is an extension of the Model[T,Q,R]
trait.
Model[T,Q,R]
The Model
trait is quite bare bones: machine learning models are viewed as objects containing two parts or components.

A training data set (of type
T
). 
A method
predict(point: Q):R
to generate a prediction of typeR
given a data point of typeQ
.
Parameterized Models¶
Many predictive models calculate predictions by formulating an expression which includes a set of parameters which are used along with the data points to generate predictions, the ParameterizedLearner[G, T, Q, R, S]
class represents a skeleton for all parametric machine learning models such as Generalized Linear Models, Neural Networks, etc.
Tip
The defining characteristic of classes which extend ParameterizedLearner
is that they must contain a member variable optimizer: RegularizedOptimizer[T, Q, R, S]
which represents a regularization enabled optimizer implementation along with a learn()
method which uses the optimizer member to calculate approximate values of the model parameters given the training data.
Linear Models¶
Linear models; represented by the LinearModel[T, P, Q , R, S]
trait are extensions of ParameterizedLearner
, this top level trait is extended to yield many useful linear prediction models.
Generalized Linear Models which are linear in parameters expression for the predictions y given a vector of processed features \phi(x) or basis functions.
Stochastic Processes¶
Stochastic processes (or random functions) are general probabilistic models which can be used to construct finite dimensional distributions over a set of sampled domain points. More specifically a stochastic process is a probabilistic function f(.) defined on any domain or index set \mathcal{X} such that for any finite collection x_i \in \mathcal{X}, i = 1 \cdots N, the finite dimensional distribution P(f(x_1), \cdots, f(x_N)) is coherently defined.
Tip
The StochasticProcessModel[T, I, Y, W]
trait extends Model[T, I, Y]
and is the top level trait for the implementation of general stochastic processes. In order to extend it, one must implement among others a function to output the posterior predictive distribution predictiveDistribution()
.
Continuous Processes¶
By continuous processes, we mean processes whose values lie on a continuous domain (such as \mathbb{R}^d). The ContinuousProcessModel[T, I, Y, W]
abstract class provides a template which can be extended to implement continuous random process models.
Tip
The ContinuousProcessModel
class contains the method predictionWithErrorBars()
which takes inputs test data and number of standard deviations, and generates predictions with upper and lower error bars around them. In order to create a subclass of ContinuousProcessModel
, you must implement the method predictionWithErrorBars()
.
Second Order Processes¶
Second order stochastic processes can be described by specifying the mean (first order statistic) and variance (second order statistic) of their finite dimensional distribution. The SecondOrderProcessModel[T, I, Y, K, M, W]
trait is an abstract skeleton which describes what elements a second order process model must have i.e. the mean and covariance functions.
Meta Models/Model Ensembles¶
Meta models use predictions from several candidate models and derive a prediction that is a meaningful combination of the individual predictions. This may be achieved in several ways some of which are.
 Average of predictions/voting
 Weighted predictions: Problem is now transferred to calculating appropriate weights.
 Learning some nontrivial functional transformation of the individual prediction, also known as gating networks.
Currently the DynaML API has the following classes providing capabilities of meta models.
Abstract Classes
Implementations