Skip to content

# v1.4.1

Version 1.4.1 of DynaML, released March 26, 2017, implements a number of new models (Extended Skew GP, student T process, generalized least squares, etc) and features.

## Pipes API¶

### Additions¶

The pipes API has been vastly extended by creating pipes which encapsulate functions of multiple arguments leading to the following end points.

• DataPipe2[A, B, C]: Pipe which takes 2 arguments
• DataPipe3[A, B, C, D] : Pipe which takes 3 arguments
• DataPipe4[A, B, C, D, E]: Pipe which takes 4 arguments

Furthermore there is now the ability to create pipes which return pipes, something akin to curried functions in functional programming.

• MetaPipe: Takes an argument returns a DataPipe
• MetaPipe21: Takes 2 arguments returns a DataPipe
• MetaPipe12: Takes an argument returns a DataPipe2

A new kind of Stream data pipe, StreamFlatMapPipe is added to represent data pipelines which can perform flat map like operations on streams.

 1 2 val mapFunc: (I) => Stream[J] = ... val streamFMPipe = StreamFlatMapPipe(mapFunc) 
• Added Data Pipes API for Apache Spark RDDs.
  1 2 3 4 5 6 7 8 9 10 11 12 val num = 20 val numbers = sc.parallelize(1 to num) val convPipe = RDDPipe((n: Int) => n.toDouble) val sqPipe = RDDPipe((x: Double) => x*x) val sqrtPipe = RDDPipe((x: Double) => math.sqrt(x)) val resultPipe = RDDPipe((r: RDD[Double]) => r.reduce(_+_).toInt) val netPipeline = convPipe > sqPipe > sqrtPipe > resultPipe netPipeline(numbers) 
• Added UnivariateGaussianScaler class for gaussian scaling of univariate data.

## Core API¶

### Additions¶

Package dynaml.models.bayes

This new package will house stochastic prior models, currently there is support for GP and Skew GP priors, to see a starting example see stochasticPriors.sc in the scripts directory of the DynaML source.

Package dynaml.kernels

• Added evaluateAt(h)(x,y) and gradientAt(h)(x,y); expressing evaluate(x,y) and gradient(x,y) in terms of them
• Added asPipe method for Covariance Functions
• For backwards compatibility users are advised to extend LocalSVMKernel in their custom Kernel implementations incase they do not want to implement the evaluateAt API endpoints.
• Added FeatureMapKernel, representing kernels which can be explicitly decomposed into feature mappings.
• Added Matern half integer kernel GenericMaternKernel[I]
• Added block(S: String*) method to block any hyper-parameters of kernels.
• Added NeuralNetworkKernel and GaussianSpectralKernel.
• Added DecomposableCovariance
  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 import io.github.mandar2812.dynaml.DynaMLPipe._ import io.github.mandar2812.dynaml.kernels._ implicit val ev = VectorField(6) implicit val sp = breezeDVSplitEncoder(2) implicit val sumR = sumReducer val kernel = new LaplacianKernel(1.5) val other_kernel = new PolynomialKernel(1, 0.05) val decompKernel = new DecomposableCovariance(kernel, other_kernel)(sp, sumReducer) val other_kernel1 = new FBMKernel(1.0) val decompKernel1 = new DecomposableCovariance(decompKernel, other_kernel1)(sp, sumReducer) val veca = DenseVector.tabulate[Double](8)(math.sin(_)) val vecb = DenseVector.tabulate[Double](8)(math.cos(_)) decompKernel1.evaluate(veca, vecb) 

Package dynaml.algebra

Partitioned Matrices/Vectors and the following operations

• Addition, Subtraction
• Matrix, vector multiplication
• LU, Cholesky
• A\y, A\Y

Added calculation of quadratic forms, namely:

• quadraticForm which calculates $\mathbf{x}^\intercal A^{-1} \mathbf{x}$
• crossQuadraticForm which calculates $\mathbf{y}^\intercal A^{-1} \mathbf{x}$

Where A is assumed to be a symmetric positive semi-definite matrix

Usage:

 1 2 3 4 5 6 7 8 import io.github.mandar2812.dynaml.algebra._ val x: DenseVector[Double] = ... val y: DenseVector[Double] = ... val a: DenseMatrix[Double] = ... quadraticForm(a,x) crossQuadraticForm(y, a, x) 

Package dynaml.modelpipe

New package created, moved all inheriting classes of ModelPipe to this package.

Added the following:

• GLMPipe2 A pipe taking two arguments and returning a GeneralizedLinearModel instance
• GeneralizedLeastSquaresPipe2:
• GeneralizedLeastSquaresPipe3:

Package dynaml.models

• Added a new Neural Networks API: NeuralNet and GenericFFNeuralNet, for an example refer to TestNNDelve in dynaml-examples.
• GeneralizedLeastSquaresModel: The GLS model.
• ESGPModel: The implementation of a skew gaussian process regression model
• Warped Gaussian Process models WIP
• Added mean function capability to Gaussian Process and Student T process models.
• Added Apache Spark implementation of Generalized Linear Models; see SparkGLM, SparkLogisticModel, SparkProbitGLM

Package dynaml.probability

• MultivariateSkewNormal as specified in Azzalani et. al
• ExtendedMultivariateSkewNormal
• UESN and MESN representing an alternative formulation of the skew gaussian family from Adcock and Shutes.
• TruncatedGaussian: Truncated version of the Gaussian distribution.
• Matrix Normal Distribution
• Added Expectation operator for RandomVariable implementations in the io.github.mandar2812.dynaml.probability package object. Usage example given below.
• SkewGaussian, ExtendedSkewGaussian: An breeze implementation of the SkewGaussian and extended Skew-Gaussian distributions respectively
• PushforwardMap, DifferentiableMap added: PushforwardMap enables creating new random variables with defined density from base random variables.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 import io.github.mandar2812.dynaml.analysis._ import io.github.mandar2812.dynaml.probability._ import io.github.mandar2812.dynaml.probability.distributions._ val g = GaussianRV(0.0, 0.25) val sg = RandomVariable(SkewGaussian(1.0, 0.0, 0.25)) //Define a determinant implementation for the Jacobian type (Double in this case) implicit val detImpl = identityPipe[Double] //Defines a homeomorphism y = exp(x) x = log(y) val h: PushforwardMap[Double, Double, Double] = PushforwardMap( DataPipe((x: Double) => math.exp(x)), DifferentiableMap( (x: Double) => math.log(x), (x: Double) => 1.0/x) ) //Creates a log-normal random variable val p = h->g //Creates a log-skew-gaussian random variable val q = h->sg //Calculate expectation of q println("E[Q] = "+E(q)) 
- Added Markov Chain Monte Carlo (MCMC) based inference schemes ContinuousMCMC and the underlying sampling implementation in GeneralMetropolisHastings. - Added implementation of Approximate Bayesian Computation (ABC) in the ApproxBayesComputation class.

 1 2 3 4 5 6 7 //The mean val center: DenseMatrix[Double] = ... //Covariance (positive semi-def) matrix among rows val sigmaRows: DenseMatrix[Double] = ... //Covariance (positive semi-def) matrix among columns val sigmaCols: DenseMatrix[Double] = ... val matD = MatrixNormal(center, sigmaRows, sigmaCols) 
- Matrix T Distribution (Experimental)

 1 2 3 4 5 6 7 8 9 //The degrees of freedom (must be > 2.0 for existence of finite moments) val mu: Double = ... //The mean val center: DenseMatrix[Double] = ... //Covariance (positive semi-def) matrix among rows val sigmaRows: DenseMatrix[Double] = ... //Covariance (positive semi-def) matrix among columns val sigmaCols: DenseMatrix[Double] = ... val matD = MatrixT(mu, center, sigmaCols, sigmaRows) 

Package dynaml.optimization

• Added ProbGPCommMachine which performs grid search or CSA and then instead of selecting a single hyper-parameter configuration calculates a weighted Gaussian Process committee where the weights correspond to probabilities or confidence on each model instance (hyper-parameter configuration).

Package dynaml.utils

 1 2 //Returns logarithm of multivariate gamma function val g = mvlgamma(5, 1.5) 

Package dynaml.dataformat

• Added support for reading MATLAB .mat files in the MAT object.

### Improvements/Bug Fixes¶

Package dynaml.probability

• Removed ProbabilityModel and replaced with JointProbabilityScheme and BayesJointProbabilityScheme, major refactoring to RandomVariable API.

Package dynaml.optimization

• Improved logging of CoupledSimulatedAnnealing
• Refactored GPMLOptimizer to GradBasedGlobalOptimizer

Package dynaml.utils - Correction to utils.getStats method used for calculating mean and variance of data sets consisting of DenseVector[Double]. - minMaxScalingTrainTest minMaxScaling of DynaMLPipe using GaussianScaler instead of MinMaxScaler for processing of features.

Package dynaml.kernels

• Fix to CoRegCauchyKernel: corrected mismatch of hyper-parameter string
• Fix to SVMKernel objects matrix gradient computation in the case when kernel dimensions are not multiples of block size.
• Correction to gradient calculation in RBF kernel family.
• Speed up of kernel gradient computation, kernel and kernel gradient matrices with respect to the model hyper-parameters now calculated in a single pass through the data.