Tour

This document describes several of the major components of the 0.2 release of ScalaNLP-Data and Scalanlp-Learn. It’s intended as something of an overview, and not a reference, for that, see the scaladocs.

Important Packages

Distributions

ScalaNLP also provides a fairly large number of probability distributions built in. These come with access to either probability density function (for discrete distributions) or pdf functions (for continuous distributions). Many distributions also have methods for giving the mean and the variance.

  scala> val poi = new Poisson(3.0);
  poi: scalanlp.stats.distributions.Poisson = <function1>

  scala> poi.sample(10);
  res21: List[Int] = List(3, 5, 5, 2, 2, 1, 1, 2, 4, 1)

  scala> res21 map { poi.probabilityOf(_) }
  res23: List[Double] = List(0.6721254229661636, 0.504094067224622, 0.504094067224622, 0.44808361531077556, 0.44808361531077556, 0.1493612051035918, 0.1493612051035918, 0.44808361531077556, 0.6721254229661628, 0.1493612051035918)
  
  scala> val doublePoi = for(x <- poi) yield x.toDouble; // meanAndVariance requires doubles, but Poisson samples over Ints
  doublePoi: java.lang.Object with scalanlp.stats.distributions.Rand[Double] = scalanlp.stats.distributions.Rand$$anon$2@2c98070c

  scala> scalanlp.stats.DescriptiveStats.meanAndVariance(doublePoi.samples.take(1000));
  res29: (Double, Double) = (3.018,2.9666426426426447)

  scala> (poi.mean,poi.variance)
  res30: (Double, Double) = (3.0,3.0)

TODO: exponential families

Optimization

ScalaNLP’s optimization package includes several convex optimization routines and a simple linear program solver. Convex optimization routines typically take a
DiffFunction[T], which is a Function1 extended to have a gradientAt method, which returns the gradient at a particular point. Most routines will require
a Scalala enabled type: something like a Vector or a Counter.

Here’s a simple DiffFunction: a parabola along each vector’s coordinate.

scala> import scalanlp.optimize._
import scalanlp.optimize._

scala> import scalala.tensor.dense._
import scalala.tensor.dense._

scala> import scalala.library.Library.norm
import scalala.library.Library.norm

scala>       val f = new DiffFunction[DenseVector[Double]] {
     |         def calculate(x: DenseVector[Double]) = {
     |           (norm((x -3) :^ 2,1),(x * 2) - 6);
     |         }
     |       }
f: java.lang.Object with scalanlp.optimize.DiffFunction[scalala.tensor.dense.DenseVector[Double]] = $anon$1@7593da36


scala> f.valueAt(DenseVector(0,0,0))
res0: Double = 27.0

scala> f.valueAt(DenseVector(3,3,3))
res1: Double = 0.0

scala> f.gradientAt(DenseVector(3,0,1))
res2: scalala.tensor.dense.DenseVector[Double] = 
 0.00000
-6.00000
-4.00000

scala> f.calculate(DenseVector(0,0))
res3: (Double, scalala.tensor.dense.DenseVector[Double]) = 
(18.0,-6.00000
-6.00000)

You can also use approximate derivatives, if your function is easy enough to compute:

scala> def g(x: DenseVector[Double]) = (x - 3.0):^2 sum
g: (x: scalala.tensor.dense.DenseVector[Double])Double

scala> g(DenseVector(0.,0.,0.))
res5: Double = 27.0

scala> val diffg = new ApproximateGradientFunction(g)
diffg: scalanlp.optimize.ApproximateGradientFunction[Int,scalala.tensor.dense.DenseVector[Double]] = <function1>

scala> diffg.gradientAt(DenseVector(3,0,1))
res6: scalala.tensor.dense.DenseVector[Double] = 
 1.00000e-05
-5.99999
-3.99999

Ok, now let’s optimize f. The easiest routine to use is just LBFGS, which is a quasi-Newton method that works well for most problems.

scala> val lbfgs = new LBFGS[DenseVector[Double]](maxIter=100, m=3) // m is the memory. anywhere between 3 and 7 is fine. The larger m, the more memory is needed.
lbfgs: scalanlp.optimize.LBFGS[scalala.tensor.dense.DenseVector[Double]] = scalanlp.optimize.LBFGS@c7d97d5

scala> lbfgs.minimize(f,DenseVector(0,0,0))
res7: scalala.tensor.dense.DenseVector[Double] = 
 3.00000
 3.00000
 3.00000


scala> f(res7)
res8: Double = 0.0