Code

Model

A model is defined in it’s own package. The model must implement interface model.Model, consisting of a single method Observe([]float64) float64. In the model’s source code:

1. Methods on the type implementing model.Model returning a single float or nothing are differentiated.
2. Within the methods, the following is differentiated:

• assignments to float64 (including parallel assignments if all values are of type float64);
• returns of float64;
• standalone calls to methods on the type implementing model.Model (apparently called for side effects on the model).

3. Imported package name ad is reserved.
4. Non-dummy identifiers starting with the prefix for generated identifiers (_ by default) are reserved.

Derivatives do not propagate through a function that is not an elemental or a call to a model method. If a derivative is not registered for an elemental, calling the elemental in a differentiated context will cause a run-time error.

Elemental model

If method Gradient() []float64 is provided for a model type, the model is treated as an ‘elemental’ model, and the gradient returned by Gradient() is used by inference algorithms. This allows to code the gradient by hand instead of relying on automatic differentiation.

Elementals

Functions are considered elementals (and must have a registered derivative) if their signature is either of kind

        func (float64, float64*) float64

that is, one or more non-variadic float64 arguments and float64 return value, or

        func ([]float64) float64

For example, functions

        func foo(float64, float64, float64) float64
        func bar([]float64) float64

are considered elementals, while functions

        func fee(...float64) float64
        func buz(int, float64) float64

are not. Gradients for selected functions from the math package are pre-defined (Sqrt, Exp, Log, Pow, Sin, Cos, Tan, Erf, Erfc). Auxiliary elemental functions with pre-defined gradients are in bitbucket.org/dtolpin/infergo/mathx.

Distributions

Distributions are models. Several distributions are provided in bitbucket.org/dtolpin/infergo/dist. In addition to the Observe method, distributions have Logp (single observation) and Logps (multiple observations) methods which accept distribution parameters and observations as individual arguments rather than in a single slice.

Differentiation

Command-line utility deriv is used to differentiate a model. The command-line syntax is:

deriv path/to/model/package

For example,

deriv examples/hello/model

Run deriv -h for the full list of command-line options. The differentiated model is put into subpackage “ad” of the model’s package, with the same name as the original package.

Inference

For inference, infergo offers

• optimization via gradient ascent methods.
• full posterior inference via Hamiltonian Monte Carlo variants.

Optimization

An optimizer implements interface infer.Grad. Interface implementations are gradient ascent with momentum and Adam. Both methods are capable to work with stochastic data (e.g. streams or batches).

Full posterior

An MCMC sampler for full posterior inference implements interface infer.MCMC. Inteface implementations are HMC and NUTS.

DepthAdapter enables adaption of NUTS step size with respect to the average tree depth.