## Code

### Model

A model is defined in it’s own package. The model must implement
interface
`model.Model`

,
consisting of a single method `Observe([]float64) float64`

. In
the model’s source code:

- Methods on the type implementing
`model.Model`

returning a single float or nothing are differentiated. - Within the methods, the following is differentiated:

- assignments to
`float64`

(including parallel assignments if all values are of type`float64`

); - returns of float64;
- standalone calls to methods on the type implementing model.Model (apparently called for side effects on the model).

- Imported package name
`ad`

is reserved. - Non-dummy identifiers starting with the prefix for
generated identifiers (
`_`

by default) are reserved.

Derivatives do not propagate through a function that is not an elemental or a call to a model method. If a derivative is not registered for an elemental, calling the elemental in a differentiated context will cause a run-time error.

#### Elemental model

If method `Gradient() []float64`

is provided for a model type,
the model is treated as an ‘elemental’ model, and the gradient
returned by `Gradient()`

is used by inference algorithms. This
allows to code the gradient by hand instead of relying on
automatic differentiation.

### Elementals

Functions are considered elementals (and must have a registered derivative) if their signature is either of kind

` func (float64, float64*) float64`

that is, one or more non-variadic `float64`

arguments and
`float64`

return value, or

` func ([]float64) float64`

For example, functions

```
func foo(float64, float64, float64) float64
func bar([]float64) float64
```

are considered elementals, while functions

```
func fee(...float64) float64
func buz(int, float64) float64
```

are not. Gradients for selected functions from the `math`

package are pre-defined (`Sqrt`

, `Exp`

, `Log`

, `Pow`

, `Sin`

,
`Cos`

, `Tan`

, `Erf`

, `Erfc`

). Auxiliary elemental functions with
pre-defined gradients are in
bitbucket.org/dtolpin/infergo/mathx.

### Distributions

Distributions are models. Several distributions are provided in
bitbucket.org/dtolpin/infergo/dist.
In addition to the `Observe`

method, distributions have `Logp`

(single observation) and `Logps`

(multiple observations) methods
which accept distribution parameters and observations as
individual arguments rather than in a single slice.

## Differentiation

Command-line utility `deriv`

is used to differentiate a model.
The command-line syntax is:

```
deriv path/to/model/package
```

For example,

```
deriv examples/hello/model
```

Run `deriv -h`

for the full list of command-line options. The
differentiated model is put into subpackage “ad” of the model’s
package, with the same name as the original package.

## Inference

For inference, `infergo`

offers

- optimization via gradient ascent methods.
- full posterior inference via Hamiltonian Monte Carlo variants.

### Optimization

An optimizer implements interface
`infer.Grad`

.
Interface implementations are gradient ascent with
momentum
and
Adam.
Both methods are capable to work with stochastic data (e.g.
streams or batches).

### Full posterior

An MCMC sampler for full posterior inference implements interface
`infer.MCMC`

. Inteface implementations are HMC and NUTS.

`DepthAdapter`

enables adaption of NUTS step size with respect to the average
tree depth.