lsbi package

lsbi: Linear Simulation Based Inference.

lsbi.model module

Gaussian models for linear Bayesian inference.

class lsbi.model.LinearModel(*args, **kwargs)

Bases: object

A multilinear model.

D|θ ~ N( m + M θ, C ) θ ~ N( μ, Σ )

Defined by:

Parameters: θ (…, n,) Data: D (…, d,) Model: M (…, d, n) Prior mean: μ (…, n,) Prior covariance: Σ (…, n, n) Data mean: m (…, d,) Data covariance: C (…, d, d)

where the ellipses indicate arbitrary (broadcastable) additional copies.

Parameters:

Marray_like, optional

if ndim>=2: model matrices if ndim==1: model matrix with vector diagonal for all components if ndim==0: scalar * rectangular identity matrix for all components Defaults to rectangular identity matrix

marray_like, optional

if ndim>=1: data means if ndim==0: scalar * unit vector for all components Defaults to 0 for all components

Carray_like, optional

if ndim>=2: data covariances if ndim==1: data covariance with vector diagonal for all components if ndim==0: scalar * identity matrix for all components Defaults to rectangular identity matrix

μ(or mu) array_like, optional

if ndim>=1: prior means if ndim==0: scalar * unit vector for all components Defaults to 0 for all components Prior mean, defaults to zero vector

Σ(or Sigma) array_like, optional

if ndim>=2: prior covariances if ndim==1: prior covariance with vector diagonal for all components if ndim==0: scalar * identity matrix for all components Defaults to k copies of identity matrices

nint, optional

Number of parameters, defaults to automatically inferred value

dint, optional

Number of data dimensions, defaults to automatically inferred value

shape(), optional

Number of mixture components, defaults to automatically inferred value

property Sigma

property d

Dimensionality of data space len(D).

property diagonal_Sigma

dkl(D, n=0)

KL divergence between the posterior and prior.

Parameters:

Darray_like, shape (…, d)

Data to form the posterior

nint, optional

Number of samples for a monte carlo estimate, defaults to 0

evidence()

P(D) as a distribution object.

D ~ N( m + M μ, C + M Σ M’ )

joint()

P(θ, D) as a distribution object.

[θ] ~ N( [ μ ] [ Σ Σ M’ ] ) [D] ( [m + M μ] , [M Σ C + M Σ M’] )

likelihood(θ)

P(D|θ) as a distribution object.

D|θ ~ N( m + M θ, C ) θ ~ N( μ, Σ )

Parameters:

θarray_like, shape (k, n)

model(θ)

Model matrix M(θ) for a given parameter vector.

M(θ) = m + M θ

Parameters:

θarray_like, shape (…, n,)

property mu

property n

Dimension of the distribution.

posterior(D)

P(θ|D) as a distribution object.

θ|D ~ N( μ + S M’C^{-1}(D - m - M μ), S ) S = (Σ^{-1} + M’C^{-1}M)^{-1}

Parameters:

Darray_like, shape (d,)

ppd(D0)

P(D|D0) as a distribution object.

prior()

P(θ) as a distribution object.

θ ~ N( μ, Σ )

property shape

Shape of the distribution.

update(D, inplace=False)

Bayesian update of the model with data.

Parameters:

Darray_like, shape (…, d)

class lsbi.model.MixtureModel(*args, **kwargs)

Bases: LinearModel

A linear mixture model.

D|θ, w ~ N( m + M θ, C ) θ|w ~ N( μ, Σ ) w ~ categorical( exp(logw) )

Defined by:

Parameters: θ (…, n,) Data: D (…, d,) Prior means: μ (…, k, n) Prior covariances: Σ (…, k, n, n) Data means: m (…, k, d) Data covariances: C (…, k, d, d) log mixture weights: logw (…, k,)

Parameters:

Marray_like, optional

if ndim>=2: model matrices if ndim==1: model matrix with vector diagonal for all components if scalar: scalar * rectangular identity matrix for all components Defaults to k copies of rectangular identity matrices

marray_like, optional

if ndim>=1: data means if scalar: scalar * unit vector for all components Defaults to 0 for all components

Carray_like, optional

if ndim>=2: data covariances if ndim==1: data covariance with vector diagonal for all components if scalar: scalar * identity matrix for all components Defaults to k copies of identity matrices

μarray_like, optional

if ndim>=1: prior means if scalar: scalar * unit vector for all components Defaults to 0 for all components Prior mean, defaults to zero vector

Σarray_like, optional

if ndim>=2: prior covariances if ndim==1: prior covariance with vector diagonal for all components if scalar: scalar * identity matrix for all components Defaults to k copies of identity matrices

logwarray_like, optional

if ndim>=1: log mixture weights if scalar: scalar * unit vector Defaults to uniform weights

nint, optional

Number of parameters, defaults to automatically inferred value

dint, optional

Number of data dimensions, defaults to automatically inferred value

dkl(D, n=0)

KL divergence between the posterior and prior.

Parameters:

Darray_like, shape (…, d)

Data to form the posterior

nint, optional

Number of samples for a monte carlo estimate, defaults to 0

evidence()

P(D) as a distribution object.

D|w ~ N( m + M μ, C + M Σ M’ ) w ~ categorical(exp(logw))

joint()

P(D, θ) as a distribution object.

[θ] | w ~ N( [ μ ] [ Σ Σ M’ ] ) [D] | ( [m + M μ] , [M Σ C + M Σ M’] )

w ~ categorical(exp(logw))

property k

Number of mixture components.

likelihood(θ)

P(D|θ) as a distribution object.

D|θ,w ~ N( m + M θ, C ) w|θ ~ categorical(…)

Parameters:

θarray_like, shape (n,)

posterior(D)

P(θ|D) as a distribution object.

θ|D, w ~ N( μ + S M’C^{-1}(D - m - M μ), S ) w|D ~ P(D|w)P(w)/P(D) S = (Σ^{-1} + M’C^{-1}M)^{-1}

Parameters:

Darray_like, shape (d,)

prior()

P(θ) as a distribution object.

θ|w ~ N( μ, Σ ) w ~ categorical(exp(logw))

property shape

Shape of the distribution.

update(D, inplace=False)

Bayesian update of the model with data.

Parameters:

Darray_like, shape (…, d)

class lsbi.model.ReducedLinearModel(*args, **kwargs)

Bases: object

A model with no data.

If a Likelihood is Gaussian in the parameters, it is sometimes more clear/efficient to phrase it in terms of a parameter covariance, parameter mean and peak value:

logL(θ) = logLmax - (θ - mu_L)’ Sigma_L^{-1} (θ - mu_L)

We can link this to a data-based model with the relations:

Sigma_L = (M’ C^{-1} M)^{-1} mu_L = Sigma_L M’ C^{-1} (D-m) logLmax = - log|2 pi C|/2 - (D-m)’C^{-1}(C - M (M’ C^{-1} M)^{-1} M’ )C^{-1}(D-m)/2

Parameters:

mu_Larray_like

Likelihood peak

Sigma_Larray_like

Likelihood covariance

logLmaxfloat, optional

Likelihood maximum, defaults to zero

mmu_piarray_like, optional

Prior mean, defaults to zero vector

Sigma_piarray_like, optional

Prior covariance, defaults to identity matrix

DKL()

D_KL(P(θ|D)||P(θ)) the Kullback-Leibler divergence.

logL(θ)

P(D|θ) as a scalar.

logP(θ)

P(θ|D) as a scalar.

logZ()

P(D) as a scalar.

logpi(θ)

P(θ) as a scalar.

posterior()

P(θ|D) as a distribution object.

prior()

P(θ) as a distribution object.

class lsbi.model.ReducedLinearModelUniformPrior(*args, **kwargs)

Bases: object

A model with no data.

Gaussian likelihood in the parameters

logL(θ) = logLmax - (θ - mu_L)’ Sigma_L^{-1} (θ - mu_L)

Uniform prior

We can link this to a data-based model with the relations:

Sigma_L = (M’ C^{-1} M)^{-1} mu_L = Sigma_L M’ C^{-1} (D-m) logLmax = -log|2 pi C|/2 - (D-m)’C^{-1}(C - M (M’ C^{-1} M)^{-1} M’ )C^{-1}(D-m)/2

Parameters:

mu_Larray_like

Likelihood peak

Sigma_Larray_like

Likelihood covariance

logLmaxfloat, optional

Likelihood maximum, defaults to zero

logVfloat, optional

log prior volume, defaults to zero

DKL()

D_KL(P(θ|D)||P(θ)) the Kullback-Leibler divergence.

logL(θ)

P(D|θ) as a scalar.

logP(θ)

P(θ|D) as a scalar.

logZ()

P(D) as a scalar.

logpi(θ)

P(θ) as a scalar.

posterior()

P(θ|D) as a distribution object.

lsbi.stats module

Extensions to scipy.stats functions.

lsbi.stats.dkl(p, q, n=0)

Kullback-Leibler divergence between two distributions.

Parameters:

plsbi.stats.multivariate_normal

qlsbi.stats.multivariate_normal

nint, optional, default=0

Number of samples to mcmc estimate the divergence.

Returns:

dklarray_like

Kullback-Leibler divergence between p and q.

class lsbi.stats.mixture_normal(logw=0, mean=0, cov=1, shape=(), dim=1, diagonal=False)

Bases: multivariate_normal

Mixture of multivariate normal distributions.

Broadcastable multivariate mixture model.

Parameters:

meanarray_like, shape (…, n, dim)

Mean of each component.

cov: array_like, shape `(…, n, dim, dim)`

Covariance matrix of each component.

logw: array_like, shape `(…, n)`

Log of the mixing weights.

shape: tuple, optional, default=()

Shape of the distribution. Useful for forcing a broadcast beyond that inferred by mean and cov shapes

dim: int, optional, default=0

Dimension of the distribution. Useful for forcing a broadcast beyond that inferred by mean and cov shapes

diagonal: bool, optional, default=False

If True, cov is interpreted as the diagonal of the covariance matrix.

bijector(x, inverse=False)

Bijector between U([0, 1])^d and the distribution.

• x in [0, 1]^d is the hypercube space.

• theta in R^d is the physical space.

Computes the transformation from x to theta or theta to x depending on the value of inverse.

Parameters:

xarray_like, shape (…, d)

if inverse: x is theta else: x is x

inversebool, optional, default=False

If True: compute the inverse transformation from physical to hypercube space.

where self.shape[:-1] is broadcastable to …

Returns:

transformed x or theta: array_like, shape (…, d)

condition(indices, values)

Condition on indices with values.

Parameters:

indicesarray_like

Indices to condition over.

valuesarray_like shape (…, len(indices))

Values to condition on.

where self.shape[:-1] is broadcastable to …

Returns:

conditioned distribution, shape (*shape, len(indices))

property k

Number of components.

logpdf(x, broadcast=False, joint=False)

Log of the probability density function.

Parameters:

xarray_like, shape (*size, dim)

Points at which to evaluate the log of the probability density function.

broadcastbool, optional, default=False

If True, broadcast x across the distribution parameters.

jointbool, optional, default=False

If True, return the joint logpdf of the mixture P(x, N)

Returns:

logpdfarray_like

Log of the probability density function evaluated at x. if not broadcast and not joint: shape (*size, *shape[:-1]) elif not broadcast and joint: shape (*size, *shape) elif not joint: shape the broadcast of (*size,) and `shape[:-1] else: shape the broadcast of (*size,) and `shape

pdf(x, broadcast=False, joint=False)

Probability density function.

Parameters:

xarray_like, shape (*size, dim)

Points at which to evaluate the probability density function.

broadcastbool, optional, default=False

If True, broadcast x across the distribution parameters.

jointbool, optional, default=False

If True, return the joint pdf of the mixture P(x, N)

Returns:

pdf

Probability density function evaluated at x. if not broadcast and not joint: shape (*size, *shape[:-1]) elif not broadcast and joint: shape (*size, *shape) elif not joint: shape the broadcast of (*size,) and `shape[:-1] else: shape the broadcast of (*size,) and `shape

rvs(size=(), broadcast=False)

Draw random samples from the distribution.

Parameters:

sizeint or tuple of ints, optional, default=1

Number of samples to draw.

broadcastbool, optional, default=False

If True, broadcast x across the distribution parameters.

Returns:

rvsarray_like, shape (*size, *shape[:-1], dim)

property shape

Shape of the distribution.

class lsbi.stats.multivariate_normal(mean=0, cov=1, shape=(), dim=1, diagonal=False)

Bases: object

Vectorised multivariate normal distribution.

This extends scipy.stats.multivariate_normal to allow for vectorisation across the distribution parameters mean and cov.

Implemented with the same style as scipy.stats.multivariate_normal, except that results are not squeezed.

mean and cov are lazily broadcasted to the same shape to improve performance.

Parameters:

meanarray_like, shape (…, dim)

Mean of each component.

cov array_like, shape `(…, dim, dim)`if diagonal is False else shape `(…, dim)`

Covariance matrix of each component.

shape: tuple, optional, default=()

Shape of the distribution. Useful for forcing a broadcast beyond that inferred by mean and cov shapes

dim: int, optional, default=0

Dimension of the distribution. Useful for forcing a broadcast beyond that inferred by mean and cov dimensions

diagonal: bool, optional, default=False

If True, cov is interpreted as the diagonal of the covariance matrix.

bijector(x, inverse=False)

Bijector between U([0, 1])^d and the distribution.

• x in [0, 1]^d is the hypercube space.

• theta in R^d is the physical space.

Computes the transformation from x to theta or theta to x depending on the value of inverse.

Parameters:

xarray_like, shape (…, dim)

if inverse: x is theta else: x is x

inversebool, optional, default=False

If True: compute the inverse transformation from physical to hypercube space.

where self.shape is broadcastable to …

Returns:

transformed x or theta: array_like, shape (…, dim)

condition(indices, values)

Condition on indices with values.

Parameters:

indicesarray_like

Indices to condition over.

valuesarray_like shape (…, len(indices))

Values to condition on.

where where self.shape is broadcastable to …

Returns:

conditioned distribution shape (…, len(indices))

property dim

Dimension of the distribution.

logpdf(x, broadcast=False)

Log of the probability density function.

Parameters:

xarray_like, shape (*size, dim)

Points at which to evaluate the log of the probability density function.

broadcastbool, optional, default=False

If True, broadcast x across the shape of the distribution

Returns:

logpdfarray_like

Log of the probability density function evaluated at x. if not broadcast: shape (*size, *self.shape) else: shape broadcast of (*size,) and `self.shape

marginalise(indices)

Marginalise over indices.

Parameters:

indicesarray_like

Indices to marginalise.

Returns:

marginalised distribution, shape (*shape, dim - len(indices))

pdf(x, broadcast=False)

Probability density function.

Parameters:

xarray_like, shape (*size, dim)

Points at which to evaluate the probability density function.

broadcastbool, optional, default=False

If True, broadcast x across the distribution parameters.

Returns:

pdfarray_like

Probability density function evaluated at x. if not broadcast: shape (*size, *self.shape) else: shape broadcast of (*size,) and `self.shape

predict(A=1, b=0, diagonal=False)

Predict the mean and covariance of a linear transformation.

if: x ~ N(μ, Σ) then: Ax + b ~ N(A μ + b, A Σ A^T)

Parameters:

Aarray_like, shape (…, k, dim)

Linear transformation matrix.

barray_like, shape (…, k), optional

Linear transformation vector.

diagonalbool, optional, default=False

If True, A is interpreted as the diagonal of the transformation matrix.

where self.shape is broadcastable to …

Returns:

transformed distribution shape (…, k)

rvs(size=(), broadcast=False)

Draw random samples from the distribution.

Parameters:

sizeint or tuple of ints, optional, default=()

Number of samples to draw.

broadcastbool, optional, default=False

If True, broadcast x across the distribution parameters.

Returns:

rvsndarray, shape (*size, *shape, dim)

Random samples from the distribution.

property shape

Shape of the distribution.

lsbi.utils module

Utility functions for lsbi.

lsbi.utils.alias(cls, name, alias)

Create an alias for a property.

Parameters:

clsclass

Class to add the alias to.

namestr

Name of the property to alias.

aliasstr

Name of the alias.

Examples

>>> class MyCls:
...     def __init__(self, name):
...         self.name = name
...
>>> alias(MyCls, 'name', 'n')
>>> obj = MyCls('will')
>>> obj.name
'will'
>>> obj.n
'will'
>>> obj.n = 'bill'
>>> obj.name
'bill'

lsbi.utils.bisect(f, a, b, args=(), tol=1e-08)

Vectorised simple bisection search.

The shape of the output is the broadcasted shape of a and b.

Parameters:

fcallable

Function to find the root of.

aarray_like

Lower bound of the search interval.

barray_like

Upper bound of the search interval.

argstuple, optional

Extra arguments to f.

tolfloat, optional

(absolute) tolerance of the solution

Returns:

xndarray

Solution to the equation f(x) = 0.

lsbi.utils.dediagonalise(x, diagonal, *args)

Optionally construct a dense matrix with x on the diagonal.

lsbi.utils.logdet(A, diagonal=False)

log(abs(det(A))).

lsbi.utils.quantise(f, x, tol=1e-08)

Quantise f(x) to zero within tolerance tol.