botorch.posteriors¶

Posterior APIs¶

Abstract Posterior API¶

Abstract base module for all botorch posteriors.

class botorch.posteriors.posterior.Posterior[source]¶

Bases: ABC

Abstract base class for botorch posteriors.

rsample_from_base_samples(sample_shape, base_samples)[source]¶

Sample from the posterior (with gradients) using base samples.

This is intended to be used with a sampler that produces the corresponding base samples, and enables acquisition optimization via Sample Average Approximation.

Parameters:

sample_shape (Size) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
base_samples (Tensor) – The base samples, obtained from the appropriate sampler. This is a tensor of shape sample_shape x base_sample_shape.

Returns:

Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).

Return type:

Tensor

abstract rsample(sample_shape=None)[source]¶

Sample from the posterior (with gradients).

Parameters:: sample_shape (Size | None) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
Returns:: Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).
Return type:: Tensor

sample(sample_shape=None)[source]¶

Sample from the posterior without gradients.

Parameters:: sample_shape (Size | None) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
Returns:: Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).
Return type:: Tensor

abstract property device: device¶: The torch device of the distribution.

abstract property dtype: dtype¶: The torch dtype of the distribution.

quantile(value)[source]¶

Compute quantiles of the distribution.

For multi-variate distributions, this may return the quantiles of the marginal distributions.

Parameters:: value (Tensor)
Return type:: Tensor

density(value)[source]¶

The probability density (or mass) of the distribution.

For multi-variate distributions, this may return the density of the marginal distributions.

Parameters:: value (Tensor)
Return type:: Tensor

property base_sample_shape: Size¶

The base shape of the base samples expected in rsample.

Informs the sampler to produce base samples of shape sample_shape x base_sample_shape.

property batch_range: tuple[int, int]¶

The t-batch range.

This is used in samplers to identify the t-batch component of the base_sample_shape. The base samples are expanded over the t-batches to provide consistency in the acquisition values, i.e., to ensure that a candidate produces same value regardless of its position on the t-batch.

Posterior List API¶

Abstract base module for all botorch posteriors.

class botorch.posteriors.posterior_list.PosteriorList(*posteriors)[source]¶

Bases: Posterior

A Posterior represented by a list of independent Posteriors.

When at least one of the posteriors is a GaussianMixturePosterior, the other posteriors are expanded to match the size of the GaussianMixturePosterior.

A Posterior represented by a list of independent Posteriors.

Parameters:: *posteriors (Posterior) – A variable number of single-outcome posteriors.

Example

>>> p_1 = model_1.posterior(test_X)
>>> p_2 = model_2.posterior(test_X)
>>> p_12 = PosteriorList(p_1, p_2)

Note: This is typically produced automatically in ModelList; it should generally not be necessary for the end user to invoke it manually.

property device: device¶: The torch device of the posterior.

property dtype: dtype¶: The torch dtype of the posterior.

property mean: Tensor¶

The mean of the posterior as a (b) x n x m-dim Tensor.

This is only supported if all posteriors provide a mean.

property variance: Tensor¶

The variance of the posterior as a (b) x n x m-dim Tensor.

This is only supported if all posteriors provide a variance.

rsample(sample_shape=None)[source]¶

Sample from the posterior (with gradients).

Parameters:: sample_shape (Size | None) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
Returns:: Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).
Return type:: Tensor

Posteriors¶

Torch Posterior¶

Posterior module to be used with PyTorch distributions.

class botorch.posteriors.torch.TorchPosterior(distribution)[source]¶

Bases: Posterior

A posterior based on a PyTorch Distribution.

NOTE: For any attribute that is not explicitly defined on the Posterior level, this returns the corresponding attribute of the distribution. This allows easy access to the distribution attributes, without having to expose them on the Posterior.

A posterior based on a PyTorch Distribution.

Parameters:: distribution (Distribution) – A PyTorch Distribution object.

rsample(sample_shape=None)[source]¶

Sample from the posterior (with gradients).

This is generally used with a sampler that produces the base samples.

Parameters:: sample_shape (Size | None) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
Returns:: Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).
Return type:: Tensor

property device: device¶: The torch device of the distribution.

property dtype: dtype¶: The torch dtype of the distribution.

quantile(value)[source]¶

Compute quantiles of the distribution.

For multi-variate distributions, this may return the quantiles of the marginal distributions.

Parameters:: value (Tensor)
Return type:: Tensor

density(value)[source]¶

The probability density (or mass if discrete) of the distribution.

For multi-variate distributions, this may return the density of the marginal distributions.

Parameters:: value (Tensor)
Return type:: Tensor

GPyTorch Posterior¶

Posterior module to be used with GPyTorch models.

class botorch.posteriors.gpytorch.GPyTorchPosterior(distribution)[source]¶

Bases: TorchPosterior

A posterior based on GPyTorch’s multi-variate Normal distributions.

Parameters:: distribution (MultivariateNormal) – A GPyTorch MultivariateNormal (single-output case) or MultitaskMultivariateNormal (multi-output case).

distribution: MultivariateNormal¶

property mvn: MultivariateNormal¶: Expose the distribution as a backwards-compatible attribute.

property base_sample_shape: Size¶: The shape of a base sample used for constructing posterior samples.

property batch_range: tuple[int, int]¶

The t-batch range.

This is used in samplers to identify the t-batch component of the base_sample_shape. The base samples are expanded over the t-batches to provide consistency in the acquisition values, i.e., to ensure that a candidate produces same value regardless of its position on the t-batch.

rsample_from_base_samples(sample_shape, base_samples)[source]¶

Sample from the posterior (with gradients) using base samples.

This is intended to be used with a sampler that produces the corresponding base samples, and enables acquisition optimization via Sample Average Approximation.

Parameters:

sample_shape (Size) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
base_samples (Tensor) – A Tensor of N(0, I) base samples of shape sample_shape x base_sample_shape, typically obtained from a Sampler. This is used for deterministic optimization.

Returns:

Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).

Return type:

Tensor

rsample(sample_shape=None)[source]¶

Sample from the posterior (with gradients).

Parameters:: sample_shape (Size | None) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
Returns:: Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).
Return type:: Tensor

property mean: Tensor¶: The posterior mean.

property variance: Tensor¶: The posterior variance.

quantile(value)[source]¶

Compute the quantiles of the marginal distributions.

Parameters:: value (Tensor)
Return type:: Tensor

density(value)[source]¶

The probability density of the marginal distributions.

Parameters:: value (Tensor)
Return type:: Tensor

botorch.posteriors.gpytorch.scalarize_posterior_gpytorch(posterior, weights, offset=0.0)[source]¶

Helper function for scalarize_posterior, producing a mean and variance.

This mean and variance are consumed by scalarize_posterior to produce a GPyTorchPosterior.

Parameters:

posterior (GPyTorchPosterior) – The posterior over m outcomes to be scalarized. Supports t-batching.
weights (Tensor) – A tensor of weights of size m.
offset (float) – The offset of the affine transformation.

Returns:

The transformed (single-output) posterior. If the input posterior has: mean mu and covariance matrix Sigma, this posterior has mean weights^T * mu and variance weights^T Sigma w.

Return type:

tuple[Tensor, Tensor | LinearOperator]

Example

Example for a model with two outcomes:

>>> X = torch.rand(1, 2)
>>> posterior = model.posterior(X)
>>> weights = torch.tensor([0.5, 0.25])
>>> mean, cov = scalarize_posterior_gpytorch(posterior, weights=weights)
>>> mvn = MultivariateNormal(mean, cov)
>>> new_posterior = GPyTorchPosterior

botorch.posteriors.gpytorch.scalarize_posterior(posterior, weights, offset=0.0)[source]¶

Affine transformation of a multi-output posterior.

Parameters:

posterior (Union[GPyTorchPosterior, PosteriorList]) – The posterior over m outcomes to be scalarized. Supports t-batching. Can be either a GPyTorchPosterior, or a PosteriorList that contains GPyTorchPosteriors all with q=1.
weights (Tensor) – A tensor of weights of size m.
offset (float) – The offset of the affine transformation.

Returns:

The transformed (single-output) posterior. If the input posterior has: mean mu and covariance matrix Sigma, this posterior has mean weights^T * mu and variance weights^T Sigma w.

Return type:

GPyTorchPosterior

Example

Example for a model with two outcomes:

>>> X = torch.rand(1, 2)
>>> posterior = model.posterior(X)
>>> weights = torch.tensor([0.5, 0.25])
>>> new_posterior = scalarize_posterior(posterior, weights=weights)

Ensemble Posterior¶

Ensemble posteriors. Used in conjunction with ensemble models.

class botorch.posteriors.ensemble.EnsemblePosterior(values)[source]¶

Bases: Posterior

Ensemble posterior, that should be used for ensemble models that compute eagerly a finite number of samples per X value as for example a deep ensemble or a random forest.

Parameters:: values (Tensor) – Values of the samples produced by this posterior as a (b) x s x q x m tensor where m is the output size of the model and s is the ensemble size.

property ensemble_size: int¶: The size of the ensemble

property weights: Tensor¶: The weights of the individual models in the ensemble. Equally weighted by default.

property device: device¶: The torch device of the posterior.

property dtype: dtype¶: The torch dtype of the posterior.

property mean: Tensor¶: The mean of the posterior as a (b) x n x m-dim Tensor.

property variance: Tensor¶

The variance of the posterior as a (b) x n x m-dim Tensor.

Computed as the sample variance across the ensemble outputs.

rsample(sample_shape=None)[source]¶

Sample from the posterior (with gradients).

Based on the sample shape, base samples are generated and passed to rsample_from_base_samples.

Parameters:: sample_shape (Size | None) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
Returns:: Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).
Return type:: Tensor

rsample_from_base_samples(sample_shape, base_samples)[source]¶

Sample from the posterior (with gradients) using base samples.

This is intended to be used with a sampler that produces the corresponding base samples, and enables acquisition optimization via Sample Average Approximation.

Parameters:

sample_shape (Size) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
base_samples (Tensor) – A Tensor of indices as base samples of shape sample_shape, typically obtained from IndexSampler. This is used for deterministic optimization. The predictions of the ensemble corresponding to the indices are then sampled.

Returns:

Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).

Return type:

Tensor

Higher Order GP Posterior¶

class botorch.posteriors.higher_order.HigherOrderGPPosterior(distribution, joint_covariance_matrix, train_train_covar, test_train_covar, train_targets, output_shape, num_outputs)[source]¶

Bases: GPyTorchPosterior

Posterior class for a Higher order Gaussian process model [Zhe2019hogp]. Extends the standard GPyTorch posterior class by overwriting the rsample method. The posterior variance is handled internally by the HigherOrderGP model. HOGP is a tensorized GP model so the posterior covariance grows to be extremely large, but is highly structured, which means that we can exploit Kronecker identities to sample from the posterior using Matheron’s rule as described in [Doucet2010sampl].

In general, this posterior should ONLY be used for HOGP models that have highly structured covariances. It should also only be used internally when called from the HigherOrderGP.posterior(…) method. At this time, the posterior does not support gradients with respect to the training data.

A Posterior for HigherOrderGP models.

Parameters:

distribution (MultivariateNormal) – Posterior multivariate normal distribution.
joint_covariance_matrix (LinearOperator) – Joint test train covariance matrix over the entire tensor.
train_train_covar (LinearOperator) – Covariance matrix of train points in the data space.
test_train_covar (LinearOperator) – Covariance matrix of test x train points in the data space.
train_targets (Tensor) – Training responses vectorized.
output_shape (Size) – Shape output training responses.
num_outputs (int) – Batch shaping of model.

property base_sample_shape¶

The shape of a base sample used for constructing posterior samples.

Overwrites the standard base_sample_shape call to inform samplers that n + 2 n_train samples need to be drawn rather than n samples.

property batch_range: tuple[int, int]¶

The t-batch range.

This is used in samplers to identify the t-batch component of the base_sample_shape. The base samples are expanded over the t-batches to provide consistency in the acquisition values, i.e., to ensure that a candidate produces same value regardless of its position on the t-batch.

rsample_from_base_samples(sample_shape, base_samples)[source]¶

Sample from the posterior (with gradients) using base samples.

As the posterior covariance is difficult to draw from in this model, we implement Matheron’s rule as described in [Doucet2010sampl]-. This may not work entirely correctly for deterministic base samples unless base samples are provided that are of shape n + 2 * n_train because the sampling method draws 2 * n_train samples as well as the standard n. samples.

Parameters:

sample_shape (Size) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
base_samples (Tensor | None) – An (optional) Tensor of N(0, I) base samples of appropriate dimension, typically obtained from a Sampler. This is used for deterministic optimization.

Returns:

Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).

Return type:

Tensor

rsample(sample_shape=None)[source]¶

Sample from the posterior (with gradients).

Parameters:: sample_shape (Size | None) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
Returns:: Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).
Return type:: Tensor

Multitask GP Posterior¶

class botorch.posteriors.multitask.MultitaskGPPosterior(distribution, joint_covariance_matrix, test_train_covar, train_diff, test_mean, train_train_covar, train_noise, test_noise=None)[source]¶

Bases: GPyTorchPosterior

Posterior class for a Kronecker Multi-task GP model using with ICM kernel. Extends the standard GPyTorch posterior class by overwriting the rsample method. In general, this posterior should ONLY be used for MTGP models that have structured covariances. It should also only be used internally when called from the KroneckerMultiTaskGP.posterior(…) method.

Parameters:

distribution (MultivariateNormal) – Posterior multivariate normal distribution.
joint_covariance_matrix (LinearOperator) – Joint test train covariance matrix over the entire tensor.
train_train_covar (LinearOperator) – Covariance matrix of train points in the data space.
test_obs_covar – Covariance matrix of test x train points in the data space.
train_diff (Tensor) – Difference between train mean and train responses.
train_noise (LinearOperator | Tensor) – Training noise covariance.
test_noise (LinearOperator | Tensor | None) – Only used if posterior should contain observation noise. Testing noise covariance.
test_train_covar (LinearOperator)
test_mean (Tensor)

property base_sample_shape: Size¶

The shape of a base sample used for constructing posterior samples.

Overwrites the standard base_sample_shape call to inform samplers that n + 2 n_train samples need to be drawn rather than n samples.

property batch_range: tuple[int, int]¶

The t-batch range.

This is used in samplers to identify the t-batch component of the base_sample_shape. The base samples are expanded over the t-batches to provide consistency in the acquisition values, i.e., to ensure that a candidate produces same value regardless of its position on the t-batch.

rsample_from_base_samples(sample_shape, base_samples, train_diff=None)[source]¶

Sample from the posterior (with gradients) using base samples.

Parameters:

sample_shape (Size) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
base_samples (Tensor | None) – An (optional) Tensor of N(0, I) base samples of appropriate dimension, typically obtained from a Sampler. This is used for deterministic optimization.
train_diff (Tensor | None) – Difference between train mean and train responses to assume during sampling.

Returns:

Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).

Return type:

Tensor

rsample(sample_shape=None)[source]¶

Sample from the posterior (with gradients).

Parameters:: sample_shape (Size | None) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
Returns:: Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).
Return type:: Tensor

Transformed Posterior¶

class botorch.posteriors.transformed.TransformedPosterior(posterior, sample_transform, mean_transform=None, variance_transform=None)[source]¶

Bases: Posterior

A generic transformation of a posterior (implicitly represented).

An implicitly represented transformed posterior.

Parameters:

posterior (Posterior) – The posterior object to be transformed.
sample_transform (Callable[[Tensor], Tensor]) – A callable applying a sample-level transform to a sample_shape x batch_shape x q x m-dim tensor of samples from the original posterior, returning a tensor of samples of the same shape.
mean_transform (Optional[Callable[[Tensor, Tensor], Tensor]]) – A callable transforming a 2-tuple of mean and variance (both of shape batch_shape x m x o) of the original posterior to the mean of the transformed posterior.
variance_transform (Optional[Callable[[Tensor, Tensor], Tensor]]) – A callable transforming a 2-tuple of mean and variance (both of shape batch_shape x m x o) of the original posterior to a variance of the transformed posterior.

property base_sample_shape: Size¶: The shape of a base sample used for constructing posterior samples.

property batch_range: tuple[int, int]¶

The t-batch range.

This is used in samplers to identify the t-batch component of the base_sample_shape. The base samples are expanded over the t-batches to provide consistency in the acquisition values, i.e., to ensure that a candidate produces same value regardless of its position on the t-batch.

property device: device¶: The torch device of the posterior.

property dtype: dtype¶: The torch dtype of the posterior.

property mean: Tensor¶: The mean of the posterior as a batch_shape x n x m-dim Tensor.

property variance: Tensor¶: The variance of the posterior as a batch_shape x n x m-dim Tensor.

rsample_from_base_samples(sample_shape, base_samples)[source]¶

Sample from the posterior (with gradients) using base samples.

This is intended to be used with a sampler that produces the corresponding base samples, and enables acquisition optimization via Sample Average Approximation.

Parameters:

sample_shape (Size) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
base_samples (Tensor) – The base samples, obtained from the appropriate sampler. This is a tensor of shape sample_shape x base_sample_shape.

Returns:

Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).

Return type:

Tensor

rsample(sample_shape=None)[source]¶

Sample from the posterior (with gradients).

Parameters:: sample_shape (Size | None) – A torch.Size object specifying the sample shape. To draw n samples, set to torch.Size([n]). To draw b batches of n samples each, set to torch.Size([b, n]).
Returns:: Samples from the posterior, a tensor of shape self._extended_shape(sample_shape=sample_shape).
Return type:: Tensor

Fully Bayesian Posterior¶

botorch.posteriors.fully_bayesian.batched_bisect(f, target, bounds, tol=1e-06, max_steps=32)[source]¶

Batched bisection with a fixed number of steps.

Parameters:

f (Callable) – Target function that takes a (b1 x … x bk)-dim tensor and returns a (b1 x … x bk)-dim tensor.
target (float) – Scalar target value of type float.
bounds (Tensor) – Lower and upper bounds, of size 2 x b1 x … x bk.
tol (float) – We termniate if all elements satisfy are within tol of the target.
max_steps (int) – Maximum number of bisection steps.

Returns:

Tensor X of size b1 x … x bk such that f(X) = target.

class botorch.posteriors.fully_bayesian.GaussianMixturePosterior(distribution)[source]¶

Bases: GPyTorchPosterior

A Gaussian mixture posterior.

The MCMC batch dimension that corresponds to the models in the mixture is located at MCMC_DIM (defined at the top of this file). Note that while each MCMC sample corresponds to a Gaussian posterior, the posterior is rather a mixture of Gaussian distributions.

A posterior for a fully Bayesian model.

Parameters:: distribution (MultivariateNormal) – A GPyTorch MultivariateNormal (single-output case)

property mixture_mean: Tensor¶: The posterior mean for the mixture of models.

property mixture_variance: Tensor¶: The posterior variance for the mixture of models.

property mixture_covariance_matrix: Tensor¶: The posterior covariance matrix for the mixture of models.

quantile(value)[source]¶

Compute the posterior quantiles for the mixture of models.

Parameters:: value (Tensor)
Return type:: Tensor

property batch_range: tuple[int, int]¶

The t-batch range.

This is used in samplers to identify the t-batch component of the base_sample_shape. The base samples are expanded over the t-batches to provide consistency in the acquisition values, i.e., to ensure that a candidate produces same value regardless of its position on the t-batch.

class botorch.posteriors.fully_bayesian.FullyBayesianPosterior(distribution)[source]¶

Bases: GaussianMixturePosterior

For backwards compatibility.

DEPRECATED.

Parameters:: distribution (MultivariateNormal)

BoTorch

stable

botorch.posteriors¶

Posterior APIs¶

Abstract Posterior API¶

Posterior List API¶

Posteriors¶

Torch Posterior¶

GPyTorch Posterior¶

Ensemble Posterior¶

Higher Order GP Posterior¶

Multitask GP Posterior¶

Transformed Posterior¶

Fully Bayesian Posterior¶

Utilities¶

Base Samples¶

BoTorch

Navigation

Related Topics