botorch.acquisition¶
Acquisition Function APIs¶
Abstract Acquisition Function APIs¶
Abstract base module for all botorch acquisition functions.

class
botorch.acquisition.acquisition.
AcquisitionFunction
(model)[source]¶ Bases:
torch.nn.modules.module.Module
,abc.ABC
Abstract base class for acquisition functions.
Constructor for the AcquisitionFunction base class.
 Parameters
model (
Model
) – A fitted model.

class
botorch.acquisition.acquisition.
OneShotAcquisitionFunction
(model)[source]¶ Bases:
botorch.acquisition.acquisition.AcquisitionFunction
,abc.ABC
Abstract base class for acquisition functions using oneshot optimization
Constructor for the AcquisitionFunction base class.
 Parameters
model (
Model
) – A fitted model.

abstract
get_augmented_q_batch_size
(q)[source]¶ Get augmented q batch size for oneshot optimzation.
 Parameters
q (
int
) – The number of candidates to consider jointly. Return type
int
 Returns
The augmented size for oneshot optimzation (including variables parameterizing the fantasy solutions).

abstract
extract_candidates
(X_full)[source]¶ Extract the candidates from a full “oneshot” parameterization.
 Parameters
X_full (
Tensor
) – A b x q_aug x ddim Tensor with b tbatches of q_aug design points each. Return type
Tensor
 Returns
A b x q x ddim Tensor with b tbatches of q design points each.
Analytic Acquisition Function API¶

class
botorch.acquisition.analytic.
AnalyticAcquisitionFunction
(model, objective=None)[source]¶ Bases:
botorch.acquisition.acquisition.AcquisitionFunction
,abc.ABC
Base class for analytic acquisition functions.
Base constructor for analytic acquisition functions.
 Parameters
model (
Model
) – A fitted singleoutcome model.objective (
Optional
[ScalarizedObjective
]) – A ScalarizedObjective (optional).
MonteCarlo Acquisition Function API¶

class
botorch.acquisition.monte_carlo.
MCAcquisitionFunction
(model, sampler=None, objective=None, X_pending=None)[source]¶ Bases:
botorch.acquisition.acquisition.AcquisitionFunction
,abc.ABC
Abstract base class for MonteCarlo based batch acquisition functions.
Constructor for the MCAcquisitionFunction base class.
 Parameters
model (
Model
) – A fitted model.sampler (
Optional
[MCSampler
]) – The sampler used to draw base samples. Defaults to SobolQMCNormalSampler(num_samples=512, collapse_batch_dims=True).objective (
Optional
[MCAcquisitionObjective
]) – The MCAcquisitionObjective under which the samples are evaluated. Defaults to IdentityMCObjective().X_pending (
Optional
[Tensor
]) – A batch_shape, m x ddim Tensor of m design points that have points that have been submitted for function evaluation but have not yet been evaluated.

abstract
forward
(X)[source]¶ Takes in a batch_shape x q x d X Tensor of tbatches with q ddim design points each, and returns a Tensor with shape batch_shape’, where batch_shape’ is the broadcasted batch shape of model and input X. Should utilize the result of set_X_pending as needed to account for pending function evaluations.
 Return type
Tensor
MultiObjective Analytic Acquisition Function API¶

class
botorch.acquisition.multi_objective.analytic.
MultiObjectiveAnalyticAcquisitionFunction
(model, objective=None)[source]¶ Bases:
botorch.acquisition.acquisition.AcquisitionFunction
Abstract base class for MultiObjective batch acquisition functions.
Constructor for the MultiObjectiveAnalyticAcquisitionFunction base class.
 Parameters
model (
Model
) – A fitted model.objective (
Optional
[AnalyticMultiOutputObjective
]) – An AnalyticMultiOutputObjective (optional).
MultiObjective MonteCarlo Acquisition Function API¶

class
botorch.acquisition.multi_objective.monte_carlo.
MultiObjectiveMCAcquisitionFunction
(model, sampler=None, objective=None, X_pending=None)[source]¶ Bases:
botorch.acquisition.acquisition.AcquisitionFunction
Abstract base class for MultiObjective batch acquisition functions.
Constructor for the MCAcquisitionFunction base class.
 Parameters
model (
Model
) – A fitted model.sampler (
Optional
[MCSampler
]) – The sampler used to draw base samples. Defaults to SobolQMCNormalSampler(num_samples=512, collapse_batch_dims=True).objective (
Optional
[MCMultiOutputObjective
]) – The MCMultiOutputObjective under which the samples are evaluated. Defaults to IdentityMultiOutputObjective().X_pending (
Optional
[Tensor
]) – A m x ddim Tensor of m design points that have points that have been submitted for function evaluation but have not yet been evaluated.

abstract
forward
(X)[source]¶ Takes in a batch_shape x q x d X Tensor of tbatches with q ddim design points each, and returns a Tensor with shape batch_shape’, where batch_shape’ is the broadcasted batch shape of model and input X. Should utilize the result of set_X_pending as needed to account for pending function evaluations.
 Return type
Tensor
Acquisition Functions¶
Analytic Acquisition Functions¶
Analytic Acquisition Functions that evaluate the posterior without performing MonteCarlo sampling.

class
botorch.acquisition.analytic.
ExpectedImprovement
(model, best_f, objective=None, maximize=True)[source]¶ Bases:
botorch.acquisition.analytic.AnalyticAcquisitionFunction
Singleoutcome Expected Improvement (analytic).
Computes classic Expected Improvement over the current best observed value, using the analytic formula for a Normal posterior distribution. Unlike the MCbased acquisition functions, this relies on the posterior at single test point being Gaussian (and require the posterior to implement mean and variance properties). Only supports the case of q=1. The model must be singleoutcome.
EI(x) = E(max(y  best_f, 0)), y ~ f(x)
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> EI = ExpectedImprovement(model, best_f=0.2) >>> ei = EI(test_X)
Singleoutcome Expected Improvement (analytic).
 Parameters
model (
Model
) – A fitted singleoutcome model.best_f (
Union
[float
,Tensor
]) – Either a scalar or a bdim Tensor (batch mode) representing the best function value observed so far (assumed noiseless).objective (
Optional
[ScalarizedObjective
]) – A ScalarizedObjective (optional).maximize (
bool
) – If True, consider the problem a maximization problem.

forward
(X)[source]¶ Evaluate Expected Improvement on the candidate set X.
 Parameters
X (
Tensor
) – A b1 x … bk x 1 x ddim batched tensor of ddim design points. Expected Improvement is computed for each point individually, i.e., what is considered are the marginal posteriors, not the joint. Return type
Tensor
 Returns
A b1 x … bkdim tensor of Expected Improvement values at the given design points X.

class
botorch.acquisition.analytic.
PosteriorMean
(model, objective=None)[source]¶ Bases:
botorch.acquisition.analytic.AnalyticAcquisitionFunction
Singleoutcome Posterior Mean.
Only supports the case of q=1. Requires the model’s posterior to have a mean property. The model must be singleoutcome.
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> PM = PosteriorMean(model) >>> pm = PM(test_X)
Base constructor for analytic acquisition functions.
 Parameters
model (
Model
) – A fitted singleoutcome model.objective (
Optional
[ScalarizedObjective
]) – A ScalarizedObjective (optional).

class
botorch.acquisition.analytic.
ProbabilityOfImprovement
(model, best_f, objective=None, maximize=True)[source]¶ Bases:
botorch.acquisition.analytic.AnalyticAcquisitionFunction
Singleoutcome Probability of Improvement.
Probability of improvment over the current best observed value, computed using the analytic formula under a Normal posterior distribution. Only supports the case of q=1. Requires the posterior to be Gaussian. The model must be singleoutcome.
PI(x) = P(y >= best_f), y ~ f(x)
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> PI = ProbabilityOfImprovement(model, best_f=0.2) >>> pi = PI(test_X)
Singleoutcome analytic Probability of Improvement.
 Parameters
model (
Model
) – A fitted singleoutcome model.best_f (
Union
[float
,Tensor
]) – Either a scalar or a bdim Tensor (batch mode) representing the best function value observed so far (assumed noiseless).objective (
Optional
[ScalarizedObjective
]) – A ScalarizedObjective (optional).maximize (
bool
) – If True, consider the problem a maximization problem.

class
botorch.acquisition.analytic.
UpperConfidenceBound
(model, beta, objective=None, maximize=True)[source]¶ Bases:
botorch.acquisition.analytic.AnalyticAcquisitionFunction
Singleoutcome Upper Confidence Bound (UCB).
Analytic upper confidence bound that comprises of the posterior mean plus an additional term: the posterior standard deviation weighted by a tradeoff parameter, beta. Only supports the case of q=1 (i.e. greedy, nonbatch selection of design points). The model must be singleoutcome.
UCB(x) = mu(x) + sqrt(beta) * sigma(x), where mu and sigma are the posterior mean and standard deviation, respectively.
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> UCB = UpperConfidenceBound(model, beta=0.2) >>> ucb = UCB(test_X)
Singleoutcome Upper Confidence Bound.
 Parameters
model (
Model
) – A fitted singleoutcome GP model (must be in batch mode if candidate sets X will be)beta (
Union
[float
,Tensor
]) – Either a scalar or a onedim tensor with b elements (batch mode) representing the tradeoff parameter between mean and covarianceobjective (
Optional
[ScalarizedObjective
]) – A ScalarizedObjective (optional).maximize (
bool
) – If True, consider the problem a maximization problem.

class
botorch.acquisition.analytic.
ConstrainedExpectedImprovement
(model, best_f, objective_index, constraints, maximize=True)[source]¶ Bases:
botorch.acquisition.analytic.AnalyticAcquisitionFunction
Constrained Expected Improvement (feasibilityweighted).
Computes the analytic expected improvement for a Normal posterior distribution, weighted by a probability of feasibility. The objective and constraints are assumed to be independent and have Gaussian posterior distributions. Only supports the case q=1. The model should be multioutcome, with the index of the objective and constraints passed to the constructor.
Constrained_EI(x) = EI(x) * Product_i P(y_i in [lower_i, upper_i]), where y_i ~ constraint_i(x) and lower_i, upper_i are the lower and upper bounds for the ith constraint, respectively.
Example
>>> # example where 0th output has a nonnegativity constraint and ... # 1st output is the objective >>> model = SingleTaskGP(train_X, train_Y) >>> constraints = {0: (0.0, None)} >>> cEI = ConstrainedExpectedImprovement(model, 0.2, 1, constraints) >>> cei = cEI(test_X)
Analytic Constrained Expected Improvement.
 Parameters
model (
Model
) – A fitted singleoutcome model.best_f (
Union
[float
,Tensor
]) – Either a scalar or a bdim Tensor (batch mode) representing the best function value observed so far (assumed noiseless).objective_index (
int
) – The index of the objective.constraints (
Dict
[int
,Tuple
[Optional
[float
],Optional
[float
]]]) – A dictionary of the form {i: [lower, upper]}, where i is the output index, and lower and upper are lower and upper bounds on that output (resp. interpreted as Inf / Inf if None)maximize (
bool
) – If True, consider the problem a maximization problem.

class
botorch.acquisition.analytic.
NoisyExpectedImprovement
(model, X_observed, num_fantasies=20, maximize=True)[source]¶ Bases:
botorch.acquisition.analytic.ExpectedImprovement
Singleoutcome Noisy Expected Improvement (via fantasies).
This computes Noisy Expected Improvement by averaging over the Expected Improvemnt values of a number of fantasy models. Only supports the case q=1. Assumes that the posterior distribution of the model is Gaussian. The model must be singleoutcome.
NEI(x) = E(max(y  max Y_baseline), 0)), (y, Y_baseline) ~ f((x, X_baseline)), where X_baseline are previously observed points.
Note: This acquisition function currently relies on using a FixedNoiseGP (required for noiseless fantasies).
Example
>>> model = FixedNoiseGP(train_X, train_Y, train_Yvar=train_Yvar) >>> NEI = NoisyExpectedImprovement(model, train_X) >>> nei = NEI(test_X)
Singleoutcome Noisy Expected Improvement (via fantasies).
 Parameters
model (
GPyTorchModel
) – A fitted singleoutcome model.X_observed (
Tensor
) – A n x d Tensor of observed points that are likely to be the best observed points so far.num_fantasies (
int
) – The number of fantasies to generate. The higher this number the more accurate the model (at the expense of model complexity and performance).maximize (
bool
) – If True, consider the problem a maximization problem.
MonteCarlo Acquisition Functions¶
Batch acquisition functions using the reparameterization trick in combination with (quasi) MonteCarlo sampling. See [Rezende2014reparam], [Wilson2017reparam] and [Balandat2019botorch].
 Rezende2014reparam
D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference in deep generative models. ICML 2014.
 Wilson2017reparam
J. T. Wilson, R. Moriconi, F. Hutter, and M. P. Deisenroth. The reparameterization trick for acquisition functions. ArXiv 2017.

class
botorch.acquisition.monte_carlo.
qExpectedImprovement
(model, best_f, sampler=None, objective=None, X_pending=None)[source]¶ Bases:
botorch.acquisition.monte_carlo.MCAcquisitionFunction
MCbased batch Expected Improvement.
This computes qEI by (1) sampling the joint posterior over q points (2) evaluating the improvement over the current best for each sample (3) maximizing over q (4) averaging over the samples
qEI(X) = E(max(max Y  best_f, 0)), Y ~ f(X), where X = (x_1,…,x_q)
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> best_f = train_Y.max()[0] >>> sampler = SobolQMCNormalSampler(1000) >>> qEI = qExpectedImprovement(model, best_f, sampler) >>> qei = qEI(test_X)
qExpected Improvement.
 Parameters
model (
Model
) – A fitted model.best_f (
Union
[float
,Tensor
]) – The best objective value observed so far (assumed noiseless). Can be a batch_shapeshaped tensor, which in case of a batched model specifies potentially different values for each element of the batch.sampler (
Optional
[MCSampler
]) – The sampler used to draw base samples. Defaults to SobolQMCNormalSampler(num_samples=500, collapse_batch_dims=True)objective (
Optional
[MCAcquisitionObjective
]) – The MCAcquisitionObjective under which the samples are evalauted. Defaults to IdentityMCObjective().X_pending (
Optional
[Tensor
]) – A m x ddim Tensor of m design points that have been submitted for function evaluation but have not yet been evaluated. Concatenated into X upon forward call. Copied and set to have no gradient.

forward
(X)[source]¶ Evaluate qExpectedImprovement on the candidate set X.
 Parameters
X (
Tensor
) – A batch_shape x q x ddim Tensor of tbatches with q ddim design points each. Return type
Tensor
 Returns
A batch_shape’dim Tensor of Expected Improvement values at the given design points X, where batch_shape’ is the broadcasted batch shape of model and input X.

class
botorch.acquisition.monte_carlo.
qNoisyExpectedImprovement
(model, X_baseline, sampler=None, objective=None, X_pending=None, prune_baseline=False)[source]¶ Bases:
botorch.acquisition.monte_carlo.MCAcquisitionFunction
MCbased batch Noisy Expected Improvement.
This function does not assume a best_f is known (which would require noiseless observations). Instead, it uses samples from the joint posterior over the q test points and previously observed points. The improvement over previously observed points is computed for each sample and averaged.
qNEI(X) = E(max(max Y  max Y_baseline, 0)), where (Y, Y_baseline) ~ f((X, X_baseline)), X = (x_1,…,x_q)
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> sampler = SobolQMCNormalSampler(1000) >>> qNEI = qNoisyExpectedImprovement(model, train_X, sampler) >>> qnei = qNEI(test_X)
qNoisy Expected Improvement.
 Parameters
model (
Model
) – A fitted model.X_baseline (
Tensor
) – A batch_shape x r x ddim Tensor of r design points that have already been observed. These points are considered as the potential best design point.sampler (
Optional
[MCSampler
]) – The sampler used to draw base samples. Defaults to SobolQMCNormalSampler(num_samples=500, collapse_batch_dims=True).objective (
Optional
[MCAcquisitionObjective
]) – The MCAcquisitionObjective under which the samples are evaluated. Defaults to IdentityMCObjective().X_pending (
Optional
[Tensor
]) – A batch_shape x m x ddim Tensor of m design points that have points that have been submitted for function evaluation but have not yet been evaluated. Concatenated into X upon forward call. Copied and set to have no gradient.prune_baseline (
bool
) – If True, remove points in X_baseline that are highly unlikely to be the best point. This can significantly improve performance and is generally recommended. In order to customize pruning parameters, instead manually call botorch.acquisition.utils.prune_inferior_points on X_baseline before instantiating the acquisition function.

forward
(X)[source]¶ Evaluate qNoisyExpectedImprovement on the candidate set X.
 Parameters
X (
Tensor
) – A batch_shape x q x ddim Tensor of tbatches with q ddim design points each. Return type
Tensor
 Returns
A batch_shape’dim Tensor of Noisy Expected Improvement values at the given design points X, where batch_shape’ is the broadcasted batch shape of model and input X.

class
botorch.acquisition.monte_carlo.
qProbabilityOfImprovement
(model, best_f, sampler=None, objective=None, X_pending=None, tau=0.001)[source]¶ Bases:
botorch.acquisition.monte_carlo.MCAcquisitionFunction
MCbased batch Probability of Improvement.
Estimates the probability of improvement over the current best observed value by sampling from the joint posterior distribution of the qbatch. MCbased estimates of a probability involves taking expectation of an indicator function; to support autodifferntiation, the indicator is replaced with a sigmoid function with temperature parameter tau.
qPI(X) = P(max Y >= best_f), Y ~ f(X), X = (x_1,…,x_q)
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> best_f = train_Y.max()[0] >>> sampler = SobolQMCNormalSampler(1000) >>> qPI = qProbabilityOfImprovement(model, best_f, sampler) >>> qpi = qPI(test_X)
qProbability of Improvement.
 Parameters
model (
Model
) – A fitted model.best_f (
Union
[float
,Tensor
]) – The best objective value observed so far (assumed noiseless). Can be a batch_shapeshaped tensor, which in case of a batched model specifies potentially different values for each element of the batch.sampler (
Optional
[MCSampler
]) – The sampler used to draw base samples. Defaults to SobolQMCNormalSampler(num_samples=500, collapse_batch_dims=True)objective (
Optional
[MCAcquisitionObjective
]) – The MCAcquisitionObjective under which the samples are evaluated. Defaults to IdentityMCObjective().X_pending (
Optional
[Tensor
]) – A m x ddim Tensor of m design points that have points that have been submitted for function evaluation but have not yet been evaluated. Concatenated into X upon forward call. Copied and set to have no gradient.tau (
float
) – The temperature parameter used in the sigmoid approximation of the step function. Smaller values yield more accurate approximations of the function, but result in gradients estimates with higher variance.

forward
(X)[source]¶ Evaluate qProbabilityOfImprovement on the candidate set X.
 Parameters
X (
Tensor
) – A batch_shape x q x ddim Tensor of tbatches with q ddim design points each. Return type
Tensor
 Returns
A batch_shape’dim Tensor of Probability of Improvement values at the given design points X, where batch_shape’ is the broadcasted batch shape of model and input X.

class
botorch.acquisition.monte_carlo.
qSimpleRegret
(model, sampler=None, objective=None, X_pending=None)[source]¶ Bases:
botorch.acquisition.monte_carlo.MCAcquisitionFunction
MCbased batch Simple Regret.
Samples from the joint posterior over the qbatch and computes the simple regret.
qSR(X) = E(max Y), Y ~ f(X), X = (x_1,…,x_q)
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> sampler = SobolQMCNormalSampler(1000) >>> qSR = qSimpleRegret(model, sampler) >>> qsr = qSR(test_X)
Constructor for the MCAcquisitionFunction base class.
 Parameters
model (
Model
) – A fitted model.sampler (
Optional
[MCSampler
]) – The sampler used to draw base samples. Defaults to SobolQMCNormalSampler(num_samples=512, collapse_batch_dims=True).objective (
Optional
[MCAcquisitionObjective
]) – The MCAcquisitionObjective under which the samples are evaluated. Defaults to IdentityMCObjective().X_pending (
Optional
[Tensor
]) – A batch_shape, m x ddim Tensor of m design points that have points that have been submitted for function evaluation but have not yet been evaluated.

forward
(X)[source]¶ Evaluate qSimpleRegret on the candidate set X.
 Parameters
X (
Tensor
) – A batch_shape x q x ddim Tensor of tbatches with q ddim design points each. Return type
Tensor
 Returns
A batch_shape’dim Tensor of Simple Regret values at the given design points X, where batch_shape’ is the broadcasted batch shape of model and input X.

class
botorch.acquisition.monte_carlo.
qUpperConfidenceBound
(model, beta, sampler=None, objective=None, X_pending=None)[source]¶ Bases:
botorch.acquisition.monte_carlo.MCAcquisitionFunction
MCbased batch Upper Confidence Bound.
Uses a reparameterization to extend UCB to qUCB for q > 1 (See Appendix A of [Wilson2017reparam].)
qUCB = E(max(mu + Y_tilde  mu)), where Y_tilde ~ N(mu, beta pi/2 Sigma) and f(X) has distribution N(mu, Sigma).
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> sampler = SobolQMCNormalSampler(1000) >>> qUCB = qUpperConfidenceBound(model, 0.1, sampler) >>> qucb = qUCB(test_X)
qUpper Confidence Bound.
 Parameters
model (
Model
) – A fitted model.beta (
float
) – Controls tradeoff between mean and standard deviation in UCB.sampler (
Optional
[MCSampler
]) – The sampler used to draw base samples. Defaults to SobolQMCNormalSampler(num_samples=500, collapse_batch_dims=True)objective (
Optional
[MCAcquisitionObjective
]) – The MCAcquisitionObjective under which the samples are evaluated. Defaults to IdentityMCObjective().X_pending (
Optional
[Tensor
]) – A batch_shape x m x ddim Tensor of m design points that have points that have been submitted for function evaluation but have not yet been evaluated. Concatenated into X upon forward call. Copied and set to have no gradient.

forward
(X)[source]¶ Evaluate qUpperConfidenceBound on the candidate set X.
 Parameters
X (
Tensor
) – A batch_sahpe x q x ddim Tensor of tbatches with q ddim design points each. Return type
Tensor
 Returns
A batch_shape’dim Tensor of Upper Confidence Bound values at the given design points X, where batch_shape’ is the broadcasted batch shape of model and input X.
MultiObjective Analytic Acquisition Functions¶
Analytic Acquisition Functions for Multiobjective Bayesian optimization.
References
 Yang2019(1,2,3,4,5)
Yang, K., Emmerich, M., Deutz, A. et al. Efficient computation of expected hypervolume improvement using box decomposition algorithms. J Glob Optim 75, 3–34 (2019)

class
botorch.acquisition.multi_objective.analytic.
ExpectedHypervolumeImprovement
(model, ref_point, partitioning, objective=None)[source]¶ Bases:
botorch.acquisition.multi_objective.analytic.MultiObjectiveAnalyticAcquisitionFunction
Expected Hypervolume Improvement supporting m>=2 outcomes.
This implements the computes EHVI using the algorithm from [Yang2019], but additionally computes gradients via autodifferentiation as proposed by [Daulton2020].
Note: this is currently inefficient in two ways due to the binary partitioning algorithm that we use for the box decomposition:
We have more boxes in our decomposition
 If we used a box decomposition that used inf as the upper bound for
the last dimension in all hypercells, then we could reduce the number of terms we need to compute from 2^m to 2^(m1). [Yang2019] do this by using DKLV17 and LKF17 for the box decomposition.
TODO: Use DKLV17 and LKF17 for the box decomposition as in [Yang2019] for greater efficiency.
TODO: Add support for outcome constraints.
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> ref_point = [0.0, 0.0] >>> EHVI = ExpectedHypervolumeImprovement(model, ref_point, partitioning) >>> ehvi = EHVI(test_X)
 Parameters
model (
Model
) – A fitted model.ref_point (
List
[float
]) – A list with m elements representing the reference point (in the outcome space) w.r.t. to which compute the hypervolume. This is a reference point for the objective values (i.e. after applying objective to the samples).partitioning (
NondominatedPartitioning
) – A NondominatedPartitioning module that provides the non dominated front and a partitioning of the nondominated space in hyper rectangles.objective (
Optional
[AnalyticMultiOutputObjective
]) – An AnalyticMultiOutputObjective.

psi
(lower, upper, mu, sigma)[source]¶ Compute Psi function.
For each cell i and outcome k:
Psi(lower_{i,k}, upper_{i,k}, mu_k, sigma_k) = ( sigma_k * PDF((upper_{i,k}  mu_k) / sigma_k) + ( mu_k  lower_{i,k} ) * (1  CDF(upper_{i,k}  mu_k) / sigma_k) )
See Equation 19 in [Yang2019] for more details.
 Parameters
lower (
Tensor
) – A num_cells x mdim tensor of lower cell boundsupper (
Tensor
) – A num_cells x mdim tensor of upper cell boundsmu (
Tensor
) – A batch_shape x 1 x mdim tensor of meanssigma (
Tensor
) – A batch_shape x 1 x mdim tensor of standard deviations (clamped).
 Return type
None
 Returns
A batch_shape x num_cells x mdim tensor of values.

nu
(lower, upper, mu, sigma)[source]¶ Compute Nu function.
For each cell i and outcome k:
nu(lower_{i,k}, upper_{i,k}, mu_k, sigma_k) = ( upper_{i,k}  lower_{i,k} ) * (1  CDF((upper_{i,k}  mu_k) / sigma_k))
See Equation 25 in [Yang2019] for more details.
 Parameters
lower (
Tensor
) – A num_cells x mdim tensor of lower cell boundsupper (
Tensor
) – A num_cells x mdim tensor of upper cell boundsmu (
Tensor
) – A batch_shape x 1 x mdim tensor of meanssigma (
Tensor
) – A batch_shape x 1 x mdim tensor of standard deviations (clamped).
 Return type
None
 Returns
A batch_shape x num_cells x mdim tensor of values.
MultiObjective MonteCarlo Acquisition Functions¶
MonteCarlo Acquisition Functions for Multiobjective Bayesian optimization.
References
 Daulton2020(1,2)
S. Daulton, M. Balandat, E. Bakshy. Differentiable Expected Hypervolume Improvement for Parallel MultiObjective Bayesian Optimization. arXiv eprints, arXiv:2006.05078, Jun. 2020.

class
botorch.acquisition.multi_objective.monte_carlo.
qExpectedHypervolumeImprovement
(model, ref_point, partitioning, sampler=None, objective=None, constraints=None, X_pending=None, eta=0.001)[source]¶ Bases:
botorch.acquisition.multi_objective.monte_carlo.MultiObjectiveMCAcquisitionFunction
qExpected Hypervolume Improvement supporting m>=2 outcomes.
See [Daulton2020] for details.
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> ref_point = [0.0, 0.0] >>> qEHVI = qExpectedHypervolumeImprovement(model, ref_point, partitioning) >>> qehvi = qEHVI(test_X)
 Parameters
model (
Model
) – A fitted model.ref_point (
List
[float
]) – A list with m elements representing the reference point (in the outcome space) w.r.t. to which compute the hypervolume. This is a reference point for the objective values (i.e. after applying objective to the samples).partitioning (
NondominatedPartitioning
) – A NondominatedPartitioning module that provides the non dominated front and a partitioning of the nondominated space in hyper rectangles. If constraints are present, this partitioning must only include feasible points.sampler (
Optional
[MCSampler
]) – The sampler used to draw base samples. Defaults to SobolQMCNormalSampler(num_samples=512, collapse_batch_dims=True).objective (
Optional
[MCMultiOutputObjective
]) – The MCMultiOutputObjective under which the samples are evaluated. Defaults to IdentityMultiOutputObjective().constraints (
Optional
[List
[Callable
[[Tensor
],Tensor
]]]) – A list of callables, each mapping a Tensor of dimension sample_shape x batchshape x q x m to a Tensor of dimension sample_shape x batchshape x q, where negative values imply feasibility. The acqusition function will compute expected feasible hypervolume.X_pending (
Optional
[Tensor
]) – A batch_shape x m x ddim Tensor of m design points that have points that have been submitted for function evaluation but have not yet been evaluated. Concatenated into X upon forward call. Copied and set to have no gradient.eta (
float
) – The temperature parameter for the sigmoid function used for the differentiable approximation of the constraints.

forward
(X)[source]¶ Takes in a batch_shape x q x d X Tensor of tbatches with q ddim design points each, and returns a Tensor with shape batch_shape’, where batch_shape’ is the broadcasted batch shape of model and input X. Should utilize the result of set_X_pending as needed to account for pending function evaluations.
 Return type
Tensor
The OneShot Knowledge Gradient¶
Batch Knowledge Gradient (KG) via oneshot optimization as introduced in [Balandat2019botorch]. For broader discussion of KG see also [Frazier2008knowledge] and [Wu2016parallelkg].
 Balandat2019botorch(1,2)
M. Balandat, B. Karrer, D. R. Jiang, S. Daulton, B. Letham, A. G. Wilson, and E. Bakshy. BoTorch: Programmable Bayesian Optimziation in PyTorch. ArXiv 2019.
 Frazier2008knowledge
P. Frazier, W. Powell, and S. Dayanik. A KnowledgeGradient policy for sequential information collection. SIAM Journal on Control and Optimization, 2008.
 Wu2016parallelkg
J. Wu and P. Frazier. The parallel knowledge gradient method for batch bayesian optimization. NIPS 2016.

class
botorch.acquisition.knowledge_gradient.
qKnowledgeGradient
(model, num_fantasies=64, sampler=None, objective=None, inner_sampler=None, X_pending=None, current_value=None)[source]¶ Bases:
botorch.acquisition.monte_carlo.MCAcquisitionFunction
,botorch.acquisition.acquisition.OneShotAcquisitionFunction
Batch Knowledge Gradient using oneshot optimization.
This computes the batch Knowledge Gradient using fantasies for the outer expectation and either the model posterior mean or MCsampling for the inner expectation.
In addition to the design variables, the input X also includes variables for the optimal designs for each of the fantasy models. For a fixed number of fantasies, all parts of X can be optimized in a “oneshot” fashion.
qKnowledge Gradient (oneshot optimization).
 Parameters
model (
Model
) – A fitted model. Must support fantasizing.num_fantasies (
Optional
[int
]) – The number of fantasy points to use. More fantasy points result in a better approximation, at the expense of memory and wall time. Unused if sampler is specified.sampler (
Optional
[MCSampler
]) – The sampler used to sample fantasy observations. Optional if num_fantasies is specified.objective (
Optional
[AcquisitionObjective
]) – The objective under which the samples are evaluated. If None or a ScalarizedObjective, then the analytic posterior mean is used, otherwise the objective is MCevaluated (using inner_sampler).inner_sampler (
Optional
[MCSampler
]) – The sampler used for inner sampling. Ignored if the objective is None or a ScalarizedObjective.X_pending (
Optional
[Tensor
]) – A m x ddim Tensor of m design points that have points that have been submitted for function evaluation but have not yet been evaluated.current_value (
Optional
[Tensor
]) – The current value, i.e. the expected best objective given the observed points D. If omitted, forward will not return the actual KG value, but the expected best objective given the data set D u X.

forward
(X)[source]¶ Evaluate qKnowledgeGradient on the candidate set X.
 Parameters
X (
Tensor
) –A b x (q + num_fantasies) x d Tensor with b tbatches of q + num_fantasies design points each. We split this X tensor into two parts in the q dimension (dim=2). The first q are the qbatch of design points and the last num_fantasies are the current solutions of the inner optimization problem.
X_fantasies = X[…, num_fantasies:, :] X_fantasies.shape = b x num_fantasies x d
X_actual = X[…, :num_fantasies, :] X_actual.shape = b x q x d
 Return type
Tensor
 Returns
 A Tensor of shape b. For tbatch b, the qKG value of the design
X_actual[b] is averaged across the fantasy models, where X_fantasies[b, i] is chosen as the final selection for the ith fantasy model. NOTE: If current_value is not provided, then this is not the true KG value of X_actual[b], and X_fantasies[b, : ] must be maximized at fixed X_actual[b].

get_augmented_q_batch_size
(q)[source]¶ Get augmented q batch size for oneshot optimzation.
 Parameters
q (
int
) – The number of candidates to consider jointly. Return type
int
 Returns
The augmented size for oneshot optimzation (including variables parameterizing the fantasy solutions).

extract_candidates
(X_full)[source]¶ We only return X as the set of candidates postoptimization.
 Parameters
X_full (
Tensor
) – A b x (q + num_fantasies) x ddim Tensor with b tbatches of q + num_fantasies design points each. Return type
Tensor
 Returns
A b x q x ddim Tensor with b tbatches of q design points each.

class
botorch.acquisition.knowledge_gradient.
qMultiFidelityKnowledgeGradient
(model, num_fantasies=64, sampler=None, objective=None, inner_sampler=None, X_pending=None, current_value=None, cost_aware_utility=None, project=<function qMultiFidelityKnowledgeGradient.<lambda>>, expand=<function qMultiFidelityKnowledgeGradient.<lambda>>)[source]¶ Bases:
botorch.acquisition.knowledge_gradient.qKnowledgeGradient
Batch Knowledge Gradient for multifidelity optimization.
A version of qKnowledgeGradient that supports multifidelity optimization via a CostAwareUtility and the project and expand operators. If none of these are set, this acquisition function reduces to qKnowledgeGradient.
MultiFidelity qKnowledge Gradient (oneshot optimization).
 Parameters
model (
Model
) – A fitted model. Must support fantasizing.num_fantasies (
Optional
[int
]) – The number of fantasy points to use. More fantasy points result in a better approximation, at the expense of memory and wall time. Unused if sampler is specified.sampler (
Optional
[MCSampler
]) – The sampler used to sample fantasy observations. Optional if num_fantasies is specified.objective (
Optional
[AcquisitionObjective
]) – The objective under which the samples are evaluated. If None or a ScalarizedObjective, then the analytic posterior mean is used, otherwise the objective is MCevaluated (using inner_sampler).inner_sampler (
Optional
[MCSampler
]) – The sampler used for inner sampling. Ignored if the objective is None or a ScalarizedObjective.X_pending (
Optional
[Tensor
]) – A m x ddim Tensor of m design points that have points that have been submitted for function evaluation but have not yet been evaluated.current_value (
Optional
[Tensor
]) – The current value, i.e. the expected best objective given the observed points D. If omitted, forward will not return the actual KG value, but the expected best objective given the data set D u X.cost_aware_utility (
Optional
[CostAwareUtility
]) – A CostAwareUtility computing the costtransformed utility from a candidate set and samples of increases in utility.project (
Callable
[[Tensor
],Tensor
]) – A callable mapping a batch_shape x q x d tensor of design points to a tensor of the same shape projected to the desired target set (e.g. the target fidelities in case of multifidelity optimization).expand (
Callable
[[Tensor
],Tensor
]) – A callable mapping a batch_shape x q x d input tensor to a batch_shape x (q + q_e)’ x ddim output tensor, where the q_e additional points in each qbatch correspond to additional (“trace”) observations.

property
cost_sampler
¶

forward
(X)[source]¶ Evaluate qMultiFidelityKnowledgeGradient on the candidate set X.
 Parameters
X (
Tensor
) –A b x (q + num_fantasies) x d Tensor with b tbatches of q + num_fantasies design points each. We split this X tensor into two parts in the q dimension (dim=2). The first q are the qbatch of design points and the last num_fantasies are the current solutions of the inner optimization problem.
X_fantasies = X[…, num_fantasies:, :] X_fantasies.shape = b x num_fantasies x d
X_actual = X[…, :num_fantasies, :] X_actual.shape = b x q x d
In addition, X may be augmented with fidelity parameteres as part of thee ddimension. Projecting fidelities to the target fidelity is handled by project.
 Return type
Tensor
 Returns
 A Tensor of shape b. For tbatch b, the qKG value of the design
X_actual[b] is averaged across the fantasy models, where X_fantasies[b, i] is chosen as the final selection for the ith fantasy model. NOTE: If current_value is not provided, then this is not the true KG value of X_actual[b], and X_fantasies[b, : ] must be maximized at fixed X_actual[b].
EntropyBased Acquisition Functions¶
Acquisition functions for maxvalue entropy search (MES) and multifidelity MES with noisy observation and trace observations.
References
 Wang2018mves
Wang, Z., Jegelka, S., Maxvalue Entropy Search for Efficient Bayesian Optimization. arXiv:1703.01968v3, 2018
 Takeno2019mfmves
Takeno, S., et al., Multifidelity Bayesian Optimization with Maxvalue Entropy Search. arXiv:1901.08275v1, 2019

class
botorch.acquisition.max_value_entropy_search.
qMaxValueEntropy
(model, candidate_set, num_fantasies=16, num_mv_samples=10, num_y_samples=128, use_gumbel=True, maximize=True, X_pending=None)[source]¶ Bases:
botorch.acquisition.monte_carlo.MCAcquisitionFunction
The acquisition function for Maxvalue Entropy Search.
This acquisition function computes the mutual information of max values and a candidate point X. See [Wang2018mves] for a detailed discussion.
The model must be singleoutcome. q > 1 is supported through cyclic optimization and fantasies.
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> candidate_set = torch.rand(1000, bounds.size(1)) >>> candidate_set = bounds[0] + (bounds[1]  bounds[0]) * candidate_set >>> MES = qMaxValueEntropy(model, candidate_set) >>> mes = MES(test_X)
Singleoutcome maxvalue entropy search acquisition function.
 Parameters
model (
Model
) – A fitted singleoutcome model.candidate_set (
Tensor
) – A n x d Tensor including n candidate points to discretize the design space. Max values are sampled from the (joint) model posterior over these points.num_fantasies (
int
) – Number of fantasies to generate. The higher this number the more accurate the model (at the expense of model complexity, wall time and memory). Ignored if X_pending is None.num_mv_samples (
int
) – Number of max value samples.num_y_samples (
int
) – Number of posterior samples at specific design point X.use_gumbel (
bool
) – If True, use Gumbel approximation to sample the max values.X_pending (
Optional
[Tensor
]) – A m x ddim Tensor of m design points that have been submitted for function evaluation but have not yet been evaluated.maximize (
bool
) – If True, consider the problem a maximization problem.

set_X_pending
(X_pending=None)[source]¶ Set pending points.
Informs the acquisition function about pending design points, fantasizes the model on the pending points and draws maxvalue samples from the fantasized model posterior.
 Parameters
X_pending (
Optional
[Tensor
]) – m x d Tensor with m ddim design points that have been submitted for evaluation but have not yet been evaluated. Return type
None

class
botorch.acquisition.max_value_entropy_search.
qMultiFidelityMaxValueEntropy
(model, candidate_set, num_fantasies=16, num_mv_samples=10, num_y_samples=128, use_gumbel=True, X_pending=None, maximize=True, cost_aware_utility=None, project=<function qMultiFidelityMaxValueEntropy.<lambda>>, expand=<function qMultiFidelityMaxValueEntropy.<lambda>>)[source]¶ Bases:
botorch.acquisition.max_value_entropy_search.qMaxValueEntropy
Multifidelity maxvalue entropy.
The acquisition function for multifidelity maxvalue entropy search with support for trace observations. See [Takeno2019mfmves] for a detailed discussion of the basic ideas on multifidelity MES (note that this implementation is somewhat different).
The model must be singleoutcome. q > 1 is supported through cyclic optimization and fantasies.
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> candidate_set = torch.rand(1000, bounds.size(1)) >>> candidate_set = bounds[0] + (bounds[1]  bounds[0]) * candidate_set >>> MF_MES = qMultiFidelityMaxValueEntropy(model, candidate_set) >>> mf_mes = MF_MES(test_X)
Singleoutcome maxvalue entropy search acquisition function.
 Parameters
model (
Model
) – A fitted singleoutcome model.candidate_set (
Tensor
) – A n x d Tensor including n candidate points to discretize the design space, which will be used to sample the max values from their posteriors.cost_aware_utility (
Optional
[CostAwareUtility
]) – A CostAwareUtility computing the costtransformed utility from a candidate set and samples of increases in utility.num_fantasies (
int
) – Number of fantasies to generate. The higher this number the more accurate the model (at the expense of model complexity and performance) and it’s only used when X_pending is not None.num_mv_samples (
int
) – Number of max value samples.num_y_samples (
int
) – Number of posterior samples at specific design point X.use_gumbel (
bool
) – If True, use Gumbel approximation to sample the max values.X_pending (
Optional
[Tensor
]) – A m x ddim Tensor of m design points that have been submitted for function evaluation but have not yet been evaluated.maximize (
bool
) – If True, consider the problem a maximization problem.cost_aware_utility – A CostAwareUtility computing the costtransformed utility from a candidate set and samples of increases in utility.
project (
Callable
[[Tensor
],Tensor
]) – A callable mapping a batch_shape x q x d tensor of design points to a tensor of the same shape projected to the desired target set (e.g. the target fidelities in case of multifidelity optimization).expand (
Callable
[[Tensor
],Tensor
]) – A callable mapping a batch_shape x q x d input tensor to a batch_shape x (q + q_e)’ x ddim output tensor, where the q_e additional points in each qbatch correspond to additional (“trace”) observations.

property
cost_sampler
¶
Active Learning Acquisition Functions¶
Active learning acquisition functions.
 Seo2014activedata
S. Seo, M. Wallat, T. Graepel, and K. Obermayer. Gaussian process regression: Active data selection and test point rejection. IJCNN 2000.
 Chen2014seqexpdesign
X. Chen and Q. Zhou. Sequential experimental designs for stochastic kriging. Winter Simulation Conference 2014.
 Binois2017repexp
M. Binois, J. Huang, R. B. Gramacy, and M. Ludkovski. Replication or exploration? Sequential design for stochastic simulation experiments. ArXiv 2017.

class
botorch.acquisition.active_learning.
qNegIntegratedPosteriorVariance
(model, mc_points, sampler=None, objective=None, X_pending=None)[source]¶ Bases:
botorch.acquisition.analytic.AnalyticAcquisitionFunction
Batch Integrated Negative Posterior Variance for Active Learning.
This acquisition function quantifies the (negative) integrated posterior variance (excluding observation noise, computed using MC integration) of the model. In that, it is a proxy for global model uncertainty, and thus purely focused on “exploration”, rather the “exploitation” of many of the classic Bayesian Optimization acquisition functions.
See [Seo2014activedata], [Chen2014seqexpdesign], and [Binois2017repexp].
qIntegrated Negative Posterior Variance.
 Parameters
model (
Model
) – A fitted model.mc_points (
Tensor
) – A batch_shape x N x d tensor of points to use for MCintegrating the posterior variance. Usually, these are qMC samples on the whole design space, but biased sampling directly allows weighted integration of the posterior variance.sampler (
Optional
[MCSampler
]) – The sampler used for drawing fantasy samples. In the basic setting of a standard GP (default) this is a dummy, since the variance of the model after conditioning does not actually depend on the sampled values.objective (
Optional
[ScalarizedObjective
]) – A ScalarizedObjective. Required for multioutput models.X_pending (
Optional
[Tensor
]) – A n’ x ddim Tensor of n’ design points that have points that have been submitted for function evaluation but have not yet been evaluated.
Objectives and CostAware Utilities¶
Objectives¶
Objective Modules to be used with acquisition functions.

class
botorch.acquisition.objective.
AcquisitionObjective
[source]¶ Bases:
torch.nn.modules.module.Module
,abc.ABC
Abstract base class for objectives.
Initializes internal Module state, shared by both nn.Module and ScriptModule.

class
botorch.acquisition.objective.
ScalarizedObjective
(weights, offset=0.0)[source]¶ Bases:
botorch.acquisition.objective.AcquisitionObjective
Affine objective to be used with analytic acquisition functions.
For a Gaussian posterior at a single point (q=1) with mean mu and covariance matrix Sigma, this yields a singleoutput posterior with mean weights^T * mu and variance weights^T Sigma w.
Example
Example for a model with two outcomes:
>>> weights = torch.tensor([0.5, 0.25]) >>> objective = ScalarizedObjective(weights) >>> EI = ExpectedImprovement(model, best_f=0.1, objective=objective)
Affine objective.
 Parameters
weights (
Tensor
) – A onedimensional tensor with m elements representing the linear weights on the outputs.offset (
float
) – An offset to be added to posterior mean.

forward
(posterior)[source]¶ Compute the posterior of the affine transformation.
 Parameters
posterior (
GPyTorchPosterior
) – A posterior with the same number of outputs as the elements in self.weights. Return type
 Returns
A singleoutput posterior.

class
botorch.acquisition.objective.
MCAcquisitionObjective
[source]¶ Bases:
botorch.acquisition.objective.AcquisitionObjective
Abstract base class for MCbased objectives.
Initializes internal Module state, shared by both nn.Module and ScriptModule.

abstract
forward
(samples)[source]¶ Evaluate the objective on the samples.
 Parameters
samples (
Tensor
) – A sample_shape x batch_shape x q x mdim Tensors of samples from a model posterior. Returns
A sample_shape x batch_shape x qdim Tensor of objective values (assuming maximization).
 Return type
Tensor
This method is usually not called directly, but via the objectives
Example
>>> # `__call__` method: >>> samples = sampler(posterior) >>> outcome = mc_obj(samples)

abstract

class
botorch.acquisition.objective.
IdentityMCObjective
[source]¶ Bases:
botorch.acquisition.objective.MCAcquisitionObjective
Trivial objective extracting the last dimension.
Example
>>> identity_objective = IdentityMCObjective() >>> samples = sampler(posterior) >>> objective = identity_objective(samples)
Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward
(samples)[source]¶ Evaluate the objective on the samples.
 Parameters
samples (
Tensor
) – A sample_shape x batch_shape x q x mdim Tensors of samples from a model posterior. Returns
A sample_shape x batch_shape x qdim Tensor of objective values (assuming maximization).
 Return type
Tensor
This method is usually not called directly, but via the objectives
Example
>>> # `__call__` method: >>> samples = sampler(posterior) >>> outcome = mc_obj(samples)


class
botorch.acquisition.objective.
LinearMCObjective
(weights)[source]¶ Bases:
botorch.acquisition.objective.MCAcquisitionObjective
Linear objective constructed from a weight tensor.
For input samples and mc_obj = LinearMCObjective(weights), this produces mc_obj(samples) = sum_{i} weights[i] * samples[…, i]
Example
Example for a model with two outcomes:
>>> weights = torch.tensor([0.75, 0.25]) >>> linear_objective = LinearMCObjective(weights) >>> samples = sampler(posterior) >>> objective = linear_objective(samples)
Linear Objective.
 Parameters
weights (
Tensor
) – A onedimensional tensor with m elements representing the linear weights on the outputs.

class
botorch.acquisition.objective.
GenericMCObjective
(objective)[source]¶ Bases:
botorch.acquisition.objective.MCAcquisitionObjective
Objective generated from a generic callable.
Allows to construct arbitrary MCobjective functions from a generic callable. In order to be able to use gradientbased acquisition function optimization it should be possible to backpropagate through the callable.
Example
>>> generic_objective = GenericMCObjective(lambda Y: torch.sqrt(Y).sum(dim=1)) >>> samples = sampler(posterior) >>> objective = generic_objective(samples)
Objective generated from a generic callable.
 Parameters
objective (
Callable
[[Tensor
],Tensor
]) – A callable mapping a sample_shape x batchshape x q x m dim Tensor to a sample_shape x batchshape x qdim Tensor of objective values.

forward
(samples)[source]¶ Evaluate the feasibilityweigthed objective on the samples.
 Parameters
samples (
Tensor
) – A sample_shape x batch_shape x q x mdim Tensors of samples from a model posterior. Return type
Tensor
 Returns
A sample_shape x batch_shape x qdim Tensor of objective values weighted by feasibility (assuming maximization).

class
botorch.acquisition.objective.
ConstrainedMCObjective
(objective, constraints, infeasible_cost=0.0, eta=0.001)[source]¶ Bases:
botorch.acquisition.objective.GenericMCObjective
Feasibilityweighted objective.
An Objective allowing to maximize some scalable objective on the model outputs subject to a number of constraints. Constraint feasibilty is approximated by a sigmoid function.
mc_acq(X) = objective(X) * prod_i (1  sigmoid(constraint_i(X))) TODO: Document functional form exactly.
See botorch.utils.objective.apply_constraints for details on the constarint handling.
Example
>>> bound = 0.0 >>> objective = lambda Y: Y[..., 0] >>> # apply nonnegativity constraint on f(x)[1] >>> constraint = lambda Y: bound  Y[..., 1] >>> constrained_objective = ConstrainedMCObjective(objective, [constraint]) >>> samples = sampler(posterior) >>> objective = constrained_objective(samples)
Feasibilityweighted objective.
 Parameters
objective (
Callable
[[Tensor
],Tensor
]) – A callable mapping a sample_shape x batchshape x q x m dim Tensor to a sample_shape x batchshape x qdim Tensor of objective values.constraints (
List
[Callable
[[Tensor
],Tensor
]]) – A list of callables, each mapping a Tensor of dimension sample_shape x batchshape x q x m to a Tensor of dimension sample_shape x batchshape x q, where negative values imply feasibility.infeasible_cost (
float
) – The cost of a design if all associated samples are infeasible.eta (
float
) – The temperature parameter of the sigmoid function approximating the constraint.

forward
(samples)[source]¶ Evaluate the feasibilityweighted objective on the samples.
 Parameters
samples (
Tensor
) – A sample_shape x batch_shape x q x mdim Tensors of samples from a model posterior. Return type
Tensor
 Returns
A sample_shape x batch_shape x qdim Tensor of objective values weighted by feasibility (assuming maximization).
MultiObjective Objectives¶

class
botorch.acquisition.multi_objective.objective.
MCMultiOutputObjective
[source]¶ Bases:
botorch.acquisition.objective.AcquisitionObjective
Abstract base class for multioutput objectives.
Initializes internal Module state, shared by both nn.Module and ScriptModule.

abstract
forward
(samples)[source]¶ Evaluate the multioutput objective on the samples.
 Parameters
samples (
Tensor
) – A sample_shape x batch_shape x q x mdim Tensors of samples from a model posterior. Return type
Tensor
 Returns
A sample_shape x batch_shape x q x m’dim Tensor of objective values with m’ the output dimension. assuming maximization in each output dimension).
This method is usually not called directly, but via the objectives
Example
>>> # `__call__` method: >>> samples = sampler(posterior) >>> outcomes = multi_obj(samples)

abstract

class
botorch.acquisition.multi_objective.objective.
IdentityMCMultiOutputObjective
[source]¶ Bases:
botorch.acquisition.multi_objective.objective.MCMultiOutputObjective
Trivial objective extracting the last dimension.
Example
>>> identity_objective = IdentityMCObjective() >>> samples = sampler(posterior) >>> objective = identity_objective(samples)
Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward
(samples)[source]¶ Evaluate the multioutput objective on the samples.
 Parameters
samples (
Tensor
) – A sample_shape x batch_shape x q x mdim Tensors of samples from a model posterior. Return type
Tensor
 Returns
A sample_shape x batch_shape x q x m’dim Tensor of objective values with m’ the output dimension. assuming maximization in each output dimension).
This method is usually not called directly, but via the objectives
Example
>>> # `__call__` method: >>> samples = sampler(posterior) >>> outcomes = multi_obj(samples)


class
botorch.acquisition.multi_objective.objective.
UnstandardizeMCMultiOutputObjective
(Y_mean, Y_std, outcomes=None)[source]¶ Bases:
botorch.acquisition.multi_objective.objective.MCMultiOutputObjective
Objective that unstandardizes the samples.
TODO: remove this when MultiTask models support outcome transforms.
Example
>>> unstd_objective = UnstandardizeMCMultiOutputObjective(Y_mean, Y_std) >>> samples = sampler(posterior) >>> objective = unstd_objective(samples)
Initialize objective.
 Parameters
Y_mean (
Tensor
) – mdim tensor of outcome means.Y_std (
Tensor
) – mdim tensor of outcome standard deviations.outcomes (
Optional
[List
[int
]]) – A list of m’ <= m indices that specifies which of the m model outputs should be considered as the outcomes for MOO. If omitted, use all model outcomes. Typically used for constrained optimization.

forward
(samples)[source]¶ Evaluate the multioutput objective on the samples.
 Parameters
samples (
Tensor
) – A sample_shape x batch_shape x q x mdim Tensors of samples from a model posterior. Return type
Tensor
 Returns
A sample_shape x batch_shape x q x m’dim Tensor of objective values with m’ the output dimension. assuming maximization in each output dimension).
This method is usually not called directly, but via the objectives
Example
>>> # `__call__` method: >>> samples = sampler(posterior) >>> outcomes = multi_obj(samples)

class
botorch.acquisition.multi_objective.objective.
AnalyticMultiOutputObjective
[source]¶ Bases:
botorch.acquisition.objective.AcquisitionObjective
Abstract base class for multioutput analyic objectives.
Initializes internal Module state, shared by both nn.Module and ScriptModule.

abstract
forward
(posterior)[source]¶ Transform the posterior
 Parameters
posterior (
GPyTorchPosterior
) – A posterior. Return type
 Returns
A transformed posterior.

abstract

class
botorch.acquisition.multi_objective.objective.
IdentityAnalyticMultiOutputObjective
[source]¶ Bases:
botorch.acquisition.multi_objective.objective.AnalyticMultiOutputObjective
Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward
(posterior)[source]¶ Transform the posterior
 Parameters
posterior (
GPyTorchPosterior
) – A posterior. Return type
 Returns
A transformed posterior.


class
botorch.acquisition.multi_objective.objective.
UnstandardizeAnalyticMultiOutputObjective
(Y_mean, Y_std, outcomes=None)[source]¶ Bases:
botorch.acquisition.multi_objective.objective.AnalyticMultiOutputObjective
Objective that unstandardizes the posterior.
TODO: remove this when MultiTask models support outcome transforms.
Example
>>> unstd_objective = UnstandardizeAnalyticMultiOutputObjective(Y_mean, Y_std) >>> unstd_posterior = unstd_objective(posterior)
Initialize objective.
 Parameters
Y_mean (
Tensor
) – mdim tensor of outcome meansY_std (
Tensor
) – mdim tensor of outcome standard deviationsoutcomes (
Optional
[List
[int
]]) – A list of m’ <= m indices that specifies which of the m model outputs should be considered as the outcomes for MOO. If omitted, use all model outcomes. Typically used for constrained optimization.

forward
(posterior)[source]¶ Transform the posterior
 Parameters
posterior (
GPyTorchPosterior
) – A posterior. Return type
Tensor
 Returns
A transformed posterior.
CostAware Utility¶
Cost functions for costaware acquisition functions, e.g. multifidelity KG. To be used in a context where there is an objective/cost tradeoff.

class
botorch.acquisition.cost_aware.
CostAwareUtility
[source]¶ Bases:
torch.nn.modules.module.Module
,abc.ABC
Abstract base class for costaware utilities.
Initializes internal Module state, shared by both nn.Module and ScriptModule.

abstract
forward
(X, deltas, **kwargs)[source]¶ Evaluate the costaware utility on the candidates and improvements.
 Parameters
X (
Tensor
) – A batch_shape x q x ddim Tensor of with q ddim design points each for each tbatch.deltas (
Tensor
) – A num_fantasies x batch_shapedim Tensor of num_fantasy samples from the marginal improvement in utility over the current state at X for each tbatch.
 Return type
Tensor
 Returns
A num_fantasies x batch_shapedim Tensor of costtransformed utilities.

abstract

class
botorch.acquisition.cost_aware.
GenericCostAwareUtility
(cost)[source]¶ Bases:
botorch.acquisition.cost_aware.CostAwareUtility
Generic costaware utility wrapping a callable.
Generic costaware utility wrapping a callable.
 Parameters
cost (
Callable
[[Tensor
,Tensor
],Tensor
]) – A callable mapping a batch_shape x q x d’dim candidate set to a batch_shapedim tensor of costs

forward
(X, deltas, **kwargs)[source]¶ Evaluate the cost function on the candidates and improvements.
 Parameters
X (
Tensor
) – A batch_shape x q x d’dim Tensor of with q ddim design points for each tbatch.deltas (
Tensor
) – A num_fantasies x batch_shapedim Tensor of num_fantasy samples from the marginal improvement in utility over the current state at X for each tbatch.
 Return type
Tensor
 Returns
A num_fantasies x batch_shapedim Tensor of costweighted utilities.

class
botorch.acquisition.cost_aware.
InverseCostWeightedUtility
(cost_model, use_mean=True, cost_objective=None, min_cost=0.01)[source]¶ Bases:
botorch.acquisition.cost_aware.CostAwareUtility
A costaware utility using inverse cost weighting based on a model.
Computes the costaware utility by inverseweighting samples U = (u_1, …, u_N) of the increase in utility. If use_mean=True, this uses the posterior mean mean_cost of the cost model, i.e. weighted utility = mean(U) / mean_cost. If use_mean=False, it uses samples C = (c_1, …, c_N) from the posterior of the cost model and performs the inverse weighting on the sample level: weighted utility = mean(u_1 / c_1, …, u_N / c_N).
The cost is additive across multiple elements of a qbatch.
Costaware utility that weights increase in utiltiy by inverse cost.
 Parameters
cost_model (
Model
) – A Model modeling the cost of evaluating a candidate set X, where X are the same features as in the model for the acquisition function this is to be used with. If no cost_objective is specified, the outputs are required to be nonnegative.use_mean (
bool
) – If True, use the posterior mean, otherwise use posterior samples from the cost model.cost_objective (
Optional
[MCAcquisitionObjective
]) – If specified, transform the posterior mean / the posterior samples from the cost model. This can be used e.g. to untransform predictions/samples of a cost model fit on the logtransformed cost (often done to ensure nonnegativity).min_cost (
float
) – A value used to clamp the cost samples so that they are not too close to zero, which may cause numerical issues.
 Returns
The inversecostweighted utiltiy.

forward
(X, deltas, sampler=None, **kwargs)[source]¶ Evaluate the cost function on the candidates and improvements.
 Parameters
X (
Tensor
) – A batch_shape x q x ddim Tensor of with q ddim design points each for each tbatch.deltas (
Tensor
) – A num_fantasies x batch_shapedim Tensor of num_fantasy samples from the marginal improvement in utility over the current state at X for each tbatch.sampler (
Optional
[MCSampler
]) – A sampler used for sampling from the posterior of the cost model (required if use_mean=False, ignored if use_mean=True).
 Return type
Tensor
 Returns
A num_fantasies x batch_shapedim Tensor of costweighted utilities.
Utilities¶
Fixed Feature Acquisition Function¶
A wrapper around AquisitionFunctions to fix certain features for optimization. This is useful e.g. for performing contextual optimization.

class
botorch.acquisition.fixed_feature.
FixedFeatureAcquisitionFunction
(acq_function, d, columns, values)[source]¶ Bases:
botorch.acquisition.acquisition.AcquisitionFunction
A wrapper around AquisitionFunctions to fix a subset of features.
Example
>>> model = SingleTaskGP(train_X, train_Y) # d = 5 >>> qEI = qExpectedImprovement(model, best_f=0.0) >>> columns = [2, 4] >>> values = X[..., columns] >>> qEI_FF = FixedFeatureAcquisitionFunction(qEI, 5, columns, values) >>> qei = qEI_FF(test_X) # d' = 3
Derived Acquisition Function by fixing a subset of input features.
 Parameters
acq_function (
AcquisitionFunction
) – The base acquisition function, operating on input tensors X_full of feature dimension d.d (
int
) – The feature dimension expected by acq_function.columns (
List
[int
]) – d_f < d indices of columns in X_full that are to be fixed to the provided values.values (
Union
[Tensor
,List
[float
]]) – The values to which to fix the columns in columns. Either a full batch_shape x q x d_f tensor of values (if values are different for each of the q input points), or an arraylike of values that is broadcastable to the input across tbatch and qbatch dimensions, e.g. a list of length d_f if values are the same across all t and qbatch dimensions.

forward
(X)[source]¶ Evaluate base acquisition function under the fixed features.
 Parameters
X (
Tensor
) – Input tensor of feature dimension d’ < d such that d’ + d_f = d. Returns
Base acquisition function evaluated on tensor X_full constructed by adding values in the appropriate places (see _construct_X_full).
General Utilities for Acquisition Functions¶
Utilities for acquisition functions.

botorch.acquisition.utils.
get_acquisition_function
(acquisition_function_name, model, objective, X_observed, X_pending=None, mc_samples=500, qmc=True, seed=None, **kwargs)[source]¶ Convenience function for initializing botorch acquisition functions.
 Parameters
acquisition_function_name (
str
) – Name of the acquisition function.model (
Model
) – A fitted model.objective (
MCAcquisitionObjective
) – A MCAcquisitionObjective.X_observed (
Tensor
) – A m1 x ddim Tensor of m1 design points that have already been observed.X_pending (
Optional
[Tensor
]) – A m2 x ddim Tensor of m2 design points whose evaluation is pending.mc_samples (
int
) – The number of samples to use for (q)MC evaluation of the acquisition function.qmc (
bool
) – If True, use quasiMonteCarlo sampling (instead of iid).seed (
Optional
[int
]) – If provided, perform deterministic optimization (i.e. the function to optimize is fixed and not stochastic).
 Return type
 Returns
The requested acquisition function.
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> obj = LinearMCObjective(weights=torch.tensor([1.0, 2.0])) >>> acqf = get_acquisition_function("qEI", model, obj, train_X)

botorch.acquisition.utils.
get_infeasible_cost
(X, model, objective=<function squeeze_last_dim>)[source]¶ Get infeasible cost for a model and objective.
 Computes an infeasible cost M such that M < min_x f(x) almost always,
so that feasible points are preferred.
 Parameters
X (
Tensor
) – A n x d Tensor of n design points to use in evaluating the minimum. These points should cover the design space well. The more points the better the estimate, at the expense of added computation.model (
Model
) – A fitted botorch model.objective (
Callable
[[Tensor
],Tensor
]) – The objective with which to evaluate the model output.
 Return type
float
 Returns
The infeasible cost M value.
Example
>>> model = SingleTaskGP(train_X, train_Y) >>> objective = lambda Y: Y[..., 1] ** 2 >>> M = get_infeasible_cost(train_X, model, obj)

botorch.acquisition.utils.
is_nonnegative
(acq_function)[source]¶ Determine whether a given acquisition function is nonnegative.
 Parameters
acq_function (
AcquisitionFunction
) – The AcquisitionFunction instance. Return type
bool
 Returns
True if acq_function is nonnegative, False if not, or if the behavior is unknown (for custom acquisition functions).
Example
>>> qEI = qExpectedImprovement(model, best_f=0.1) >>> is_nonnegative(qEI) # returns True

botorch.acquisition.utils.
prune_inferior_points
(model, X, objective=None, num_samples=2048, max_frac=1.0)[source]¶ Prune points from an input tensor that are unlikely to be the best point.
Given a model, an objective, and an input tensor X, this function returns the subset of points in X that have some probability of being the best point under the objective. This function uses sampling to estimate the probabilities, the higher the number of points n in X the higher the number of samples num_samples should be to obtain accurate estimates.
 Parameters
model (
Model
) – A fitted model. Batched models are currently not supported.X (
Tensor
) – An input tensor of shape n x d. Batched inputs are currently not supported.objective (
Optional
[MCAcquisitionObjective
]) – The objective under which to evaluate the posterior.num_samples (
int
) – The number of samples used to compute empirical probabilities of being the best point.max_frac (
float
) – The maximum fraction of points to retain. Must satisfy 0 < max_frac <= 1. Ensures that the number of elements in the returned tensor does not exceed ceil(max_frac * n).
 Return type
Tensor
 Returns
A n’ x d with subset of points in X, where
n’ = min(N_nz, ceil(max_frac * n))
with N_nz the number of points in X that have nonzero (empirical, under num_samples samples) probability of being the best point.

botorch.acquisition.utils.
project_to_target_fidelity
(X, target_fidelities=None)[source]¶ Project X onto the target set of fidelities.
This function assumes that the set of feasible fidelities is a box, so projecting here just means setting each fidelity parameter to its target value.
 Parameters
X (
Tensor
) – A batch_shape x q x ddim Tensor of with q ddim design points for each tbatch.target_fidelities (
Optional
[Dict
[int
,float
]]) – A dictionary mapping a subset of columns of X (the fidelity parameters) to their respective target fidelity value. If omitted, assumes that the last column of X is the fidelity parameter with a target value of 1.0.
 Return type
Tensor
 Returns
 A batch_shape x q x ddim Tensor X_proj with fidelity parameters
projected to the provided fidelity values.

botorch.acquisition.utils.
expand_trace_observations
(X, fidelity_dims=None, num_trace_obs=0)[source]¶ Expand X with trace observations.
Expand a tensor of inputs with “trace observations” that are obtained during the evaluation of the candidate set. This is used in multifidelity optimization. It can be though of as augmenting the qbatch with additional points that are the expected trace observations.
Let f_i be the ith fidelity parameter. Then this functions assumes that for each element of the qbatch, besides the fidelity f_i, we will observe additonal fidelities f_i1, …, f_iK, where K = num_trace_obs, during evaluation of the candidate set X. Specifically, this function assumes that f_ij = (Kj) / (num_trace_obs + 1) * f_i for all i. That is, the expansion is performed in parallel for all fidelities (it does not expand out all possible combinations).
 Parameters
X (
Tensor
) – A batch_shape x q x ddim Tensor of with q ddim design points (incl. the fidelity parameters) for each tbatch.fidelity_dims (
Optional
[List
[int
]]) – The indices of the fidelity parameters. If omitted, assumes that the last column of X contains the fidelity parameters.num_trace_obs (
int
) – The number of trace observations to use.
 Return type
Tensor
 Returns
 A batch_shape x (q + num_trace_obs x q) x d Tensor X_expanded that
expands X with trace observations.