botorch.generation¶
Candidate Generation Utilities for Acquisition Functions¶
Candidate generation utilities.

botorch.generation.gen.
gen_candidates_scipy
(initial_conditions, acquisition_function, lower_bounds=None, upper_bounds=None, inequality_constraints=None, equality_constraints=None, options=None, fixed_features=None)[source]¶ Generate a set of candidates using scipy.optimize.minimize.
Optimizes an acquisition function starting from a set of initial candidates using scipy.optimize.minimize via a numpy converter.
 Parameters
initial_conditions (
Tensor
) – Starting points for optimization.acquisition_function (
Module
) – Acquisition function to be used.lower_bounds (
Union
[float
,Tensor
,None
]) – Minimum values for each column of initial_conditions.upper_bounds (
Union
[float
,Tensor
,None
]) – Maximum values for each column of initial_conditions.constraints (equality) – A list of tuples (indices, coefficients, rhs), with each tuple encoding an inequality constraint of the form sum_i (X[indices[i]] * coefficients[i]) >= rhs.
constraints – A list of tuples (indices, coefficients, rhs), with each tuple encoding an inequality constraint of the form sum_i (X[indices[i]] * coefficients[i]) = rhs.
options (
Optional
[Dict
[str
,Any
]]) – Options used to control the optimization including “method” and “maxiter”. Select method for scipy.minimize using the method” key. By default uses LBFGSB for boxconstrained problems and SLSQP if inequality or equality constraints are present.fixed_features (
Optional
[Dict
[int
,Optional
[float
]]]) – This is a dictionary of feature indices to values, where all generated candidates will have features fixed to these values. If the dictionary value is None, then that feature will just be fixed to the clamped value and not optimized. Assumes values to be compatible with lower_bounds and upper_bounds!
 Return type
Tuple
[Tensor
,Tensor
] Returns
2element tuple containing
The set of generated candidates.
The acquisition value for each tbatch.
Example
>>> qEI = qExpectedImprovement(model, best_f=0.2) >>> bounds = torch.tensor([[0., 0.], [1., 2.]]) >>> Xinit = gen_batch_initial_conditions( >>> qEI, bounds, q=3, num_restarts=25, raw_samples=500 >>> ) >>> batch_candidates, batch_acq_values = gen_candidates_scipy( initial_conditions=Xinit, acquisition_function=qEI, lower_bounds=bounds[0], upper_bounds=bounds[1], )

botorch.generation.gen.
gen_candidates_torch
(initial_conditions, acquisition_function, lower_bounds=None, upper_bounds=None, optimizer=<class 'torch.optim.adam.Adam'>, options=None, verbose=True, fixed_features=None)[source]¶ Generate a set of candidates using a torch.optim optimizer.
Optimizes an acquisition function starting from a set of initial candidates using an optimizer from torch.optim.
 Parameters
initial_conditions (
Tensor
) – Starting points for optimization.acquisition_function (
Callable
) – Acquisition function to be used.lower_bounds (
Union
[float
,Tensor
,None
]) – Minimum values for each column of initial_conditions.upper_bounds (
Union
[float
,Tensor
,None
]) – Maximum values for each column of initial_conditions.optimizer (Optimizer) – The pytorch optimizer to use to perform candidate search.
options (
Optional
[Dict
[str
,Union
[float
,str
]]]) – Options used to control the optimization. Includes maxiter: Maximum number of iterationsverbose (
bool
) – If True, provide verbose output.fixed_features (
Optional
[Dict
[int
,Optional
[float
]]]) – This is a dictionary of feature indices to values, where all generated candidates will have features fixed to these values. If the dictionary value is None, then that feature will just be fixed to the clamped value and not optimized. Assumes values to be compatible with lower_bounds and upper_bounds!
 Return type
Tuple
[Tensor
,Tensor
] Returns
2element tuple containing
The set of generated candidates.
The acquisition value for each tbatch.
Example
>>> qEI = qExpectedImprovement(model, best_f=0.2) >>> bounds = torch.tensor([[0., 0.], [1., 2.]]) >>> Xinit = gen_batch_initial_conditions( >>> qEI, bounds, q=3, num_restarts=25, raw_samples=500 >>> ) >>> batch_candidates, batch_acq_values = gen_candidates_torch( initial_conditions=Xinit, acquisition_function=qEI, lower_bounds=bounds[0], upper_bounds=bounds[1], )

botorch.generation.gen.
get_best_candidates
(batch_candidates, batch_values)[source]¶ Extract best (qbatch) candidate from batch of candidates
 Parameters
batch_candidates (
Tensor
) – A b x q x d tensor of b qbatch candidates, or a b x d tensor of b singlepoint candidates.batch_values (
Tensor
) – A tensor with b elements containing the value of the respective candidate (higher is better).
 Return type
Tensor
 Returns
A tensor of size q x d (if qbatch mode) or d from batch_candidates with the highest associated value.
Example
>>> qEI = qExpectedImprovement(model, best_f=0.2) >>> bounds = torch.tensor([[0., 0.], [1., 2.]]) >>> Xinit = gen_batch_initial_conditions( >>> qEI, bounds, q=3, num_restarts=25, raw_samples=500 >>> ) >>> batch_candidates, batch_acq_values = gen_candidates_scipy( initial_conditions=Xinit, acquisition_function=qEI, lower_bounds=bounds[0], upper_bounds=bounds[1], ) >>> best_candidates = get_best_candidates(batch_candidates, batch_acq_values)
Sampling Strategies¶
Samplingbased generation strategies.
A SamplingStrategy returns samples from the input points (i.e. Tensors in feature space), rather than the value for a set of tensors, as acquisition functions do. The qbatch dimension has similar semantics as for acquisition functions in that the points across the qbatch are considered jointly for sampling (where as for qacquisition functions we evaluate the joint value of the qbatch).

class
botorch.generation.sampling.
SamplingStrategy
[source]¶ Bases:
torch.nn.modules.module.Module
,abc.ABC
Abstract base class for samplingbased generation strategies.
Initializes internal Module state, shared by both nn.Module and ScriptModule.

abstract
forward
(X, num_samples=1, **kwargs)[source]¶ Sample according to the SamplingStrategy.
 Parameters
X (
Tensor
) – A batch_shape x N x ddim Tensor from which to sample (in the N dimension).num_samples (
int
) – The number of samples to draw.kwargs (
Any
) – Additional implementationspecific kwargs.
 Return type
Tensor
 Returns
A batch_shape x num_samples x ddim Tensor of samples from X, where X[…, i, :] is the ith sample.

abstract

class
botorch.generation.sampling.
MaxPosteriorSampling
(model, objective=None, replacement=True)[source]¶ Bases:
botorch.generation.sampling.SamplingStrategy
Sample from a set of points according to their max posterior value.
Example
>>> MPS = MaxPosteriorSampling(model) # model w/ feature dim d=3 >>> X = torch.rand(2, 100, 3) >>> sampled_X = MPS(X, n=5)
Constructor for the SamplingStrategy base class.
 Parameters
model (
Model
) – A fitted model.objective (
Optional
[AcquisitionObjective
]) – The objective. Typically, the AcquisitionObjective under which the samples are evaluated. If a ScalarizedObjective, samples from the scalarized posterior are used. Defaults to IdentityMCObjective().replacement (
bool
) – If True, sample with replacement.

forward
(X, num_samples=1, observation_noise=False)[source]¶ Sample from the model posterior.
 Parameters
X (
Tensor
) – A batch_shape x N x ddim Tensor from which to sample (in the N dimension) according to the maximum posterior value under the objective.num_samples (
int
) – The number of samples to draw.observation_noise (
bool
) – If True, sample with observation noise.
 Return type
Tensor
 Returns
A batch_shape x num_samples x ddim Tensor of samples from X, where X[…, i, :] is the ith sample.

class
botorch.generation.sampling.
BoltzmannSampling
(acq_func, eta=1.0, replacement=True)[source]¶ Bases:
botorch.generation.sampling.SamplingStrategy
Sample from a set of points according to a tempered acquisition value.
Given an acquisition function acq_func, this sampling strategies draws samples from a batch_shape x N x ddim tensor X according to a multinomial distribution over its indices given by
weight(X[…, i, :]) ~ exp(eta * standardize(acq_func(X[…, i, :])))
where standardize(Y) standardizes Y to zero mean and unit variance. As the temperature parameter eta > 0, this approaches uniform sampling, while as eta > infty, this approaches selecting the maximizer(s) of the acquisition function acq_func.
Example
>>> UCB = UpperConfidenceBound(model, beta=0.1) >>> BMUCB = BoltzmannSampling(UCB, eta=0.5) >>> X = torch.rand(2, 100, 3) >>> sampled_X = BMUCB(X, n=5)
Boltzmann Acquisition Value Sampling.
 Parameters
acq_func (
AcquisitionFunction
) – The acquisition function; to be evaluated in batch at the individual points of a qbatch (not jointly, as is the case for acquisition functions). Can be analytic or MonteCarlo.eta (
float
) – The temperature parameter in the softmax.replacement (
bool
) – If True, sample with replacement.

forward
(X, num_samples=1)[source]¶ Sample from a tempered value of the acquisition function value.
 Parameters
X (
Tensor
) – A batch_shape x N x ddim Tensor from which to sample (in the N dimension) according to the maximum posterior value under the objective. Note that if a batched model is used in the underlying acquisition function, then its batch shape must be broadcastable to batch_shape.num_samples (
int
) – The number of samples to draw.
 Return type
Tensor
 Returns
A batch_shape x num_samples x ddim Tensor of samples from X, where X[…, i, :] is the ith sample.