BayesianOptimization

Bayesian optimization results

Description

A BayesianOptimization object is the output of the bayesopt function and contains the results of a Bayesian optimization. In addition, the object contains data for each iteration of bayesopt that can be accessed by a plot function or an output function.

Creation

Create a BayesianOptimization object by using the bayesopt function. For more information, see Ways to Perform Bayesian Optimization.

Properties

Problem Definition Properties

ObjectiveFcn — ObjectiveFcn argument used by bayesopt

Read-only: function handle

This property is read-only.

ObjectiveFcn argument used by bayesopt, returned as a function handle. When you call bayesopt directly, ObjectiveFcn is the bayesopt objective function argument.

Data Types: function_handle

VariableDescriptions — VariableDescriptions argument that bayesopt used

Read-only: vector of optimizableVariable objects

This property is read-only.

VariableDescriptions argument that bayesopt used, returned as a vector of optimizableVariable objects. When you call bayesopt directly, VariableDescriptions is the bayesopt variable description argument.

Options — Options used by bayesopt

Read-only: structure

This property is read-only.

Options used by bayesopt, returned as a structure. When you call bayesopt directly, Options is the set of options used in bayesopt, which correspond to the name-value arguments. See Input Arguments of bayesopt.

Options contains the following fields.

Option Name	Meaning
AcquisitionFunctionName	Acquisition function name (with dashes removed). See Acquisition Function Types.
IsObjectiveDeterministic	true means the objective function is deterministic, false otherwise.
ExplorationRatio	Used only when AcquisitionFunctionName is 'expectedimprovementplus' or 'expectedimprovement-persecondplus'. See Plus.
MaxObjectiveEvaluations	Objective function evaluation limit.
MaxTime	Time limit.
XConstraintFcn	Deterministic constraints on variables. See Deterministic Constraints — XConstraintFcn.
ConditionalVariableFcn	Conditional variable constraints. See Conditional Constraints — ConditionalVariableFcn.
NumCoupledConstraints	Number of coupled constraints. See Coupled Constraints.
CoupledConstraintTolerances	Coupled constraint tolerances. See Coupled Constraints.
AreCoupledConstraintsDeterministic	Logical vector specifying whether each coupled constraint is deterministic.
Verbose	Command-line display level.
OutputFcn	Function called after each iteration. See Bayesian Optimization Output Functions.
SaveVariableName	Variable name for the @assignInBase output function.
SaveFileName	File name for the @saveToFile output function.
PlotFcn	Plot function called after each iteration. See Bayesian Optimization Plot Functions
InitialX	Points where bayesopt evaluated the objective function.
InitialObjective	Objective function values at InitialX.
InitialConstraintViolations	Coupled constraint function values at InitialX.
InitialErrorValues	Error values at InitialX.
InitialObjectiveEvaluationTimes	Objective function evaluation times at InitialX.
InitialIterationTimes	Time for each iteration, including objective function evaluation and other computations.

Data Types: struct

Solution Properties

MinObjective — Minimum observed value of objective function

Read-only: real scalar

This property is read-only.

Minimum observed value of the objective function, returned as a real scalar. When the optimization includes coupled constraints or evaluation errors, this value is the minimum over all observed points that are feasible according to the final constraint and error models.

Data Types: double

XAtMinObjective — Observed point with minimum objective function value

Read-only: 1-by-D table

This property is read-only.

Observed point with the minimum objective function value, returned as a 1-by-D table, where D is the number of variables.

Data Types: table

MinEstimatedObjective — Estimated objective function value

Read-only: real scalar

This property is read-only.

Estimated objective function value at XAtMinEstimatedObjective, returned as a real scalar.

MinEstimatedObjective is the mean value of the posterior distribution of the final objective model. The software estimates the MinEstimatedObjective value by passing XAtMinEstimatedObjective to the object function predictObjective.

Data Types: double

XAtMinEstimatedObjective — Point with minimum upper confidence bound of objective function value

Read-only: 1-by-D table

This property is read-only.

Point with the minimum upper confidence bound of the objective function value among the visited points, returned as a 1-by-D table, where D is the number of variables. The software uses the final objective model to find the upper confidence bounds of the visited points.

XAtMinEstimatedObjective is the same as the best point returned by the bestPoint function with the default criterion ('min-visited-upper-confidence-interval').

Data Types: table

NumObjectiveEvaluations — Number of objective function evaluations

Read-only: positive integer

This property is read-only.

Number of objective function evaluations, returned as a positive integer. This number includes the initial evaluations used to form a posterior model, as well as evaluations during the optimization iterations.

Data Types: double

TotalElapsedTime — Total elapsed time of optimization in seconds

Read-only: positive scalar

This property is read-only.

Total elapsed time of the optimization in seconds, returned as a positive scalar.

Data Types: double

NextPoint — Next point to evaluate

Read-only: 1-by-D table

This property is read-only.

Next point to evaluate if the optimization continues, returned as a 1-by-D table, where D is the number of variables.

Data Types: table

Trace Properties

XTrace — Points at which objective function is evaluated

Read-only: T-by-D table

This property is read-only.

Points at which the objective function is evaluated, returned as a T-by-D table, where T is the number of evaluation points and D is the number of variables.

Data Types: table

ObjectiveTrace — Objective function values

Read-only: column vector of length T

This property is read-only.

Objective function values, returned as a column vector of length T, where T is the number of evaluation points. ObjectiveTrace contains the history of the objective function evaluations.

Data Types: double

ObjectiveEvaluationTimeTrace — Objective function evaluation times

Read-only: column vector of length T

This property is read-only.

Objective function evaluation times, returned as a column vector of length T, where T is the number of evaluation points. ObjectiveEvaluationTimeTrace includes the time required to evaluate coupled constraints, because the objective function computes these constraints.

Data Types: double

IterationTimeTrace — Iteration times

Read-only: column vector of length T

This property is read-only.

Iteration times, returned as a column vector of length T, where T is the number of evaluation points. IterationTimeTrace includes the objective function evaluation time and other overhead.

Data Types: double

ConstraintsTrace — Coupled constraint values

Read-only: T-by-K array

This property is read-only.

Coupled constraint values, returned as a T-by-K array, where T is the number of evaluation points and K is the number of coupled constraints.

Data Types: double

ErrorTrace — Error indications

Read-only: column vector of length T containing -1 or 1 entries

This property is read-only.

Error indications, returned as a column vector of length T containing -1 or 1 entries, where T is the number of evaluation points. Each 1 entry indicates that the objective function issues an error or returns NaN on the corresponding point in XTrace. Each -1 entry indicates that the software computes the objective function value.

Data Types: double

FeasibilityTrace — Feasibility indications

Read-only: logical column vector of length T

This property is read-only.

Feasibility indications, returned as a logical column vector of length T, where T is the number of evaluation points. Each 1 entry indicates that the final constraint model predicts feasibility at the corresponding point in XTrace.

Data Types: logical

FeasibilityProbabilityTrace — Probability that evaluation point is feasible

Read-only: column vector of length T

This property is read-only.

Probability that an evaluation point is feasible, returned as a column vector of length T, where T is the number of evaluation points. The probabilities come from the final constraint model, including the error constraint model, on the corresponding points in XTrace.

Data Types: double

IndexOfMinimumTrace — Evaluation that gives minimum feasible objective

Read-only: column vector of integer indices of length T

This property is read-only.

Evaluation that gives the minimum feasible objective, returned as a column vector of integer indices of length T, where T is the number of evaluation points. Feasibility is determined with respect to the constraint models at each iteration, including the error constraint model.

Data Types: double

ObjectiveMinimumTrace — Minimum observed objective

Read-only: column vector of length T

This property is read-only.

Minimum observed objective, returned as a column vector of length T, where T is the number of evaluation points.

Data Types: double

EstimatedObjectiveMinimumTrace — Estimated objective

Read-only: column vector of length T

This property is read-only.

Estimated objective, returned as a column vector of length T, where T is the number of evaluation points. The estimated objective at each iteration is determined with respect to the objective model at that iteration. At each iteration, the software uses the object function predictObjective to estimate the objective function value at the point with the minimum upper confidence bound of the objective function among the visited points.

Data Types: double

UserDataTrace — Auxiliary data from objective function

Read-only: cell array of length T

This property is read-only.

Auxiliary data from the objective function, returned as a cell array of length T, where T is the number of evaluation points. Each entry in the cell array is the UserData value returned in the third output of the objective function.

Data Types: cell

Object Functions

bestPoint	Best point in a Bayesian optimization according to a criterion
plot	Plot Bayesian optimization results
predictConstraints	Predict coupled constraint violations at a set of points
predictError	Predict error value at a set of points
predictObjective	Predict objective function at a set of points
predictObjectiveEvaluationTime	Predict objective function run times at a set of points
resume	Resume a Bayesian optimization

Examples

Create a BayesianOptimization Object Using bayesopt

This example shows how to create a BayesianOptimization object by using bayesopt to minimize cross-validation loss.

Optimize hyperparameters of a KNN classifier for the ionosphere data, that is, find KNN hyperparameters that minimize the cross-validation loss. Have bayesopt minimize over the following hyperparameters:

• Nearest-neighborhood sizes from 1 to 30

• Distance functions 'chebychev', 'euclidean', and 'minkowski'.

For reproducibility, set the random seed, set the partition, and set the AcquisitionFunctionName option to 'expected-improvement-plus'. To suppress iterative display, set 'Verbose' to 0. Pass the partition c and fitting data X and Y to the objective function fun by creating fun as an anonymous function that incorporates this data. See Parameterizing Functions.

load ionosphere
rng default
num = optimizableVariable('n',[1,30],'Type','integer');
dst = optimizableVariable('dst',{'chebychev','euclidean','minkowski'},'Type','categorical');
c = cvpartition(351,'Kfold',5);
fun = @(x)kfoldLoss(fitcknn(X,Y,'CVPartition',c,'NumNeighbors',x.n,...
    'Distance',char(x.dst),'NSMethod','exhaustive'));
results = bayesopt(fun,[num,dst],'Verbose',0,...
    'AcquisitionFunctionName','expected-improvement-plus')

results = 
  BayesianOptimization with properties:

                      ObjectiveFcn: @(x)kfoldLoss(fitcknn(X,Y,'CVPartition',c,'NumNeighbors',x.n,'Distance',char(x.dst),'NSMethod','exhaustive'))
              VariableDescriptions: [1×2 optimizableVariable]
                           Options: [1×1 struct]
                      MinObjective: 0.1197
                   XAtMinObjective: [1×2 table]
             MinEstimatedObjective: 0.1213
          XAtMinEstimatedObjective: [1×2 table]
           NumObjectiveEvaluations: 30
                  TotalElapsedTime: 30.3867
                         NextPoint: [1×2 table]
                            XTrace: [30×2 table]
                    ObjectiveTrace: [30×1 double]
                  ConstraintsTrace: []
                     UserDataTrace: {30×1 cell}
      ObjectiveEvaluationTimeTrace: [30×1 double]
                IterationTimeTrace: [30×1 double]
                        ErrorTrace: [30×1 double]
                  FeasibilityTrace: [30×1 logical]
       FeasibilityProbabilityTrace: [30×1 double]
               IndexOfMinimumTrace: [30×1 double]
             ObjectiveMinimumTrace: [30×1 double]
    EstimatedObjectiveMinimumTrace: [30×1 double]

Version History

Introduced in R2016b