Bayesian optimization results
A BayesianOptimization
object contains the
results of a Bayesian optimization. It is the output of bayesopt
or a fit function that accepts the
OptimizeHyperparameters
namevalue pair such as fitcdiscr
. In addition, a BayesianOptimization
object contains data for each iteration of
bayesopt
that can be accessed by a plot function or an output
function.
Create a BayesianOptimization
object using the
bayesopt
function or a fit function with
the OptimizeHyperparameters
namevalue pair.
ObjectiveFcn
— ObjectiveFcn
argument used by
bayesopt
This property is readonly.
ObjectiveFcn
argument used by
bayesopt
, returned as a function handle.
If you call bayesopt
directly,
ObjectiveFcn
is the
bayesopt
objective function
argument.
If you call a fit function containing the
'OptimizeHyperparameters'
namevalue pair
argument, ObjectiveFcn
is a function handle
that returns the misclassification rate for classification or
returns the logarithm of one plus the crossvalidation loss for
regression, measured by fivefold crossvalidation.
Data Types: function_handle
VariableDescriptions
— VariableDescriptions
argument that bayesopt
usedoptimizableVariable
objectsThis property is readonly.
VariableDescriptions
argument that
bayesopt
used, returned as a vector of optimizableVariable
objects.
If you called bayesopt
directly,
VariableDescriptions
is the
bayesopt
variable description
argument.
If you called a fit function with the
OptimizeHyperparameters
namevalue pair,
VariableDescriptions
is the vector of
hyperparameters.
Options
— Options that bayesopt
usedThis property is readonly.
Options that bayesopt
used, returned as a
structure.
If you called bayesopt
directly,
Options
is the options used in
bayesopt
, which are the namevalue
pairs See bayesopt
Input Arguments.
If you called a fit function with the
OptimizeHyperparameters
namevalue pair,
Options
are the default
bayesopt
options, modified by the
HyperparameterOptimizationOptions
namevalue pair.
Options
is a readonly structure containing the
following fields.
Option Name  Meaning 

AcquisitionFunctionName  Acquisition function name. See Acquisition Function Types. 
IsObjectiveDeterministic  true means the objective function
is deterministic, false
otherwise. 
ExplorationRatio  Used only when
AcquisitionFunctionName is
'expectedimprovementplus' or
'expectedimprovementpersecondplus' .
See Plus. 
MaxObjectiveEvaluations  Objective function evaluation limit. 
MaxTime  Time limit. 
XConstraintFcn  Deterministic constraints on variables. See Deterministic Constraints — XConstraintFcn. 
ConditionalVariableFcn  Conditional variable constraints. See Conditional Constraints — ConditionalVariableFcn. 
NumCoupledConstraints  Number of coupled constraints. See Coupled Constraints. 
CoupledConstraintTolerances  Coupled constraint tolerances. See Coupled Constraints. 
AreCoupledConstraintsDeterministic  Logical vector specifying whether each coupled constraint is deterministic. 
Verbose  Commandline display level. 
OutputFcn  Function called after each iteration. See Bayesian Optimization Output Functions. 
SaveVariableName  Variable name for the
@assignInBase output function.

SaveFileName  File name for the @saveToFile
output function. 
PlotFcn  Plot function called after each iteration. See Bayesian Optimization Plot Functions 
InitialX  Points where bayesopt evaluated
the objective function. 
InitialObjective  Objective function values at
InitialX . 
InitialConstraintViolations  Coupled constraint function values at
InitialX . 
InitialErrorValues  Error values at InitialX . 
InitialObjectiveEvaluationTimes  Objective function evaluation times at
InitialX . 
InitialIterationTimes  Time for each iteration, including objective function evaluation and other computations. 
Data Types: struct
MinObjective
— Minimum observed value of objective functionThis property is readonly.
Minimum observed value of objective function, returned as a real scalar. When there are coupled constraints or evaluation errors, this value is the minimum over all observed points that are feasible according to the final constraint and Error models.
Data Types: double
XAtMinObjective
— Observed point with minimum objective function value1
byD
tableThis property is readonly.
Observed point with minimum objective function value, returned as a
1
byD
table, where
D
is the number of variables.
Data Types: table
MinEstimatedObjective
— Minimum estimated value of objective functionThis property is readonly.
Minimum estimated value of objective function, returned as a real
scalar. MinEstimatedObjective
uses the final
objective model.
MinEstimatedObjective
is the same as the CriterionValue
result of bestPoint
with default
criterion.
Data Types: double
XAtMinEstimatedObjective
— Point with minimum estimated objective function value1
byD
tableThis property is readonly.
Point with minimum estimated objective function value, returned as a
1
byD
table, where
D
is the number of variables.
XAtMinEstimatedObjective
uses the final objective
model.
Data Types: table
NumObjectiveEvaluations
— Number of objective function evaluationsThis property is readonly.
Number of objective function evaluations, returned as a positive integer. This includes the initial evaluations to form a posterior model as well as evaluation during the optimization iterations.
Data Types: double
TotalElapsedTime
— Total elapsed time of optimization in secondsThis property is readonly.
Total elapsed time of optimization in seconds, returned as a positive scalar.
Data Types: double
NextPoint
— Next point to evaluate if optimization continues1
byD
tableThis property is readonly.
Next point to evaluate if optimization continues, returned as a
1
byD
table, where
D
is the number of variables.
Data Types: table
XTrace
— Points where the objective function was evaluatedT
byD
tableThis property is readonly.
Points where the objective function was evaluated, returned as a
T
byD
table, where
T
is the number of evaluation points and
D
is the number of variables.
Data Types: table
ObjectiveTrace
— Objective function valuesT
This property is readonly.
Objective function values, returned as a column vector of length
T
, where T
is the number of
evaluation points. ObjectiveTrace
contains the
history of objective function evaluations.
Data Types: double
ObjectiveEvaluationTimeTrace
— Objective function evaluation timesT
This property is readonly.
Objective function evaluation times, returned as a column vector of
length T
, where T
is the number of
evaluation points. ObjectiveEvaluationTimeTrace
includes the time in evaluating coupled constraints, because the
objective function computes these constraints.
Data Types: double
IterationTimeTrace
— Iteration timesT
This property is readonly.
Iteration times, returned as a column vector of length
T
, where T
is the number of
evaluation points. IterationTimeTrace
includes both
objective function evaluation time and other overhead.
Data Types: double
ConstraintsTrace
— Coupled constraint valuesT
byK
arrayThis property is readonly.
Coupled constraint values, returned as a
T
byK
array, where
T
is the number of evaluation points and
K
is the number of coupled constraints.
Data Types: double
ErrorTrace
— Error indicationsT
of 1
or 1
entriesThis property is readonly.
Error indications, returned as a column vector of length
T
of 1
or
1
entries, where T
is the
number of evaluation points. Each 1
entry indicates
that the objective function errored or returned NaN
on the corresponding point in XTrace
. Each
1
entry indicates that the objective function
value was computed.
Data Types: double
FeasibilityTrace
— Feasibility indicationsT
This property is readonly.
Feasibility indications, returned as a logical column vector of length
T
, where T
is the number of
evaluation points. Each 1
entry indicates that the
final constraint model predicts feasibility at the corresponding point
in XTrace
.
Data Types: logical
FeasibilityProbabilityTrace
— Probability that evaluation point is feasibleT
This property is readonly.
Probability that evaluation point is feasible, returned as a column
vector of length T
, where T
is the
number of evaluation points. The probabilities come from the final
constraint model, including the error constraint model, on the
corresponding points in XTrace
.
Data Types: double
IndexOfMinimumTrace
— Which evaluation gave minimum feasible objectiveT
This property is readonly.
Which evaluation gave minimum feasible objective, returned as a column
vector of integer indices of length T
, where
T
is the number of evaluation points. Feasibility
is determined with respect to the constraint models that existed at each
iteration, including the error constraint model.
Data Types: double
ObjectiveMinimumTrace
— Minimum observed objectiveT
This property is readonly.
Minimum observed objective, returned as a column vector of integer
indices of length T
, where T
is
the number of evaluation points.
Data Types: double
EstimatedObjectiveMinimumTrace
— Minimum estimated objectiveT
This property is readonly.
Minimum estimated objective, returned as a column vector of integer
indices of length T
, where T
is
the number of evaluation points. The estimated objective at each
iteration is determined with respect to the objective model that existed
at that iteration.
Data Types: double
UserDataTrace
— Auxiliary data from the objective functionT
This property is readonly.
Auxiliary data from the objective function, returned as a cell array
of length T
, where T
is the number
of evaluation points. Each entry in the cell array is the
UserData
returned in the third output of the
objective function.
Data Types: cell
bestPoint  Best point in a Bayesian optimization according to a criterion 
plot  Plot Bayesian optimization results 
predictConstraints  Predict coupled constraint violations at a set of points 
predictError  Predict error value at a set of points 
predictObjective  Predict objective function at a set of points 
predictObjectiveEvaluationTime  Predict objective function run times at a set of points 
resume  Resume a Bayesian optimization 
BayesianOptimization
Object Using bayesopt
This example shows how to create a BayesianOptimization
object by using bayesopt
to minimize crossvalidation loss.
Optimize hyperparameters of a KNN classifier for the ionosphere
data, that is, find KNN hyperparameters that minimize the crossvalidation loss. Have bayesopt
minimize over the following hyperparameters:
Nearestneighborhood sizes from 1 to 30
Distance functions 'chebychev'
, 'euclidean'
, and 'minkowski'
.
For reproducibility, set the random seed, set the partition, and set the AcquisitionFunctionName
option to 'expectedimprovementplus'
. To suppress iterative display, set 'Verbose'
to 0
. Pass the partition c
and fitting data X
and Y
to the objective function fun
by creating fun
as an anonymous function that incorporates this data. See Parameterizing Functions (MATLAB).
load ionosphere rng default num = optimizableVariable('n',[1,30],'Type','integer'); dst = optimizableVariable('dst',{'chebychev','euclidean','minkowski'},'Type','categorical'); c = cvpartition(351,'Kfold',5); fun = @(x)kfoldLoss(fitcknn(X,Y,'CVPartition',c,'NumNeighbors',x.n,... 'Distance',char(x.dst),'NSMethod','exhaustive')); results = bayesopt(fun,[num,dst],'Verbose',0,... 'AcquisitionFunctionName','expectedimprovementplus')
results = BayesianOptimization with properties: ObjectiveFcn: [function_handle] VariableDescriptions: [1x2 optimizableVariable] Options: [1x1 struct] MinObjective: 0.1197 XAtMinObjective: [1x2 table] MinEstimatedObjective: 0.1213 XAtMinEstimatedObjective: [1x2 table] NumObjectiveEvaluations: 30 TotalElapsedTime: 60.8003 NextPoint: [1x2 table] XTrace: [30x2 table] ObjectiveTrace: [30x1 double] ConstraintsTrace: [] UserDataTrace: {30x1 cell} ObjectiveEvaluationTimeTrace: [30x1 double] IterationTimeTrace: [30x1 double] ErrorTrace: [30x1 double] FeasibilityTrace: [30x1 logical] FeasibilityProbabilityTrace: [30x1 double] IndexOfMinimumTrace: [30x1 double] ObjectiveMinimumTrace: [30x1 double] EstimatedObjectiveMinimumTrace: [30x1 double]
BayesianOptimization
Object Using a Fit FunctionThis example shows how to minimize the crossvalidation loss in the ionosphere
data using Bayesian optimization of an SVM classifier.
Load the data.
load ionosphere
Optimize the classification using the 'auto'
parameters.
rng default % For reproducibility Mdl = fitcsvm(X,Y,'OptimizeHyperparameters','auto')
=====================================================================================================  Iter  Eval  Objective  Objective  BestSoFar  BestSoFar  BoxConstraint KernelScale    result   runtime  (observed)  (estim.)    =====================================================================================================  1  Best  0.21652  17.227  0.21652  0.21652  64.836  0.0015729 
 2  Accept  0.35897  0.076634  0.21652  0.22539  0.036335  5.5755 
 3  Best  0.13105  6.789  0.13105  0.14152  0.0022147  0.0023957 
 4  Accept  0.35897  0.076591  0.13105  0.13108  5.1259  98.62 
 5  Best  0.12251  0.083261  0.12251  0.12253  0.0010264  0.042908 
 6  Accept  0.12536  0.11182  0.12251  0.12242  0.0010276  0.01796 
 7  Accept  0.13105  1.3745  0.12251  0.12363  0.04548  0.028371 
 8  Accept  0.12821  0.092434  0.12251  0.12489  0.001004  0.030765 
 9  Accept  0.1339  0.078771  0.12251  0.12489  0.0010115  0.074186 
 10  Accept  0.12536  0.14329  0.12251  0.12493  0.0010018  0.012976 
 11  Accept  0.12821  0.097635  0.12251  0.12596  0.001089  0.029027 
 12  Accept  0.13675  0.27617  0.12251  0.12541  0.0010255  0.0089521 
 13  Accept  0.1339  0.088394  0.12251  0.12704  0.0010116  0.035171 
 14  Accept  0.35897  0.078646  0.12251  0.12704  0.0013423  76.712 
 15  Accept  0.35897  0.075086  0.12251  0.127  0.13735  993.22 
 16  Accept  0.35897  0.075732  0.12251  0.12698  994.11  990.15 
 17  Accept  0.35897  0.093507  0.12251  0.12763  0.0010027  0.85485 
 18  Accept  0.12536  0.083076  0.12251  0.12657  0.0010101  0.048845 
 19  Accept  0.12821  0.083382  0.12251  0.12701  0.0010938  0.047844 
 20  Accept  0.12536  0.11448  0.12251  0.12556  0.0010146  0.017572 
=====================================================================================================  Iter  Eval  Objective  Objective  BestSoFar  BestSoFar  BoxConstraint KernelScale    result   runtime  (observed)  (estim.)    =====================================================================================================  21  Accept  0.12251  0.11771  0.12251  0.12461  0.0010166  0.018502 
 22  Accept  0.12251  0.12403  0.12251  0.12405  0.001065  0.018515 
 23  Accept  0.35897  0.079806  0.12251  0.12416  0.03993  21.731 
 24  Accept  0.35897  0.075101  0.12251  0.12429  2.3338  336.56 
 25  Accept  0.12536  0.17006  0.12251  0.12423  992.83  14.69 
 26  Accept  0.13675  0.30995  0.12251  0.12421  987.13  5.8314 
 27  Accept  0.13105  0.19598  0.12251  0.12442  0.0029454  0.017583 
 28  Accept  0.12251  0.1239  0.12251  0.12445  0.0062605  0.059597 
 29  Accept  0.12821  0.14179  0.12251  0.12451  0.022617  0.065617 
 30  Accept  0.13105  0.16587  0.12251  0.12458  995.38  10.651 
__________________________________________________________ Optimization completed. MaxObjectiveEvaluations of 30 reached. Total function evaluations: 30 Total elapsed time: 55.6497 seconds. Total objective function evaluation time: 28.6241 Best observed feasible point: BoxConstraint KernelScale _____________ ___________ 0.0010264 0.042908 Observed objective function value = 0.12251 Estimated objective function value = 0.12703 Function evaluation time = 0.083261 Best estimated feasible point (according to models): BoxConstraint KernelScale _____________ ___________ 0.0010146 0.017572 Estimated objective function value = 0.12458 Estimated function evaluation time = 0.11721
Mdl = ClassificationSVM ResponseName: 'Y' CategoricalPredictors: [] ClassNames: {'b' 'g'} ScoreTransform: 'none' NumObservations: 351 HyperparameterOptimizationResults: [1×1 BayesianOptimization] Alpha: [90×1 double] Bias: 5.7329 KernelParameters: [1×1 struct] BoxConstraints: [351×1 double] ConvergenceInfo: [1×1 struct] IsSupportVector: [351×1 logical] Solver: 'SMO' Properties, Methods
The fit achieved about 12% loss for the default 5fold cross validation.
Examine the BayesianOptimization
object that is returned in the HyperparameterOptimizationResults
property of the returned model.
disp(Mdl.HyperparameterOptimizationResults)
BayesianOptimization with properties: ObjectiveFcn: @createObjFcn/inMemoryObjFcn VariableDescriptions: [5×1 optimizableVariable] Options: [1×1 struct] MinObjective: 0.1225 XAtMinObjective: [1×2 table] MinEstimatedObjective: 0.1246 XAtMinEstimatedObjective: [1×2 table] NumObjectiveEvaluations: 30 TotalElapsedTime: 55.6497 NextPoint: [1×2 table] XTrace: [30×2 table] ObjectiveTrace: [30×1 double] ConstraintsTrace: [] UserDataTrace: {30×1 cell} ObjectiveEvaluationTimeTrace: [30×1 double] IterationTimeTrace: [30×1 double] ErrorTrace: [30×1 double] FeasibilityTrace: [30×1 logical] FeasibilityProbabilityTrace: [30×1 double] IndexOfMinimumTrace: [30×1 double] ObjectiveMinimumTrace: [30×1 double] EstimatedObjectiveMinimumTrace: [30×1 double]
bayesopt
 fitcdiscr
 fitcecoc
 fitcensemble
 fitcknn
 fitclinear
 fitcnb
 fitcsvm
 fitctree
 fitrensemble
 fitrgp
 fitrlinear
 fitrsvm
 fitrtree
A modified version of this example exists on your system. Do you want to open this version instead?
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
Select web siteYou can also select a web site from the following list:
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.