# Stopping Criteria in bayesopt for Early Termination

22 views (last 30 days)
Commented: Muhammad on 29 Apr 2024 at 11:02
I'm working with bayesopt for Bayesian optimization in MATLAB. I want to implement a stopping criterion that terminates the optimization process when a specific condition is met like minimum tolerence level 1e-3, allowing for early termination if improvement becomes insignificant. Here's the relevant portion of my bayesopt command:
results = bayesopt(@optimizeParameters, parRange, ...
'IsObjectiveDeterministic', false, ...
'AcquisitionFunctionName', 'expected-improvement-plus', ...
'Verbose', 0, ...
'ExplorationRatio', 0.6, ...
'MaxObjectiveEvaluations', 100);
Could you provide guidance on how to best achieve this using bayesopt.

Tejas on 25 Apr 2024
It seems you want a method to halt Bayesian optimization prematurely when a certain parameter value falls beneath a specified tolerance level.
This objective can be achieved by crafting a custom function designed to halt the optimization process once a specific condition is satisfied. This function is then invoked using the OutputFcn property of bayesopt. The OutputFcn triggers this custom function at the end of each iteration.
I have created a custom function named terminationFunc to demonstrate this approach. Here is how it operates:
• The function receives the results and state of the current iteration as input.
• It monitors the ‘ObjectiveMinimumTrace’ parameter across each iteration.
• Optimization is halted early if this parameter's value drops below the tolerance threshold of 1e-3.
function stop = terminationFunc(results, state)
persistent bestObjective
stop = false;
if strcmp(state, 'iteration')
if isempty(bestObjective) % True for first iteration
bestObjective = results.ObjectiveMinimumTrace(end);
fprintf('Initializing bestObjective: %e\n', bestObjective);
else
currentObjective = results.ObjectiveMinimumTrace(end);
improvement = abs(currentObjective - bestObjective);
fprintf('Current Improvement: %e\n', improvement);
% Check if improvement is less than the tolerance
if improvement < 1e-3
fprintf('Stopping Optimization: Improvement below minimum tolerance level of 1e-3.\n');
stop = true; % Signal to stop optimization
else
bestObjective = currentObjective;
end
end
elseif strcmp(state, 'initial')
fprintf('Optimization initializing...\n');
elseif strcmp(state, 'done')
fprintf('Optimization completed.\n');
end
end
The function can be executed using the following syntax:
results = bayesopt(@optimizeParameters, parRange, ...
'IsObjectiveDeterministic', false, ...
'AcquisitionFunctionName', 'expected-improvement-plus', ...
'Verbose', 0, ...
'ExplorationRatio', 0.6, ...
'MaxObjectiveEvaluations', 100,'OutputFcn',@terminationFunc);
The effectiveness of this approach was evaluated using a sample Bayesian optimization:
rng default
num = optimizableVariable('n',[1,30],'Type','integer');
dst = optimizableVariable('dst',{'chebychev','euclidean','minkowski'},'Type','categorical');
c = cvpartition(351,'Kfold',5);
fun = @(x)kfoldLoss(fitcknn(X,Y,'CVPartition',c,'NumNeighbors',x.n,'Distance',char(x.dst),'NSMethod','exhaustive'));
results = bayesopt(fun,[num,dst],'Verbose',0,'AcquisitionFunctionName','expected-improvement-plus','OutputFcn',@terminationFunc);
File is saved as ‘bayesian.m’. Below is the screenshot of the output:
To know more about the OutputFcn property of bayesopt function, refer to this documentation:
Hope it helps!
Muhammad on 29 Apr 2024 at 11:02