Best practices for setting penalty values in objective functions for bayesopt in MATLAB

Hi all,

I’m using bayesop, and trying to design a “safe” objective function: whenever my function encounters out-of-domain inputs,or some kind of errors, I return a large penalty value instead of letting the algorithm fail, although I understand that this algorithm never fails but keeps on running even if it encounters an error.

My typical objective values are in the range of -2 to 2. I’m wondering, does the magnitude of the penalty value influence how bayesopt performs?

For example, would assigning a penalty of 20 have a noticeably different impact compared to a penalty of 1e6? Is there an optimal approach or rule of thumb for choosing penalty values so that invalid points are discouraged, yet the optimization routine remains numerically stable and efficient?

I’d appreciate any insights. Thanks!

Answer 1

[Matt J]

You might see slower convergence when the penalty is very large.

[Sauradeep]

Thanks. So, curious to know what's the rule of thumb for choosing penalty values? Given my typical objective values are in the range of -2 to 2, what should I be using?

[Matt J]

There is no rule of thumb. You will have to experiment.

[John D'Errico]

From what I have read, bayesopt can handle a barrier penalty to avoid constraint regions. It seems clearly, the penalty must be a value larger than any you will encounter insider the valid domain. So if you know absolutely the range of the objective inside the valid domain will be [-2,2], then in theory, I think anything larger than 2 would be an acceptable penalty to return, and fully sufficient. If the upper end is a bit fuzzy, then just use something clearly larger than you would ever see inside, and all should be good.

Will it help or hinder things if the penalty value returned from the constraint region is 1e25, instead of just 3? That is not clear. But I don't see why a hugely large value will help, and it might cause numerical problems. It would be worth deeper investigation. Or take the opportunity to contact tech support directly, hoping to find the person who wrote bayesopt. That person would likely have good insights to offer.