How to skip an optimization iteration if the constraint is violated ?

Hello, I am trying to minimizing an objectif function with Surrogate optimisation.

The objectif function depends on 2 parameters , having the following lower and upper limits:

lb = [10, 2 ];

ub = [20, 4 ];

And I have a constraint , which, if it is not complied with, prevents the proper functioning of the function objective. The problem with surrogate optimisation is that the constraint is only check after the objectif function was evaluated. So my question is "how can I forbid certain combinations of parameters"? As proposed in this answer (https://nl.mathworks.com/matlabcentral/answers/49677-how-to-make-optimization-to-redo-or-skip-one-iteration), I tried to add in my objectif function, an if statement:

if then output.Fval = NaN

Hower this does not seems to work for surrogate optimisation. So if you have any idea how i could solve this problem I am a taker. Thanks.

Answer 1

[Matt J]

And I have a constraint , which, if it is not complied with, prevents the proper functioning of the function objective.

In your objective function, make the execution of the main routine conditional on the constraints being satisfied.

if x1<3*x2
    fval=inf; return
else
    fval=...%evaluate normaly
end

or you can post-correct the output,

    fval=%evaluate normally
    
    if ~(isfinite(fval)&&isreal(fval))
       
        fval=inf; return
       
    end

[Matthieu Bernard]

Hello, thank you for your fast response. The problem with your solution is that it still consider an iteration even the constrain was not respected. So I would like a solution, where if the constraint is violated, the objective function doesn't even start.

[Matt J]

That is the first of the two solutions that I gave you. Once you test the constraints, a decision is made whether to continue to do the computationally intensive part of the evaluation.

[Matthieu Bernard]

I understand what you mean, it is clear that the "computationally intensive part of the objectif function " will not be launched with this if statement. But the problem is that it still enter in the objectif function (even the intensive part is not performed), so it means that the iteration number will increase, which gives a not very nice Optimization plot function.

[Matt J]

Well, you can replot at the end of the optimization with the undesired points removed.

It doesn't make sense to interfere with the search for the optimum just to make plots look nicer.

[Matthieu Bernard]

yes I agree with you, but it strange that a basic constraint cannot be simply added such that the objectif function is not run if this constraint is violated.

[Matt J]

Well, what if the constraints are never satisfied, except at convergence? How would you want the code to behave?

[Matthieu Bernard]

For your remark "what if the constraints are never satisfied, except at convergence" I think it will not change anything to have the constraint oustide the objectif function or inside the objectif function. I even wonder if giving fval an infinite unrealistic value does not distort the surrogate model ?

[Matt J]

Only one way to find out.

[Walter Roberson]

it strange that a basic constraint cannot be simply added such that the objectif function is not run if this constraint is violated.

The objective function is already not run if the bounds or linear constraints are violated.

Most of the other optimizers have separate nonlinear constraint functions and objective functions. It is not immediately obvious to me why surrogateopt() chose to combine the two into a single function -- but given that they are combined, then the only way to find out of a nonlinear constraint was violated is to run the objective function.

The one thing that comes to mind is that it is not uncommon for nonlinear constraints to depend upon expensive calculations that the objective function also needs, and because the objective functions for fmincon() and similar do not share local variables with the nonlinear constraint functions, work-arounds have to be found to avoid repeating the same (expensive) calculation between the two. I could see that potentially this could be an important factor that might lead to wanting to combine the two.

[Matt J]

The objective function is already not run if the bounds or linear constraints are violated.

The bounds perhaps, but surrogateopt has no mechanism by which to distinguish between nonlinear and general linear constraints.

[Matthieu Bernard]

Thank you for your response Walter. Your explanation seems coherent, so if I understand well, the objectif function always need to be run in the case of surrogate optimisation. But do you think that assign an infinite value to fval does not distort the surrogate model ?

[Walter Roberson]

BlackBoxOptimizer.m specifically tests for nan returned for the function value. Also for empty being returned.

[Weihan Lin]

I had a really complex case and if I use if statement like this, the results were always be 'no feasible points'

[Walter Roberson]

@Matt J

I am not sure why you say that surrogateopt has no mechanism by which to distinguish between nonlinear and general linear constraints? It supports A, b, Aeq, beq constraints, and if I recall correctly, it handles those without calling the the objective function.

[Walter Roberson]

@Weihan Lin

When the objective function returns a struct that includes Ineq then surrogate optimization can reason about where the boundaries are likely to be; however it cannot do that if you just return a numeric value (such as inf or nan) instead of the struct with Ineq information.

[Weihan Lin]

@Walter Roberson

Thank you so much for your suggestions! But I'm still not fully understanding how to solve it. Would you mind to explain more on how to let the objective function returns a 'Ineq'?

Or do you have any other suggestions on solving this kind of problems? My problem is a bit different from this but the main goal is same. I have a optimization problem and I want to set a feasible range for design parameters. However, I have more than 20 different conditions and each condition has its own constraints, and I want to use 'if' statement to seperate those 20 conditions and write everything in one function file.

[Walter Roberson]

In your objective function, prepare a struct to return, with fields named Fval and Ineq (all other field names will be ignored.)

Calculate the amount of constraint violation for each of the 20 different conditions, and assign the results to the Ineq field, with the usual convention that positive is a constraint violation and non-positive is no violation.

Having done that,

if any(TheResult.Ineq > 0)
  TheResult.Fval = nan
else
  calculate the function value and assign to TheResult.Fval
end

[Weihan Lin]

@Walter Roberson

Thank you so much for you fast and valuable response! It did work on my code! However, it might because I have too many design parameters, the optimization takes hours to calculate the optimal value. Also, the optimal value was returned to be 'nan' but not a number. How to delete or unsave the iterations with value of 'nan'?

Also is it possible to define or set a feasible region with equations and conditioanl operators before those parameters go into the objective function?

[Matt J]

@MattJ I am not sure why you say that surrogateopt has no mechanism by which to distinguish between nonlinear and general linear constraints? It supports A, b, Aeq, beq constraints,

Not prior to R2021a.

[Walter Roberson]

@Weihan Lin

If the optimization returns nan in that setup, it means that the optmizer did not find even one point that satisfied all of your constraints.

[Weihan Lin]

@Walter Roberson

Yeah, I see. It couldn't find any feasible point when the population size is small. I change th population size from 50 to 1000 and only one point was found in the feasible region after 30 iterations. I probably will increase the population size more and see if the results will get any better and I'm so appreciate for your help.