I'd argue you are thinking about this the wrong way.You see that fzero has a problem, so you think you need to do something to fzero. But that is wrong, the wrong approach.
You CAN constrain the parameters in the PSO however! Nothing stops you from doing that. Why do you need to use optimoptions? (Which would be of absolutely no use anyway.) Note that particleswarm does allow bound constraints on the parameters.
help particleswarm
particleswarm - Particle swarm optimization
This MATLAB function attempts to find a vector x that achieves a local
minimum of fun.
Syntax
x = particleswarm(fun,nvars)
x = particleswarm(fun,nvars,lb,ub)
x = particleswarm(fun,nvars,lb,ub,options)
x = particleswarm(problem)
[x,fval,exitflag,output,points] = particleswarm(___)
Input Arguments
fun - Objective function
function handle | function name
nvars - Number of variables
positive integer
lb - Lower bounds
[] (default) | real vector or array
ub - Upper bounds
[] (default) | real vector or array
options - Options for particleswarm
options created using optimoptions
problem - Optimization problem
structure
Output Arguments
x - Solution
real vector
fval - Objective value
real scalar
exitflag - Algorithm stopping condition
integer
output - Solution process summary
structure
points - Final swarm positions and objective function values
structure
Examples
openExample('globaloptim/MinimizeaSimpleFunctionExample')
openExample('globaloptim/MinimizeaSimpleFunctionwithBoundsExample')
openExample('globaloptim/MinimizeUsingNondefaultPSwarmOptionsExample')
openExample('globaloptim/ExaminetheSolutionProcessExample')
See also ga, patternsearch, Optimize
Introduced in Global Optimization Toolbox in R2014b
Documentation for particleswarm
doc particleswarm
If you are using some other tool, and you never told us, then you are on your own, since we cannot know what the capabilities of some unknown tool would be.
Finally, you have code that calls fzero. Surely you can pretest the parameters just before you call fzero. If the parameter would cause a problem, then do not call fzero, and just return some large value for the objective for that set of parameters. Again, there is absolutely no problem. All it takes is an if statement to test the value, BEFORE fzero is ever called.
And even if all of the above ideas don't seem to work for you (which they absolutely must, but assuming they don't) then you could call fzero using a try/catch clause to catch the times when fzero would fail.
Do you see that instead of looking at fzero, you need to think about how to deal with the problem before fzero ever sees it as an issue.
I've outlined 4 different solutions you can use. One of them will work for you. Actually, ALL of them should work.