Functions with same evaluation outputs and same starting point have different fminsearch outcomes?

Hi guys,

I have two numerical functions f1 and f2 and for some reason, I cannot show the detailed codes for them. Basically, f2 is the vectorized version of f1.

These two functions have the same values for any inputs. Given the same starting point, however, the fminsearch gives different minimizer and different minimized function value.

What could be the reason behind it?

Thanks,

Jialin

Answer 1

[Mukul Rao]

Hi,

Vectorizing an expression can change its final value due to optimizations from the execution engine and in some cases, due to a differences in algorithm. For example, MATLAB uses a different algorithm for vector-matrix multiplication as opposed to matrix-matrix multiplication. So if one computes the product of two matrices in a for loop with vector-matrix multiplications, the answer would differ slightly from computing the same expression with the * operator for matrix product.

    >> A = rand(3,3);
    >> B = rand(3,3);
    >> [A(1,:)*B ; A(2,:)*B; A(3,:)*B] - A*B %The first term can be evaluated in a for loop

ans =

     1.0e-15 *

           0    0.2220         0
     -0.0555    0.1110         0
           0         0    0.1110

In your case, the value of the vectorized expression might remain the same as far as the math is concerned, but the way this expression is evaluated by the MATLAB engine might differ due to the aforementioned factors.

The optimization solver is most likely returning a different solution due to the vectorized objective expression returning a slightly different value. If the problem is ill-conditioned, the final results could be drastically different.

Answer 2

[Walter Roberson]

fminsearch uses a random number generator so it will not produce the same results unless the random seed is rese

[Mukul Rao]

Hi Walter, thanks for pitching in. While random number generators are used by many of our solvers, especially global optimization solvers, it is not the case with fminsearch. It's based on the Nelder Mead algorithm, MATLAB's implementation of this algorithm is deterministic. I have verified this looking at the source code for "fminsearch" and also getting in touch with my colleagues in development. You can also try running the fminsearch algorithm on the same problem with the same initial conditions several times, it always returns the same solution (I've tried this on R2017a).