Nonlinear regressions alternatives to nlinfit

Hi,

What are the available alternatives to 'nlinfit' in Matlab ? Is there a ridge type of regressions to be used for non-linear model ?

I am not satisfied with the nlinfit since, is very sensitive to the initial conditions given and in my case cannot make an educated guess on those. So, I am wondering where there are other nonlinear regression options in matlab suited for non-linear models.

UPDATE

I have an algorithm where the sample for y (dependent variable) and the sample for x's (explanatory variables) of the regression are generated from a particular economic model.

Now to solve my model, I have a loop where my data are generated. So in each loop I have a new sample and therefore a new regression. I use the nlinfit for those regressions, where in each iteration I use as initial guesses the coefficients found by the regression in the PREVIOUS iteration.

So it very easy now to understand where my algorithm crashes. There are instances where in a particular iteration step the initial guesses might be so bad making the nlinfit to crash (for example problems with the Jacobian)

Many thanks

Answer 1

[Star Strider]

You haven’t gone into any detail as to the problems you’re having with your regression, but if it is particularly difficult, use a Genetic Algorithm or other robust optimisation approach to estimate the parameters. Even if you don’t have the Global Optimization Toolbox, GAs are relatively easy to write, especially with MATLAB’s functionality. They can ‘guess’ the parameters with sufficient accuracy that nlinfit would be able to converge quickly with the GA output as initial parameter estimates.

[msh]

Your idea, sounds promising. I did not provide more details since I was afraid that might not be necessary. I will update my answer accordingly. But let me explain to you what I do.

I have an algorithm where the sample for y (dependent variable) and the sample for x's (explanatory variables) of the regression are generated from a particular economic model.

Now to solve my model, I have a loop where my data are generated. So in each loop I have a new sample and therefore a new regression. I use the nlinfit for those regressions, where in each iteration I use as initial guesses the coefficients founds by the regression in the PREVIOUS iteration.

So it very easy now to understand where my algorithm crashes. There are instances where in a particular iteration step the initial guesses might be so bad, crashes the nlinfit (for example problems with the Jacobian)

I hope now makes more clear the overall pic.

Could you please elaborate what is a Genetic Algorithm or where I can find sources to implement one in Matlab? And I really I am not aware what are the other "robust optimization approaches" to estimations as you mention.

[Star Strider]

The genetic algorithm function in MATLAB, ga, is in the Global Optimization Toolbox. It is optimised, so I would use it rather than write my own if possible, but simple GAs are not difficult to write and ‘tune’.

The best description is How The Genetic Algorithm Works. It is better than any description I could write.

You would then take those parameter estimates and if necessary, ‘fine tune’ them with nlinfit. The fitness function for the GA would be the same function you would use in nlinfit.

I am not certain how time-efficient it would be if you are running several iterations, since you would have to run the GA each time as well. But the GA is as close as it’s possible to get for code providing an ‘intelligent guess’ of initial parameter estimates.

[msh]

Thanks. I will try it. At the moment I do not care about the computational cost.

[Star Strider]

My pleasure.

If you don’t mind computational cost or time, then the GA will likely work well for you. I suggest the MATLAB ga and related functions, but there are some third-party GA algorithms for MATLAB available as well.

Answer 2

[John D'Errico]

There is no magic wand that works perfectly. If there was, it would be in use. What a surprise!

While you CAN create a simple ridge type estimator for a nonlinear problem, they just bias the result towards zero. As arbitrarily, you could use it to bias the result towards any number you want, but if you already know the answer, then why bother to estimate it? At least if you know the answer, then you have good starting values for it, and if the estimation process drives away from there, then it means you have crap for data.

So get better starting values. Or get better data. Or use a model that actually fits your data, instead of trying to force some arbitrary model to your data.

In the end, I can't give you better advice without seeing your data, without seeing your model. I'd need to understand why it was that you chose the model you are trying to use. For example, it might be that you should be using a spline instead. Or maybe a partitioned least squares might be appropriate. But to just ask for other methods without knowing anything more would mean I need to write a book here. I'd be wasting my time to describe methods that are totally useless for you.