problem in curve fitting using summation of sine functions

i wanted to fit the summation of sine functions in the following data: https://drive.google.com/file/d/1OzD-Dcra6j4Xn6nCWy4zFUUTObcIHbwq/view?usp=drive_link and https://drive.google.com/file/d/1gC00X5qEbqPlIBzsFNk9kQSmhVM44lEr/view?usp=drive_link, i am using curve fitter tool box for this, but fitted curve is not coming what i want. i am attaching the pic for reference. please any one help.

[John D'Errico]

First, you need to actually attach your data, not as a broken Google drive link that we cannot access. Since that is trivial to do, then just do it. You were able to attach the picture. So attach your data too!

Second, you need to show what you did get. Show what you tried. Did you use only a sum of sines, or sines and cosines? How many terms, and what you don't like about the result. (We can guess of course, but that would be just a wild guess.) If we don't know what you saw as inadequate, then we cannot easily help you to improve on what you did try.

[nihal]

sorry for causing the misunderstanding, i am attaching the data to look for it and i tried both sum of sine and sines and cosines in both case i am getting wavy like curve at the bottom as i attached the in above picture. i am using matlab toolbox for curve fitting. But i want to fit sum of sine curve for my further analysis

[nihal]

i am also attaching the screen shot of the curve tool box

[John D'Errico]

So what was wrong with the multiplle answers you did get here? Was there a compelling reason to keep on asking the same question?

[nihal]

Because I want to know @Alex done this curve fitting. I am facing same issue with theta_dot data set. Answer provided by you is also correct.

Answer 1

[Alex Sha]

@nihal if don't mind fitting function other than summation of sine, much better result will be achieved.

For phi_dot data:

Result:

Sum Squared Error (SSE): 5.9320020125328
Root of Mean Square Error (RMSE): 0.034854373653761
Correlation Coef. (R): 0.998694235877059
R-Square: 0.997390176774062

Parameter	Best Estimate     
---------	-------------     
p1       	-6.47104419670817 
p2       	-5.98261682333558 
p3       	6.97174098381172  
p4       	2.12628676769052  
p5       	2.46804771859176  
p6       	1.86197168986149  
p7       	45.118903356855   
p8       	-0.275472905476552
p9       	-24.2903853518025 
p10      	-3.04940813819318 

For theta_dot data:

Result:

Sum Squared Error (SSE): 17477.3796297918
Root of Mean Square Error (RMSE): 2.03435911509887
Correlation Coef. (R): 0.99924491377999
R-Square: 0.998490397715179

Parameter	Best Estimate    
---------	-------------    
p1       	16.6617050215943 
p2       	13.9664063412695 
p3       	-14.6357178726065
p4       	8.23236922174309 
p5       	-14.4468349720996
p6       	-92.0366152695814
p7       	99.9974678533004 
p8       	-98.6269424999215
p9       	-33512.5352364955
p10      	98.8356843265421 
p11      	-57.1951273322505
p12      	-24.1093307498757
p13      	42.2121329296961 
p14      	6.50567474900366 
p15      	16.8130006928835 
p16      	33512.1242507784 

[nihal]

thanks for the help @Alex Sha. i will use this for the further calculation and let you know if its is useful or not.

[nihal]

can you help out with this data because previous data is not working correctly. how you fit this equation?

[Alex Sha]

Sum Squared Error (SSE): 18453.9953695898
Root of Mean Square Error (RMSE): 2.07815892759393
Correlation Coef. (R): 0.999185081837055
R-Square: 0.998370827765723

Parameter	Best Estimate    
---------	-------------    
p1       	6.94198095204219 
p2       	7.32054888851064 
p3       	-7.22481844471732
p4       	4.10543902982487 
p5       	8.3880277156189  
p6       	100.67594697447  
p7       	99.0111230787527 
p8       	99.2795172276182 
p9       	-476284.666173349
p10      	-91.8736794718403
p11      	-23.9658037935652
p12      	-42.2268253727175
p13      	16.823353938126  
p14      	9.15801008621451 
p15      	-57.5778970416515
p16      	476184.830260837 

[nihal]

Please send me reply asap

[Alex Sha]

If taking one more term like below, the result will be better （the more the term, the better the result）:

Sum Squared Error (SSE): 15998.7964870748
Root of Mean Square Error (RMSE): 1.93498325286753
Correlation Coef. (R): 0.999293540379408
R-Square: 0.998587579844012

Parameter	Best Estimate    
---------	-------------    
p1       	9.8908910796797  
p2       	8.95888782995696 
p3       	4.10355352427305 
p4       	7.25393626351966 
p5       	-7.30283717193789
p6       	4.02100761706011 
p7       	71.5096735441434 
p8       	-88.3838068430202
p9       	-10448.9839956201
p10      	98.394740589154  
p11      	98.587554856128  
p12      	27.7869699687812 
p13      	-57.3380149156091
p14      	-61.483027988291 
p15      	5.33374786313033 
p16      	-25.0156591535756
p17      	16.9906486506356 
p18      	-22.3735278114893
p19      	10349.793921589  

[nihal]

Which tool you are using? Please can you tell me it will be very useful for my future applications.

[Alex Sha]

Hi, it is a package named 1stOpt, a global optimization solver, other than Matlab. The fitting functions above actually come from the Neural Network showing below:

[nihal]

CAN YOU TELL ME FROM WHERE IT CAN BE DOWNLOADED , I TRIED FROM SITES BUT NOT ABLE TO DO IT. PLEASE HELP ME ON THIS REGARD

[Alex Sha]

Please refer to this web link：

7D Software (7d-soft.com)

As far as I know, there is no download link。

Answer 2

[John D'Errico]

Ok. Thank you for the data. The plot does help a lot actually. At first glance, the plot of the fit itself looks quite reasonable. But then I focused my old bleary eyes on it, and I see the sinusoidal oscillations in the curve near the flat regions. (When I said I might guess what the problem was, this is what I was thinking.)

That is a common consequence of trying to fit a flat curve (or a curve with flat or nearly flat sections) with a sum of sine waves. Higher frequency sine functions will just introduce more (higher frequency) wiggles in the flats. Take enough terms and eventually the bumps go away, at least to the point you do not see them. But this just argues you did not take enough terms in the model.

It is exactly like the presence of Gibbs phenomena in a fit. In fact, if you look at the figures in the wiki page I link here, you will see that same behavior.

https://en.wikipedia.org/wiki/Gibbs_phenomenon

Again, taking more terms in that series will improve the fit, beause the oscillations will eventually cancel out. You don't have any sharp transition in your data, so there are no large overshoots near the edges. But even so, there will still be the same characteristic oscillations in the region of the curve that want to be nearly flat.

So, how can you improve on that result? Again, the simple solution is to just throw more terms at the problem. If you insist on having a model that is a sum of sines (and cosines usually) that will work, to drive down the oscillations. They will still be there of course, only at a finer level. Nothing you can do will prevent that in such a model.

Alternatively, you could use a spline model of sorts. Perhaps a smoothing spline would be a good choice, or a least squares spline. But splines have their own issues of course. People always want to write down the function, and you simply cannot write down some nice looking function from a spline. You can use them to evaluate, or make a nice smooth plot.

[nihal]

Thank you for the response, but the curve tool box matlab only go upto terms. Same issue is there when fourier function is used. If i use spline then it can not be helped in my further analysis

Answer 3

[John D'Errico]

I'll post this as a separate answer, so as to not get it lost in the comments, and since I will show you how to solve it as a trig series. I'll use backslash to perform the solve. I could have used many other tools, but this is actually a purely linear problem! (In the parameters, that is.)

phidot = readtable('phi_dot.csv');
x = phidot.Var1;
y = phidot.Var2;
plot(x,y)
yline(0)

per = 6.9; % approximate length of the period, determined by observation, where the curve crosses the x axis.

nterms = 8; 
n = 1:nterms;
% model as a sum of sine and cosine terms
% the first coefficient is just an additive constant,
% the next nterms coefficients apply to the sine terms,
% then the last nterms coefficients apply to cosine terms
A = [ones(size(x)),sin(2*pi*x*n/per),cos(2*pi*x*n/per)];
coeffs = A\y;
ypred = A*coeffs;
plot(x,y,'b-',x,ypred,'-r')

With 8 terms, this will be approximately what the curve fitting toolbox would have done (using the 'sin8' option for fittype. Next, I'll redo the fit, using 20 terms in the model.

nterms = 20; 
n = 1:nterms;
% model as a sum of sine and cosine terms
A = [ones(size(x)),sin(2*pi*x*n/per),cos(2*pi*x*n/per)];
coeffs = A\y;
ypred = A*coeffs;
plot(x,ypred,'-r')

I plotted only the curve this time, since you would miss what it does in the flat spots otherwise. As you can see, the oscillations decrease. No matter what you do, with more terms the wiggles will get smaller, but this is a FUNDAMENTAL characteristic of a trig series when used to fit such a problem.

nterms = 50; 
n = 1:nterms;
% model as a sum of sine and cosine terms
A = [ones(size(x)),sin(2*pi*x*n/per),cos(2*pi*x*n/per)];
coeffs = A\y;
ypred = A*coeffs;
plot(x,ypred,'-r')

The last plot was with 50 terms in the series. You should see there the wiggles are almost impossible to see. Again, this is just a fundamental characterisitic of such a model.

Finally, note that you can ALWAYS transform a pair of terms of the form

a1*sin(b*x) + a2*cos(b*x)

combining them both into a single phase shifted sine term only. That is just a basic trig identity. I won't show how to do that unless there is some need for it.

So while the model I fit used both sine and cosine terms, you could do the transformation I mention easily enough to reduce the number of terms.