Parameter Estimation Example

Kurt Bryan and SIMIODE

A very simple example of fitting a function or model to data by using least squares.

The Data: Here are some hypothetical data in the form of (t,y) pairs:

data = {{1.1,1.24},{1.9,0.83},{2.3,0.71},{4.1,0.29},{5.5,0.15}}

A plot of the data:

plt1 = ListPlot[data]

The Model and Sum of Squares: Let's fit a model f(t) = a*exp(b*t) to this data by adjusting a and b. First form a sum of squares

f[t_] = a*Exp[b*t]
SS[a_,b_] = Sum[(f[data[[j,1]]]-data[[j,2]])^2,{j,1,5}]

With only two parameters "a" and "b", a visual estimate of the best choice (the choices of a and b that minimize SS) can be found by plotting. It's clear that b<0 since the data decays, and also that a>1. A plot of log(SS) is more informative, though:

Plot3D[Log[SS[a,b]],{a,1,3},{b,-1,0}]

Rotate the graph around. Something around a = 2, b = -0.5 looks promising.

Minimizing the Sum of Squares:  The multivariable calculus approach is to find a critical point. Form the appropriate derivatives

dSSda = D[SS[a,b],a]
dSSdb = D[SS[a,b],b]
optab = FindRoot[{dSSda==0,dSSdb==0},{a,2.0},{b,-0.5}]

The residual sum of squares is

SS[a,b]/.optab

A plot the best-fit f(t) to compare to the data:

bestf[t_] = f[t]/.optab
plt2 = Plot[bestf[t],{t,1.1,5.5},PlotStyle->{Red}];
Show[plt1,plt2]