posted by mathstudent
I'm trying to follow a research paper
The paper shows an equation to minimize. That makes perfect sense. Then, the paper says:
"The optimal solution to the least squares problem [above] is found by differentiation as a solution of a linear system of equations."
I am very familiar with traditional linear algebra least squares:
y = xb + e
solve for coefficients b by:
b = (x^T * x)^-1 * x^T * y
I understand the equation to be minimized but I don't understand the formulas that follow, and I think the paper is using some alternative least squares approach that I am not familiar with. It doesn't look like the paper is doing regular least squares (like I know). Any ideas what the paper is referring to? How do you use differentiation as a solution to a linear system of equations?
Differentiation is used when the equation is nonlinear. To parameterize the equation, partial derivatives are used to construct the design matrix (in your case, that would be the b matrix.) x would be the unknown vector, e would be the residual vector and y would be your observation vector.