I'm trying to work through the proof for
SST = SSM + SSE
MEAN = ∑(X)/N
SST = ∑((x - MEAN)^2)
= ∑(x^2 - 2 * x1 * MEAN + MEAN^2)
= ∑(x^2) - 2 * MEAN * ∑(x) + N * MEAN^2
= ∑(x^2) - 2 * ∑(x)^2/N + ∑(x)^2/N
= ∑(x^2) - ∑(x)^2/N
SSM = ∑((MODEL - MEAN)^2)
= ∑(MODEL^2 - 2 * MODEL * MEAN + MEAN^2)
= ∑(MODEL^2) - 2 * MEAN * ∑(MODEL) + N * MEAN^2
= ∑(MODEL^2) - 2 * MEAN * ∑(MODEL) + N * MEAN^2
= ∑(MODEL^2) - 2/N * ∑(x) * ∑(MODEL) + ∑(x)^2/N
SSE = ∑((x - MODEL)^2)
= ∑(x^2 - 2 * x * MODEL + MODEL^2)
= ∑(x^2) - 2 * ∑(x * MODEL) + ∑(MODEL^2)
SST = SSM + SSE
∑(x^2) - ∑(x)^2/N = ∑(MODEL^2) - 2/N * ∑(x) * ∑(MODEL) + ∑(x)^2/N + ∑(x^2) - 2 * ∑(x * MODEL) + ∑(MODEL^2)
2 * ∑(MODEL^2) - 2/N * ∑(x) * ∑(MODEL) + 2 * ∑(x)^2/N + - 2 * ∑(x * MODEL) = 0
∑(MODEL^2) - 1/N * ∑(x) * ∑(MODEL) + ∑(x)^2/N + - ∑(x * MODEL) = 0
I can't complete the proof. What am I missing? Thanks!
Divide the sum of squares by N and work with the averages. Let's use the notation:
<X> for the average of X. E.g.:
<X> = ∑(X)/N = Mean
And:
<(X-<X>)^2> =
<X^2 - 2X<X> + <X>^2> =
<X^2> - <X>^2
Note that <a X> = a <X> for a constant factor a. In an average like <X <X>>, the inner <X> is a constant when carrying out the outer average, so you can take it out of the outer average sign. So, you have <X <X>> = <X>^2. The average of a constant is, of course, the same constant so e.g. <<X>^2> = <X^2> because once the inner average is carried out it is a constant w.r.t. the outer average.
If you work with averages and use these rules then you can derive the desired result in just one line. If you use summations, you'll tend to re-derive these rules in every step you make, so you'll get a complicated mess.
Derivation:
<(X - <X>)^2> =
<(X - m + m - <X>)^2> =
<(X-m)^2> + <(m - <X>)^2>
+ 2 <X-m><m-<X>>
The last term is zero if the average of X equals the average of the Model.
I don't follow this:
<<X>^2> = <X^2>
Of course, if k is constant and x is variable:
<kx> = k<x>
<k> = k
<k^2> = k^2
but...
<x^2> != <x>^2
Sorry, that was a typo.
I meant to write:
<<X>^2> = <X>^2
I don't follow this at all:
<(X - m + m - <X>)^2> = <(X-m)^2> + <(m - <X>)^2> + 2 <X-m><m-<X>>
Trying to follow your logic, for the left side:
<(X - m + m - <X>)^2>
= <(X - <X>)^2>
= <x^2> - <x>^2
= SST
For SSM + SSE:
<(x - m>^2> + <(m - <x>)^2>
= <x^2 - 2xm + m^2> + <m^2 - 2m<x> + <x>^2>
= <x^2> - 2<xm> + <m^2> + <m^2> - 2<m><x> + <x>^2>
= 2<x^2> + 2<m^2> - 2<xm> - 2<m><x>
And I'm stuck...
To complete the proof, you need to simplify further and use the fact that the sum of squares of deviations from the mean (SST) can be decomposed into the sum of squares of deviations explained by the model (SSM) and the sum of squares of deviations unexplained by the model (SSE).
Let's simplify the equation further:
∑(MODEL^2) - 1/N * ∑(x) * ∑(MODEL) + ∑(x)^2/N + - ∑(x * MODEL) = 0
Now, let's examine each term separately:
1. ∑(MODEL^2):
This term represents the sum of squares of the predicted values (MODEL). It measures the variation explained by the model.
2. - 1/N * ∑(x) * ∑(MODEL):
This term represents the product of the sum of the observed values (x) and the sum of the predicted values (MODEL) divided by the sample size (N). It captures the relationship between the observed and predicted values.
3. ∑(x)^2/N:
This term represents the sum of squares of the observed values (x) divided by the sample size (N). It measures the variation of the observed values.
4. - ∑(x * MODEL):
This term represents the sum of the product of the observed values (x) and the predicted values (MODEL). It captures the deviations between the actual and predicted values.
The equation states that the sum of squares of deviations from the mean (SST) is equal to the sum of squares of deviations explained by the model (SSM) plus the sum of squares of deviations unexplained by the model (SSE).
Therefore, the equation is expressing the fundamental principle of decomposition of total variation (SST) into the variation explained by the model (SSM) and the unexplained variation (SSE).
You haven't missed anything in the proof. The equation you have derived is indeed correct, and it demonstrates the decomposition of SST into SSM and SSE.