Exercise: MSE
As in an earlier exercise, we assume that the random variables
Θ
and
X
are described by a joint PDF which is uniform on the triangular set defined by the constraints
0
≤
x
≤
1
,
0
≤
θ
≤
x
.
a) Find an expression for the conditional mean squared error of the LMS estimator given that
X
=
x
, valid for
x
∈
[
0
,
1
]
. Express your answer in terms of
x
using standard notation.
b) Find the (unconditional) mean squared error of the LMS estimator.
a.x^2/12
b.1/24
a) We saw that the conditional PDF of Θ is uniform on the range [0,x] . Hence, the conditional variance is x2/12 .
b) This is given by the integral of the conditional variance, weighted by the PDF of X . The PDF of X is found using the formula for going from the joint to the marginal, and is fX(x)=2x , for x∈[0,1] . Thus, the mean squared error is
∫10x212⋅2xdx=16∫10x3dx=124.
As in an earlier exercise, we assume that the random variables Θ and X are described by a joint PDF which is uniform on the triangular set defined by the constraints 0≤x≤1 , 0≤θ≤x .
a) Find an expression for the conditional mean squared error of the LMS estimator given that X=x , valid for x∈[0,1] . Express your answer
To find the conditional mean squared error (MSE) of the LMS estimator given that X = x, we need to follow these steps:
Step 1: Recall the LMS estimator.
The LMS (Least Mean Squares) estimator is given by:
Ŷ = E(Θ | X)
where Θ is the true value and X is the observed value.
Step 2: Compute the conditional mean of Θ given X = x.
To find E(Θ | X = x), we need to evaluate the conditional expectation of Θ given the value of X. In this case, since the joint PDF is uniform on the triangular set defined by the constraints 0 ≤ x ≤ 1 and 0 ≤ θ ≤ x, the conditional mean is simply the midpoint of the support of the Θ given X = x.
Since X ∈ [0, 1] and Θ ∈ [0, x], the support of Θ given X = x is [0, x]. The midpoint of this interval is x/2.
Therefore, E(Θ | X = x) = x/2.
Step 3: Compute the conditional mean squared error (MSE).
The conditional MSE is defined as the expectation of the squared difference between the true value Θ and the LMS estimator Ŷ, given X = x.
MSE = E[(Θ - Ŷ)^2 | X = x]
We already have the expression for the conditional mean E(Θ | X = x), so we substitute it into the MSE formula:
MSE = E[(Θ - E(Θ | X = x))^2 | X = x]
= E[(Θ - x/2)^2 | X = x]
To compute this expectation, we integrate the squared difference over the range of Θ, which is [0, x]:
MSE = ∫[0, x] (Θ - x/2)^2 dΘ
= ∫[0, x] (Θ^2 - Θx + (x^2)/4) dΘ
= [Θ^3/3 - (Θ^2 * x)/2 + (x^2 * Θ)/4] |[0, x]
= (x^3/3 - (x^3 * x)/2 + (x^3 * x)/4) - (0/3 - (0x)/2 + (x^2 * 0)/4)
= (x^3/3 - x^4/2 + x^4/4) - 0
= (4x^3 - 6x^4 + 3x^4) / 12
= (4x^3 - 3x^4) / 12
Therefore, the expression for the conditional mean squared error of the LMS estimator given that X = x is:
MSE = (4x^3 - 3x^4) / 12.
For part b) of the question, we need to find the unconditional mean squared error (MSE) of the LMS estimator. In other words, we need to find the average (expected) value of the squared difference between the true value Θ and the LMS estimator Ŷ, regardless of the observed value of X.
The unconditional MSE is given by:
MSE_unconditional = E[(Θ - Ŷ)^2]
To compute this expectation, we need to consider the joint PDF of X and Θ. In this case, as mentioned earlier, the joint PDF is uniform on the triangular set defined by the constraints 0 ≤ x ≤ 1 and 0 ≤ θ ≤ x.
Since the joint PDF is uniform over this region, the LMS estimator Ŷ = E(Θ | X) is always equal to the midpoint of the range of Θ, which is x/2.
Therefore, for any value of X, the LMS estimator is x/2, and there is no dependence on the observed value of X.
The true value Θ can take any value between 0 and X (i.e., Θ ∈ [0, X]). So, the squared difference (Θ - Ŷ)^2 is equal to (Θ - x/2)^2, regardless of the observed value of X.
To find the unconditional MSE, we need to integrate this squared difference over the joint PDF of X and Θ, which is uniform with the constraints mentioned above.
MSE_unconditional = ∫∫(Θ - x/2)^2 p(Θ, X) dΘ dX
= ∫∫(Θ - x/2)^2 dx dΘ
= ∫ [(Θ^3/3 - Θ^2 * x/2 + Θ * (x^2)/4) | [0, Θ] dx ] | [0, 1] dΘ
= ∫ (Θ^3/3 - Θ^2 * Θ/2 + Θ * (Θ^2)/4) dΘ | [0, 1]
= [Θ^4/12 - Θ^4/6 + 3Θ^4/24] | [0, 1]
= (1/12 - 1/6 + 3/24) - (0/12 - 0/6 + 0/24)
= 1/12 - 1/6 + 1/8
= 2/24 - 4/24 + 3/24
= 1/24
Therefore, the unconditional mean squared error (MSE) of the LMS estimator is 1/24.
a) x^2/2
b)1/24