As in the previous exercise, let È be the bias of a coin, i.e., the probability of Heads at each toss. We assume that È is uniformly distributed on [0,1]. Let K be the number of Heads in 9 independent tosses. We have seen that the LMS estimate of K is E[K∣È=è]=nè.
a) Find the conditional mean squared error E[(K−E[K∣È=è])2∣È=è] if è=1/3.
b) Find the overall mean squared error of this estimation procedure.
a) = var(K/o)= np(1-p) = 9*(1/3)*(2/3)= 2
b) 3/2
b) integrate n*p*(1-p) , from 0 to 1
a) Well, if è=1/3, then I guess we can call that an 'unbiased' coin... or maybe just a very indecisive one. Anyway, let's crunch some numbers. The conditional mean squared error can be calculated as follows:
E[(K−E[K∣È=è])2∣È=è] = E[(K−3/3)2∣È=1/3]
Now, since K is the number of Heads in 9 tosses, it follows a binomial distribution with parameters n = 9 and p = 1/3. The variance of a binomial distribution is np(1-p), so:
Var(K∣È=1/3) = 9 * 1/3 * (1 - 1/3)
From there, we can calculate the mean squared error as:
E[(K−E[K∣È=è])2∣È=è] = Var(K∣È=1/3) + (E[K∣È=1/3] - 3/3)2
Substituting the values we get:
E[(K−E[K∣È=è])2∣È=è] = 9 * 1/3 * (2/3) + (1/3 - 3/3)2
Simplifying:
E[(K−E[K∣È=è])2∣È=è] = 6/9 + (-2/3)2
Calculating further:
E[(K−E[K∣È=è])2∣È=è] = 6/9 + 4/9
E[(K−E[K∣È=è])2∣È=è] = 10/9
So, when è=1/3, the conditional mean squared error is 10/9. Which means our estimation might be a bit biased in this case.
b) To find the overall mean squared error, we need to take the expectation over all possible values of È. Since È is uniformly distributed on [0,1], we can integrate over this interval:
E[(K−E[K∣È=è])2] = ∫ E[(K−E[K∣È=è])2∣È=è] * f(È) dÈ
In this case, f(È) is the probability density function of È, which is 1 for È in the interval [0,1] (since it's uniformly distributed).
Simplifying the integral:
E[(K−E[K∣È=è])2] = ∫ (10/9) * 1 dÈ
= 10/9 * ∫ 1 dÈ
= 10/9 * (È)|[0,1]
= 10/9 * (1 - 0)
= 10/9
So, the overall mean squared error of this estimation procedure is 10/9. It seems like our coin's indecisiveness is causing a bit of a problem here.
To find the conditional mean squared error E[(K−E[K∣È=è])2∣È=è] when è=1/3, we first need to calculate the conditional expectation E[K∣È=è] when è=1/3.
Given that È is uniformly distributed on [0,1], we know that the probability density function (PDF) of È, denoted as f(è), is constant within the range [0,1]. Thus, f(è) = 1 for 0 ≤ è ≤ 1, and f(è) = 0 for è < 0 or è > 1.
To find the conditional expectation E[K∣È=è], we multiply the number of tosses (9) by the value of è, resulting in E[K∣È=è] = 9 * è.
With è=1/3, we have E[K∣È=1/3] = 9 * (1/3) = 3.
To find the conditional mean squared error E[(K−E[K∣È=è])2∣È=è] when è=1/3, we substitute the given value of è into the expression.
E[(K−E[K∣È=è])2∣È=è] = E[(K−3)2∣È=1/3]
Since K follows a binomial distribution with parameters n and È, we have
E[(K−3)2∣È=1/3] = ∑[(k−3)2 * Pr(K=k∣È=1/3)] for k = 0 to 9.
We calculate Pr(K=k∣È=1/3) using the formula for the binomial distribution:
Pr(K=k∣È=1/3) = (9 choose k) * (1/3)^k * (2/3)^(9-k) for k = 0 to 9.
Substituting these values into the expression for E[(K−3)2∣È=1/3], we can find the value of the conditional mean squared error.
For part b), to find the overall mean squared error of this estimation procedure, we need to calculate the average of the conditional mean squared errors over all possible values of È.
The overall mean squared error is given by:
E[E[(K−E[K∣È=è])2∣È]] = E[(K−E[K∣È])2]
We can find the overall mean squared error by integrating the conditional mean squared error E[(K−E[K∣È=è])2∣È=è] with respect to the density function f(è) over the range [0,1].
This will give us the average of the conditional mean squared errors weighted by the probability distribution of È.
Integrating E[(K−E[K∣È=è])2∣È=è] with respect to f(è) from 0 to 1 will yield the overall mean squared error.