Consider a fire alarm that senses the environment constantly to figure out if there is smoke in the air and hence to conclude whether there is a fire or not. Consider a simple model for this phenomenon. Let Θ be the unknown true state of the environment: Θ=1 means that there is a fire and Θ=0 means that there is no fire. The signal observed by the alarm at time n is Xn=Θ+Wn, where the random variable Wn represents noise. Assume that Wn is Gaussian with mean 0 and variance 1 and is independent of Θ. Furthermore, assume that for i≠j, Wi and Wj are independent. Suppose that Θ is 1 with probability 0.1 and 0 with probability 0.9.
Give numerical answers for all parts below.
Given the observation X1=0.5, calculate the posterior distribution of Θ. That is, find the conditional distribution of Θ given X1=0.5.
P(Θ=0∣X1=0.5)= ?
P(Θ=1∣X1=0.5)= ?
What is the LMS estimate of Θ given X1=0.5?
θˆLMS= ?
What is the resulting conditional mean squared error of the LMS estimator given X1=0.5? - ?
P(Θ=0∣X1=0.5)= 0.9
P(Θ=1∣X1=0.5)= 0.1
MSE= 0.09
θˆLMS= ?
èLMS= ?
èLMS = 0.1
To find the posterior distribution of Θ given the observation X1=0.5, we need to use Bayes' Theorem.
Bayes' Theorem states that:
P(Θ|X1) = (P(X1|Θ) * P(Θ)) / P(X1)
Where P(X1) is the marginal probability of X1, which can be obtained by summing over all possible values of Θ:
P(X1) = P(X1|Θ=0) * P(Θ=0) + P(X1|Θ=1) * P(Θ=1)
In this case, P(Θ=0) = 0.9 and P(Θ=1) = 0.1.
Now let's calculate the values:
P(Θ=0|X1=0.5) = (P(X1=0.5|Θ=0) * P(Θ=0)) / P(X1)
P(Θ=1|X1=0.5) = (P(X1=0.5|Θ=1) * P(Θ=1)) / P(X1)
To find the conditional mean squared error (MSE) of the LMS estimator, we need to calculate the expected squared difference between the estimate θˆLMS and the true value of Θ, i.e., E[(θˆLMS-Θ)^2].
To find θˆLMS, we need to use the minimum mean squared error (MMSE) criterion, which states that θˆLMS is the conditional mean of Θ given the observations, i.e., E[Θ|X1].
Let's calculate the values step by step:
1. Calculate P(X1):
P(X1) = P(X1|Θ=0) * P(Θ=0) + P(X1|Θ=1) * P(Θ=1)
= P(W1=0.5) * P(Θ=0) + P(Θ=1) * P(W1=0.5|Θ=1)
Since W1 is Gaussian with mean 0 and variance 1:
P(W1=0.5) = 1 / sqrt(2π) * exp(-(0.5-0)^2 / 2)
= 0.352
P(W1=0.5|Θ=1) = 1 / sqrt(2π) * exp(-(0.5-1)^2 / 2)
= 0.193
P(X1) = 0.352 * 0.9 + 0.193 * 0.1
≈ 0.336
2. Calculate P(Θ=0|X1=0.5):
P(Θ=0|X1=0.5) = (P(X1=0.5|Θ=0) * P(Θ=0)) / P(X1)
= (0.352 * 0.9) / 0.336
≈ 0.941
3. Calculate P(Θ=1|X1=0.5):
P(Θ=1|X1=0.5) = (P(X1=0.5|Θ=1) * P(Θ=1)) / P(X1)
= (0.193 * 0.1) / 0.336
≈ 0.057
Therefore:
P(Θ=0∣X1=0.5) ≈ 0.941
P(Θ=1∣X1=0.5) ≈ 0.057
Next, we need to calculate θˆLMS.
4. Calculate θˆLMS:
θˆLMS = E[Θ|X1]
= Θ * P(Θ|X1)
= 0 * P(Θ=0|X1=0.5) + 1 * P(Θ=1|X1=0.5)
≈ 0.057
Finally, we can calculate the MSE of the LMS estimator:
5. Calculate the conditional mean squared error (MSE):
MSE = E[(θˆLMS-Θ)^2]
= (θˆLMS - 0)^2 * P(Θ=0|X1=0.5) + (θˆLMS - 1)^2 * P(Θ=1|X1=0.5)
= (0.057 - 0)^2 * 0.941 + (0.057 - 1)^2 * 0.057
≈ 0.430