Let Θ1 and Θ2 be some unobserved Bernoulli random variables and let X be an observation. Conditional on X=x, the posterior joint PMF of Θ1 and Θ2 is given by
pΘ1,Θ2∣X(θ1,θ2∣x)=
0.26, if θ1=0,θ2=0,
0.26, if θ1=0,θ2=1,
0.21, if θ1=1,θ2=0,
0.27, if θ1=1,θ2=1,
0, otherwise.
We can view this as a hypothesis testing problem where we choose between four alternative hypotheses: the four possible values of (Θ1,Θ2).
a) What is the estimate of (Θ1,Θ2) provided by the MAP rule?
b) Once you calculate the estimate (θ^1,θ^2) of (Θ1,Θ2), you may report the first component, θ^1, as your estimate of Θ1. With this procedure, your estimate of Θ1 will be
c) What is the probability that Θ1 is estimated incorrectly (the probability of error) when you use the procedure in part (b)?
d) What is the MAP estimate of Θ1 based on X, that is, the one that maximizes pΘ1|X(θ1∣x)?
e) The moral of this example is that an estimate of Θ1 obtained by identifying the maximum of the joint PMF of all unknown random variables is
the MAP estimate of Θ1.
a) (1,1)
b) 1
c) 0.52
d) 0
e) can be different from
a) The estimate of (Θ1,Θ2) provided by the MAP rule is (θ1=1, θ2=1), because it has the highest posterior probability of 0.27.
b) Once I calculate the estimate (θ^1,θ^2) of (Θ1,Θ2), I can report the first component, θ^1, as my estimate of Θ1. So, my estimate of Θ1 will be θ^1 = 1.
c) The probability that Θ1 is estimated incorrectly (the probability of error) when using the procedure in part (b) is the sum of the probabilities when Θ1 is estimated as 0. Since the joint PMF gives 0.26 probability for (θ1=0, θ2=0) and 0.26 probability for (θ1=0, θ2=1), the probability of error is 0.26 + 0.26 = 0.52.
d) The MAP estimate of Θ1 based on X, that maximizes pΘ1|X(θ1|x), is θ1 = 1, because pΘ1|X(1|x) = 0.21 + 0.27 = 0.48, which is the highest among the probabilities for θ1.
e) The moral of this example is that an estimate of Θ1 obtained by identifying the maximum of the joint PMF of all unknown random variables is not necessarily the MAP estimate of Θ1.
a) The Maximum A Posteriori (MAP) rule estimates the values of Θ1 and Θ2 that maximize the posterior joint PMF. From the given probabilities, we can see that the maximum probability occurs when θ1=1 and θ2=1. Therefore, the MAP estimate of (Θ1,Θ2) is (1, 1).
b) Since the MAP estimate of (Θ1,Θ2) is (1, 1), we can report the first component, θ^1, as the estimate of Θ1. Therefore, the estimate of Θ1 is 1.
c) To calculate the probability of error, we need to sum the probabilities of all incorrect estimates of Θ1. From the given probabilities, we see that the only incorrect estimate is when θ1=0. The probability of this incorrect estimate is 0.26 + 0.21 = 0.47. Therefore, the probability that Θ1 is estimated incorrectly is 0.47.
d) The MAP estimate of Θ1 based on X can be obtained by maximizing the posterior distribution p(θ1|X=x). From the given probabilities, we can see that the maximum probability occurs when θ1=1. Therefore, the MAP estimate of Θ1 based on X is θ^1 = 1.
e) The moral of this example is that an estimate of Θ1 obtained by identifying the maximum of the joint PMF of all unknown random variables is the MAP estimate of Θ1. In other words, the MAP estimate considers both the observed data (X) and the prior information (priors on Θ1 and Θ2) in order to provide the most likely estimate.
a) The Maximum A Posteriori (MAP) rule estimates the values of (Θ1,Θ2) based on the posterior joint PMF. To find the MAP estimate, we look for the combination of (θ1,θ2) that maximizes the posterior joint PMF pΘ1,Θ2∣X(θ1,θ2∣x).
From the given posterior joint PMF, we see that the combination (θ1=1, θ2=1) has the highest probability of 0.27. Therefore, the estimate provided by the MAP rule is (Θ1,Θ2) = (1, 1).
b) Once we have the estimate (θ^1,θ^2) of (Θ1,Θ2), we can report the first component, θ^1, as the estimate of Θ1. From part (a), the estimate provided by the MAP rule is (Θ1,Θ2) = (1, 1). So, the estimate of Θ1 is θ^1 = 1.
c) To find the probability that Θ1 is estimated incorrectly, we need to consider the cases when the estimated value of Θ1 is different from its actual value. From part (b), we have estimated Θ1 = θ^1 = 1.
From the given posterior joint PMF, we see that the probability of estimating Θ1 = 1 correctly is 0.27, and the probability of estimating Θ1 = 0 incorrectly is 0.26. Therefore, the probability of estimating Θ1 incorrectly is 0.26.
d) The MAP estimate of Θ1 based on X is the value of θ1 that maximizes the conditional PMF pΘ1|X(θ1∣x).
From the given posterior joint PMF, the conditional PMF pΘ1|X(θ1∣x) can be calculated by dividing each entry of the posterior joint PMF by the sum of the entries for the corresponding value of θ1.
For θ1 = 0, the sum of the entries is 0.26 + 0.26 = 0.52, giving pΘ1|X(0∣x) = 0.26/0.52 = 0.5.
For θ1 = 1, the sum of the entries is 0.21 + 0.27 = 0.48, giving pΘ1|X(1∣x) = (0.21 + 0.27)/0.48 = 0.96/0.48 = 2.
Therefore, the MAP estimate of Θ1 based on X is the value of θ1 that maximizes pΘ1|X(θ1∣x), which is θ1 = 1.
e) The moral of this example is that an estimate of Θ1 obtained by identifying the maximum of the joint PMF of all unknown random variables is the MAP estimate of Θ1. In other words, the MAP estimate is the value that maximizes the posterior probability given the observed data. It takes into account both the prior information (represented by the prior PMF) and the likelihood of the observed data to provide a robust estimate.
a) (1,1)
b) 1
c) 0.52
d) 1
e) is always the same as