In this problem, we study a simple noisy communication channel. Suppose that X is a binary signal that takes value -1 and 1 with equal probability. This signal X is sent through a noisy communication channel, and the medium of transmission adds an independent noise term. More precisely, the received signal is Y=X+N, where N is standard normal, indpendendent of X.
The decoder receives the value y of Y, and decides whether X was 1 or -1, using the following decoding rule: it decides in favor of 1 if and only if
\text {}\quad \mathbf{P}(X=1|Y=y)>2\mathbf{P}(X=-1|Y=y).
It turns out that the decoding rule can be expressed in the form: decide in favor of 1 if and only if Y>t, for some threshold t. Find the threshhold t.
As an intermediate step, find p_1\triangleq \mathbf{P}(X=1|Y=y).
Now find t.
To find the threshold t, we need to first find p_1, which is the probability that X=1 given Y=y.
Using Bayes' theorem, we can express p_1 as:
p_1 = P(X=1|Y=y) = (P(Y=y|X=1)*P(X=1)) / P(Y=y)
Since X is equally likely to be -1 or 1, we have P(X=1) = P(X=-1) = 0.5.
To find P(Y=y), we can use the law of total probability:
P(Y=y) = P(Y=y|X=1)*P(X=1) + P(Y=y|X=-1)*P(X=-1)
Since N is standard normal and independent of X, we have P(Y=y|X=1) = P(N=y-1) and P(Y=y|X=-1) = P(N=y+1). The probability density function of a standard normal distribution can be denoted as f(x), so we have:
P(Y=y) = f(y-1)*0.5 + f(y+1)*0.5
Now, substituting these values into the expression for p_1, we have:
p_1 = (P(Y=y|X=1)*P(X=1)) / P(Y=y)
= (P(N=y-1)*0.5) / (f(y-1)*0.5 + f(y+1)*0.5)
= P(N=y-1) / (f(y-1) + f(y+1))
Now, we need to find the threshold t such that p_1 > 2p_-1, where p_-1 = P(X=-1|Y=y).
Using Bayes' theorem, we can express p_-1 as:
p_-1 = P(X=-1|Y=y) = (P(Y=y|X=-1)*P(X=-1)) / P(Y=y)
Similarly as before, we have P(Y=y|X=-1) = P(N=y+1), so we have:
p_-1 = (P(N=y+1)*0.5) / (f(y-1)*0.5 + f(y+1)*0.5)
= P(N=y+1) / (f(y-1) + f(y+1))
Now, we want to find the threshold t such that p_1 > 2p_-1:
p_1 > 2p_-1
P(N=y-1) / (f(y-1) + f(y+1)) > 2*P(N=y+1) / (f(y-1) + f(y+1))
P(N=y-1) > 2P(N=y+1)
f(y-1) > 2f(y+1)
Now, we can solve for t by finding the value of y that satisfies this inequality. Since f(x) is the probability density function of a standard normal distribution, we can use standard tables or calculators to find the value of t.
To find the threshold t, we will first calculate the probability p₁, which is defined as P(X = 1 | Y = y).
Given that Y = X + N, where N is standard normal and independent of X, we can rewrite this as:
P(X = 1 | Y = y) = P(X = 1, Y = y) / P(Y = y)
Using Bayes' theorem, we have:
P(X = 1 | Y = y) = P(Y = y | X = 1) * P(X = 1) / P(Y = y)
Since X takes the values -1 and 1 with equal probability, we have P(X = 1) = P(X = -1) = 1/2.
Now, let's calculate the numerators and denominators separately:
1. Calculation of the numerator P(Y = y | X = 1):
Since Y = X + N, and N is standard normal, we have:
Y = 1 + N (when X = 1)
P(Y = y | X = 1) = P(1 + N = y) = P(N = y - 1)
2. Calculation of the numerator P(Y = y | X = -1):
Since Y = X + N, and N is standard normal, we have:
Y = -1 + N (when X = -1)
P(Y = y | X = -1) = P(-1 + N = y) = P(N = y + 1)
3. Calculation of the denominator P(Y = y):
P(Y = y) = P(Y = y | X = 1) * P(X = 1) + P(Y = y | X = -1) * P(X = -1)
Now, let's substitute the values into the formula for p₁ and calculate it:
p₁ = P(X = 1 | Y = y) = P(N = y - 1) / P(N = y - 1) + P(N = y + 1)
Now that we have p₁, we can find the threshold t.
The decoding rule states that we decide in favor of 1 if and only if P(X = 1 | Y = y) > 2 * P(X = -1 | Y = y).
Since p₁ is P(X = 1 | Y = y), we can rewrite the decoding rule as:
p₁ > 2(1 - p₁)
Simplifying the inequality, we get:
p₁ > 2 - 2p₁
3p₁ > 2
p₁ > 2/3
Since p₁ is the probability, it lies between 0 and 1.
Therefore, we can conclude that the threshold t is 2/3.