Suppose we have the improper prior [mathjaxinline]\pi (\lambda ) \propto e^{-a\lambda }[/mathjaxinline], [mathjaxinline]\lambda \in \mathbb {R}[/mathjaxinline] (and [mathjaxinline]a \geq 0[/mathjaxinline]). Conditional on [mathjaxinline]\lambda[/mathjaxinline], we have observations [mathjaxinline]X _1[/mathjaxinline], [mathjaxinline]X _2[/mathjaxinline], [mathjaxinline]\cdots[/mathjaxinline], [mathjaxinline]X _{n}[/mathjaxinline] [mathjaxinline]\stackrel{\text {i.i.d}}{\sim }[/mathjaxinline] [mathjaxinline]\textsf{N}(\lambda , 1)[/mathjaxinline]. Compute the posterior distribution [mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n)[/mathjaxinline], then provide the following statistics on the posterior distribution.
Use SumXi for [mathjaxinline]\sum _{i=1}^ n X_ i[/mathjaxinline].
mean:
variance:
[mathjaxinline]q_{0.025}[/mathjaxinline] (cutoff for highest 2.5%):
The posterior distribution can be calculated using Bayes' theorem:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n) = \frac{\pi (\lambda) \cdot L(X_1, X_2, \ldots , X_ n|\lambda)}{\int_{-\infty}^{\infty} \pi (\lambda) \cdot L(X_1, X_2, \ldots , X_ n|\lambda) d\lambda}[/mathjaxinline]
where [mathjaxinline]L(X_1, X_2, \ldots , X_ n|\lambda)[/mathjaxinline] is the likelihood function for [mathjaxinline]\lambda[/mathjaxinline] given the observations [mathjaxinline]X_1, X_2, \ldots , X_ n[/mathjaxinline]. In this case, since the observations are independent and normally distributed with mean [mathjaxinline]\lambda[/mathjaxinline] and variance 1, the likelihood function can be written as:
[mathjaxinline]L(X_1, X_2, \ldots , X_ n|\lambda) = \prod_{i=1}^{n} \frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2}(X_i - \lambda)^2}[/mathjaxinline]
Substituting the prior and likelihood into Bayes' theorem, we have:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n) \propto e^{-a\lambda} \cdot \left(\prod_{i=1}^{n} \frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2}(X_i - \lambda)^2}\right)[/mathjaxinline]
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n) \propto e^{-a\lambda} \cdot e^{-\frac{1}{2} \sum_{i=1}^{n} (X_i - \lambda)^2}[/mathjaxinline]
Simplifying further, we get:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n) \propto e^{-a\lambda -\frac{1}{2} \left(\sum_{i=1}^{n} X_i^2 - 2\lambda \sum_{i=1}^{n} X_i + n\lambda^2\right)}[/mathjaxinline]
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n) \propto e^{-\left(a +\frac{1}{2}\sum_{i=1}^{n} X_i^2 - \lambda \left(\sum_{i=1}^{n} X_i + n\lambda\right)\right)}[/mathjaxinline]
This is a Gaussian distribution, so we can complete the square to find the mean and variance:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n) \propto e^{-\frac{1}{2}(\lambda^2 - 2\lambda \mu + \mu^2/\tau^2)}[/mathjaxinline]
where [mathjaxinline]\mu = \frac{\sum_{i=1}^{n} X_i}{n+1}[/mathjaxinline] and [mathjaxinline]\tau^2 = \frac{1}{n+1+a}[/mathjaxinline].
Therefore, the posterior distribution is a Gaussian distribution with mean [mathjaxinline]\mu[/mathjaxinline] and variance [mathjaxinline]\tau^2[/mathjaxinline]:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n) = \frac{1}{\sqrt{2\pi \tau^2}} e^{-\frac{1}{2\tau^2}(\lambda - \mu)^2}[/mathjaxinline]
Now we can compute the required statistics on the posterior distribution:
Mean: The mean of the posterior distribution is equal to [mathjaxinline]\mu[/mathjaxinline], which is [mathjaxinline]\frac{\sum_{i=1}^{n} X_i}{n+1}[/mathjaxinline].
Variance: The variance of the posterior distribution is equal to [mathjaxinline]\tau^2[/mathjaxinline], which is [mathjaxinline]\frac{1}{n+1+a}[/mathjaxinline].
Cutoff for highest 2.5% ([mathjaxinline]q_{0.025}[/mathjaxinline]):
To find the cutoff for the highest 2.5% of the distribution, we need to find the [mathjaxinline]\alpha[/mathjaxinline] value such that [mathjaxinline]P(\lambda > \alpha) = 0.025[/mathjaxinline].
Using the cumulative distribution function (CDF) of the Gaussian distribution, we can find the cutoff value [mathjaxinline]\alpha[/mathjaxinline] such that [mathjaxinline]P(\lambda \leq \alpha) = 1 - 0.025 = 0.975[/mathjaxinline].
Since the CDF of a Gaussian distribution with mean [mathjaxinline]\mu[/mathjaxinline] and variance [mathjaxinline]\tau^2[/mathjaxinline] is given by [mathjaxinline]\Phi\left(\frac{\lambda - \mu}{\sqrt{\tau^2}}\right)[/mathjaxinline], where [mathjaxinline]\Phi[/mathjaxinline] is the standard normal CDF, we need to find [mathjaxinline]\alpha[/mathjaxinline] such that [mathjaxinline]\Phi\left(\frac{\alpha - \mu}{\sqrt{\tau^2}}\right) = 0.975[/mathjaxinline].
Using inverse of the standard normal CDF, we can find [mathjaxinline]\alpha[/mathjaxinline]:
[mathjaxinline]\frac{\alpha - \mu}{\sqrt{\tau^2}} = \Phi^{-1}(0.975)[/mathjaxinline]
Finally, plug in the values of [mathjaxinline]\mu[/mathjaxinline], [mathjaxinline]\tau^2[/mathjaxinline], and [mathjaxinline]\Phi^{-1}(0.975)[/mathjaxinline] to get the specific values for mean, variance, and [mathjaxinline]q_{0.025}[/mathjaxinline].