Suppose we have the prior [mathjaxinline]\pi (\lambda ) \sim[/mathjaxinline] [mathjaxinline]\textsf{Gamma}(a, b)[/mathjaxinline] (where [mathjaxinline]a, b > 0[/mathjaxinline], and conditional on [mathjaxinline]\lambda[/mathjaxinline], we have observations [mathjaxinline]X _1[/mathjaxinline], [mathjaxinline]X _2[/mathjaxinline], [mathjaxinline]\cdots[/mathjaxinline], [mathjaxinline]X _{n}[/mathjaxinline] [mathjaxinline]\stackrel{\text {i.i.d}}{\sim }[/mathjaxinline] [mathjaxinline]\textsf{Exp}(\lambda )[/mathjaxinline]. Compute the posterior distribution [mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n)[/mathjaxinline].
Yet again, the posterior distribution for [mathjaxinline]\lambda[/mathjaxinline] is an Gamma distribution. What are its parameters? Enter your answer in terms of [mathjaxinline]a[/mathjaxinline], [mathjaxinline]b[/mathjaxinline], [mathjaxinline]n[/mathjaxinline], and [mathjaxinline]\displaystyle \sum _{i=1}^{n} X_ i[/mathjaxinline].
(Enter Sigma_i(X_i) for [mathjaxinline]\displaystyle \sum _{i=1}^{n} X_ i[/mathjaxinline] . Do not worry if the parser does not render properly; the grader works independently. If you wish to have proper rendering, enclose Sigma_i(X_i) by brackets. )
[mathjaxinline]\theta =[/mathjaxinline]
To compute the posterior distribution [mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n)[/mathjaxinline], we will use Bayes' theorem:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n) = \frac{{\textsf{Gamma}(a, b) f(X_1, X_2, \ldots , X_ n | \lambda)}}{{\int_{0}^{\infty} \textsf{Gamma}(a, b) f(X_1, X_2, \ldots , X_ n | \lambda) \, d\lambda}}[/mathjaxinline]
where [mathjaxinline]f(X_1, X_2, \ldots , X_ n | \lambda)[/mathjaxinline] is the joint density of [mathjaxinline]X_1, X_2, \ldots , X_ n[/mathjaxinline] given [mathjaxinline]\lambda[/mathjaxinline].
Since [mathjaxinline]X_1, X_2, \ldots , X_ n[/mathjaxinline] are i.i.d. exponential random variables with rate [mathjaxinline]\lambda[/mathjaxinline], we have:
[mathjaxinline]f(X_1, X_2, \ldots , X_ n | \lambda) = \prod_{i=1}^{n} \textsf{Exp}(X_i | \lambda) = \prod_{i=1}^{n} \lambda e^{-\lambda X_i} = \lambda^n e^{-\lambda \Sigma_{i=1}^{n} X_i}[/mathjaxinline]
Now, let's substitute these values back into the posterior distribution:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n) = \frac{{\textsf{Gamma}(a, b) \cdot \lambda^n e^{-\lambda \Sigma_{i=1}^{n} X_i}}}{{\int_{0}^{\infty} \textsf{Gamma}(a, b) \cdot \lambda^n e^{-\lambda \Sigma_{i=1}^{n} X_i} \, d\lambda}}[/mathjaxinline]
To simplify the computation of the integral in the denominator, we notice that the expression inside the integral is proportional to the pdf of a gamma distribution with parameters [mathjaxinline]a + n[/mathjaxinline] and [mathjaxinline]b + \Sigma_{i=1}^{n} X_i[/mathjaxinline]. Therefore, we can rewrite the posterior distribution as:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots , X_ n) \propto \textsf{Gamma}(a, b) \cdot \lambda^n e^{-\lambda \Sigma_{i=1}^{n} X_i}[/mathjaxinline]
Thus, the posterior distribution for [mathjaxinline]\lambda[/mathjaxinline] is also a gamma distribution with parameters [mathjaxinline]a + n[/mathjaxinline] and [mathjaxinline]b + \Sigma_{i=1}^{n} X_i[/mathjaxinline]. Therefore, [mathjaxinline]\theta = (a + n, b + \Sigma_{i=1}^{n} X_i )[/mathjaxinline].
To find the parameters of the posterior distribution, we need to update the parameters of the prior distribution using the observed data.
The prior distribution for [mathjaxinline]\lambda[/mathjaxinline] is given by [mathjaxinline]\pi (\lambda ) \sim \textsf{Gamma}(a, b)[/mathjaxinline].
The likelihood function is given by [mathjaxinline]f(X_1, X_2, \ldots , X_n | \lambda) = \prod_{i=1}^n \textsf{Exp}(\lambda) = \lambda^n e^{-\lambda \sum_{i=1}^n X_i}[/mathjaxinline].
The posterior distribution is given by [mathjaxinline]\pi (\lambda | X_1, X_2, \ldots, X_n) \propto f(X_1, X_2, \ldots, X_n | \lambda) \cdot \pi (\lambda)[/mathjaxinline].
Multiplying the likelihood and the prior, we get:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots, X_n) \propto \lambda^n e^{-\lambda \sum_{i=1}^n X_i} \cdot \lambda^{a-1} e^{-b\lambda}[/mathjaxinline].
Simplifying the expression by combining like terms, we have:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots, X_n) \propto \lambda^{n+a-1} e^{-\lambda(\sum_{i=1}^n X_i + b)}[/mathjaxinline].
Now we recognize that this is the kernel of a Gamma distribution.
Comparing the expression to the Gamma probability density function [mathjaxinline]\textsf{Gamma}(\lambda | c, d)[/mathjaxinline]:
[mathjaxinline]\pi (\lambda | X_1, X_2, \ldots, X_n) = \textsf{Gamma}(\lambda | n+a, \sum_{i=1}^n X_i + b)[/mathjaxinline].
Therefore, the posterior distribution for [mathjaxinline]\lambda[/mathjaxinline] is [mathjaxinline]\textsf{Gamma}(n+a, \sum_{i=1}^n X_i + b)[/mathjaxinline].
So, [mathjaxinline]\theta = n+a[/mathjaxinline] and [mathjaxinline]\displaystyle \sum _{i=1}^{n} X_ i + b[/mathjaxinline].