In this problem we will consider a model which has a two-dimensional parameter. Then you will calculate Jeffrey's prior using the Fisher information matrix.
Suppose that X_1, \ldots , X_ n \stackrel{iid}{\sim } N(\mu ,\sigma ^2), where \mu and \sigma ^2 are unknown. In this case, the Fisher information matrix I(\theta ) for \theta =(\mu ,\sigma ^2)^ T will be a 2\times 2 matrix, where the off-diagonal entries are 0.
Find (I(\theta ))_{11}.
n/theta^2
unanswered
Find (I(\theta ))_{22}.
n/(2*theta^4)
unanswered
[Math Processing Error]
Using your answers to the previous part, determine Jeffreys prior, \pi (\theta ), in terms of \mu and \sigma. Express your answer in such a form that \pi ((1, 1)^ T) = 1.
To find (I(θ))11, we need to determine the first diagonal entry of the Fisher information matrix.
The Fisher information matrix for a two-dimensional parameter θ = (μ, σ^2)^T is given by:
I(θ) = [[I11, I12],
[I21, I22]]
where Iij is the (i,j)th entry of the matrix.
In this case, we have:
I(θ) = [[n/σ^2, 0],
[0, n/(2σ^4)]]
So, (I(θ))11 = n/σ^2.
To find (I(θ))22, we need to determine the second diagonal entry of the Fisher information matrix.
In this case, (I(θ))22 = n/(2σ^4).
Now, to determine Jeffrey's prior, we need to calculate the square root of the determinant of the Fisher information matrix:
√(det(I(θ))) = √(n/σ^2 * n/(2σ^4)) = √(n^2/(2σ^6)) = n/(√(2)σ^3)
Jeffrey's prior, π(θ), is then given by:
π(θ) = k * √(det(I(θ)))
where k is a constant.
To satisfy π((1, 1)^T) = 1, we substitute μ = 1 and σ = 1 into the expression for π(θ):
π((1, 1)^T) = k * √(1/(√(2))) = k/(√(2)) = 1
Solving for k, we find:
k = √(2)
Therefore, Jeffrey's prior, π(θ), in terms of μ and σ, is:
π(θ) = √(2) * √(det(I(θ))) = √(2) * (n/(√(2)σ^3)) = n/σ^3.
To find (I(θ))11, we need to calculate the first diagonal entry of the Fisher information matrix for the parameter vector θ = (μ, σ^2)^T.
The Fisher information matrix is given by:
I(θ) = [ E[(∂/∂μ) log f(X|θ) * (∂/∂μ) log f(X|θ)], E[(∂/∂μ) log f(X|θ) * (∂/∂σ^2) log f(X|θ)]
E[(∂/∂σ^2) log f(X|θ) * (∂/∂μ) log f(X|θ)], E[(∂/∂σ^2) log f(X|θ) * (∂/∂σ^2) log f(X|θ)] ]
where f(X|θ) is the probability density function of the random variables X_1, ..., X_n.
In this case, we have n independent and identically distributed (iid) observations from a normal distribution N(μ, σ^2). The probability density function is given by:
f(X|θ) = (1 / sqrt(2πσ^2))^n * exp(- ∑[(Xi - μ)^2 / (2σ^2)])
Taking the log of f(X|θ), we have:
log f(X|θ) = - (n/2) log(2πσ^2) - ∑[(Xi - μ)^2 / (2σ^2)]
Differentiating log f(X|θ) with respect to μ, we get:
(∂/∂μ) log f(X|θ) = ∑[(Xi - μ) / σ^2]
Now, we need to calculate the expected value of (∂/∂μ) log f(X|θ) * (∂/∂μ) log f(X|θ). Since X_1, ..., X_n are iid, the expected value can be calculated as:
E[(∂/∂μ) log f(X|θ) * (∂/∂μ) log f(X|θ)] = E[∑[(Xi - μ) / σ^2] * ∑[(Xj - μ) / σ^2]]
Taking the expectation inside the double summation, we get:
E[(∂/∂μ) log f(X|θ) * (∂/∂μ) log f(X|θ)] = ∑ E[(Xi - μ)^2 / σ^4]
Since E[(Xi - μ)^2] = σ^2 (the variance of X_i), we have:
E[(∂/∂μ) log f(X|θ) * (∂/∂μ) log f(X|θ)] = nσ^2 / σ^4 = n/σ^2
Therefore, (I(θ))11 = n/σ^2.
To find (I(θ))22, we need to calculate the second diagonal entry of the Fisher information matrix.
Differentiating log f(X|θ) with respect to σ^2, we get:
(∂/∂σ^2) log f(X|θ) = - (n/2)(1/σ^2) + ∑[(Xi - μ)^2 / (2σ^4)]
Now, we need to calculate the expected value of (∂/∂σ^2) log f(X|θ) * (∂/∂σ^2) log f(X|θ). Using the same approach as before, we get:
E[(∂/∂σ^2) log f(X|θ) * (∂/∂σ^2) log f(X|θ)] = ∑ E[((Xi - μ)^2 / (2σ^4))]
Since E[(Xi - μ)^2] = σ^2 (the variance of X_i), we have:
E[(∂/∂σ^2) log f(X|θ) * (∂/∂σ^2) log f(X|θ)] = nσ^2 / (2σ^4) = n/(2σ^2)
Therefore, (I(θ))22 = n/(2σ^2).
Now, to determine Jeffreys prior π(θ), we use the formula:
π(θ) = sqrt(det(I(θ))) = sqrt((I(θ))11 * (I(θ))22)
Substituting the values we calculated earlier:
π(θ) = sqrt((n/σ^2) * (n/(2σ^2))) = sqrt((n^2)/(2σ^4))
To express the prior in terms of μ and σ in a way that π((1, 1)^T) = 1, we can substitute μ = 1 and σ = 1:
π((μ, σ)) = sqrt((n^2)/(2σ^4)) = sqrt((n^2)/(2 * 1^4)) = sqrt(n^2/2)
Therefore, Jeffreys prior π((μ, σ)) = sqrt(n^2/2).