Although the \, X_ i \, are discrete, we can also use a logistic regression model to analyze the data. That is, now we assume
Y_ i | X_ i \sim \textsf{Ber}\left( \frac{1}{1 + \mathbf e^{-(X_ i \beta _1 + \beta _0})} \right),
for \, \beta _0, \beta _1 \in \mathbb {R} \,, and that given \, X_ i \,, the \, Y_ i \, are independent.
Calculate the maximum likelihood estimator \, \widehat{\beta }_0 \,, \, \widehat{\beta }_1 \, for \, \beta _0 \, and \, \beta _1 \,, where we again assume that all \, f_{kl} > 0 \,. Express your answer in terms of f_{00} (enter “A"), f_{01} (enter “B"), f_{10} (enter “C"), f_{11} (enter “D") and n.
\widehat{\beta }_{0}
unanswered
\widehat{\beta }_{1}
To calculate the maximum likelihood estimators for β₀ and β₁, we need to find values that maximize the likelihood function. The likelihood function is the joint probability of the observed Yᵢ given the values of Xᵢ and the parameters β₀ and β₁.
Let's denote the observed values of Yᵢ as yᵢ and the corresponding Xᵢ as xᵢ. Since Yᵢ follows a Bernoulli distribution with parameter pᵢ = 1 / (1 + exp(-(xᵢβ₁ + β₀))), the likelihood function can be written as:
L(β₀, β₁) = Π [ pᵢ^(yᵢ) * (1 - pᵢ)^(1 - yᵢ) ]
To maximize this function, it's easier to maximize the log-likelihood function:
log(L(β₀, β₁)) = Σ [ yᵢ * log(pᵢ) + (1 - yᵢ) * log(1 - pᵢ) ]
Now, let's differentiate this log-likelihood function with respect to β₀ and β₁ and set the derivatives equal to zero to find the maximum likelihood estimators:
∂(log(L))/∂β₀ = Σ [ (yᵢ - pᵢ) ]
∂(log(L))/∂β₁ = Σ [ xᵢ(yᵢ - pᵢ) ]
Setting both of these derivatives equal to zero gives us the following equations:
Σ [ (yᵢ - pᵢ) ] = 0
Σ [ xᵢ(yᵢ - pᵢ) ] = 0
Using these equations, we can solve for the maximum likelihood estimators, denoted as ĥ₀ and ĥ₁:
Let's substitute for pᵢ using the expression pᵢ = 1 / (1 + exp(-(xᵢβ₁ + β₀))):
Σ [ (yᵢ - 1 / (1 + exp(-(xᵢβ₁ + β₀)))) ] = 0
Σ [ xᵢ(yᵢ - 1 / (1 + exp(-(xᵢβ₁ + β₀)))) ] = 0
Now, let's express the sum in terms of the frequencies f₀₀, f₀₁, f₁₀, f₁₁, and n:
Σ [ (yᵢ - 1 / (1 + exp(-(xᵢβ₁ + β₀)))) ] = f₀₁ - (f₁₀ + f₁₁) / (1 + exp(-β₀)) - (f₀₁ + f₁₁) / (1 + exp(-β₀ - β₁))
Σ [ xᵢ(yᵢ - 1 / (1 + exp(-(xᵢβ₁ + β₀)))) ] = f₁₁ - (f₀₁ + f₁₁) * Σ [ xᵢ / (1 + exp(-(xᵢβ₁ + β₀))) ]
Setting both equations equal to zero, we get:
f₀₁ - (f₁₀ + f₁₁) / (1 + exp(-ĥ₀)) - (f₀₁ + f₁₁) / (1 + exp(-ĥ₀ - ĥ₁)) = 0
f₁₁ - (f₀₁ + f₁₁) * Σ [ xᵢ / (1 + exp(-(xᵢĥ₁ + ĥ₀))) ] = 0
These are the equations for estimating ĥ₀ and ĥ₁.
To find the maximum likelihood estimators for β₀ and β₁, we need to maximize the likelihood function based on the given assumptions.
The likelihood function can be written as:
L(β₀, β₁) = ∏[f₀₀(1 + e^(β₁) / (1 + e^(β₀ + β₁)), f₀₁(1 / (1 + e^(β₀ + β₁))), f₁₀(e^(β₀ + β₁) / (1 + e^(β₀ + β₁))), f₁₁(1 + e^(β₀ + β₁)) / (1 + e^(β₀ + β₁)))]
Taking the logarithm of the likelihood function, we have:
log L(β₀, β₁) = ∑[log(f₀₀) + f₀₁(log(1) - log(1 + e^(β₀ + β₁))) + f₁₀(log(1 + e^(β₀ + β₁)) - log(1)) + log(f₁₁)]
To find the maximum likelihood estimators, we need to find the values of β₀ and β₁ that maximize log L(β₀, β₁). This can be done either analytically or numerically using optimization techniques such as gradient descent or Newton's method.
Analytically solving for the maximum likelihood estimators involves finding the values of β₀ and β₁ that satisfy the first-order conditions (∂/∂β₀) log L(β₀, β₁) = 0 and (∂/∂β₁) log L(β₀, β₁) = 0.
However, since the equation is nonlinear and involves exponential functions, it may be more practical to use numerical methods to obtain an approximate solution.
Therefore, the maximum likelihood estimators for β₀ and β₁ are best obtained using optimization algorithms.