How are logistic regression coefficients estimated?
Logistic regression coefficients are estimated using maximum likelihood estimation (MLE) or iteratively reweighted least squares (IRLS) algorithm.
In MLE, the goal is to maximize the likelihood function, which represents the probability of observing the given data sample under the logistic regression model. The model parameters, i.e., the coefficients, are adjusted iteratively to maximize the likelihood. This is typically done using optimization techniques such as gradient descent.
IRLS is an iterative algorithm that updates the coefficient estimates to fit the model to the data. It works by approximating the likelihood function through a series of weighted linear regression models. In each iteration, it computes the estimates by assuming linear relationship between the predictors and their log-odds ratios. The weights are adjusted based on the residuals and the estimated probabilities from the previous iteration.
Overall, both MLE and IRLS aim to find the coefficient values that best fit the logistic regression model to the given data by maximizing the likelihood or minimizing the deviance of the observed outcomes from the predicted probabilities.