Consider the distribution [mathjaxinline]\text {Ber}(0.25)[/mathjaxinline]. Consider the categorical statistical model [mathjaxinline]\left( \{ a_1, \ldots , a_ K \} , \{ \mathbf{P}_{\mathbf{p}}\} \right)[/mathjaxinline] for this Bernoulli distribution.
If we let [mathjaxinline]a_1 = 1[/mathjaxinline] and [mathjaxinline]a_2 = 0[/mathjaxinline], then this corresponds to a categorical distribution [mathjaxinline]\mathbf{P}_{\mathbf{p}}[/mathjaxinline] with parameter vector [mathjaxinline]\mathbf{p}[/mathjaxinline] given by...
[mathjaxinline]0.25[/mathjaxinline]
[mathjaxinline]0.75[/mathjaxinline]
[mathjaxinline][0.25~ ~ 0.75]^ T[/mathjaxinline]
[mathjaxinline][0.75~ ~ 0.25]^ T[/mathjaxinline]
correct
Let [mathjaxinline]a_ i = i[/mathjaxinline] for [mathjaxinline]i = 1, \ldots , K[/mathjaxinline]. The uniform distribution on [mathjaxinline]E = \{ 1, 2, \ldots , K\}[/mathjaxinline] can be expressed as a categorical distribution [mathjaxinline]\mathbf{P}_{\mathbf{p}}[/mathjaxinline] for some choice of parameter [mathjaxinline]\mathbf{p}[/mathjaxinline].
What is [mathjaxinline]\sum _{i = 1}^ K p_ i^2[/mathjaxinline]?
The uniform distribution on a set of size [mathjaxinline]K[/mathjaxinline] corresponds to a categorical distribution with [mathjaxinline]K[/mathjaxinline] categories, where each category has equal probability. Therefore, the parameter vector [mathjaxinline]\mathbf{p}[/mathjaxinline] for the uniform distribution is given by [mathjaxinline]\mathbf{p} = \left(\frac{1}{K}, \frac{1}{K}, \ldots, \frac{1}{K}\right)[/mathjaxinline].
Substituting this into the expression for [mathjaxinline]\sum _{i = 1}^ K p_ i^2[/mathjaxinline]:
[mathjaxinline]\sum _{i = 1}^ K p_ i^2 = \sum _{i = 1}^ K \left(\frac{1}{K}\right)^2 = K \cdot \frac{1}{K^2} = \frac{1}{K}[/mathjaxinline].
Therefore, [mathjaxinline]\sum _{i = 1}^ K p_ i^2 = \frac{1}{K}[/mathjaxinline].
The sum of squares of the parameter vector [mathjaxinline]\mathbf{p}[/mathjaxinline] can be calculated using the formula [mathjaxinline]\sum _{i = 1}^ K p_ i^2[/mathjaxinline].
Since we have a uniform distribution, which means all the probabilities are equal, the parameter vector [mathjaxinline]\mathbf{p}[/mathjaxinline] will be a vector of equal probabilities.
Let's assume the uniform distribution has [mathjaxinline]K[/mathjaxinline] categories. In this case, the parameter vector [mathjaxinline]\mathbf{p}[/mathjaxinline] will have [mathjaxinline]K[/mathjaxinline] equal probabilities, each denoted by [mathjaxinline]p_i[/mathjaxinline].
Since the sum of probabilities in a probability distribution should be equal to 1, we have:
[mathjaxinline]p_1 + p_2 + \ldots + p_K = 1[/mathjaxinline]
Since all the probabilities are equal, we can rewrite the equation as:
[mathjaxinline]K \cdot p_i = 1[/mathjaxinline]
Solving for [mathjaxinline]p_i[/mathjaxinline], we get:
[mathjaxinline]p_i = \frac{1}{K}[/mathjaxinline]
Substituting this result back into the sum of squares formula, we get:
[mathjaxinline]\sum _{i = 1}^ K \left (\frac{1}{K} \right )^2 = \sum _{i = 1}^ K \frac{1}{K^2}[/mathjaxinline]
Since there are [mathjaxinline]K[/mathjaxinline] terms in the summation, we can rewrite it as:
[mathjaxinline]K \cdot \frac{1}{K^2} = \frac{1}{K}[/mathjaxinline]
Therefore, [mathjaxinline]\sum _{i = 1}^ K p_ i^2 = \frac{1}{K}[/mathjaxinline].