Let \mathrm{{\boldsymbol X}}_1, \ldots , \mathrm{{\boldsymbol X}}_ n \in \mathbb {R}^ d denote a data set and let
\mathbb {X} = \begin{pmatrix} \longleftarrow & \mathbf{X}_1^ T & \longrightarrow \\ \longleftarrow & \mathbf{X}_2^ T & \longrightarrow \\ \vdots & \vdots & \vdots \\ \longleftarrow & \mathbf{X}_ n^ T & \longrightarrow \\ \end{pmatrix}.
Recall that the empirical covariance matrix S of this data set can be expressed as
S = \frac{1}{n} \mathbb {X}^ T H \mathbb {X}
where
H = I_ n - \frac{1}{n} \mathbf{1} \mathbf{1}^ T.
The matrix H \in \mathbb {R}^{n \times n} is an orthogonal projection .
In general, we say that a matrix M is an orthogonal projection onto a subspace S if
M is symmetric,
M^2 = M, and
S = \{ \mathrm{{\boldsymbol y}} : \, M \mathbf{x}= y \, \, \text {for some} \, \, \mathbf{x}\in \mathbb {R}^ n \}
Which of the following are true about the matrix H? (Choose all that apply.)
For any positive integer k and any vector \mathbf{x}\in \mathbb {R}^ n, we have H^ k \mathbf{x}= H \mathbf{x}.
For any positive integer k and any vector \mathbf{x}\in \mathbb {R}^ n, we have H^ k \mathbf{x}= \mathbf{x}.
The matrix H is a projection onto the subspace of vectors perpendicular to the vector \mathbf{1} \in \mathbb {R}^ n, which has all of its entries equal to 1.
The matrix H is a projections onto the subspace \{ \mathbf{x}: \frac{1}{n} \sum _{i = 1}^ n \mathbf{x}^ i = 0\} \subset \mathbb {R}^ n. (In other words, this is the set of vectors having coordinate-wise average equal to 0.)
The correct statements are:
1. For any positive integer k and any vector 𝐱∈ℝ𝑛 , we have 𝐻^𝑘𝐱=𝐻𝐱.
This is true because 𝐻 is an orthogonal projection, so raising it to any power does not change its action on vectors.
3. The matrix 𝐻 is a projection onto the subspace of vectors perpendicular to the vector 𝟏∈ℝ𝑛 , which has all of its entries equal to 1.
This is true because 𝐻 subtracts the average of the vector from each component, effectively making the vector perpendicular to the constant vector 𝟏 .
4. The matrix 𝐻 is a projection onto the subspace {𝐱:1𝑛∑𝑖=1𝑛𝐱𝑖=0}⊂ℝ𝑛 .
(In other words, this is the set of vectors having coordinate-wise average equal to 0.)
This is true because the matrix 𝐻 subtracts the average of the vector from each component, effectively ensuring that the coordinate-wise average of the resulting vector is zero.
To determine which of the statements are true about the matrix H, let's analyze each statement one by one:
1. For any positive integer k and any vector x∈R^n, we have H^kx=Hx.
This statement is true. To verify this, notice that H is an orthogonal projection matrix, which means it is idempotent (H^2 = H) and symmetric. Therefore, raising H to any power k (k > 0) will yield the same matrix H. Hence, H^kx = Hx.
2. For any positive integer k and any vector x∈R^n, we have H^kx=x.
This statement is false. Since H is an orthogonal projection matrix, H^kx will always project vector x onto a subspace perpendicular to vector 1 ∈ R^n. Therefore, H^kx will not equal x in general.
3. The matrix H is a projection onto the subspace of vectors perpendicular to the vector 1∈R^n, which has all of its entries equal to 1.
This statement is true. The matrix H is defined as H = I_n − (1/n)11^T, where 1 is a vector of all ones. Multiplying H by any vector x will give a projection of x onto the subspace of vectors that are perpendicular to the vector 1, i.e., vectors that sums to zero when dotted with vector 1. This makes H a projection onto the subspace of vectors perpendicular to vector 1.
4. The matrix H is a projection onto the subspace {x: (1/n)∑_{i=1}^n xi = 0} ⊆ R^n.
This statement is true. As mentioned in the previous statement, H is a projection onto the subspace of vectors perpendicular to vector 1. This subspace corresponds to the set of vectors {x: (1/n)∑_{i=1}^n xi = 0}, which are the vectors with coordinate-wise average equal to zero. Therefore, H is indeed a projection onto this subspace.
In conclusion, the following statements are true:
- For any positive integer k and any vector x∈R^n, we have H^kx=Hx.
- The matrix H is a projection onto the subspace of vectors perpendicular to the vector 1∈R^n, which has all of its entries equal to 1.
- The matrix H is a projection onto the subspace {x: (1/n)∑_{i=1}^n xi = 0} ⊆ R^n.
To determine which of the given statements are true about the matrix H, we can go through each statement one by one.
Statement 1: For any positive integer k and any vector x ∈ R^n, we have H^kx = Hx.
This statement is true. The matrix H is an orthogonal projection matrix, which means it satisfies the property H^2 = H. Therefore, by applying this property repeatedly, H^kx = H(H(H...(Hx))) = Hx, regardless of the positive integer k.
Statement 2: For any positive integer k and any vector x ∈ R^n, we have H^kx = x.
This statement is not true. As explained in the previous statement, H^kx = Hx, so it is not equal to x unless H is the identity matrix. In general, H != I, so H^kx will not be equal to x.
Statement 3: The matrix H is a projection onto the subspace of vectors perpendicular to the vector 1 ∈ R^n, which has all of its entries equal to 1.
This statement is true. The matrix H is defined as H = I_n - 1/n11^T, where 1 is a vector with all entries equal to 1. In this case, H is a projection matrix that projects vectors onto the subspace orthogonal to the vector 1. This is because the column space of 1 is the set of vectors whose entries sum to a constant, and subtracting this projection from the identity matrix effectively projects onto the orthogonal subspace.
Statement 4: The matrix H is a projection onto the subspace {x: (1/n)Σ_{i=1}^nxi = 0} ⊂ R^n.
This statement is not true. The matrix H is not a projection onto the subspace described in statement 4. The subspace described is the set of vectors whose coordinate-wise average is equal to 0, but H does not have this property. H is a projection onto the subspace orthogonal to the vector 1, as explained in statement 3.
So, the correct statements are:
- Statement 1: For any positive integer k and any vector x ∈ R^n, we have H^kx = Hx.
- Statement 3: The matrix H is a projection onto the subspace of vectors perpendicular to the vector 1 ∈ R^n, which has all of its entries equal to 1.