Theorem 5

Given a set of variables X₁, X₂, ...,X_p, with nonsingular covariance matrix S, we can always derive a set of uncorrelated variables Y₁, Y₂, ..., Y_p by a set of linear transforamtions corresponding to the principal-axes rotation. The covariance matrix of this new set of variables is the diagonal matrix L = V'SV

Theorem 6

Given a set of variables X₁, X₂, ...,X_p, with nonsingular covariance matrix S_x, a new set of variables Y₁, Y₂, ..., Y_p is defined by the transformation Y' = X'V, where V is an orthogonal matrix. If the covariance matrix of the Y's is S_y, then the following relation holds:

In other words, the quadratic form is invariant under rigid rotation.

Definition

A set of vectors v₁, v₂, ..., v_k is said to be linearly dependent if scalars a₁, a₂,..., a_k, not all zero, can be found such that a₁v₁ + a₂v₂ +...+ a_kv_k =0

On the other hand, if Sa_iv_i = 0 only when a₁ = a₂ =...= a_k = 0, the set of vectors is said to be linearly independent.

Another Definition

The rank of a pxq matrix is the largest number r such that there exists a set of r rows or columns which is linearly independent.

Theorem 7

(a) The rank of a product C = A*B is less than or equal to the rank of A or B, whichever is smaller.

(b) The rank of AA' equals the rank of A'A equals the rank of A.

Yet Another Definition

A principal minor of a matrix A is a submatrix obtained by deleting any number of its corresponding row-and-column pairs.

Example

has principal minors:

Definition

A matrix G, is said to be Gramian if every principal minor determinant is non-negative.

Some Properties

If G is a nonsingular Gramian matix than all principal minor determinants are positive.

If G is a nonsingular Gramian matrix, the G^-1 is also a nonsingular Gramian matrix.

More Properties

If G is a Gramian matrix, then for all vectors x, the quadratic form Q = x'Gx has non-negative values, and is said to be positive semidefinite.

If G is a nonsingular Gramian matrix then for all vectors x, the quadratic form Q = x'Gx has positive values, and is said to be positive definite.

Terminology

The terms positive semi-definite and positive definite are often applied to the Gramian matrix itself, not just to the quadratic form.

Yet More Properties

All the eigenvlaues of Gramian matrices are non-negative. All the eigenvalues of nonsingular Gramian matrices are positive.

Any symmetric matrix whose principal minor determinants are all non-negative is a Gramian matrix (This is the Existence Theorem).

Theorem 8

If the matrix A has l as an eigenvalue and v as the associated eigenvector then the matrix bA has bl as an eigenvalue, with v as the associated eigenvector.

Theorem 9

If the matrix A has l as an eigenvalue and v as the associated eigenvector then he matrix A + cI has (l + c) as an eigenvalue, with v as the associated eigenvector.

Theorem 10

If the matrix A has l as an eigenvalue and v as the associated eigenvector then the matrix A^m has l^m as an eigenvalue, with v as the associated eigenvector.

Theorem 11

If the matrix A has l as an eigenvalue and v as the associated eigenvector then the matrix A^-1 has 1/l as an eigenvalue, with v as the associated eigenvector.

Ed231A Page

UCLA Department of Education