a logo for the course

Matrix Computations

David Gleich

Purdue University

Fall 2017

Course number CS-51500

Tuesday and Thursday, 10:30-11:45am

Location Forney B124


In this lecture, we'll look into the inverse of a matrix, and find out which matrices are invertible. These notes roughly follow Trefethen and Bau, section 1.


Let . Recall that

We call full rank if .

Full rank matrices

Full rank matrices have the important property that they give rise to one-to-one maps between and . Let's show this.

Theorem (From Trefethen and Bau, Theorem 1.2) Let , a matrix is full rank if and only if it maps no two distinct vectors to the same vector.

Proof If a matrix is full rank, then it has linearly independent column vectors. These vectors are a basis for . This, in turn, implies that any vector in has a unique representation in this basis. (If not, then and so has linearly dependent columns, which it can't!) Thus, any vector corresponds to a unique .

We also have to prove the reverse direction, but this is easier to prove via the contrapositive. If is not full rank, then it's columns are linearly dependent. Hence, there exists a vector such that . Let be any vector in , then and so we have two distinct vectors that give us the same result.

There's a great picture I could put here, but it's too tricky. The point is that we have a one-to-one map between and , which is a subset of when . Because this map is one-to-one, it's invertible! So we can take any vector and find for some .

Linear systems

It's worth repeating this equation because it's so fundamental to the rest of the class -- and the entire field.

We call a linear system of equations.

Usually these are defined with squares matrices .

Square, full rank matrices

Let be a full-rank matrix. What we've shown above is that any vector in can be written as for some unique .

Thus, we can find the following vectors: If we write this as a matrix equation, we have:

The matrix is called the inverse and usually written .

The matrix inverse

We've shown that and full rank has an inverse such that Let's study a few properties of this inverse.

First, does too? We'll show this is the case. Let and let . Then but also Thus, .

Second, assuming that and are square. The standard way to check that you have the inverse of a particular matrix is just to show that it satifies . In this case: where we've canceled all the inverse pairs represented by parentheses.

When is a matrix full rank?

The following set of statements from Trefthen and Bau helps to characterize when a matrix has an inverse. Let , these statements are all equivalent to each other:

  1. has an inverse
  2. has rank
  3. .
  4. (not fully covered) (null is the nullspace).
  5. (not fully covered) 0 is not an eigenvalue of
  6. (not fully covered) 0 is not a singular value of
  7. (not fully covered)

Solving a linear system

Let be full-rank. Then the linear system: has solution

Be warned, this is not a good way to find unless is very special. In this class, we will see many superior procedures to find that satifies this linear system. A good way to demonstrate that you have not learned the material is to utilize this idea in your programs.