Scientific Computing Using Python - PHYS:4905 - Fall 2018

Lecture #11 - 9/27/2018 - Prof. Kaaret

These notes borrow from Linear Algebra by Cherney, Denton, Thomas, and Waldron.


Inverse of a 2×2 matrix

If we have a 2×2 matrix  M=(abcd)M = \begin{pmatrix} a & b \\ c & d \end{pmatrix}  then its inverse is   M-1=1ad-bc(d-b-ca)M^{-1} = \frac{1}{ad-bc} \begin{pmatrix} d & -b \\ -c & a \end{pmatrix}.

Some of you may already know this formula.  Or you can verify that it is correct by doing the matrix multiplication.

Here, we write it down because it is useful for figuring out when a 2×2 matrix has an inverse.  The matrix will have an inverse as long as the expression in the denominator of the fraction does not equal zero, specifically ad-bc≠0ad - bc \neq 0.  We call that expressing the determinant.  (Mostly because it determines whether or not the matrix has an inverse.)

Determinant

If we write the determinant for a 2×2 matrix using the index notation for the matrix elements, we have

det(M)=m11m22-m21m12\det(M) = m_1^1 m_2^2 - m_2^1 m_1^2
The inverse of M exists as long as det(M)≠0\det(M) \neq 0.

How do we generalize this to more dimensions?

First, we need a little background thinking on permutations or ways to re-arranged things. 

Let's consider an ordered set of 3 objects, that we'll call (1, 2, 3).  How many ways can we shuffle those 3 objects?

We can build up all the different arrangements by swapping pairs of objects.
Start with (1, 2, 3). 
Swap the last two to get (1, 3, 2) - odd.
Swap the first two to get (3, 1, 2) - even.
Swap the last two to get (3, 2, 1) - odd.
Swap the first two to get (2, 3, 1) - even.
Swap the last two to get (2, 1, 3) - odd.

Can we get any more permutations?  How about swapping the last two? How about swapping the first two? How about swapping the first and the last?

So, we have 6 permutations of 3 objects.  In general, there are n! permutations of n objects since there are n choices for the first object, n-1 choices for the second once the fi rst has been chosen, and so on.

We also define the sign sgn(σ) for the different permutations σ.  The sign is +1 if the permutation is even, meaning it takes an even number of swaps to reach it from the original ordering, and -1 if the permutation is odd.  Above, we recorded the parity (whether odd or even) of the swaps.

Now we can write down an equation for the determinant,

det(M)=∑σsgn(σ)mσ(1)1mσ(2)2⋯mσ(n)n\det(M) = \underset{σ}{∑} sgn(σ) \, m_{σ(1)}^1 \, m_{σ(2)}^2 \, ⋯ \, m_{σ(n)}^n
The sum is over all permutations of the set (1, 2, ... n) where n is the dimension of the matrix.  Each term in the sum for a determinant is a product of n entries, each from a different row of the matrix.  In the different terms, the column indexes are shuffled by the different permutations σ.

Does this work for a 2×2 matrix?

We start with the set (1, 2). The only permutation is (2,1), which is odd.  Note 2! = 2.  We have to sum over σ = (1,2) and (2,1) with the latter coming in with a minus sign.  So, we get

det(M)=+m11m22-m21m12\det(M) = +m_1^1 \, m_2^2 - m_2^1 \, m_1^2which matches what we found above.

How many terms are in the sum for the determinant of a 3×3 matrix?  How about a 5×5 matrix?

Exercise: what is the determinant of a 3×3 matrix?

Fun facts about determinants

If the matrix has a row consisting entirely of zeros, then det(M) = 0.  You can see this by going back to the description of the terms above.  Each term in the sum for a determinant is a product of n entries, each from a different row of the matrix.  If there is a row with all zeros, then every term will have at least one factor that equals zero.  Therefore, the product will be zero.  Since this is true for every term, the sum will also be zero.

If M is diagonal, then the determinant is the product of the diagonal entries, or det(M)=m12m22⋯mnn\det(M) = m_1^2 \, m_2^2 \, ⋯ \, m_n^n.  This is true since all of the summands involving off-diagonal entries are zero.

The determinant of the identity matrix is 1.  All of the diagonal entries are 1, so their product is 1.  This is true regardless of the dimension of the matrix.

If we swap two rows of the matrix, then the determinant changes sign.  This one is best considered in a quiet room.  However, you can easily check that it is true for a 2×2 matrix.

This implies that if two rows of the matrix are identical, then the determinant is zero.  Since zero is the only number that equals its negative.

The determinant of the product of two matrices is the product of the determinants,

det(MN)=det(M)det(N)\det(MN) = \det(M) \, \det(N)
This turns out to be very useful.  For a proof, see section 8.2.4 of the Linear Algebra textbook.

If M is a square matrix, then det(MT)=det(M)\det(M^T) = \det( M)

If the inverse of M exists, then det(M-1)=1det(M)\det(M^{-1}) = \frac{1}{\det(M)}

Since we are fond of elementary row operations, we can figure out the determinants of the corresponding matrices.

Row swap - we did this one above. The determinant of a row swap is -1.

Row multiplication by a scalar - The matrix for this is the identity matrix with one element on the diagonal replaced by a number, λ, not equal to one.  The matrix is diagonal, so the determinant is the product of the diagonal entries, which equal to λ.

Row addition - The matrix for this is the identity matrix with one element off the diagonal replaced by a number, λ, not equal to zero.  The determinant is 1.  This is worked out in section 8.2.3 of the Linear Algebra textbook.


Minors, adjoints, and inverses

A minor of an n×n matrix M is the determinant of any square matrix obtained from M by deleting one row and one column. If we pick entry mjim_j^i, you then delete row i and column j of M and the resulting matrix is the minor associated with that entry.

The cofactor of M corresponding to the entry mjim_j^i  is the product of the minor associated to mjim_j^i  and
(-1)i+j . This is written as cofactor(mji)(m_j^i).

The adjoint matrix of M is found by taking the cofactor of each entry of M and then taking the transpose,

adj(M) = (cofactor(mji)(m_j^i))T

where i and j run from 1 to n.

An example is


Run i and j run from 1 to 3 and check each entry.

All of this is useful because we can use the adjoint along with the determinant to find the inverse,

M-1=1det(M)adj(M)M^{-1} = \frac{1}{\det(M)} adj(M)
This gives an explicit formula for the inverse.  This is sometimes called Cramer's rule.  Cramer's rule is the basis of the formula given at the beginning of the lecture for the inverse of a 2×2 matrix.  That formula is quicker than doing Gaussian elimination.  Finding the inverse of 3×3 matrix using Cramer's rule is also a reasonable task. 

However, let's consider a 5×5 matrix.  The first bit of calculation would be to find the determinant in the denominator of the fraction.  How many terms are there in the sum for the determinant of a 5×5 matrix?  The answer is 120.  Then one would have to find determinants for 25 different 4×4 matrices, each with 24 terms.  Along with the first determinant, we're looking at 720 terms each with 4 or 5 factors.  Cramer's rule rapidly become computationally tedious for large matrices.  Gaussian elimination also become tedious, but less so.  The number of operations needed for Gaussian elimination is usually equal to the number operations needed to calculate the first determinant for Cramer's rule.