Scientific Computing Using Python - PHYS:4905 - Fall 2018
Lecture Notes #20 - 11/8/2018 - Prof. Kaaret
These notes borrow from Linear Algebra
by Cherney, Denton, Thomas, and Waldron.
Standard basis
Our standard notion of length of a vector is the dot product of a
vector with itself
The set of basis vectors for the standard basis written in the
standard basis are
Each of the basis vectors has unit length,
In addition, each of the basis vectors is orthogonal, or
perpendicular, to all of the others,
when
i ≠
j.
To write this more compactly, we introduce the Kronecker delta,
which is a deposit of sediments at the mouth of the Kronecker river
and apparently also a sexibeat.
We use a lower case delta to write the Kronecker delta which is
defined as
The relations above for the basis vectors of the standard basis can
then be summarized as .
The dot product is a special operator for vectors. We can form
an equivalent calculation using standard matrix multiplication by
taking the transpose of the first vector,
This is called the inner product. Sometimes the inner
product is written as
and
physicists doing quantum mechanics like to use a closely related
notation of .
We can also form the outer product by taking the transpose of the
second vector. In this case, the resultant is a square matrix
with dimensions equal to the number of components of the vector,
If you do the multiplication, you'll see that is a
diagonal square matrix with a 1 in the ith
diagonal position and zeros everywhere else.
We can write a diagonal matrix D with diagonal entries as
Orthonormal bases
These properties of the basis can also hold for other bases.
Orthogonal bases have basis vectors that are mutually
perpendicular,
for i ≠ j.
Orthonormal bases have the additional property that all basis
vectors are unit vectors, .
If we have an orthonormal basis, then it is easy to find the
components of any vector v in that basis.
We can always write v as a linear combination of the basis
vectors,
Then if we take the dot product of v with the basis vector
, we find
Thus, the jth component of the vector v in the
basis is and we
can write the vector as
Or if you prefer in terms of the inner product
Inner product in an orthonormal basis
You are familiar with the dot product from using vectors in physics
and it makes great sense as a way to measure the lengths of vectors
and the angles between vectors. However, in some more general
vector spaces, it makes no sense and we must use an appropriate
inner product instead. It turns out that by using an
orthonormal bases, we can always relate the inner product to the dot
product.
For example, consider the space of first order polynominals p
defined on the interval [0, 1]. We define the inner product as
We can use as a basis B, the functions 1 and x. Then a
vector in our space
describes the first order polynomial
.
This basis is neither ortho nor normal.
Instead, let's use a basis consisting of the functions 1 and
.
The vectors are orthogonal because
The vectors also have unit length
and
An arbitrary vector
can be written in the orthonormal basis O as
To see this, note that the coefficients are equal to the inner
product of the vector with the basis vector
We can evaluate the first coefficient as
We'll leave evaluating the second coefficient as an exercise for the
reader.
In an orthonormal basis, the inner product of
with
is equal to the dot product of the two vectors, which is
We can check by doing the inner product on the two vectors using the
definition in terms of the integral,
Changing between orthonormal bases
The change of basis matrix from an orthonormal basis to another
orthonormal basis is
The columns of P are made from an orthonormal set of
vectors, since they consist of the vectors in the orthonormal basis
R.
It can be shown that P is an orthogonal matrix, meaning that
We'll leave the proof to the linear algebra textbook.
If you start with a diagonal matrix D and we change to a new
basis using an orthogonal matrix P, then the matrix M in the
new basis is
Interestingly, the matrix M must be symmetric,
.
Let's try to check this.
since the transpose of product of matrices equals the product of the
transposes with the order swapped (from lecture #10),
since the transpose of the transpose is the original matrix and the
transpose of a diagonal matrix is itself,
since P is orthogonal and using our equation for the transformation
of matrices from lecture #19.
This has interesting implications.
Diagonalizing Symmetric Matrices
A matrix M is symmetric if MT = M.
Symmetric matrices arise quite often in real world
applications. For example, if you create a table of distances
between cities, it will be symmetric since the distance from city A
to city B is the same as the distance from city B to city A.
Or, if you have a collection of objects and a matrix representing
the magnitudes of the forces between them, it will be symmetric
because the force exerts by object A on object B must be equal and
opposite to the force exerted on object B by object A due to
Newton's third law.
Symmetric matrices always have real eigenvalues. Let's look at
a general 2×2 matrix with real elements. To find the
eigenvalues, we find the roots of the characteristic polynomial,
We find the roots of the quadratic polynomial using the quadratic
equation,
The discriminant
has two terms that are both squares. Thus, it must be
positive. Therefore, the square root will always be real and
the eigenvalues must be real. (Ya gotta be real man!).
But, it gets even better. Let's suppose that our symmetric
matrix M has two distinct eigenvalues λ ≠ μ and eigenvectors
x and y, so
Let's see if the eigenvectors are orthogonal. The dot product
is symmetric in its operands,
Let's calculate
we used the eigenvalue-eigenvector equation for y, and then
moved μ out of the inner product since it is a scalar.
Now let's flip things around (transpose them),
We can do the first step because the inner product gives a scalar
and the transpose of a scalar is always equal to itself. The
second step is our usual rule that the transpose of product of
matrices equals the product of the transposes with the order swapped
along with the fact that M is symmetric. Now, we can
use the eigenvalue-eigenvector equation for x,
In the first step, we move the scalar out of the transpose. In
the second step, we again have the transpose of a product of
matrices.
Then, we can write zero in a funny way,
We assumed that λ ≠ μ, so we must have x•y = 0, thus
our eigenvectors are orthogonal.
In general, the eigenvectors of a symmetric matrix with distinct
eigenvalues are orthogonal. Since we can also change the
length of an eigenvector to be whatever we want, this means that we
can always form an orthonormal basis using the eigenvectors of a
symmetric matrix.