You’ve probably heard the word eigenvectors hundreds of times in class and have been asked to calculate it hundreds more. But why is this concept so central in Linear Algebra? Why is it that we study the eigenvector of a matrix so often?
In this post, we’ll see how Eigenvectors help us immediately understand what a linear function will do to an input. We’ll do so by playing with an interactive visualization that allows us to see just that.
Here, applying on its eigenvector leades to a new vector is in the same direction as . Image Source: Wikipedia.
Let’s do a quick refresher to begin with. The eigenvector of a linear function is just the vector s.t. for some constant which we call the eigenvalue. At a high level, the eigenvector is just a dimension along which the linear function only stretches its input (for real valued eigenvalues).
This is great and all but what can we do once we find an Eigenvector? What does it tell us about the underlying Matrix?
Linear Functions Pull Inputs Towards the Dominant Eigenvector
Let’s start with an example. Take the following linear function:
Let’s see what happens when we apply repeatedly on the input .
To play with this visualization, do the following:
- Drag the slider to increase the number of times we apply .
- Notice how the output vector tilts towards , an eigenvector of .
So just by knowing an eigenvector of (namely the dominant eigenvector), we can get a sense of what does - pulls its input towards the axis of the dominant eigenvector.
How is this happening? And why is it only towards one eigenvector?
Breaking Up the Input With an Eigenbasis
You’ll be surprised to see that this behavior comes very naturally from the properties of linear functions. Let’s see this with an example.
Let’s keep the same matrix/linear function , and analyze applying three times on the input (i.e. ).
In the standard way, we'd just use standard matrix multiplication to find
The standard way to do this is to simply follow the rules of multiplication and carry out But instead, let’s do this a different way using eigenvectors.
In the following discussion we will split into a linear combination of 's eigenvectors. We then apply to each of these pieces and combine the result.
We know that that any vector can be written as the sum of the eigenvectors of . After all, eigenvectors are linearly independent and form a basis for the space (if the matrix is diagonalizable, which it is). If and are the eigenvectors of we can break up as:
for some constants and
We first split into its eigenvector subcomponents.
When we have this representation, we can then rethink as:
or more simply:
We then carry out and
We then carry out the computation of and . Thanks to and being eigenvectors, we have:
We finally combine the results to get
We then finally combine the results to get We find:
Dominant Eignevalues and Eigenvectors
Now, what happens when is larger than (i.e. there exists a dominant eigenvalue)? In this example, and Let’s now display what it would look like to carry out when we have this difference in eigenvalues.
The interaction below shows this setup:
- Drag the slider to increase or decrease the number of times we apply on
- Notice how “Output Eigenvector 1” and “Output Eigenvector 2” change at different rates.
- Notice how “Final Output Vector” tilts towards “Output Eigenvector 1” as you drag the slider to the right.
We thus see that when there’s one eigenvalue larger than the other (), the linear function pushes its inputs towards the eigenvector associated with that large eigenvalue (“Output EigenVector One”). The more times we apply , the larger this effect.
Note this “push” effect will only happen towards this eigenvector with the largest eignevalue - not any of the other eigenvectors.
Why this happens
This tilt towards “Output Vector One” happens due to exponential growth. grows much faster than . As such the more times we apply ( in our exponentials), the bigger the difference between and Hence the term has much more weight in the final sum. This increasing difference is shown in the plot below.
Due to the power of exponentials, the dominant eigenvector will play a bigger and bigger role the more times we apply A. Notice how the distance between the two expontial functions increases with x.
So, just using the properties of linear functions, we are able to see why eigenvectors are so important. They show us where a linear function will “push” its inputs.
If you’ve enjoyed this post on eigenvectors, check out the following additional posts on the topic I’ve written:
- You could have come up with eigenvectors. Here’s how.
- How Eigenvectors Power PageRank - the algorithm behind Google Search.
Thanks for reading!
- Everything I’ve discussed is for real eigenvalues.
- This only applies for matrices that are diagonalizable.
Thanks to Luis Serrano, Rouzbeh Shirvani, and Pranav Ramkrishnan for feedback.