Eigenvalues & Eigenvectors

An eigenvector is a direction a matrix does not rotate, it only stretches. Finding these invariant directions is how you discover the natural coordinate system of a linear system, in which a tangled transformation becomes a simple set of independent scalings. For quants this is the whole content of principal component analysis, factor models, covariance structure, and the convergence of Markov Chains: the eigenvalues tell you the risk concentrations and the eigenvectors tell you the directions they live in. It is the payoff of the Linear Algebra for Quants machinery.

Definition and the characteristic equation

For a square matrix $A \in \mathbb{R}^{n\times n}$, a nonzero vector $v$ is an eigenvector with eigenvalue $\lambda$ if $Av = \lambda v.$ Applying $A$ to $v$ just rescales it. Rearranging, $(A - \lambda I)v = 0$ has a nonzero solution iff $A - \lambda I$ is singular, i.e. $\det(A - \lambda I) = 0.$ This characteristic polynomial is degree $n$, so there are $n$ eigenvalues (with multiplicity, possibly complex). Two invariants read off the roots without solving: the trace equals the sum of eigenvalues, $\operatorname{tr}(A) = \sum_i \lambda_i$, and the determinant equals their product, $\det(A) = \prod_i \lambda_i$. These give instant sanity checks and, for a covariance matrix, connect total variance (trace) to the eigenvalues that partition it.

The spectral theorem: symmetry is everything

General matrices can have complex eigenvalues, defective (non-diagonalizable) structure, and non-orthogonal eigenvectors. But the matrices quants care about most, covariance matrices, correlation matrices, Hessians, are symmetric, and symmetry buys the cleanest theorem in linear algebra.

Spectral theorem. A real symmetric matrix $A = A^\top$ has (i) all real eigenvalues and (ii) an orthonormal basis of eigenvectors. Hence it factorises as $A = Q\Lambda Q^\top = \sum_{i=1}^n \lambda_i\, q_i q_i^\top,$ where $Q = [q_1,\dots,q_n]$ is orthogonal ($Q^\top Q = I$) and $\Lambda = \operatorname{diag}(\lambda_1,\dots,\lambda_n)$. The second form, a sum of rank-one projections weighted by eigenvalues, is the one to internalise: a symmetric matrix is a superposition of orthogonal "modes", each a direction $q_i$ with a strength $\lambda_i$. For a covariance matrix these are exactly the principal components: $q_i$ is the $i$-th uncorrelated risk direction and $\lambda_i$ is the variance along it. This is why the top eigenvector of an equity covariance matrix is "the market", the direction of greatest common variance.

Diagonalization: computing in the natural basis

Diagonalization is the operational value of eigenvectors. If $A = Q\Lambda Q^{-1}$ (with $Q^{-1} = Q^\top$ in the symmetric case), then in the eigenbasis $A$ acts as independent scalings, which makes matrix functions trivial: $A^k = Q\Lambda^k Q^{-1}, \qquad e^{A} = Q e^{\Lambda} Q^{-1},$ because $\Lambda^k$ just raises each diagonal entry to the $k$. This is how you compute matrix powers (transition-matrix limits, $P^n$ for a Markov Chains chain), matrix exponentials (continuous-time generators), and matrix square roots ($A^{1/2} = Q\Lambda^{1/2}Q^\top$, used to generate correlated Gaussians). The dynamics of $x_{k+1} = Ax_k$ are governed entirely by the eigenvalues: components along $q_i$ evolve as $\lambda_i^k$, so $|\lambda| > 1$ directions explode, $|\lambda| < 1$ decay, and the spectral radius $\max_i|\lambda_i|$ decides stability. The gap between the top two eigenvalues sets the rate, precisely the mixing rate of a Markov chain.

Worked example, the eigenvalues of a 2×2 covariance

Take the correlation-like matrix $A = \begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}.$ The characteristic equation is $(1-\lambda)^2 - \rho^2 = 0$, so $1 - \lambda = \pm\rho$ and the eigenvalues are $\lambda = 1+\rho$ and $\lambda = 1-\rho$. The eigenvectors are $q_1 = \tfrac{1}{\sqrt2}(1,1)$ (the "common mode", both assets moving together, variance $1+\rho$) and $q_2 = \tfrac{1}{\sqrt2}(1,-1)$ (the "spread mode", variance $1-\rho$). Notice: as $\rho \to 1$ the spread-mode eigenvalue $\to 0$, the matrix becomes singular and rank-one, meaning the two assets are effectively one risk factor. This tiny example is PCA: the first principal component is the market direction with the larger eigenvalue, and a small second eigenvalue signals redundancy and an ill-conditioned covariance, the seed of unstable optimisation. It also foreshadows Positive Semidefinite Matrices: eigenvalues stay $\ge 0$ only while $|\rho| \le 1$.

Power iteration: the algorithm behind PCA and PageRank

You rarely need all $n$ eigenpairs; often just the dominant one. Power iteration exploits diagonalization directly: start with a random $x_0$ and repeatedly apply and normalise, $x_{k+1} = \frac{Ax_k}{\lVert Ax_k\rVert}.$ Writing $x_0 = \sum c_i q_i$ in the eigenbasis, $A^k x_0 = \sum c_i \lambda_i^k q_i$; the largest $|\lambda|$ term dominates as $k$ grows, so $x_k \to q_1$ (the top eigenvector) and the associated Rayleigh quotient $x_k^\top A x_k / x_k^\top x_k \to \lambda_1$. Convergence speed is governed by the ratio $|\lambda_2/\lambda_1|$, the spectral gap again. This is how the leading principal component of a huge covariance matrix is extracted without a full decomposition, how Google's original PageRank found the dominant eigenvector of the web's link matrix, and (with deflation or shifts) how successive components follow. For symmetric matrices, more robust variants (Lanczos, QR algorithm) are production standard, but power iteration is the conceptual core.

Failure modes and subtleties

• Non-symmetric matrices misbehave. They can have complex eigenvalues, non-orthogonal eigenvectors, or be defective (fewer independent eigenvectors than $n$, so no diagonalization, only Jordan form). Transition matrices of Markov Chains are non-symmetric; their eigenvectors are not orthogonal, so PCA intuition does not transfer directly.
• Eigenvalues are unstable under noise for close spectra. When two eigenvalues are nearly equal, their individual eigenvectors are extremely sensitive to sampling error, a real problem for estimated covariance matrices where trailing eigenvalues are dominated by noise (random matrix theory / Marchenko–Pastur describes the noise floor).
• Sign and ordering are conventions. Eigenvectors are defined up to sign (and scale); PCA loadings can flip sign between samples, which is not instability, just convention. Order by eigenvalue explicitly.
• Multiplicity gives eigen-spaces, not eigen-vectors. A repeated eigenvalue has a whole subspace of eigenvectors; any orthonormal basis of it is valid, so individual directions are not identified.
• Trace/determinant checks catch errors. If your computed eigenvalues don't sum to the trace or multiply to the determinant, the computation is wrong, a cheap, always-available validation.

In interviews

Expect to compute the eigenvalues of a small (often $2\times2$) matrix from the characteristic equation, and to use trace-and-determinant as a shortcut. The symmetric-matrix facts are prized: "what do you know about the eigenvalues and eigenvectors of a covariance matrix?", real, non-negative, orthogonal eigenvectors, and they are the principal components. Be ready to explain diagonalization as computing in the eigenbasis and why it makes $A^k$ easy, connecting to Markov-chain convergence via the spectral gap. Power iteration is a favourite "how would you find the top principal component of a giant matrix?" answer. The unifying message: eigen-decomposition finds the coordinate system in which risk is a set of independent variances, which is the direct on-ramp to Singular Value Decomposition and PCA (Principal Component Analysis).