Orthonormality

Inlinear algebra,twovectorsin aninner product spaceareorthonormalif they areorthogonal unit vectors.A unit vector means that the vector has a length of 1, which is also known as normalized. Orthogonal means that the vectors are all perpendicular to each other. A set of vectors form anorthonormal setif all vectors in the set are mutually orthogonal and all of unit length. An orthonormal set which forms abasisis called anorthonormal basis.

Intuitive overview

The construction oforthogonalityof vectors is motivated by a desire to extend the intuitive notion of perpendicular vectors to higher-dimensional spaces. In theCartesian plane,twovectorsare said to beperpendicularif the angle between them is 90° (i.e. if they form aright angle). This definition can be formalized in Cartesian space by defining thedot productand specifying that two vectors in the plane are orthogonal if their dot product is zero.

Similarly, the construction of thenormof a vector is motivated by a desire to extend the intuitive notion of thelengthof a vector to higher-dimensional spaces. In Cartesian space, thenormof a vector is the square root of the vector dotted with itself. That is,

\|\mathbf {x} \|={\sqrt {\mathbf {x} \cdot \mathbf {x} }}

Many important results inlinear algebradeal with collections of two or more orthogonal vectors. But often, it is easier to deal with vectors ofunit length.That is, it often simplifies things to only consider vectors whose norm equals 1. The notion of restricting orthogonal pairs of vectors to only those of unit length is important enough to be given a special name. Two vectors which are orthogonal and of length 1 are said to beorthonormal.

Simple example

What does a pair of orthonormal vectors in 2-D Euclidean space look like?

Letu= (x₁,y₁) andv= (x₂,y₂). Consider the restrictions on x₁,x₂,y₁,y₂required to makeuandvform an orthonormal pair.

From the orthogonality restriction,u•v= 0.
From the unit length restriction onu,||u|| = 1.
From the unit length restriction onv,||v|| = 1.

Expanding these terms gives 3 equations:

$x_{1}x_{2}+y_{1}y_{2}=0\quad$
${\sqrt {{x_{1}}^{2}+{y_{1}}^{2}}}=1$
${\sqrt {{x_{2}}^{2}+{y_{2}}^{2}}}=1$

Converting from Cartesian topolar coordinates,and considering Equation $(2)$ and Equation $(3)$ immediately gives the result r₁= r₂= 1. In other words, requiring the vectors be of unit length restricts the vectors to lie on theunit circle.

After substitution, Equation $(1)$ becomes $\cos \theta _{1}\cos \theta _{2}+\sin \theta _{1}\sin \theta _{2}=0$ .Rearranging gives $\tan \theta _{1}=-\cot \theta _{2}$ .Using atrigonometric identityto convert thecotangentterm gives

\tan(\theta _{1})=\tan \left(\theta _{2}+{\tfrac {\pi }{2}}\right)

\Rightarrow \theta _{1}=\theta _{2}+{\tfrac {\pi }{2}}

It is clear that in the plane, orthonormal vectors are simply radii of the unit circle whose difference in angles equals 90°.

Definition

Let ${\mathcal {V}}$ be aninner-product space.A set of vectors

\left\{u_{1},u_{2},\ldots,u_{n},\ldots \right\}\in {\mathcal {V}}

is calledorthonormalif and only if

\forall i,j:\langle u_{i},u_{j}\rangle =\delta _{ij}

where $\delta _{ij}\,$ is theKronecker deltaand $\langle \cdot,\cdot \rangle$ is theinner productdefined over ${\mathcal {V}}$ .

Significance

Orthonormal sets are not especially significant on their own. However, they display certain features that make them fundamental in exploring the notion ofdiagonalizabilityof certainoperatorson vector spaces.

Properties

Orthonormal sets have certain very appealing properties, which make them particularly easy to work with.

Theorem.If {e₁,e₂,...,e_n} is an orthonormal list of vectors, then $\forall {\textbf {a}}:=[a_{1},\cdots,a_{n}];\ \|a_{1}{\textbf {e}}_{1}+a_{2}{\textbf {e}}_{2}+\cdots +a_{n}{\textbf {e}}_{n}\|^{2}=|a_{1}|^{2}+|a_{2}|^{2}+\cdots +|a_{n}|^{2}$
Theorem.Every orthonormal list of vectors islinearly independent.

Existence

Gram-Schmidt theorem.If {v₁,v₂,...,v_n} is a linearly independent list of vectors in an inner-product space ${\mathcal {V}}$ ,then there exists an orthonormal list {e₁,e₂,...,e_n} of vectors in ${\mathcal {V}}$ such thatspan(e₁,e₂,...,e_n) =span(v₁,v₂,...,v_n).

Proof of the Gram-Schmidt theorem isconstructive,anddiscussed at lengthelsewhere. The Gram-Schmidt theorem, together with theaxiom of choice,guarantees that every vector space admits an orthonormal basis. This is possibly the most significant use of orthonormality, as this fact permitsoperatorson inner-product spaces to be discussed in terms of their action on the space's orthonormal basis vectors. What results is a deep relationship between the diagonalizability of an operator and how it acts on the orthonormal basis vectors. This relationship is characterized by theSpectral Theorem.

Examples

Standard basis

Thestandard basisfor thecoordinate spaceFⁿis

{e₁,e₂,...,e_n} where	e₁= (1, 0,..., 0)
	e₂= (0, 1,..., 0)
	$\vdots$
	e_n= (0, 0,..., 1)

Any two vectorse_i,e_jwhere i≠j are orthogonal, and all vectors are clearly of unit length. So {e₁,e₂,...,e_n} forms an orthonormal basis.

Real-valued functions

When referring toreal-valuedfunctions,usually theL²inner product is assumed unless otherwise stated. Two functions $\phi (x)$ and $\psi (x)$ are orthonormal over theinterval $[a,b]$ if

(1)\quad \langle \phi (x),\psi (x)\rangle =\int _{a}^{b}\phi (x)\psi (x)dx=0,\quad {\rm {and}}

(2)\quad ||\phi (x)||_{2}=||\psi (x)||_{2}=\left[\int _{a}^{b}|\phi (x)|^{2}dx\right]^{\frac {1}{2}}=\left[\int _{a}^{b}|\psi (x)|^{2}dx\right]^{\frac {1}{2}}=1.

Fourier series

TheFourier seriesis a method of expressing a periodic function in terms of sinusoidalbasisfunctions. TakingC[−π,π] to be the space of all real-valued functions continuous on the interval [−π,π] and taking the inner product to be

\langle f,g\rangle =\int _{-\pi }^{\pi }f(x)g(x)dx

it can be shown that

\left\{{\frac {1}{\sqrt {2\pi }}},{\frac {\sin(x)}{\sqrt {\pi }}},{\frac {\sin(2x)}{\sqrt {\pi }}},\ldots,{\frac {\sin(nx)}{\sqrt {\pi }}},{\frac {\cos(x)}{\sqrt {\pi }}},{\frac {\cos(2x)}{\sqrt {\pi }}},\ldots,{\frac {\cos(nx)}{\sqrt {\pi }}}\right\},\quad n\in \mathbb {N}

forms an orthonormal set.

However, this is of little consequence, becauseC[−π,π] is infinite-dimensional, and a finite set of vectors cannot span it. But, removing the restriction thatnbe finite makes the setdenseinC[−π,π] and therefore an orthonormal basis ofC[−π,π].

Sources

Axler, Sheldon(1997),Linear Algebra Done Right(2nd ed.), Berlin, New York:Springer-Verlag,p.106–110,ISBN 978-0-387-98258-8
Chen, Wai-Kai (2009),Fundamentals of Circuits and Filters(3rd ed.),Boca Raton:CRC Press,p.62,ISBN 978-1-4200-5887-1