Linear algebra

Linear algebrais the branch ofmathematicsconcerninglinear equationssuch as:

a_{1}x_{1}+\cdots +a_{n}x_{n}=b,

linear mapssuch as:

(x_{1},\ldots,x_{n})\mapsto a_{1}x_{1}+\cdots +a_{n}x_{n},

and their representations invector spacesand throughmatrices.^[1]^[2]^[3]

Linear algebra is central to almost all areas of mathematics. For instance, linear algebra is fundamental in modern presentations ofgeometry,including for defining basic objects such aslines,planesandrotations.Also,functional analysis,a branch ofmathematical analysis,may be viewed as the application of linear algebra tofunction spaces.

Linear algebra is also used in most sciences and fields ofengineering,because it allowsmodelingmany natural phenomena, and computing efficiently with such models. Fornonlinear systems,which cannot be modeled with linear algebra, it is often used for dealing withfirst-order approximations,using the fact that thedifferentialof amultivariate functionat a point is the linear map that best approximates the function near that point.

History

The procedure (using counting rods) for solving simultaneous linear equations now calledGaussian eliminationappears in the ancient Chinese mathematical textChapter Eight:Rectangular ArraysofThe Nine Chapters on the Mathematical Art.Its use is illustrated in eighteen problems, with two to five equations.^[4]

Systems of linear equationsarose in Europe with the introduction in 1637 byRené Descartesofcoordinatesingeometry.In fact, in this new geometry, now calledCartesian geometry,lines and planes are represented by linear equations, and computing their intersections amounts to solving systems of linear equations.

The first systematic methods for solving linear systems useddeterminantsand were first considered byLeibnizin 1693. In 1750,Gabriel Cramerused them for giving explicit solutions of linear systems, now calledCramer's rule.Later,Gaussfurther described the method of elimination, which was initially listed as an advancement ingeodesy.^[5]

In 1844Hermann Grassmannpublished his "Theory of Extension" which included foundational new topics of what is today called linear algebra. In 1848,James Joseph Sylvesterintroduced the termmatrix,which is Latin forwomb.

Linear algebra grew with ideas noted in thecomplex plane.For instance, two numbers $w$ and $z$ in $\mathbb {C}$ have a difference $w - z$ ,and the line segments $wz$ and $0(w - z)$ are of the same length and direction. The segments areequipollent.The four-dimensional system $\mathbb {H}$ ofquaternionswas discovered byW.R. Hamiltonin 1843.^[6]The termvectorwas introduced as $v = x i + y j + z k$ representing a point in space. The quaternion difference $p - q$ also produces a segment equipollent to $pq$ .Otherhypercomplex numbersystems also used the idea of a linear space with abasis.

Arthur Cayleyintroducedmatrix multiplicationand theinverse matrixin 1856, making possible thegeneral linear group.The mechanism ofgroup representationbecame available for describing complex and hypercomplex numbers. Crucially, Cayley used a single letter to denote a matrix, thus treating a matrix as an aggregate object. He also realized the connection between matrices and determinants, and wrote "There would be many things to say about this theory of matrices which should, it seems to me, precede the theory of determinants".^[5]

Benjamin Peircepublished hisLinear Associative Algebra(1872), and his sonCharles Sanders Peirceextended the work later.^[7]

Thetelegraphrequired an explanatory system, and the 1873 publication ofA Treatise on Electricity and Magnetisminstituted afield theoryof forces and requireddifferential geometryfor expression. Linear algebra is flat differential geometry and serves in tangent spaces tomanifolds.Electromagnetic symmetries of spacetime are expressed by theLorentz transformations,and much of the history of linear algebra is thehistory of Lorentz transformations.

The first modern and more precise definition of a vector space was introduced byPeanoin 1888;^[5]by 1900, a theory of linear transformations of finite-dimensional vector spaces had emerged. Linear algebra took its modern form in the first half of the twentieth century, when many ideas and methods of previous centuries were generalized asabstract algebra.The development of computers led to increased research in efficientalgorithmsfor Gaussian elimination and matrix decompositions, and linear algebra became an essential tool for modelling and simulations.^[5]

Vector spaces

Until the 19th century, linear algebra was introduced throughsystems of linear equationsandmatrices.In modern mathematics, the presentation throughvector spacesis generally preferred, since it is moresynthetic,more general (not limited to the finite-dimensional case), and conceptually simpler, although more abstract.

A vector space over afield $F$ (often the field of thereal numbers) is aset $V$ equipped with twobinary operations.Elementsof $V$ are calledvectors,and elements ofFare calledscalars.The first operation,vector addition,takes any two vectors $v$ and $w$ and outputs a third vector $v + w$ .The second operation,scalar multiplication,takes any scalar $a$ and any vector $v$ and outputs a newvector $a v$ .The axioms that addition and scalar multiplication must satisfy are the following. (In the list below, $u, v$ and $w$ are arbitrary elements of $V$ ,and $a$ and $b$ are arbitrary scalars in the field $F$ .)^[8]

Axiom	Signification
Associativityof addition	$u + (v + w) = (u + v) + w$
Commutativityof addition	$u + v = v + u$
Identity elementof addition	There exists an element $0$ in $V$ ,called thezero vector(or simplyzero), such that $v + 0 = v$ for all $v$ in $V$ .
Inverse elementsof addition	For every $v$ in $V$ ,there exists an element $- v$ in $V$ ,called theadditive inverseof $v$ ,such that $v + (- v) = 0$
Distributivityof scalar multiplication with respect to vector addition	$a (u + v) = a u + a v$
Distributivity of scalar multiplication with respect to field addition	$(a + b) v = a v + b v$
Compatibility of scalar multiplication with field multiplication	$a (b v) = (ab) v$ ^[a]
Identity element of scalar multiplication	$1 v = v$ ,where $1$ denotes themultiplicative identityof $F$ .

The first four axioms mean that $V$ is anabelian groupunder addition.

An element of a specific vector space may have various nature; for example, it could be asequence,afunction,apolynomialor amatrix.Linear algebra is concerned with those properties of such objects that are common to all vector spaces.

Linear maps

Linear mapsaremappingsbetween vector spaces that preserve the vector-space structure. Given two vector spaces $V$ and $W$ over a field $F$ ,a linear map (also called, in some contexts, linear transformation or linear mapping) is amap

T:V\to W

that is compatible with addition and scalar multiplication, that is

T(\mathbf {u} +\mathbf {v} )=T(\mathbf {u} )+T(\mathbf {v} ),\quad T(a\mathbf {v} )=aT(\mathbf {v} )

for any vectors $u, v$ in $V$ and scalar $a$ in $F$ .

This implies that for any vectors $u, v$ in $V$ and scalars $a, b$ in $F$ ,one has

T(a\mathbf {u} +b\mathbf {v} )=T(a\mathbf {u} )+T(b\mathbf {v} )=aT(\mathbf {u} )+bT(\mathbf {v} )

When $V = W$ are the same vector space, a linear map $T : V \to V$ is also known as alinear operatoron $V$ .

Abijectivelinear map between two vector spaces (that is, every vector from the second space is associated with exactly one in the first) is anisomorphism.Because an isomorphism preserves linear structure, two isomorphic vector spaces are "essentially the same" from the linear algebra point of view, in the sense that they cannot be distinguished by using vector space properties. An essential question in linear algebra is testing whether a linear map is an isomorphism or not, and, if it is not an isomorphism, finding itsrange(or image) and the set of elements that are mapped to the zero vector, called thekernelof the map. All these questions can be solved by usingGaussian eliminationor some variant of thisalgorithm.

Subspaces, span, and basis

The study of those subsets of vector spaces that are in themselves vector spaces under the induced operations is fundamental, similarly as for many mathematical structures. These subsets are calledlinear subspaces.More precisely, a linear subspace of a vector space $V$ over a field $F$ is asubset $W$ of $V$ such that $u + v$ and $a u$ are in $W$ ,for every $u$ , $v$ in $W$ ,and every $a$ in $F$ .(These conditions suffice for implying that $W$ is a vector space.)

For example, given a linear map $T : V \to W$ ,theimage $T (V)$ of $V$ ,and theinverse image $T -1 (0)$ of $0$ (calledkernelor null space), are linear subspaces of $W$ and $V$ ,respectively.

Another important way of forming a subspace is to considerlinear combinationsof a set $S$ of vectors: the set of all sums

a_{1}\mathbf {v} _{1}+a_{2}\mathbf {v} _{2}+\cdots +a_{k}\mathbf {v} _{k},

where $v 1, v 2,..., v k$ are in $S$ ,and $a 1, a 2,..., a k$ are in $F$ form a linear subspace called thespanof $S$ .The span of $S$ is also the intersection of all linear subspaces containing $S$ .In other words, it is the smallest (for the inclusion relation) linear subspace containing $S$ .

A set of vectors islinearly independentif none is in the span of the others. Equivalently, a set $S$ of vectors is linearly independent if the only way to express the zero vector as a linear combination of elements of $S$ is to take zero for every coefficient $a i$ .

A set of vectors that spans a vector space is called aspanning setorgenerating set.If a spanning set $S$ islinearly dependent(that is not linearly independent), then some element $w$ of $S$ is in the span of the other elements of $S$ ,and the span would remain the same if one remove $w$ from $S$ .One may continue to remove elements of $S$ until getting alinearly independent spanning set.Such a linearly independent set that spans a vector space $V$ is called abasisof $V$ .The importance of bases lies in the fact that they are simultaneously minimal generating sets and maximal independent sets. More precisely, if $S$ is a linearly independent set, and $T$ is a spanning set such that $S \subseteq T$ ,then there is a basis $B$ such that $S \subseteq B \subseteq T$ .

Any two bases of a vector space $V$ have the samecardinality,which is called thedimensionof $V$ ;this is thedimension theorem for vector spaces.Moreover, two vector spaces over the same field $F$ areisomorphicif and only if they have the same dimension.^[9]

If any basis of $V$ (and therefore every basis) has a finite number of elements, $V$ is afinite-dimensional vector space.If $U$ is a subspace of $V$ ,then $dim U \leq dim V$ .In the case where $V$ is finite-dimensional, the equality of the dimensions implies $U = V$ .

If $U 1$ and $U 2$ are subspaces of $V$ ,then

\dim(U_{1}+U_{2})=\dim U_{1}+\dim U_{2}-\dim(U_{1}\cap U_{2}),

where $U 1 + U 2$ denotes the span of $U 1 \cup U 2$ .^[10]

Matrices

Matrices allow explicit manipulation of finite-dimensional vector spaces andlinear maps.Their theory is thus an essential part of linear algebra.

Let $V$ be a finite-dimensional vector space over a field $F$ ,and $(v 1, v 2,..., v m)$ be a basis of $V$ (thus $m$ is the dimension of $V$ ). By definition of a basis, the map

{\begin{aligned}(a_{1},\ldots,a_{m})&\mapsto a_{1}\mathbf {v} _{1}+\cdots a_{m}\mathbf {v} _{m}\\F^{m}&\to V\end{aligned}}

is abijectionfrom $F m$ ,the set of thesequencesof $m$ elements of $F$ ,onto $V$ .This is anisomorphismof vector spaces, if $F m$ is equipped of its standard structure of vector space, where vector addition and scalar multiplication are done component by component.

This isomorphism allows representing a vector by itsinverse imageunder this isomorphism, that is by thecoordinate vector $(a 1,..., a m)$ or by thecolumn matrix

{\begin{bmatrix}a_{1}\\\vdots \\a_{m}\end{bmatrix}}.

If $W$ is another finite dimensional vector space (possibly the same), with a basis $(w 1,..., w n)$ ,a linear map $f$ from $W$ to $V$ is well defined by its values on the basis elements, that is $(f (w 1),..., f (w n))$ .Thus, $f$ is well represented by the list of the corresponding column matrices. That is, if

f(w_{j})=a_{1,j}v_{1}+\cdots +a_{m,j}v_{m},

for $j = 1,..., n$ ,then $f$ is represented by the matrix

{\begin{bmatrix}a_{1,1}&\cdots &a_{1,n}\\\vdots &\ddots &\vdots \\a_{m,1}&\cdots &a_{m,n}\end{bmatrix}},

with $m$ rows and $n$ columns.

Matrix multiplicationis defined in such a way that the product of two matrices is the matrix of thecompositionof the corresponding linear maps, and the product of a matrix and a column matrix is the column matrix representing the result of applying the represented linear map to the represented vector. It follows that the theory of finite-dimensional vector spaces and the theory of matrices are two different languages for expressing exactly the same concepts.

Two matrices that encode the same linear transformation in different bases are calledsimilar.It can be proved that two matrices are similar if and only if one can transform one into the other byelementary row and column operations.For a matrix representing a linear map from $W$ to $V$ ,the row operations correspond to change of bases in $V$ and the column operations correspond to change of bases in $W$ .Every matrix is similar to anidentity matrixpossibly bordered by zero rows and zero columns. In terms of vector spaces, this means that, for any linear map from $W$ to $V$ ,there are bases such that a part of the basis of $W$ is mapped bijectively on a part of the basis of $V$ ,and that the remaining basis elements of $W$ ,if any, are mapped to zero.Gaussian eliminationis the basic algorithm for finding these elementary operations, and proving these results.

Linear systems

A finite set of linear equations in a finite set of variables, for example, $x 1, x 2,..., x n$ ,or $x, y,..., z$ is called asystem of linear equationsor alinear system.^[11]^[12]^[13]^[14]^[15]

Systems of linear equations form a fundamental part of linear algebra. Historically, linear algebra and matrix theory has been developed for solving such systems. In the modern presentation of linear algebra through vector spaces and matrices, many problems may be interpreted in terms of linear systems.

For example, let

{\begin{alignedat}{7}2x&&\;+\;&&y&&\;-\;&&z&&\;=\;&&8\\-3x&&\;-\;&&y&&\;+\;&&2z&&\;=\;&&-11\\-2x&&\;+\;&&y&&\;+\;&&2z&&\;=\;&&-3\end{alignedat}}

(S)

be a linear system.

To such a system, one may associate its matrix

M=\left[{\begin{array}{rrr}2&1&-1\\-3&-1&2\\-2&1&2\end{array}}\right].

and its right member vector

\mathbf {v} ={\begin{bmatrix}8\\-11\\-3\end{bmatrix}}.

Let $T$ be the linear transformation associated to the matrix $M$ .A solution of the system (S) is a vector

\mathbf {X} ={\begin{bmatrix}x\\y\\z\end{bmatrix}}

such that

T(\mathbf {X} )=\mathbf {v},

that is an element of thepreimageof $v$ by $T$ .

Let (S′) be the associatedhomogeneous system,where the right-hand sides of the equations are put to zero:

{\begin{alignedat}{7}2x&&\;+\;&&y&&\;-\;&&z&&\;=\;&&0\\-3x&&\;-\;&&y&&\;+\;&&2z&&\;=\;&&0\\-2x&&\;+\;&&y&&\;+\;&&2z&&\;=\;&&0\end{alignedat}}

(S′)

The solutions of (S′) are exactly the elements of thekernelof $T$ or, equivalently, $M$ .

TheGaussian-eliminationconsists of performingelementary row operationson theaugmented matrix

\left[\!{\begin{array}{c|c}M&\mathbf {v} \end{array}}\!\right]=\left[{\begin{array}{rrr|r}2&1&-1&8\\-3&-1&2&-11\\-2&1&2&-3\end{array}}\right]

for putting it inreduced row echelon form.These row operations do not change the set of solutions of the system of equations. In the example, the reduced echelon form is

\left[\!{\begin{array}{c|c}M&\mathbf {v} \end{array}}\!\right]=\left[{\begin{array}{rrr|r}1&0&0&2\\0&1&0&3\\0&0&1&-1\end{array}}\right],

showing that the system (S) has the unique solution

{\begin{aligned}x&=2\\y&=3\\z&=-1.\end{aligned}}

It follows from this matrix interpretation of linear systems that the same methods can be applied for solving linear systems and for many operations on matrices and linear transformations, which include the computation of theranks,kernels,matrix inverses.

Endomorphisms and square matrices

A linearendomorphismis a linear map that maps a vector space $V$ to itself. If $V$ has a basis of $n$ elements, such an endomorphism is represented by a square matrix of size $n$ .

With respect to general linear maps, linear endomorphisms and square matrices have some specific properties that make their study an important part of linear algebra, which is used in many parts of mathematics, includinggeometric transformations,coordinate changes,quadratic forms,and many other part of mathematics.

Determinant

Thedeterminantof a square matrix $A$ is defined to be^[16]

\sum _{\sigma \in S_{n}}(-1)^{\sigma }a_{1\sigma (1)}\cdots a_{n\sigma (n)},

where $S n$ is thegroup of all permutationsof $n$ elements, $σ$ is a permutation, and $(-1) σ$ theparityof the permutation. A matrix isinvertibleif and only if the determinant is invertible (i.e., nonzero if the scalars belong to a field).

Cramer's ruleis aclosed-form expression,in terms of determinants, of the solution of asystem of $n$ linear equations in $n$ unknowns.Cramer's rule is useful for reasoning about the solution, but, except for $n = 2$ or $3$ ,it is rarely used for computing a solution, sinceGaussian eliminationis a faster algorithm.

Thedeterminant of an endomorphismis the determinant of the matrix representing the endomorphism in terms of some ordered basis. This definition makes sense, since this determinant is independent of the choice of the basis.

Eigenvalues and eigenvectors

If $f$ is a linear endomorphism of a vector space $V$ over a field $F$ ,aneigenvectorof $f$ is a nonzero vector $v$ of $V$ such that $f (v) = av$ for some scalar $a$ in $F$ .This scalar $a$ is aneigenvalueof $f$ .

If the dimension of $V$ is finite, and a basis has been chosen, $f$ and $v$ may be represented, respectively, by a square matrix $M$ and a column matrix $z$ ;the equation defining eigenvectors and eigenvalues becomes

Mz=az.

Using theidentity matrix $I$ ,whose entries are all zero, except those of the main diagonal, which are equal to one, this may be rewritten

(M-aI)z=0.

As $z$ is supposed to be nonzero, this means that $M - aI$ is asingular matrix,and thus that its determinant $det (M - aI)$ equals zero. The eigenvalues are thus therootsof thepolynomial

\det(xI-M).

If $V$ is of dimension $n$ ,this is amonic polynomialof degree $n$ ,called thecharacteristic polynomialof the matrix (or of the endomorphism), and there are, at most, $n$ eigenvalues.

If a basis exists that consists only of eigenvectors, the matrix of $f$ on this basis has a very simple structure: it is adiagonal matrixsuch that the entries on themain diagonalare eigenvalues, and the other entries are zero. In this case, the endomorphism and the matrix are said to bediagonalizable.More generally, an endomorphism and a matrix are also said diagonalizable, if they become diagonalizable afterextendingthe field of scalars. In this extended sense, if the characteristic polynomial issquare-free,then the matrix is diagonalizable.

Asymmetric matrixis always diagonalizable. There are non-diagonalizable matrices, the simplest being

{\begin{bmatrix}0&1\\0&0\end{bmatrix}}

(it cannot be diagonalizable since its square is thezero matrix,and the square of a nonzero diagonal matrix is never zero).

When an endomorphism is not diagonalizable, there are bases on which it has a simple form, although not as simple as the diagonal form. TheFrobenius normal formdoes not need of extending the field of scalars and makes the characteristic polynomial immediately readable on the matrix. TheJordan normal formrequires to extend the field of scalar for containing all eigenvalues, and differs from the diagonal form only by some entries that are just above the main diagonal and are equal to 1.

Duality

Alinear formis a linear map from a vector space $V$ over a field $F$ to the field of scalars $F$ ,viewed as a vector space over itself. Equipped bypointwiseaddition and multiplication by a scalar, the linear forms form a vector space, called thedual spaceof $V$ ,and usually denoted $V*$ ^[17]or $V'$ .^[18]^[19]

If $v 1,..., v n$ is a basis of $V$ (this implies that $V$ is finite-dimensional), then one can define, for $i = 1,..., n$ ,a linear map $v i *$ such that $v i *(v i) = 1$ and $v i *(v j) = 0$ if $j \neq i$ .These linear maps form a basis of $V *$ ,called thedual basisof $v 1,..., v n$ .(If $V$ is not finite-dimensional, the $v i *$ may be defined similarly; they are linearly independent, but do not form a basis.)

For $v$ in $V$ ,the map

f\to f(\mathbf {v} )

is a linear form on $V*$ .This defines thecanonical linear mapfrom $V$ into $(V *)*$ ,the dual of $V*$ ,called thebidualof $V$ .This canonical map is anisomorphismif $V$ is finite-dimensional, and this allows identifying $V$ with its bidual. (In the infinite dimensional case, the canonical map is injective, but not surjective.)

There is thus a complete symmetry between a finite-dimensional vector space and its dual. This motivates the frequent use, in this context, of thebra–ket notation

\langle f,\mathbf {x} \rangle

for denoting $f (x)$ .

Dual map

Let

f:V\to W

be a linear map. For every linear form $h$ on $W$ ,thecomposite function $h \circ f$ is a linear form on $V$ .This defines a linear map

f^{*}:W^{*}\to V^{*}

between the dual spaces, which is called thedualor thetransposeof $f$ .

If $V$ and $W$ are finite dimensional, and $M$ is the matrix of $f$ in terms of some ordered bases, then the matrix of $f*$ over the dual bases is thetranspose $M T$ of $M$ ,obtained by exchanging rows and columns.

If elements of vector spaces and their duals are represented by column vectors, this duality may be expressed inbra–ket notationby

\langle h^{\mathsf {T}},M\mathbf {v} \rangle =\langle h^{\mathsf {T}}M,\mathbf {v} \rangle.

For highlighting this symmetry, the two members of this equality are sometimes written

\langle h^{\mathsf {T}}\mid M\mid \mathbf {v} \rangle.

Inner-product spaces

Besides these basic concepts, linear algebra also studies vector spaces with additional structure, such as aninner product.The inner product is an example of abilinear form,and it gives the vector space a geometric structure by allowing for the definition of length and angles. Formally, aninner productis a map

\langle \cdot,\cdot \rangle:V\times V\to F

that satisfies the following threeaxiomsfor all vectors $u, v, w$ in $V$ and all scalars $a$ in $F$ :^[20]^[21]

Conjugatesymmetry:
$\langle \mathbf {u},\mathbf {v} \rangle ={\overline {\langle \mathbf {v},\mathbf {u} \rangle }}.$

In

\mathbb {R}

,it is symmetric.

Linearityin the first argument:
${\begin{aligned}\langle a\mathbf {u},\mathbf {v} \rangle &=a\langle \mathbf {u},\mathbf {v} \rangle.\\\langle \mathbf {u} +\mathbf {v},\mathbf {w} \rangle &=\langle \mathbf {u},\mathbf {w} \rangle +\langle \mathbf {v},\mathbf {w} \rangle.\end{aligned}}$
Positive-definiteness:
$\langle \mathbf {v},\mathbf {v} \rangle \geq 0$

with equality only for

v = 0

.

We can define the length of a vectorvinVby

\|\mathbf {v} \|^{2}=\langle \mathbf {v},\mathbf {v} \rangle,

and we can prove theCauchy–Schwarz inequality:

|\langle \mathbf {u},\mathbf {v} \rangle |\leq \|\mathbf {u} \|\cdot \|\mathbf {v} \|.

In particular, the quantity

{\frac {|\langle \mathbf {u},\mathbf {v} \rangle |}{\|\mathbf {u} \|\cdot \|\mathbf {v} \|}}\leq 1,

and so we can call this quantity the cosine of the angle between the two vectors.

Two vectors are orthogonal if $⟨ u, v ⟩ = 0$ .An orthonormal basis is a basis where all basis vectors have length 1 and are orthogonal to each other. Given any finite-dimensional vector space, an orthonormal basis could be found by theGram–Schmidtprocedure. Orthonormal bases are particularly easy to deal with, since ifv=a₁v₁+ ⋯ +a_nv_n,then

a_{i}=\langle \mathbf {v},\mathbf {v} _{i}\rangle.

The inner product facilitates the construction of many useful concepts. For instance, given a transform $T$ ,we can define itsHermitian conjugate $T*$ as the linear transform satisfying

\langle T\mathbf {u},\mathbf {v} \rangle =\langle \mathbf {u},T^{*}\mathbf {v} \rangle.

If $T$ satisfies $TT* = T*T$ ,we call $T$ normal.It turns out that normal matrices are precisely the matrices that have an orthonormal system of eigenvectors that span $V$ .

Relationship with geometry

There is a strong relationship between linear algebra andgeometry,which started with the introduction byRené Descartes,in 1637, ofCartesian coordinates.In this new (at that time) geometry, now calledCartesian geometry,points are represented byCartesian coordinates,which are sequences of three real numbers (in the case of the usualthree-dimensional space). The basic objects of geometry, which arelinesandplanesare represented by linear equations. Thus, computing intersections of lines and planes amounts to solving systems of linear equations. This was one of the main motivations for developing linear algebra.

Mostgeometric transformation,such astranslations,rotations,reflections,rigid motions,isometries,andprojectionstransform lines into lines. It follows that they can be defined, specified and studied in terms of linear maps. This is also the case ofhomographiesandMöbius transformations,when considered as transformations of aprojective space.

Until the end of the 19th century, geometric spaces were defined byaxiomsrelating points, lines and planes (synthetic geometry). Around this date, it appeared that one may also define geometric spaces by constructions involving vector spaces (see, for example,Projective spaceandAffine space). It has been shown that the two approaches are essentially equivalent.^[22]In classical geometry, the involved vector spaces are vector spaces over the reals, but the constructions may be extended to vector spaces over any field, allowing considering geometry over arbitrary fields, includingfinite fields.

Presently, most textbooks introduce geometric spaces from linear algebra, and geometry is often presented, at elementary level, as a subfield of linear algebra.

Usage and applications

Linear algebra is used in almost all areas of mathematics, thus making it relevant in almost all scientific domains that use mathematics. These applications may be divided into several wide categories.

Functional analysis

Functional analysisstudiesfunction spaces.These are vector spaces with additional structure, such asHilbert spaces.Linear algebra is thus a fundamental part of functional analysis and its applications, which include, in particular,quantum mechanics(wave functions) andFourier analysis(orthogonal basis).

Scientific computation

Nearly allscientific computationsinvolve linear algebra. Consequently, linear algebra algorithms have been highly optimized.BLASandLAPACKare the best known implementations. For improving efficiency, some of them configure the algorithms automatically, at run time, for adapting them to the specificities of the computer (cachesize, number of availablecores,...).

Someprocessors,typicallygraphics processing units(GPU), are designed with a matrix structure, for optimizing the operations of linear algebra.^{[citation needed]}

Geometry of ambient space

Themodelingofambient spaceis based ongeometry.Sciences concerned with this space use geometry widely. This is the case withmechanicsandrobotics,for describingrigid body dynamics;geodesyfor describingEarth shape;perspectivity,computer vision,andcomputer graphics,for describing the relationship between a scene and its plane representation; and many other scientific domains.

In all these applications,synthetic geometryis often used for general descriptions and a qualitative approach, but for the study of explicit situations, one must compute withcoordinates.This requires the heavy use of linear algebra.

Study of complex systems

Most physical phenomena are modeled bypartial differential equations.To solve them, one usually decomposes the space in which the solutions are searched into small, mutually interactingcells.Forlinear systemsthis interaction involveslinear functions.Fornonlinear systems,this interaction is often approximated by linear functions.^[b]This is called a linear model or first-order approximation. Linear models are frequently used for complex nonlinear real-world systems because it makesparametrizationmore manageable.^[23]In both cases, very large matrices are generally involved.Weather forecasting(or more specifically,parametrization for atmospheric modeling) is a typical example of a real-world application, where the whole Earthatmosphereis divided into cells of, say, 100 km of width and 100 km of height.

Fluid Mechanics, Fluid Dynamics, and Thermal Energy Systems

^[24]^[25]^[26]

Linear algebra, a branch of mathematics dealing withvector spacesandlinear mappingsbetween these spaces, plays a critical role in various engineering disciplines, includingfluid mechanics,fluid dynamics,andthermal energysystems. Its application in these fields is multifaceted and indispensable for solving complex problems.

Influid mechanics,linear algebra is integral to understanding and solving problems related to the behavior of fluids. It assists in the modeling and simulation of fluid flow, providing essential tools for the analysis offluid dynamicsproblems. For instance, linear algebraic techniques are used to solve systems ofdifferential equationsthat describe fluid motion. These equations, often complex andnon-linear,can be linearized using linear algebra methods, allowing for simpler solutions and analyses.

In the field of fluid dynamics, linear algebra finds its application incomputational fluid dynamics(CFD), a branch that usesnumerical analysisanddata structuresto solve and analyze problems involving fluid flows. CFD relies heavily on linear algebra for the computation of fluid flow andheat transferin various applications. For example, theNavier-Stokes equations,fundamental influid dynamics,are often solved using techniques derived from linear algebra. This includes the use ofmatricesandvectorsto represent and manipulate fluid flow fields.

Furthermore, linear algebra plays a crucial role inthermal energysystems, particularly inpower systemsanalysis. It is used to model and optimize the generation,transmission,anddistributionof electric power. Linear algebraic concepts such as matrix operations andeigenvalueproblems are employed to enhance the efficiency, reliability, and economic performance ofpower systems.The application of linear algebra in this context is vital for the design and operation of modernpower systems,includingrenewable energysources andsmart grids.

Overall, the application of linear algebra influid mechanics,fluid dynamics,andthermal energysystems is an example of the profound interconnection betweenmathematicsandengineering.It provides engineers with the necessary tools to model, analyze, and solve complex problems in these domains, leading to advancements in technology and industry.

Extensions and generalizations

This section presents several related topics that do not appear generally in elementary textbooks on linear algebra, but are commonly considered, in advanced mathematics, as parts of linear algebra.

Module theory

The existence of multiplicative inverses in fields is not involved in the axioms defining a vector space. One may thus replace the field of scalars by aring $R$ ,and this gives the structure called amoduleover $R$ ,or $R$ -module.

The concepts of linear independence, span, basis, and linear maps (also calledmodule homomorphisms) are defined for modules exactly as for vector spaces, with the essential difference that, if $R$ is not a field, there are modules that do not have any basis. The modules that have a basis are thefree modules,and those that are spanned by a finite set are thefinitely generated modules.Module homomorphisms between finitely generated free modules may be represented by matrices. The theory of matrices over a ring is similar to that of matrices over a field, except thatdeterminantsexist only if the ring iscommutative,and that a square matrix over a commutative ring isinvertibleonly if its determinant has amultiplicative inversein the ring.

Vector spaces are completely characterized by their dimension (up to an isomorphism). In general, there is not such a complete classification for modules, even if one restricts oneself to finitely generated modules. However, every module is acokernelof a homomorphism of free modules.

Modules over the integers can be identified withabelian groups,since the multiplication by an integer may be identified to a repeated addition. Most of the theory of abelian groups may be extended to modules over aprincipal ideal domain.In particular, over a principal ideal domain, every submodule of a free module is free, and thefundamental theorem of finitely generated abelian groupsmay be extended straightforwardly to finitely generated modules over a principal ring.

There are many rings for which there are algorithms for solving linear equations and systems of linear equations. However, these algorithms have generally acomputational complexitythat is much higher than the similar algorithms over a field. For more details, seeLinear equation over a ring.

Multilinear algebra and tensors

Inmultilinear algebra,one considers multivariable linear transformations, that is, mappings that are linear in each of a number of different variables. This line of inquiry naturally leads to the idea of thedual space,the vector space $V*$ consisting of linear maps $f : V \to F$ whereFis the field of scalars. Multilinear maps $T : V n \to F$ can be described viatensor productsof elements of $V*$ .

If, in addition to vector addition and scalar multiplication, there is a bilinear vector product $V \times V \to V$ ,the vector space is called analgebra;for instance, associative algebras are algebras with an associate vector product (like the algebra of square matrices, or the algebra of polynomials).

Topological vector spaces

Vector spaces that are not finite dimensional often require additional structure to be tractable. Anormed vector spaceis a vector space along with a function called anorm,which measures the "size" of elements. The norm induces ametric,which measures the distance between elements, and induces atopology,which allows for a definition of continuous maps. The metric also allows for a definition oflimitsandcompleteness– a normed vector space that is complete is known as aBanach space.A complete metric space along with the additional structure of aninner product(a conjugate symmetricsesquilinear form) is known as aHilbert space,which is in some sense a particularly well-behaved Banach space.Functional analysisapplies the methods of linear algebra alongside those ofmathematical analysisto study various function spaces; the central objects of study in functional analysis are $L p$ spaces,which are Banach spaces, and especially the $L 2$ space of square integrable functions, which is the only Hilbert space among them. Functional analysis is of particular importance to quantum mechanics, the theory of partial differential equations, digital signal processing, and electrical engineering. It also provides the foundation and theoretical framework that underlies the Fourier transform and related methods.

Explanatory notes

^This axiom is not asserting the associativity of an operation, since there are two operations in question, scalar multiplication $b v$ ;and field multiplication: $ab$ .
^This may have the consequence that some physically interesting solutions are omitted.

Citations

^Banerjee, Sudipto; Roy, Anindya (2014).Linear Algebra and Matrix Analysis for Statistics.Texts in Statistical Science (1st ed.). Chapman and Hall/CRC.ISBN 978-1420095388.
^Strang, Gilbert (July 19, 2005).Linear Algebra and Its Applications(4th ed.). Brooks Cole.ISBN 978-0-03-010567-8.
^Weisstein, Eric."Linear Algebra".MathWorld.Wolfram.Retrieved16 April2012.
^Hart, Roger (2010).The Chinese Roots of Linear Algebra.JHU Press.ISBN 9780801899584.
^^a ^b ^c ^dVitulli, Marie."A Brief History of Linear Algebra and Matrix Theory".Department of Mathematics.University of Oregon. Archived fromthe originalon 2012-09-10.Retrieved2014-07-08.
^Koecher, M., Remmert, R. (1991). Hamilton’s Quaternions. In: Numbers. Graduate Texts in Mathematics, vol 123. Springer, New York, NY.https://doi.org/10.1007/978-1-4612-1005-4_10
^Benjamin Peirce(1872)Linear Associative Algebra,lithograph, new edition with corrections, notes, and an added 1875 paper by Peirce, plus notes by his sonCharles Sanders Peirce,published in theAmerican Journal of Mathematicsv. 4, 1881, Johns Hopkins University, pp. 221–226,GoogleEprintand as an extract, D. Van Nostrand, 1882,GoogleEprint.
^Roman (2005,ch. 1, p. 27)
^Axler (2015)p. 82, §3.59
^Axler (2015)p. 23, §1.45
^Anton (1987,p. 2)
^Beauregard & Fraleigh (1973,p. 65)
^Burden & Faires (1993,p. 324)
^Golub & Van Loan (1996,p. 87)
^Harper (1976,p. 57)
^Katznelson & Katznelson (2008)pp. 76–77, § 4.4.1–4.4.6
^Katznelson & Katznelson (2008)p. 37 §2.1.3
^Halmos (1974)p. 20, §13
^Axler (2015)p. 101, §3.94
^P. K. Jain, Khalil Ahmad (1995)."5.1 Definitions and basic properties of inner product spaces and Hilbert spaces".Functional analysis(2nd ed.). New Age International. p. 203.ISBN 81-224-0801-X.
^Eduard Prugovec̆ki (1981)."Definition 2.1".Quantum mechanics in Hilbert space(2nd ed.). Academic Press. pp. 18ff.ISBN 0-12-566060-X.
^Emil Artin(1957)Geometric AlgebraInterscience Publishers
^Savov, Ivan (2017).No Bullshit Guide to Linear Algebra.MinireferenceCo. pp. 150–155.ISBN 9780992001025.
^"MIT OpenCourseWare. Special Topics in Mathematics with Applications: Linear Algebra and the Calculus of Variations - Mechanical Engineering".
^"FAMU-FSU College of Engineering. ME Undergraduate Curriculum".
^"University of Colorado Denver. Energy and Power Systems".

General and cited sources

Anton, Howard (1987),Elementary Linear Algebra(5th ed.), New York:Wiley,ISBN 0-471-84819-0
Axler, Sheldon(18 December 2014),Linear Algebra Done Right,Undergraduate Texts in Mathematics(3rd ed.),Springer Publishing(published 2015),ISBN 978-3-319-11079-0,MR 3308468
Beauregard, Raymond A.; Fraleigh, John B. (1973),A First Course In Linear Algebra: with Optional Introduction to Groups, Rings, and Fields,Boston:Houghton Mifflin Company,ISBN 0-395-14017-X
Burden, Richard L.; Faires, J. Douglas (1993),Numerical Analysis(5th ed.), Boston:Prindle, Weber and Schmidt,ISBN 0-534-93219-3
Golub, Gene H.;Van Loan, Charles F.(1996),Matrix Computations,Johns Hopkins Studies in Mathematical Sciences (3rd ed.), Baltimore:Johns Hopkins University Press,ISBN 978-0-8018-5414-9
Halmos, Paul Richard(1974),Finite-Dimensional Vector Spaces,Undergraduate Texts in Mathematics(1958 2nd ed.),Springer Publishing,ISBN 0-387-90093-4,OCLC 1251216
Harper, Charlie (1976),Introduction to Mathematical Physics,New Jersey:Prentice-Hall,ISBN 0-13-487538-9
Katznelson, Yitzhak;Katznelson, Yonatan R. (2008),A (Terse) Introduction to Linear Algebra,American Mathematical Society,ISBN 978-0-8218-4419-9
Roman, Steven(March 22, 2005),Advanced Linear Algebra,Graduate Texts in Mathematics(2nd ed.), Springer,ISBN 978-0-387-24766-3

External links

Online Resources

MIT Linear Algebra Video Lectures,a series of 34 recorded lectures by ProfessorGilbert Strang(Spring 2010)
International Linear Algebra Society
"Linear algebra",Encyclopedia of Mathematics,EMS Press,2001 [1994]
Linear AlgebraonMathWorld
Matrix and Linear Algebra TermsonEarliest Known Uses of Some of the Words of Mathematics
Earliest Uses of Symbols for Matrices and VectorsonEarliest Uses of Various Mathematical Symbols
Essence of linear algebra,a video presentation from3Blue1Brownof the basics of linear algebra, with emphasis on the relationship between the geometric, the matrix and the abstract points of view

Online books

Beezer, Robert A. (2009) [2004].A First Course in Linear Algebra.Gainesville, Florida:University Press of Florida.ISBN 9781616100049.
Connell, Edwin H. (2004) [1999].Elements of Abstract and Linear Algebra.University of Miami,Coral Gables, Florida:Self-published.
Hefferon, Jim(2020).Linear Algebra(4th ed.).Ann Arbor, Michigan:Orthogonal Publishing.ISBN 978-1-944325-11-4.OCLC 1178900366.OL 30872051M.
Margalit, Dan;Rabinoff, Joseph (2019).Interactive Linear Algebra.Georgia Institute of Technology,Atlanta, Georgia:Self-published.
Matthews, Keith R. (2013) [1991].Elementary Linear Algebra.University of Queensland,Brisbane, Australia:Self-published.
Mikaelian, Vahagn H. (2020) [2017].Linear Algebra: Theory and Algorithms.Yerevan, Armenia:Self-published – viaResearchGate.
Sharipov, Ruslan,Course of linear algebra and multidimensional geometry
Treil, Sergei,Linear Algebra Done Wrong

[9] This axiom is not asserting the associativity of an operation, since there are two operations in question, scalar multiplication $b v$ ;and field multiplication: $ab$ .

[24] This may have the consequence that some physically interesting solutions are omitted.

[1] Banerjee, Sudipto; Roy, Anindya (2014).Linear Algebra and Matrix Analysis for Statistics.Texts in Statistical Science (1st ed.). Chapman and Hall/CRC.ISBN 978-1420095388.

[2] Strang, Gilbert (July 19, 2005).Linear Algebra and Its Applications(4th ed.). Brooks Cole.ISBN 978-0-03-010567-8.

[3] Weisstein, Eric."Linear Algebra".MathWorld.Wolfram.Retrieved16 April2012.

[4] Hart, Roger (2010).The Chinese Roots of Linear Algebra.JHU Press.ISBN 9780801899584.

[Vitulli,_Marie-5] Vitulli, Marie."A Brief History of Linear Algebra and Matrix Theory".Department of Mathematics.University of Oregon. Archived fromthe originalon 2012-09-10.Retrieved2014-07-08.

[6] Koecher, M., Remmert, R. (1991). Hamilton’s Quaternions. In: Numbers. Graduate Texts in Mathematics, vol 123. Springer, New York, NY.https://doi.org/10.1007/978-1-4612-1005-4_10

[7] Benjamin Peirce(1872)Linear Associative Algebra,lithograph, new edition with corrections, notes, and an added 1875 paper by Peirce, plus notes by his sonCharles Sanders Peirce,published in theAmerican Journal of Mathematicsv. 4, 1881, Johns Hopkins University, pp. 221–226,GoogleEprintand as an extract, D. Van Nostrand, 1882,GoogleEprint.

[8] Roman (2005,ch. 1, p. 27)

[10] Axler (2015)p. 82, §3.59

[11] Axler (2015)p. 23, §1.45

[12] Anton (1987,p. 2)

[13] Beauregard & Fraleigh (1973,p. 65)

[14] Burden & Faires (1993,p. 324)

[15] Golub & Van Loan (1996,p. 87)

[16] Harper (1976,p. 57)

[17] Katznelson & Katznelson (2008)pp. 76–77, § 4.4.1–4.4.6

[18] Katznelson & Katznelson (2008)p. 37 §2.1.3

[19] Halmos (1974)p. 20, §13

[20] Axler (2015)p. 101, §3.94

[Jain-21] P. K. Jain, Khalil Ahmad (1995)."5.1 Definitions and basic properties of inner product spaces and Hilbert spaces".Functional analysis(2nd ed.). New Age International. p. 203.ISBN 81-224-0801-X.

[Prugovec̆ki-22] Eduard Prugovec̆ki (1981)."Definition 2.1".Quantum mechanics in Hilbert space(2nd ed.). Academic Press. pp. 18ff.ISBN 0-12-566060-X.

[23] Emil Artin(1957)Geometric AlgebraInterscience Publishers

[25] Savov, Ivan (2017).No Bullshit Guide to Linear Algebra.MinireferenceCo. pp. 150–155.ISBN 9780992001025.

[26] "MIT OpenCourseWare. Special Topics in Mathematics with Applications: Linear Algebra and the Calculus of Variations - Mechanical Engineering".

[27] "FAMU-FSU College of Engineering. ME Undergraduate Curriculum".

[28] "University of Colorado Denver. Energy and Power Systems".

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[a]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[b]

[23]

[24]

[25]

[26]

v t e Linear algebra
Outline Glossary
Basic concepts	Scalar Vector Vector space Scalar multiplication Vector projection Linear span Linear map Linear projection Linear independence Linear combination Multilinear map Basis Change of basis Row and column vectors Row and column spaces Kernel Eigenvalues and eigenvectors Transpose Linear equations
Matrices	Block Decomposition Invertible Minor Multiplication Rank Transformation Cramer's rule Gaussian elimination
Bilinear	Orthogonality Dot product Hadamard product Inner product space Outer product Kronecker product Gram–Schmidt process
Multilinear algebra	Determinant Cross product Triple product Seven-dimensional cross product Geometric algebra Exterior algebra Bivector Multivector Tensor Outermorphism
Vector spaceconstructions	Dual Direct sum Function space Quotient Subspace Tensor product
Numerical	Floating-point Numerical stability Basic Linear Algebra Subprograms Sparse matrix Comparison of linear algebra libraries
Category

v t e Majormathematicsareas
History Timeline Future Lists Glossary
Foundations	Category theory Information theory Mathematical logic Philosophy of mathematics Set theory Type theory
Algebra	Abstract Commutative Elementary Group theory Linear Multilinear Universal Homological
Analysis	Calculus Real analysis Complex analysis Hypercomplex analysis Differential equations Functional analysis Harmonic analysis Measure theory
Discrete	Combinatorics Graph theory Order theory
Geometry	Algebraic Analytic Arithmetic Differential Discrete Euclidean Finite
Number theory	Arithmetic Algebraic number theory Analytic number theory Diophantine geometry
Topology	General Algebraic Differential Geometric Homotopy theory
Applied	Engineering mathematics Mathematical biology Mathematical chemistry Mathematical economics Mathematical finance Mathematical physics Mathematical psychology Mathematical sociology Mathematical statistics Probability Statistics Systems science Control theory Game theory Operations research
Computational	Computer science Theory of computation Computational complexity theory Numerical analysis Optimization Computer algebra
Related topics	Mathematicians lists Informal mathematics Films about mathematicians Recreational mathematics Mathematics and art Mathematics education
Mathematics portal Category Commons WikiProject

Authority control databases
International	FAST
National	Spain France BnF data Germany Israel United States Latvia Japan Czech Republic

Linear algebra

History

Vector spaces

Linear maps

Subspaces, span, and basis

Matrices

Linear systems

Endomorphisms and square matrices

Determinant

Eigenvalues and eigenvectors

Duality

Dual map

Inner-product spaces

Relationship with geometry

Usage and applications

Functional analysis

Scientific computation

Geometry of ambient space

Study of complex systems

Fluid Mechanics, Fluid Dynamics, and Thermal Energy Systems

Extensions and generalizations

Module theory

Multilinear algebra and tensors

Topological vector spaces

See also

Explanatory notes

Citations

General and cited sources

Further reading

History

Introductory textbooks

Advanced textbooks

Study guides and outlines

External links

Online Resources

Online books