Block matrix

Inmathematics,ablock matrixor apartitioned matrixis amatrixthat is interpreted as having been broken into sections calledblocksorsubmatrices.

Intuitively, a matrix interpreted as a block matrix can be visualized as the original matrix with a collection of horizontal and vertical lines, which break it up, orpartitionit, into a collection of smaller matrices.For example, the 3x4 matrix presented below is divided by horizontal and vertical lines into four blocks: the top-left 2x3 block, the top-right 2x1 block, the bottom-left 1x3 block, and the bottom-right 1x1 block.

$\left[{\begin{array}{ccc|c}a_{11}&a_{12}&a_{13}&b_{1}\\a_{21}&a_{22}&a_{23}&b_{2}\\\hline c_{1}&c_{2}&c_{3}&d\end{array}}\right]$

Any matrix may be interpreted as a block matrix in one or more ways, with each interpretation defined by how its rows and columns are partitioned.

This notion can be made more precise for an$n$by$m$matrix$M$by partitioning$n$into a collection${\text{rowgroups}}$,and then partitioning$m$into a collection${\text{colgroups}}$.The original matrix is then considered as the "total" of these groups, in the sense that the$(i,j)$entry of the original matrix corresponds in a1-to-1way with some$(s,t)$offsetentry of some$(x,y)$,where$x\in {\text{rowgroups}}$and$y\in {\text{colgroups}}$.

Block matrix algebra arises in general frombiproductsincategoriesof matrices.

A 168×168 element block matrix with 12×12, 12×24, 24×12, and 24×24 sub-matrices. Non-zero elements are in blue, zero elements are grayed.

Example

The matrix

$\mathbf {P} ={\begin{bmatrix}1&2&2&7\\1&5&6&2\\3&3&4&5\\3&3&6&7\end{bmatrix}}$

can be visualized as divided into four blocks, as

$\mathbf {P} =\left[{\begin{array}{cc|cc}1&2&2&7\\1&5&6&2\\\hline 3&3&4&5\\3&3&6&7\end{array}}\right]$.

The horizontal and vertical lines have no special mathematical meaning,but are a common way to visualize a partition.By this partition,$P$is partitioned into four 2×2 blocks, as

$\mathbf {P} _{11}={\begin{bmatrix}1&2\\1&5\end{bmatrix}},\quad \mathbf {P} _{12}={\begin{bmatrix}2&7\\6&2\end{bmatrix}},\quad \mathbf {P} _{21}={\begin{bmatrix}3&3\\3&3\end{bmatrix}},\quad \mathbf {P} _{22}={\begin{bmatrix}4&5\\6&7\end{bmatrix}}.$

The partitioned matrix can then be written as

$\mathbf {P} ={\begin{bmatrix}\mathbf {P} _{11}&\mathbf {P} _{12}\\\mathbf {P} _{21}&\mathbf {P} _{22}\end{bmatrix}}.$

Formal definition

Let$A\in \mathbb {C} ^{m\times n}$.Apartitioningof$A$is a representation of$A$in the form

$A={\begin{bmatrix}A_{11}&A_{12}&\cdots &A_{1q}\\A_{21}&A_{22}&\cdots &A_{2q}\\\vdots &\vdots &\ddots &\vdots \\A_{p1}&A_{p2}&\cdots &A_{pq}\end{bmatrix}}$,

where$A_{ij}\in \mathbb {C} ^{m_{i}\times n_{j}}$are contiguous submatrices,$\sum _{i=1}^{p}m_{i}=m$,and$\sum _{j=1}^{q}n_{j}=n$.The elements$A_{ij}$of the partition are calledblocks.

By this definition, the blocks in any one column must all have the same number of columns.Similarly, the blocks in any one row must have the same number of rows.

Partitioning methods

A matrix can be partitioned in many ways.For example, a matrix$A$is said to bepartitioned by columnsif it is written as

$A=(a_{1}\ a_{2}\ \cdots \ a_{n})$,

where$a_{j}$is the$j$th column of$A$.A matrix can also bepartitioned by rows:

$A={\begin{bmatrix}a_{1}^{T}\\a_{2}^{T}\\\vdots \\a_{m}^{T}\end{bmatrix}}$,

where$a_{i}^{T}$is the$i$th row of$A$.

Common partitions

Often,we encounter the 2x2 partition

$A={\begin{bmatrix}A_{11}&A_{12}\\A_{21}&A_{22}\end{bmatrix}}$,

particularly in the form where$A_{11}$is a scalar:

$A={\begin{bmatrix}a_{11}&a_{12}^{T}\\a_{21}&A_{22}\end{bmatrix}}$.

Block matrix operations

Transpose

Let

$A={\begin{bmatrix}A_{11}&A_{12}&\cdots &A_{1q}\\A_{21}&A_{22}&\cdots &A_{2q}\\\vdots &\vdots &\ddots &\vdots \\A_{p1}&A_{p2}&\cdots &A_{pq}\end{bmatrix}}$

where$A_{ij}\in \mathbb {C} ^{k_{i}\times \ell _{j}}$.(This matrix$A$will be reused in§ Additionand§ Multiplication.) Then its transpose is

$A^{T}={\begin{bmatrix}A_{11}^{T}&A_{21}^{T}&\cdots &A_{p1}^{T}\\A_{12}^{T}&A_{22}^{T}&\cdots &A_{p2}^{T}\\\vdots &\vdots &\ddots &\vdots \\A_{1q}^{T}&A_{2q}^{T}&\cdots &A_{pq}^{T}\end{bmatrix}}$,

and the same equation holds with the transpose replaced by the conjugate transpose.

Block transpose

A special form of matrixtransposecan also be defined for block matrices, where individual blocks are reordered but not transposed. Let$A=(B_{ij})$be a$k\times l$block matrix with$m\times n$blocks$B_{ij}$,the block transpose of$A$is the$l\times k$block matrix$A^{\mathcal {B}}$with$m\times n$blocks$\left(A^{\mathcal {B}}\right)_{ij}=B_{ji}$.As with the conventional trace operator, the block transpose is alinear mappingsuch that$(A+C)^{\mathcal {B}}=A^{\mathcal {B}}+C^{\mathcal {B}}$.However, in general the property$(AC)^{\mathcal {B}}=C^{\mathcal {B}}A^{\mathcal {B}}$does not hold unless the blocks of$A$and$C$commute.

Addition

Let

$B={\begin{bmatrix}B_{11}&B_{12}&\cdots &B_{1s}\\B_{21}&B_{22}&\cdots &B_{2s}\\\vdots &\vdots &\ddots &\vdots \\B_{r1}&B_{r2}&\cdots &B_{rs}\end{bmatrix}}$,

where$B_{ij}\in \mathbb {C} ^{m_{i}\times n_{j}}$,and let$A$be the matrix defined in§ Transpose.(This matrix$B$will be reused in§ Multiplication.) Then if$p=r$,$q=s$,$k_{i}=m_{i}$,and$\ell _{j}=n_{j}$,then

$A+B={\begin{bmatrix}A_{11}+B_{11}&A_{12}+B_{12}&\cdots &A_{1q}+B_{1q}\\A_{21}+B_{21}&A_{22}+B_{22}&\cdots &A_{2q}+B_{2q}\\\vdots &\vdots &\ddots &\vdots \\A_{p1}+B_{p1}&A_{p2}+B_{p2}&\cdots &A_{pq}+B_{pq}\end{bmatrix}}$.

Multiplication

It is possible to use a block partitioned matrix product that involves only algebra on submatrices of the factors. The partitioning of the factors is not arbitrary, however, and requires "conformablepartitions "between two matrices$A$and$B$such that all submatrix products that will be used are defined.

Two matrices$A$and$B$are said to be partitioned conformally for the product$AB$,when$A$and$B$are partitioned into submatrices and if the multiplication$AB$is carried out treating the submatrices as if they are scalars, but keeping the order, and when all products and sums of submatrices involved are defined.

— Arak M. Mathai and Hans J. Haubold,Linear Algebra: A Course for Physicists and Engineers

Let$A$be the matrix defined in§ Transpose,and let$B$be the matrix defined in§ Addition.Then the matrix product

$C=AB$

can be performed blockwise, yielding$C$as an$(p\times s)$matrix. The matrices in the resulting matrix$C$are calculated by multiplying:

$C_{ij}=\sum _{k=1}^{q}A_{ik}B_{kj}.$

Or, using theEinstein notationthat implicitly sums over repeated indices:

$C_{ij}=A_{ik}B_{kj}.$

Depicting$C$as a matrix, we have

$C=AB={\begin{bmatrix}\sum _{i=1}^{q}A_{1i}B_{i1}&\sum _{i=1}^{q}A_{1i}B_{i2}&\cdots &\sum _{i=1}^{q}A_{1i}B_{is}\\\sum _{i=1}^{q}A_{2i}B_{i1}&\sum _{i=1}^{q}A_{2i}B_{i2}&\cdots &\sum _{i=1}^{q}A_{2i}B_{is}\\\vdots &\vdots &\ddots &\vdots \\\sum _{i=1}^{q}A_{pi}B_{i1}&\sum _{i=1}^{q}A_{pi}B_{i2}&\cdots &\sum _{i=1}^{q}A_{pi}B_{is}\end{bmatrix}}$.

Inversion

For more details and derivation using block LDU decomposition, seeSchur complement.

See also:Helmert–Wolf blocking

If a matrix is partitioned into four blocks, it can beinverted blockwiseas follows:

${P}={\begin{bmatrix}{A}&{B}\\{C}&{D}\end{bmatrix}}^{-1}={\begin{bmatrix}{A}^{-1}+{A}^{-1}{B}\left({D}-{CA}^{-1}{B}\right)^{-1}{CA}^{-1}&-{A}^{-1}{B}\left({D}-{CA}^{-1}{B}\right)^{-1}\\-\left({D}-{CA}^{-1}{B}\right)^{-1}{CA}^{-1}&\left({D}-{CA}^{-1}{B}\right)^{-1}\end{bmatrix}},$

whereAandDare square blocks of arbitrary size, andBandCareconformablewith them for partitioning. Furthermore,Aand the Schur complement ofAinP:P/A=D−CA⁻¹Bmust be invertible.

Equivalently, by permuting the blocks:

${P}={\begin{bmatrix}{A}&{B}\\{C}&{D}\end{bmatrix}}^{-1}={\begin{bmatrix}\left({A}-{BD}^{-1}{C}\right)^{-1}&-\left({A}-{BD}^{-1}{C}\right)^{-1}{BD}^{-1}\\-{D}^{-1}{C}\left({A}-{BD}^{-1}{C}\right)^{-1}&\quad {D}^{-1}+{D}^{-1}{C}\left({A}-{BD}^{-1}{C}\right)^{-1}{BD}^{-1}\end{bmatrix}}.$

Here,Dand the Schur complement ofDinP:P/D=A−BD⁻¹Cmust be invertible.

IfAandDare both invertible, then:

${\begin{bmatrix}{A}&{B}\\{C}&{D}\end{bmatrix}}^{-1}={\begin{bmatrix}\left({A}-{B}{D}^{-1}{C}\right)^{-1}&{0}\\{0}&\left({D}-{C}{A}^{-1}{B}\right)^{-1}\end{bmatrix}}{\begin{bmatrix}{I}&-{B}{D}^{-1}\\-{C}{A}^{-1}&{I}\end{bmatrix}}.$

By theWeinstein–Aronszajn identity,one of the two matrices in the block-diagonal matrix is invertible exactly when the other is.

Determinant

The formula for the determinant of a$2\times 2$-matrix above continues to hold, under appropriate further assumptions, for a matrix composed of four submatrices$A,B,C,D$.The easiest such formula, which can be proven using either the Leibniz formula or a factorization involving theSchur complement,is

$\det {\begin{bmatrix}A&0\\C&D\end{bmatrix}}=\det(A)\det(D)=\det {\begin{bmatrix}A&B\\0&D\end{bmatrix}}.$

Using this formula, we can derive thatcharacteristic polynomialsof${\begin{bmatrix}A&0\\C&D\end{bmatrix}}$and${\begin{bmatrix}A&B\\0&D\end{bmatrix}}$are same and equal to the product of characteristic polynomials of$A$and$D$.Furthermore, If${\begin{bmatrix}A&0\\C&D\end{bmatrix}}$or${\begin{bmatrix}A&B\\0&D\end{bmatrix}}$isdiagonalizable,then$A$and$D$are diagonalizable too. The converse is false; simply check${\begin{bmatrix}1&1\\0&1\end{bmatrix}}$.

If$A$isinvertible,one has

$\det {\begin{bmatrix}A&B\\C&D\end{bmatrix}}=\det(A)\det \left(D-CA^{-1}B\right),$

and if$D$is invertible, one has

$\det {\begin{bmatrix}A&B\\C&D\end{bmatrix}}=\det(D)\det \left(A-BD^{-1}C\right).$

If the blocks are square matrices of thesamesize further formulas hold. For example, if$C$and$D$commute(i.e.,$CD=DC$), then

$\det {\begin{bmatrix}A&B\\C&D\end{bmatrix}}=\det(AD-BC).$

This formula has been generalized to matrices composed of more than$2\times 2$blocks, again under appropriate commutativity conditions among the individual blocks.

For$A=D$and$B=C$,the following formula holds (even if$A$and$B$do not commute)

$\det {\begin{bmatrix}A&B\\B&A\end{bmatrix}}=\det(A-B)\det(A+B).$

Special types of block matrices

Direct sums and block diagonal matrices

Direct sum

See also:Matrix addition § Direct sum

For any arbitrary matricesA(of sizem×n) andB(of sizep×q), we have thedirect sumofAandB,denoted byA$\oplus$Band defined as

${A}\oplus {B}={\begin{bmatrix}a_{11}&\cdots &a_{1n}&0&\cdots &0\\\vdots &\ddots &\vdots &\vdots &\ddots &\vdots \\a_{m1}&\cdots &a_{mn}&0&\cdots &0\\0&\cdots &0&b_{11}&\cdots &b_{1q}\\\vdots &\ddots &\vdots &\vdots &\ddots &\vdots \\0&\cdots &0&b_{p1}&\cdots &b_{pq}\end{bmatrix}}.$

For instance,

${\begin{bmatrix}1&3&2\\2&3&1\end{bmatrix}}\oplus {\begin{bmatrix}1&6\\0&1\end{bmatrix}}={\begin{bmatrix}1&3&2&0&0\\2&3&1&0&0\\0&0&0&1&6\\0&0&0&0&1\end{bmatrix}}.$

This operation generalizes naturally to arbitrary dimensioned arrays (provided thatAandBhave the same number of dimensions).

Note that any element in thedirect sumof twovector spacesof matrices could be represented as a direct sum of two matrices.

Block diagonal matrices

See also:Diagonal matrix

Ablock diagonal matrixis a block matrix that is asquare matrixsuch that the main-diagonal blocks are square matrices and all off-diagonal blocks are zero matrices.That is, a block diagonal matrixAhas the form

${A}={\begin{bmatrix}{A}_{1}&{0}&\cdots &{0}\\{0}&{A}_{2}&\cdots &{0}\\\vdots &\vdots &\ddots &\vdots \\{0}&{0}&\cdots &{A}_{n}\end{bmatrix}}$

whereA_kis a square matrix for allk= 1,...,n.In other words, matrixAis thedirect sumofA₁,...,A_n.It can also be indicated asA₁⊕A₂⊕... ⊕A_nor diag(A₁,A₂,...,A_n)(the latter being the same formalism used for adiagonal matrix). Any square matrix can trivially be considered a block diagonal matrix with only one block.

For thedeterminantandtrace,the following properties hold:

${\begin{aligned}\det {A}&=\det {A}_{1}\times \cdots \times \det {A}_{n},\end{aligned}}$and

${\begin{aligned}\operatorname {tr} {A}&=\operatorname {tr} {A}_{1}+\cdots +\operatorname {tr} {A}_{n}.\end{aligned}}$

A block diagonal matrix is invertibleif and only ifeach of its main-diagonal blocks are invertible, and in this case its inverse is another block diagonal matrix given by

${\begin{bmatrix}{A}_{1}&{0}&\cdots &{0}\\{0}&{A}_{2}&\cdots &{0}\\\vdots &\vdots &\ddots &\vdots \\{0}&{0}&\cdots &{A}_{n}\end{bmatrix}}^{-1}={\begin{bmatrix}{A}_{1}^{-1}&{0}&\cdots &{0}\\{0}&{A}_{2}^{-1}&\cdots &{0}\\\vdots &\vdots &\ddots &\vdots \\{0}&{0}&\cdots &{A}_{n}^{-1}\end{bmatrix}}.$

Theeigenvaluesand eigenvectorsof${A}$are simply those of the${A}_{k}$s combined.

Block tridiagonal matrices

See also:Tridiagonal matrix

Ablock tridiagonal matrixis another special block matrix, which is just like the block diagonal matrix asquare matrix,having square matrices (blocks) in the lower diagonal,main diagonaland upper diagonal, with all other blocks being zero matrices. It is essentially atridiagonal matrixbut has submatrices in places of scalars. A block tridiagonal matrix$A$has the form

${A}={\begin{bmatrix}{B}_{1}&{C}_{1}&&&\cdots &&{0}\\{A}_{2}&{B}_{2}&{C}_{2}&&&&\\&\ddots &\ddots &\ddots &&&\vdots \\&&{A}_{k}&{B}_{k}&{C}_{k}&&\\\vdots &&&\ddots &\ddots &\ddots &\\&&&&{A}_{n-1}&{B}_{n-1}&{C}_{n-1}\\{0}&&\cdots &&&{A}_{n}&{B}_{n}\end{bmatrix}}$

where${A}_{k}$,${B}_{k}$and${C}_{k}$are square sub-matrices of the lower, main and upper diagonal respectively.

Block tridiagonal matrices are often encountered in numerical solutions of engineering problems (e.g.,computational fluid dynamics). Optimized numerical methods forLU factorizationare availableand hence efficient solution algorithms for equation systems with a block tridiagonal matrix as coefficient matrix. TheThomas algorithm,used for efficient solution of equation systems involving atridiagonal matrixcan also be applied using matrix operations to block tridiagonal matrices (see alsoBlock LU decomposition).

Block triangular matrices

See also:Triangular matrix

Upper block triangular

A matrix$A$isupper block triangular(orblock upper triangular) if

$A={\begin{bmatrix}A_{11}&A_{12}&\cdots &A_{1k}\\0&A_{22}&\cdots &A_{2k}\\\vdots &\vdots &\ddots &\vdots \\0&0&\cdots &A_{kk}\end{bmatrix}}$,

where$A_{ij}\in \mathbb {F} ^{n_{i}\times n_{j}}$for all$i,j=1,\ldots,k$.

Lower block triangular

A matrix$A$islower block triangularif

$A={\begin{bmatrix}A_{11}&0&\cdots &0\\A_{21}&A_{22}&\cdots &0\\\vdots &\vdots &\ddots &\vdots \\A_{k1}&A_{k2}&\cdots &A_{kk}\end{bmatrix}}$,

where$A_{ij}\in \mathbb {F} ^{n_{i}\times n_{j}}$for all$i,j=1,\ldots,k$.

Block Toeplitz matrices

See also:Toeplitz matrix

Ablock Toeplitz matrixis another special block matrix, which contains blocks that are repeated down the diagonals of the matrix, as aToeplitz matrixhas elements repeated down the diagonal.

A matrix$A$isblock Toeplitzif$A_{(i,j)}=A_{(k,l)}$for all$k-i=l-j$,that is,

$A={\begin{bmatrix}A_{1}&A_{2}&A_{3}&\cdots \\A_{4}&A_{1}&A_{2}&\cdots \\A_{5}&A_{4}&A_{1}&\cdots \\\vdots &\vdots &\vdots &\ddots \end{bmatrix}}$,

where$A_{i}\in \mathbb {F} ^{n_{i}\times m_{i}}$.

Block Hankel matrices

See also:Hankel matrix

A matrix$A$isblock Hankelif$A_{(i,j)}=A_{(k,l)}$for all$i+j=k+l$,that is,

$A={\begin{bmatrix}A_{1}&A_{2}&A_{3}&\cdots \\A_{2}&A_{3}&A_{4}&\cdots \\A_{3}&A_{4}&A_{5}&\cdots \\\vdots &\vdots &\vdots &\ddots \end{bmatrix}}$,

where$A_{i}\in \mathbb {F} ^{n_{i}\times m_{i}}$.

See also

• Kronecker product(matrix direct product resulting in a block matrix)
• Jordan normal form(canonical form of a linear operator on a finite-dimensional complex vector space)
• Strassen algorithm(algorithm for matrix multiplication that is faster than the conventional matrix multiplication algorithm)

References

• Strang, Gilbert(1999)."Lecture 3: Multiplication and inverse matrices".MIT Open Course ware. 18:30–21:10.