Mercer's theorem

Inmathematics,specificallyfunctional analysis,Mercer's theoremis a representation of a symmetricpositive-definitefunction on a square as a sum of a convergent sequence of product functions. This theorem, presented in (Mercer 1909), is one of the most notable results of the work ofJames Mercer(1883–1932). It is an important theoretical tool in the theory ofintegral equations;it is used in theHilbert spacetheory ofstochastic processes,for example theKarhunen–Loève theorem;and it is also used in thereproducing kernel Hilbert spacetheory where it characterizes a symmetricpositive-definite kernelas a reproducing kernel.

Introduction

To explain Mercer'stheorem,we first consider an important special case; seebelowfor a more general formulation. Akernel,in this context, is a symmetric continuous function

$K:[a,b]\times [a,b]\rightarrow \mathbb {R}$

where symmetric means that$K(x,y)=K(y,x)$for all$x,y\in [a,b]$.

Kis said to be apositive-definite kernelif and only if

$\sum _{i=1}^{n}\sum _{j=1}^{n}K(x_{i},x_{j})c_{i}c_{j}\geq 0$

for all finite sequences of pointsx₁,...,x_nof [a,b] and all choices of real numbersc₁,...,c_n.Note that the term "positive-definite" is well-established in literature despite the weak inequality in the definition.

Associated toKis alinear operator(more specifically aHilbert–Schmidt integral operator) on functions defined by the integral

$[T_{K}\varphi ](x)=\int _{a}^{b}K(x,s)\varphi (s)\,ds.$

For technical considerations we assume$\varphi$can range through the space L²[a,b] (seeLp space) of square-integrable real-valued functions. SinceT_Kis a linear operator, we can talk abouteigenvaluesandeigenfunctionsofT_K.

Theorem.SupposeKis a continuous symmetric positive-definite kernel. Then there is anorthonormal basis {e_i}_iofL²[a,b] consisting of eigenfunctions ofT_Ksuch that the corresponding sequence of eigenvalues {λ_i}_iis nonnegative. The eigenfunctions corresponding to non-zero eigenvalues are continuous on [a,b] andKhas the representation

$K(s,t)=\sum _{j=1}^{\infty }\lambda _{j}\,e_{j}(s)\,e_{j}(t)$

where the convergence is absolute and uniform.

Details

We now explain in greater detail the structure of the proof of Mercer's theorem, particularly how it relates tospectral theory of compact operators.

• The mapK↦T_Kisinjective.
• T_Kis a non-negative symmetric compact operator onL²[a,b]; moreoverK(x,x) ≥ 0.

To show compactness, show that the image of theunit ballofL²[a,b] underT_Kequicontinuousand applyAscoli's theorem,to show that the image of the unit ball is relatively compact in C([a,b]) with theuniform normanda fortioriinL²[a,b].

Now apply thespectral theoremfor compact operators on Hilbert spaces toT_Kto show the existence of the orthonormal basis {e_i}_iof L²[a,b]

$\lambda _{i}e_{i}(t)=[T_{K}e_{i}](t)=\int _{a}^{b}K(t,s)e_{i}(s)\,ds.$

If λ_i≠ 0, the eigenvector (eigenfunction)e_iis seen to be continuous on [a,b]. Now

$\sum _{i=1}^{\infty }\lambda _{i}|e_{i}(t)e_{i}(s)|\leq \sup _{x\in [a,b]}|K(x,x)|,$

which shows that the sequence

$\sum _{i=1}^{\infty }\lambda _{i}e_{i}(t)e_{i}(s)$

converges absolutely and uniformly to a kernelK₀which is easily seen to define the same operator as the kernelK.HenceK=K₀from which Mercer's theorem follows.

Finally, to show non-negativity of the eigenvalues one can write$\lambda \langle f,f\rangle =\langle f,T_{K}f\rangle$and expressing the right hand side as an integral well-approximated by its Riemann sums, which are non-negative by positive-definiteness ofK,implying$\lambda \langle f,f\rangle \geq 0$,implying$\lambda \geq 0$.

Trace

The following is immediate:

Theorem.SupposeKis a continuous symmetric positive-definite kernel;T_Khas a sequence of nonnegative eigenvalues {λ_i}_i.Then

$\int _{a}^{b}K(t,t)\,dt=\sum _{i}\lambda _{i}.$

This shows that the operatorT_Kis atrace classoperator and

$\operatorname {trace} (T_{K})=\int _{a}^{b}K(t,t)\,dt.$

Generalizations

Mercer's theorem itself is a generalization of the result that anysymmetricpositive-semidefinite matrixis theGramian matrixof a set of vectors.

The first generalizationreplaces the interval [a,b] with anycompact Hausdorff spaceand Lebesgue measure on [a,b] is replaced by a finite countably additive measure μ on theBorel algebraofXwhose support isX.This means that μ(U) > 0 for any nonempty open subsetUofX.

A recent generalizationreplaces these conditions by the following: the setXis afirst-countabletopological space endowed with a Borel (complete) measure μ.Xis the support of μ and, for allxinX,there is an open setUcontainingxand having finite measure. Then essentially the same result holds:

Theorem.SupposeKis a continuous symmetric positive-definite kernel onX.If the function κ isL¹_μ(X), where κ(x)=K(x,x), for allxinX,then there is anorthonormal set {e_i}_iofL²_μ(X) consisting of eigenfunctions ofT_Ksuch that corresponding sequence of eigenvalues {λ_i}_iis nonnegative. The eigenfunctions corresponding to non-zero eigenvalues are continuous onXandKhas the representation

$K(s,t)=\sum _{j=1}^{\infty }\lambda _{j}\,e_{j}(s)\,e_{j}(t)$

where the convergence is absolute and uniform on compact subsets ofX.

The next generalizationdeals with representations ofmeasurablekernels.

Let (X,M,μ) be a σ-finite measure space. AnL²(or square-integrable) kernel onXis a function

$K\in L_{\mu \otimes \mu }^{2}(X\times X).$

L²kernels define a bounded operatorT_Kby the formula

$\langle T_{K}\varphi,\psi \rangle =\int _{X\times X}K(y,x)\varphi (y)\psi (x)\,d[\mu \otimes \mu ](y,x).$

T_Kis a compact operator (actually it is even aHilbert–Schmidt operator). If the kernelKis symmetric, by thespectral theorem,T_Khas an orthonormal basis of eigenvectors. Those eigenvectors that correspond to non-zero eigenvalues can be arranged in a sequence {e_i}_i(regardless of separability).

Theorem.IfKis a symmetric positive-definite kernel on (X,M,μ), then

$K(y,x)=\sum _{i\in \mathbb {N} }\lambda _{i}e_{i}(y)e_{i}(x)$

where the convergence in theL²norm. Note that when continuity of the kernel is not assumed, the expansion no longer converges uniformly.

Mercer's condition

Inmathematics,areal-valuedfunctionK(x,y) is said to fulfillMercer's conditionif for allsquare-integrable functionsg(x) one has

$\iint g(x)K(x,y)g(y)\,dx\,dy\geq 0.$

Discrete analog

This is analogous to the definition of apositive-semidefinite matrix.This is a matrix$K$of dimension$N$,which satisfies, for all vectors$g$,the property

$(g,Kg)=g^{T}{\cdot }Kg=\sum _{i=1}^{N}\sum _{j=1}^{N}\,g_{i}\,K_{ij}\,g_{j}\geq 0$.

Examples

A positive constant function

$K(x,y)=c\,$

satisfies Mercer's condition, as then the integral becomes byFubini's theorem

$\iint g(x)\,c\,g(y)\,dx\,dy=c\int \!g(x)\,dx\int \!g(y)\,dy=c\left(\int \!g(x)\,dx\right)^{2}$

which is indeednon-negative.

See also

• Kernel trick
• Representer theorem
• Reproducing kernel Hilbert space
• Spectral theory

References

• Adriaan Zaanen,Linear Analysis,North Holland Publishing Co., 1960,
• Ferreira, J. C., Menegatto, V. A.,Eigenvalues of integral operators defined by smooth positive definite kernels,Integral equation and Operator Theory, 64 (2009), no. 1, 61–81. (Gives the generalization of Mercer's theorem for metric spaces. The result is easily adapted to first countable topological spaces)
• Konrad Jörgens,Linear integral operators,Pitman, Boston, 1982,
• Richard CourantandDavid Hilbert,Methods of Mathematical Physics,vol 1, Interscience 1953,
• Robert Ash,Information Theory,Dover Publications, 1990,
• Mercer, J. (1909), "Functions of positive and negative type and their connection with the theory of integral equations",Philosophical Transactions of the Royal Society A,209(441–458): 415–446,Bibcode:1909RSPTA.209..415M,doi:10.1098/rsta.1909.0016,
• "Mercer theorem",Encyclopedia of Mathematics,EMS Press,2001 [1994]
• H. König,Eigenvalue distribution of compact operators,Birkhäuser Verlag, 1986. (Gives the generalization of Mercer's theorem for finite measures μ.)