Whittle likelihood

Instatistics,Whittle likelihoodis an approximation to thelikelihood functionof a stationary Gaussiantime series.It is named after the mathematician and statisticianPeter Whittle,who introduced it in his PhD thesis in 1951. It is commonly used intime series analysisandsignal processingfor parameter estimation and signal detection.

Context

In astationary Gaussian time series model,thelikelihood functionis (as usual in Gaussian models) a function of the associated mean and covariance parameters. With a large number ($N$) of observations, the ($N\times N$) covariance matrix may become very large, making computations very costly in practice. However, due to stationarity, the covariance matrix has a rather simple structure, and by using an approximation, computations may be simplified considerably (from$O(N^{2})$to$O(N\log(N))$).The idea effectively boils down to assuming aheteroscedasticzero-mean Gaussian model inFourier domain;the model formulation is based on the time series'discrete Fourier transformand itspower spectral density.

Definition

Let$X_{1},\ldots,X_{N}$be a stationary Gaussian time series with (one-sided) power spectral density$S_{1}(f)$,where$N$is even and samples are taken at constant sampling intervals$\Delta _{t}$. Let${\tilde {X}}_{1},\ldots,{\tilde {X}}_{N/2+1}$be the (complex-valued)discrete Fourier transform(DFT) of the time series. Then for the Whittle likelihood one effectively assumes independent zero-meanGaussian distributionsfor all${\tilde {X}}_{j}$with variances for the real and imaginary parts given by

$\operatorname {Var} \left(\operatorname {Re} ({\tilde {X}}_{j})\right)=\operatorname {Var} \left(\operatorname {Im} ({\tilde {X}}_{j})\right)=S_{1}(f_{j})$

where$f_{j}={\frac {j}{N\,\Delta _{t}}}$is the$j$th Fourier frequency. This approximate model immediately leads to the (logarithmic) likelihood function

$\log \left(P(x_{1},\ldots,x_{N})\right)\propto -\sum _{j}\left(\log \left(S_{1}(f_{j})\right)+{\frac {|{\tilde {x}}_{j}|^{2}}{{\frac {N}{2\,\Delta _{t}}}S_{1}(f_{j})}}\right)$

where$|\cdot |$denotes the absolute value with$|{\tilde {x}}_{j}|^{2}=\left(\operatorname {Re} ({\tilde {x}}_{j})\right)^{2}+\left(\operatorname {Im} ({\tilde {x}}_{j})\right)^{2}$.

Special case of a known noise spectrum

In case the noise spectrum is assumed a-prioriknown,and noise properties are not to be inferred from the data, the likelihood function may be simplified further by ignoring constant terms, leading to the sum-of-squares expression

$\log \left(P(x_{1},\ldots,x_{N})\right)\;\propto \;-\sum _{j}{\frac {|{\tilde {x}}_{j}|^{2}}{{\frac {N}{2\,\Delta _{t}}}S_{1}(f_{j})}}$

This expression also is the basis for the commonmatched filter.

Accuracy of approximation

The Whittle likelihood in general is only an approximation, it is only exact if the spectrum is constant, i.e., in the trivial case ofwhite noise. Theefficiencyof the Whittle approximation always depends on the particular circumstances.

Note that due tolinearityof the Fourier transform, Gaussianity in Fourier domain implies Gaussianity in time domain and vice versa. What makes the Whittle likelihood only approximately accurate is related to thesampling theorem—the effect of Fourier-transforming only afinitenumber of data points, which also manifests itself asspectral leakagein related problems (and which may be ameliorated using the same methods, namely,windowing). In the present case, the implicit periodicity assumption implies correlation between the first and last samples ($x_{1}$and$x_{N}$), which are effectively treated as "neighbouring" samples (like$x_{1}$and$x_{2}$).

Applications

Parameter estimation

Whittle's likelihood is commonly used to estimate signal parameters for signals that are buried in non-white noise. Thenoise spectrumthen may be assumed known, or it may be inferred along with the signal parameters.

Signal detection

Signal detection is commonly performed with thematched filter,which is based on the Whittle likelihood for the case of aknownnoise power spectral density. The matched filter effectively does amaximum-likelihoodfit of the signal to the noisy data and uses the resultinglikelihood ratioas the detection statistic.

The matched filter may be generalized to an analogous procedure based on aStudent-t distributionby also considering uncertainty (e.g.estimationuncertainty) in the noise spectrum. On the technical side, this entails repeated or iterative matched-filtering.

Spectrum estimation

The Whittle likelihood is also applicable for estimation of thenoise spectrum,either alone or in conjunction with signal parameters.

See also

• Coloured noise
• Discrete Fourier transform
• Likelihood function
• Matched filter
• Power spectral density
• Statistical signal processing
• Weighted least squares