Explained variation

Instatistics,explained variationmeasures the proportion to which a mathematical model accounts for the variation (dispersion) of a given data set. Often, variation is quantified asvariance;then, the more specific termexplained variancecan be used.

The complementary part of the total variation is calledunexplainedorresidualvariation;likewise, when discussing variance as such, this is referred to asunexplainedorresidual variance.

Definition in terms of information gain

Information gain by better modelling

Following Kent (1983),we use the Fraser information (Fraser 1965)

$F(\theta )=\int {\textrm {d}}r\,g(r)\,\ln f(r;\theta )$

where$g(r)$is the probability density of a random variable$R\,$,and$f(r;\theta )\,$with$\theta \in \Theta _{i}$($i=0,1\,$) are two families of parametric models. Model family 0 is the simpler one, with a restricted parameter space$\Theta _{0}\subset \Theta _{1}$.

Parameters are determined bymaximum likelihood estimation,

$\theta _{i}=\operatorname {argmax} _{\theta \in \Theta _{i}}F(\theta ).$

The information gain of model 1 over model 0 is written as

$\Gamma (\theta _{1}:\theta _{0})=2[F(\theta _{1})-F(\theta _{0})]\,$

where a factor of 2 is included for convenience. Γ is always nonnegative; it measures the extent to which the best model of family 1 is better than the best model of family 0 in explainingg(r).

Information gain by a conditional model

Assume a two-dimensional random variable$R=(X,Y)$whereXshall be considered as an explanatory variable, andYas a dependent variable. Models of family 1 "explain"Yin terms ofX,

$f(y\mid x;\theta )$,

whereas in family 0,XandYare assumed to be independent. We define the randomness ofYby$D(Y)=\exp[-2F(\theta _{0})]$,and the randomness ofY,givenX,by$D(Y\mid X)=\exp[-2F(\theta _{1})]$.Then,

$\rho _{C}^{2}=1-D(Y\mid X)/D(Y)$

can be interpreted as proportion of the data dispersion which is "explained" byX.

Special cases and generalized usage

Linear regression

Main article:Fraction of variance unexplained

The fraction of variance unexplained is an established concept in the context oflinear regression.The usual definition of thecoefficient of determinationis based on the fundamental concept of explained variance.

Correlation coefficient as measure of explained variance

LetXbe a random vector, andYa random variable that is modeled by a normal distribution with centre$\mu =\Psi ^{\textrm {T}}X$.In this case, the above-derived proportion of explained variation$\rho _{C}^{2}$equals the squaredcorrelation coefficient$R^{2}$.

Note the strong model assumptions: the centre of theYdistribution must be a linear function ofX,and for any givenx,theYdistribution must be normal. In other situations, it is generally not justified to interpret$R^{2}$as proportion of explained variance.

In principal component analysis

Explained variance is routinely used inprincipal component analysis.The relation to the Fraser–Kent information gain remains to be clarified.

Criticism

As the fraction of "explained variance" equals the squared correlation coefficient$R^{2}$,it shares all the disadvantages of the latter: it reflects not only the quality of the regression, but also the distribution of the independent (conditioning) variables.

In the words of one critic: "Thus$R^{2}$gives the 'percentage of variance explained' by the regression, an expression that, for most social scientists, is of doubtful meaning but great rhetorical value. If this number is large, the regression gives a good fit, and there is little point in searching for additional variables. Other regression equations on different data sets are said to be less satisfactory or less powerful if their$R^{2}$is lower. Nothing about$R^{2}$supports these claims ".And, after constructing an example where$R^{2}$is enhanced just by jointly considering data from two different populations: "'Explained variance' explains nothing."

See also

• Analysis of variance
• Variance reduction
• Variance-based sensitivity analysis

External links

• Explained and Unexplained Variance on a graph