Mallows'sC_p

Instatistics,Mallows's ${\textstyle {\boldsymbol {C_{p}}}}$ ,^[1]^[2]named forColin Lingwood Mallows,is used to assess thefitof aregression modelthat has been estimated usingordinary least squares.It is applied in the context ofmodel selection,where a number ofpredictor variablesare available for predicting some outcome, and the goal is to find the best model involving a subset of these predictors. A small value of ${\textstyle C_{p}}$ means that the model is relatively precise.

Mallows'sC_phas been shown to be equivalent toAkaike information criterionin the special case of Gaussianlinear regression.^[3]

Definition and properties[edit]

Mallows'sC_paddresses the issue ofoverfitting,in which model selection statistics such as the residual sum of squares always get smaller as more variables are added to a model. Thus, if we aim to select the model giving the smallest residual sum of squares, the model including all variables would always be selected. Instead, theC_pstatistic calculated on asampleof data estimates thesum squared prediction error(SSPE) as itspopulationtarget

E\sum _{i}({\hat {Y}}_{i}-E(Y_{i}\mid X_{i}))^{2}/\sigma ^{2},

where ${\hat {Y}}_{i}$ is the fitted value from the regression model for theith case,E(Y_i|X_i) is the expected value for theith case, and σ²is the error variance (assumed constant across the cases). Themean squared prediction error(MSPE) will not automatically get smaller as more variables are added. The optimum model under this criterion is a compromise influenced by the sample size, theeffect sizesof the different predictors, and the degree ofcollinearitybetween them.

IfPregressorsare selected from a set ofK>P,theC_pstatistic for that particular set of regressors is defined as:

C_{p}={SSE_{p} \over S^{2}}-N+2(P+1),

where

$SSE_{p}=\sum _{i=1}^{N}(Y_{i}-{\hat {Y}}_{pi})^{2}$ is theerror sum of squaresfor the model withPregressors,
Y_piis thepredictedvalue of theith observation ofYfrom thePregressors,
S²is the estimation of residuals variance afterregressionon the complete set ofKregressorsand can be estimated by ${1 \over N-K}\sum _{i=1}^{N}(Y_{i}-{\hat {Y}}_{i})^{2}$ ,^[4]
andNis thesample size.

Alternative definition[edit]

Given a linear model such as:

Y=\beta _{0}+\beta _{1}X_{1}+\cdots +\beta _{p}X_{p}+\varepsilon

where:

$\beta _{0},\ldots,\beta _{p}$ are coefficients for predictor variables $X_{1},\ldots,X_{p}$
$\varepsilon$ represents error

An alternate version ofC_pcan also be defined as:^[5]

C_{p}={\frac {1}{n}}(\operatorname {RSS} +2p{\hat {\sigma }}^{2})

where

RSS is the residual sum of squares on a training set of data
$p$ is the number of predictors
and ${\hat {\sigma }}^{2}$ refers to an estimate of the variance associated with each response in the linear model (estimated on a model containing all predictors)

Note that this version of theC_pdoes not give equivalent values to the earlier version, but the model with the smallestC_pfrom this definition will also be the same model with the smallestC_pfrom the earlier definition.

Limitations[edit]

TheC_pcriterion suffers from two main limitations^[6]

theC_papproximation is only valid for large sample size;
theC_pcannot handle complex collections of models as in the variable selection (orfeature selection) problem.^[6]

Practical use[edit]

TheC_pstatistic is often used as a stopping rule for various forms ofstepwise regression.Mallows proposed the statistic as a criterion for selecting among many alternative subset regressions. Under a model not suffering from appreciable lack of fit (bias),C_phas expectation nearly equal toP;otherwise the expectation is roughlyPplus a positive bias term. Nevertheless, even though it has expectation greater than or equal toP,there is nothing to preventC_p<Por evenC_p< 0 in extreme cases. It is suggested that one should choose a subset that hasC_papproachingP,^[7]from above, for a list of subsets ordered by increasingP.In practice, the positive bias can be adjusted for by selecting a model from the ordered list of subsets, such thatC_p< 2P.

Since the sample-basedC_pstatistic is an estimate of the MSPE, usingC_pfor model selection does not completely guard against overfitting. For instance, it is possible that the selected model will be one in which the sampleC_pwas a particularly severe underestimate of the MSPE.

Model selection statistics such asC_pare generally not used blindly, but rather information about the field of application, the intended use of the model, and any known biases in the data are taken into account in the process of model selection.

References[edit]

^Mallows, C. L. (1973). "Some Comments onC_P".Technometrics.15(4): 661–675.doi:10.2307/1267380.JSTOR 1267380.
^Gilmour, Steven G. (1996). "The interpretation of Mallows'sC_p-statistic ".Journal of the Royal Statistical Society, Series D.45(1): 49–56.JSTOR 2348411.
^Boisbunon, Aurélie; Canu, Stephane; Fourdrinier, Dominique; Strawderman, William; Wells, Martin T. (2013). "AIC,C_pand estimators of loss for elliptically symmetric distributions ".arXiv:1308.2766[math.ST].
^Mallows, C. L. (1973). "Some Comments onC_P".Technometrics.15(4): 661–675.doi:10.2307/1267380.JSTOR 1267380.
^James, Gareth; Witten; Hastie; Tibshirani (2013-06-24).An Introduction to Statistical Learning.Springer.ISBN 978-1-4614-7138-7.
^^a ^bGiraud, C. (2015),Introduction to high-dimensional statistics,Chapman & Hall/CRC,ISBN 9781482237948
^Daniel, C.; Wood, F. (1980).Fitting Equations to Data(Rev. ed.). New York: Wiley & Sons, Inc.