Margin of error

This article is about the statistical precision of estimates from sample surveys. For observational errors, seeObservational error.For safety margins in engineering, seeFactor of safety.For tolerance in engineering, seeEngineering tolerance.For the eponymous movie, seeMargin for error (film).

Probability densitiesof polls of different sizes, each color-coded to its 95%confidence interval(below), margin of error (left), and sample size (right). Each interval reflects the range within which one may have 95% confidence that thetruepercentage may be found, given a reported percentage of 50%. Themargin of erroris half the confidence interval (also, theradiusof the interval). The larger the sample, the smaller the margin of error. Also, the further from 50% the reported percentage, the smaller the margin of error.

Themargin of erroris a statistic expressing the amount of randomsampling errorin the results of asurvey.The larger the margin of error, the less confidence one should have that a poll result would reflect the result of a census of the entirepopulation.The margin of error will be positive whenever a population is incompletely sampled and the outcome measure has positivevariance,which is to say, whenever the measurevaries.

The termmargin of erroris often used in non-survey contexts to indicateobservational errorin reporting measured quantities.

Concept

Consider a simpleyes/nopoll$P$as a sample of$n$respondents drawn from a population$N{\text{, }}(n\ll N)$reporting the percentage$p$ofyesresponses. We would like to know how close$p$is to the true result of a survey of the entire population$N$,without having to conduct one. If, hypothetically, we were to conduct poll$P$over subsequent samples of$n$respondents (newly drawn from$N$), we would expect those subsequent results$p_{1},p_{2},\ldots$to be normally distributed about${\overline {p}}$,the true but unknown percentage of the population. Themargin of errordescribes the distance within which a specified percentage of these results is expected to vary from${\overline {p}}$.

According to the68-95-99.7 rule,we would expect that 95% of the results$p_{1},p_{2},\ldots$will fall withinabouttwostandard deviations($\pm 2\sigma _{P}$) either side of the true mean${\overline {p}}$.This interval is called theconfidence interval,and theradius(half the interval) is called themargin of error,corresponding to a 95%confidence level.

Generally, at a confidence level$\gamma$,a sample sized$n$of a population having expected standard deviation$\sigma$has a margin of error

$MOE_{\gamma }=z_{\gamma }\times {\sqrt {\frac {\sigma ^{2}}{n}}}$

where$z_{\gamma }$denotes thequantile(also, commonly, az-score), and${\sqrt {\frac {\sigma ^{2}}{n}}}$is thestandard error.

Standard deviation and standard error

We would expect the average of normally distributed values$p_{1},p_{2},\ldots$to have a standard deviation which somehow varies with$n$.The smaller$n$,the wider the margin. This is called the standard error$\sigma _{\overline {p}}$.

For the single result from our survey, weassumethat$p={\overline {p}}$,and thatallsubsequent results$p_{1},p_{2},\ldots$together would have a variance$\sigma _{P}^{2}=P(1-P)$.

${\text{Standard error}}=\sigma _{\overline {p}}\approx {\sqrt {\frac {\sigma _{P}^{2}}{n}}}\approx {\sqrt {\frac {p(1-p)}{n}}}$

Note that$p(1-p)$corresponds to the variance of aBernoulli distribution.

Maximum margin of error at different confidence levels

For a confidencelevel$\gamma$,there is a corresponding confidenceintervalabout the mean$\mu \pm z_{\gamma }\sigma$,that is, the interval$[\mu -z_{\gamma }\sigma,\mu +z_{\gamma }\sigma ]$within which values of$P$should fall with probability$\gamma$.Precise values of$z_{\gamma }$are given by thequantile function of the normal distribution(which the 68-95-99.7 rule approximates).

Note that$z_{\gamma }$is undefined for$|\gamma |\geq 1$,that is,$z_{1.00}$is undefined, as is$z_{1.10}$.

$\gamma$	$z_{\gamma }$		$\gamma$	$z_{\gamma }$
0.84	0.994457883210		0.9995	3.290526731492
0.95	1.644853626951		0.99995	3.890591886413
0.975	1.959963984540		0.999995	4.417173413469
0.99	2.326347874041		0.9999995	4.891638475699
0.995	2.575829303549		0.99999995	5.326723886384
0.9975	2.807033768344		0.999999995	5.730728868236
0.9985	2.967737925342		0.9999999995	6.109410204869

Log-log graphs of$MOE_{\gamma }(0.5)$vs sample sizenand confidence levelγ.The arrows show that the maximum margin error for a sample size of 1000 is ±3.1% at 95% confidence level, and ±4.1% at 99%.
The inset parabola$\sigma _{p}^{2}=p-p^{2}$illustrates the relationship between$\sigma _{p}^{2}$at$p=0.71$and$\sigma _{max}^{2}$at$p=0.5$.In the example,MOE₉₅(0.71) ≈0.9 × ±3.1%≈ ±2.8%.

Since$\max \sigma _{P}^{2}=\max P(1-P)=0.25$at$p=0.5$,we can arbitrarily set$p={\overline {p}}=0.5$,calculate$\sigma _{P}$,$\sigma _{\overline {p}}$,and$z_{\gamma }\sigma _{\overline {p}}$to obtain themaximummargin of error for$P$at a given confidence level$\gamma$and sample size$n$,even before having actual results. With$p=0.5,n=1013$

$MOE_{95}(0.5)=z_{0.95}\sigma _{\overline {p}}\approx z_{0.95}{\sqrt {\frac {\sigma _{P}^{2}}{n}}}=1.96{\sqrt {\frac {.25}{n}}}=0.98/{\sqrt {n}}=\pm 3.1\%$

$MOE_{99}(0.5)=z_{0.99}\sigma _{\overline {p}}\approx z_{0.99}{\sqrt {\frac {\sigma _{P}^{2}}{n}}}=2.58{\sqrt {\frac {.25}{n}}}=1.29/{\sqrt {n}}=\pm 4.1\%$

Also, usefully, for any reported$MOE_{95}$

$MOE_{99}={\frac {z_{0.99}}{z_{0.95}}}MOE_{95}\approx 1.3\times MOE_{95}$

Specific margins of error

If a poll has multiple percentage results (for example, a poll measuring a single multiple-choice preference), the result closest to 50% will have the highest margin of error. Typically, it is this number that is reported as the margin of error for the entire poll. Imagine poll$P$reports$p_{a},p_{b},p_{c}$as$71\%,27\%,2\%,n=1013$

$MOE_{95}(P_{a})=z_{0.95}\sigma _{\overline {p_{a}}}\approx 1.96{\sqrt {\frac {p_{a}(1-p_{a})}{n}}}=0.89/{\sqrt {n}}=\pm 2.8\%$(as in the figure above)

$MOE_{95}(P_{b})=z_{0.95}\sigma _{\overline {p_{b}}}\approx 1.96{\sqrt {\frac {p_{b}(1-p_{b})}{n}}}=0.87/{\sqrt {n}}=\pm 2.7\%$

$MOE_{95}(P_{c})=z_{0.95}\sigma _{\overline {p_{c}}}\approx 1.96{\sqrt {\frac {p_{c}(1-p_{c})}{n}}}=0.27/{\sqrt {n}}=\pm 0.8\%$

As a given percentage approaches the extremes of 0% or 100%, its margin of error approaches ±0%.

Comparing percentages

Imagine multiple-choice poll$P$reports$p_{a},p_{b},p_{c}$as$46\%,42\%,12\%,n=1013$.As described above, the margin of error reported for the poll would typically be$MOE_{95}(P_{a})$,as$p_{a}$is closest to 50%. The popular notion ofstatistical tieorstatistical dead heat,however, concerns itself not with the accuracy of the individual results, but with that of therankingof the results. Which is in first?

If, hypothetically, we were to conduct poll$P$over subsequent samples of$n$respondents (newly drawn from$N$), and report result$p_{w}=p_{a}-p_{b}$,we could use thestandard error of differenceto understand how$p_{w_{1}},p_{w_{2}},p_{w_{3}},\ldots$is expected to fall about${\overline {p_{w}}}$.For this, we need to apply thesum of variancesto obtain a new variance,$\sigma _{P_{w}}^{2}$,

$\sigma _{P_{w}}^{2}=\sigma _{P_{a}-P_{b}}^{2}=\sigma _{P_{a}}^{2}+\sigma _{P_{b}}^{2}-2\sigma _{P_{a},P_{b}}=p_{a}(1-p_{a})+p_{b}(1-p_{b})+2p_{a}p_{b}$

where$\sigma _{P_{a},P_{b}}=-P_{a}P_{b}$is thecovarianceof$P_{a}$and$P_{b}$.

Thus (after simplifying),

${\text{Standard error of difference}}=\sigma _{\overline {w}}\approx {\sqrt {\frac {\sigma _{P_{w}}^{2}}{n}}}={\sqrt {\frac {p_{a}+p_{b}-(p_{a}-p_{b})^{2}}{n}}}=0.029,P_{w}=P_{a}-P_{b}$

$MOE_{95}(P_{a})=z_{0.95}\sigma _{\overline {p_{a}}}\approx \pm {3.1\%}$

$MOE_{95}(P_{w})=z_{0.95}\sigma _{\overline {w}}\approx \pm {5.8\%}$

Note that this assumes that$P_{c}$is close to constant, that is, respondents choosing either A or B would almost never choose C (making$P_{a}$and$P_{b}$close toperfectly negatively correlated). With three or more choices in closer contention, choosing a correct formula for$\sigma _{P_{w}}^{2}$becomes more complicated.

Effect of finite population size

The formulae above for the margin of error assume that there is an infinitely large population and thus do not depend on the size of population$N$,but only on the sample size$n$.According tosampling theory,this assumption is reasonable when thesampling fractionis small. The margin of error for a particular sampling method is essentially the same regardless of whether the population of interest is the size of a school, city, state, or country, as long as the samplingfractionis small.

In cases where the sampling fraction is larger (in practice, greater than 5%), analysts might adjust the margin of error using afinite population correctionto account for the added precision gained by sampling a much larger percentage of the population. FPC can be calculated using the formula

$\operatorname {FPC} ={\sqrt {\frac {N-n}{N-1}}}$

...and so, if poll$P$were conducted over 24% of, say, an electorate of 300,000 voters,

$MOE_{95}(0.5)=z_{0.95}\sigma _{\overline {p}}\approx {\frac {0.98}{\sqrt {72,000}}}=\pm 0.4\%$

$MOE_{95_{FPC}}(0.5)=z_{0.95}\sigma _{\overline {p}}{\sqrt {\frac {N-n}{N-1}}}\approx {\frac {0.98}{\sqrt {72,000}}}{\sqrt {\frac {300,000-72,000}{300,000-1}}}=\pm 0.3\%$

Intuitively, for appropriately large$N$,

$\lim _{n\to 0}{\sqrt {\frac {N-n}{N-1}}}\approx 1$

$\lim _{n\to N}{\sqrt {\frac {N-n}{N-1}}}=0$

In the former case,$n$is so small as to require no correction. In the latter case, the poll effectively becomes a census and sampling error becomes moot.

See also

• Engineering tolerance
• Key relevance
• Measurement uncertainty
• Random error

Sources

• Sudman, Seymour and Bradburn, Norman (1982).Asking Questions: A Practical Guide to Questionnaire Design.San Francisco: Jossey Bass.ISBN0-87589-546-8
• Wonnacott, T.H.; R.J. Wonnacott (1990).Introductory Statistics(5th ed.). Wiley.ISBN0-471-61518-8.

External links

• "Errors, theory of",Encyclopedia of Mathematics,EMS Press,2001 [1994]
• Weisstein, Eric W."Margin of Error".MathWorld.