Propagation of uncertainty
Instatistics,propagation of uncertainty(orpropagation of error) is the effect ofvariables'uncertainties(orerrors,more specificallyrandom errors) on the uncertainty of afunctionbased on them. When the variables are the values of experimental measurements they haveuncertainties due to measurement limitations(e.g., instrumentprecision) which propagate due to the combination of variables in the function.
The uncertaintyucan be expressed in a number of ways. It may be defined by theabsolute errorΔx.Uncertainties can also be defined by therelative error(Δx)/x,which is usually written as a percentage. Most commonly, the uncertainty on a quantity is quantified in terms of thestandard deviation,σ,which is the positive square root of thevariance.The value of a quantity and its error are then expressed as an intervalx±u. However, the most general way of characterizing uncertainty is by specifying itsprobability distribution. If theprobability distributionof the variable is known or can be assumed, in theory it is possible to get any of its statistics. In particular, it is possible to deriveconfidence limitsto describe the region within which the true value of the variable may be found. For example, the 68% confidence limits for a one-dimensional variable belonging to anormal distributionare approximately ± one standard deviationσfrom the central valuex,which means that the regionx±σwill cover the true value in roughly 68% of cases.
If the uncertainties arecorrelatedthencovariancemust be taken into account. Correlation can arise from two different sources. First, themeasurement errorsmay be correlated. Second, when the underlying values are correlated across a population, theuncertainties in the group averageswill be correlated.[1]
In a general context where a nonlinear function modifies the uncertain parameters (correlated or not), the standard tools to propagate uncertainty, and infer resulting quantity probability distribution/statistics, are sampling techniques from theMonte Carlo methodfamily.[2]For very large datasets or complex functions, the calculation of the error propagation may be very expensive so that asurrogate model[3]or aparallel computingstrategy[4][5][6]may be necessary.
In some particular cases, the uncertainty propagation calculation can be done through simplistic algebraic procedures. Some of these scenarios are described below.
Linear combinations
[edit]Letbe a set ofmfunctions, which are linear combinations ofvariableswith combination coefficients: or in matrix notation,
Also let thevariance–covariance matrixofx= (x1,...,xn)be denoted byand let the mean value be denoted by: is theouter product.
Then, the variance–covariance matrixoffis given by
In component notation, the equation reads
This is the most general expression for the propagation of error from one set of variables onto another. When the errors onxare uncorrelated, the general expression simplifies to whereis the variance ofk-th element of thexvector. Note that even though the errors onxmay be uncorrelated, the errors onfare in general correlated; in other words, even ifis a diagonal matrix,is in general a full matrix.
The general expressions for a scalar-valued functionfare a little simpler (hereais a row vector):
Each covariance termcan be expressed in terms of thecorrelation coefficientby,so that an alternative expression for the variance offis
In the case that the variables inxare uncorrelated, this simplifies further to
In the simple case of identical coefficients and variances, we find
For the arithmetic mean,,the result is thestandard error of the mean:
Non-linear combinations
[edit]Whenfis a set of non-linear combination of the variablesx,aninterval propagationcould be performed in order to compute intervals which contain all consistent values for the variables. In a probabilistic approach, the functionfmust usually be linearised by approximation to a first-orderTaylor seriesexpansion, though in some cases, exact formulae can be derived that do not depend on the expansion as is the case for the exact variance of products.[7]The Taylor expansion would be: wheredenotes thepartial derivativeoffkwith respect to thei-th variable, evaluated at the mean value of all components of vectorx.Or inmatrix notation, where J is theJacobian matrix.Since f0is a constant it does not contribute to the error on f. Therefore, the propagation of error follows the linear case, above, but replacing the linear coefficients,AkiandAkjby the partial derivatives,and.In matrix notation,[8]
That is, the Jacobian of the function is used to transform the rows and columns of the variance-covariance matrix of the argument. Note this is equivalent to the matrix expression for the linear case with.
Simplification
[edit]Neglecting correlations or assuming independent variables yields a common formula among engineers and experimental scientists to calculate error propagation, the variance formula:[9] whererepresents the standard deviation of the function,represents the standard deviation of,represents the standard deviation of,and so forth.
This formula is based on the linear characteristics of the gradient ofand therefore it is a good estimation for the standard deviation ofas long asare small enough. Specifically, the linear approximation ofhas to be close toinside a neighbourhood of radius.[10]
Example
[edit]Any non-linear differentiable function,,of two variables,and,can be expanded as If we take the variance on both sides and use the formula[11]for the variance of a linear combination of variables then we obtain whereis the standard deviation of the function,is the standard deviation of,is the standard deviation ofandis the covariance betweenand.
In the particular case that,,.Then or whereis the correlation betweenand.
When the variablesandare uncorrelated,.Then
Caveats and warnings
[edit]Error estimates for non-linear functions arebiasedon account of using a truncated series expansion. The extent of this bias depends on the nature of the function. For example, the bias on the error calculated for log(1+x) increases asxincreases, since the expansion toxis a good approximation only whenxis near zero.
For highly non-linear functions, there exist five categories of probabilistic approaches for uncertainty propagation;[12]seeUncertainty quantificationfor details.
Reciprocal and shifted reciprocal
[edit]In the special case of the inverse or reciprocal,wherefollows astandard normal distribution,the resulting distribution is a reciprocal standard normal distribution, and there is no definable variance.[13]
However, in the slightly more general case of a shifted reciprocal functionforfollowing a general normal distribution, then mean and variance statistics do exist in aprincipal valuesense, if the difference between the poleand the meanis real-valued.[14]
Ratios
[edit]Ratios are also problematic; normal approximations exist under certain conditions.
Example formulae
[edit]This table shows the variances and standard deviations of simple functions of the real variableswith standard deviationscovarianceand correlationThe real-valued coefficientsandare assumed exactly known (deterministic), i.e.,
In the right-hand columns of the table,andareexpectation values,andis the value of the function calculated at those values.
Function | Variance | Standard deviation |
---|---|---|
[15][16] | ||
[17] | ||
[18] | ||
[18] | ||
[19] | ||
For uncorrelated variables (,) expressions for more complicated functions can be derived by combining simpler functions. For example, repeated multiplication, assuming no correlation, gives
For the casewe also have Goodman's expression[7]for the exact variance: for the uncorrelated case it is and therefore we have
Effect of correlation on differences
[edit]IfAandBare uncorrelated, their differenceA−Bwill have more variance than either of them. An increasing positive correlation () will decrease the variance of the difference, converging to zero variance for perfectly correlated variables with thesame variance.On the other hand, a negative correlation () will further increase the variance of the difference, compared to the uncorrelated case.
For example, the self-subtractionf=A−Ahas zero varianceonly if the variate is perfectlyautocorrelated(). IfAis uncorrelated,then the output variance is twice the input variance,And ifAis perfectly anticorrelated,then the input variance is quadrupled in the output,(noticeforf=aA−aAin the table above).
Example calculations
[edit]Inverse tangent function
[edit]We can calculate the uncertainty propagation for the inverse tangent function as an example of using partial derivatives to propagate error.
Define whereis the absolute uncertainty on our measurement ofx.The derivative off(x)with respect toxis
Therefore, our propagated uncertainty is whereis the absolute propagated uncertainty.
Resistance measurement
[edit]A practical application is anexperimentin which one measurescurrent,I,andvoltage,V,on aresistorin order to determine theresistance,R,usingOhm's law,R=V/I.
Given the measured variables with uncertainties,I±σIandV±σV,and neglecting their possible correlation, the uncertainty in the computed quantity,σR,is:
See also
[edit]- Accuracy and precision
- Automatic differentiation
- Bienaymé's identity
- Delta method
- Dilution of precision (navigation)
- Errors and residuals in statistics
- Experimental uncertainty analysis
- Interval finite element
- Measurement uncertainty
- Numerical stability
- Probability bounds analysis
- Uncertainty quantification
- Random-fuzzy variable
- Variance § Propagation
References
[edit]- ^Kirchner, James."Data Analysis Toolkit #5: Uncertainty Analysis and Error Propagation"(PDF).Berkeley Seismology Laboratory.University of California.Retrieved22 April2016.
- ^Kroese, D. P.; Taimre, T.; Botev, Z. I. (2011).Handbook of Monte Carlo Methods.John Wiley & Sons.
- ^Ranftl, Sascha; von der Linden, Wolfgang (2021-11-13)."Bayesian Surrogate Analysis and Uncertainty Propagation".Physical Sciences Forum.3(1): 6.arXiv:2101.04038.doi:10.3390/psf2021003006.ISSN2673-9984.
- ^Atanassova, E.; Gurov, T.; Karaivanova, A.; Ivanovska, S.; Durchova, M.; Dimitrov, D. (2016). "On the parallelization approaches for Intel MIC architecture".AIP Conference Proceedings.1773(1): 070001.Bibcode:2016AIPC.1773g0001A.doi:10.1063/1.4964983.
- ^Cunha Jr, A.; Nasser, R.; Sampaio, R.; Lopes, H.; Breitman, K. (2014). "Uncertainty quantification through the Monte Carlo method in a cloud computing setting".Computer Physics Communications.185(5):1355–1363.arXiv:2105.09512.Bibcode:2014CoPhC.185.1355C.doi:10.1016/j.cpc.2014.01.006.S2CID32376269.
- ^Lin, Y.; Wang, F.; Liu, B. (2018). "Random number generators for large-scale parallel Monte Carlo simulations on FPGA".Journal of Computational Physics.360:93–103.Bibcode:2018JCoPh.360...93L.doi:10.1016/j.jcp.2018.01.029.
- ^abGoodman, Leo(1960). "On the Exact Variance of Products".Journal of the American Statistical Association.55(292):708–713.doi:10.2307/2281592.JSTOR2281592.
- ^Ochoa1, Benjamin; Belongie, Serge"Covariance Propagation for Guided Matching"Archived2011-07-20 at theWayback Machine
- ^Ku, H. H. (October 1966)."Notes on the use of propagation of error formulas".Journal of Research of the National Bureau of Standards.70C(4): 262.doi:10.6028/jres.070c.025.ISSN0022-4316.Retrieved3 October2012.
- ^Clifford, A. A. (1973).Multivariate error analysis: a handbook of error propagation and calculation in many-parameter systems.John Wiley & Sons.ISBN978-0470160558.[page needed]
- ^Soch, Joram (2020-07-07)."Variance of the linear combination of two random variables".The Book of Statistical Proofs.Retrieved2022-01-29.
- ^Lee, S. H.; Chen, W. (2009). "A comparative study of uncertainty propagation methods for black-box-type problems".Structural and Multidisciplinary Optimization.37(3):239–253.doi:10.1007/s00158-008-0234-7.S2CID119988015.
- ^Johnson, Norman L.; Kotz, Samuel; Balakrishnan, Narayanaswamy (1994).Continuous Univariate Distributions, Volume 1.Wiley. p. 171.ISBN0-471-58495-9.
- ^Lecomte, Christophe (May 2013). "Exact statistics of systems with uncertainties: an analytical theory of rank-one stochastic dynamic systems".Journal of Sound and Vibration.332(11):2750–2776.doi:10.1016/j.jsv.2012.12.009.
- ^"A Summary of Error Propagation"(PDF).p. 2. Archived fromthe original(PDF)on 2016-12-13.Retrieved2016-04-04.
- ^"Propagation of Uncertainty through Mathematical Operations"(PDF).p. 5.Retrieved2016-04-04.
- ^"Strategies for Variance Estimation"(PDF).p. 37.Retrieved2013-01-18.
- ^abHarris, Daniel C. (2003),Quantitative chemical analysis(6th ed.), Macmillan, p. 56,ISBN978-0-7167-4464-1
- ^"Error Propagation tutorial"(PDF).Foothill College.October 9, 2009.Retrieved2012-03-01.
Further reading
[edit]- Bevington, Philip R.; Robinson, D. Keith (2002),Data Reduction and Error Analysis for the Physical Sciences(3rd ed.), McGraw-Hill,ISBN978-0-07-119926-1
- Fornasini, Paolo (2008),The uncertainty in physical measurements: an introduction to data analysis in the physics laboratory,Springer, p. 161,ISBN978-0-387-78649-0
- Meyer, Stuart L. (1975),Data Analysis for Scientists and Engineers,Wiley,ISBN978-0-471-59995-1
- Peralta, M. (2012),Propagation Of Errors: How To Mathematically Predict Measurement Errors,CreateSpace
- Rouaud, M. (2013),Probability, Statistics and Estimation: Propagation of Uncertainties in Experimental Measurement(PDF)(short ed.)
- Taylor, J. R. (1997),An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements(2nd ed.), University Science Books
- Wang, C. M.; Iyer, Hari K. (2005-09-07). "On higher-order corrections for propagating uncertainties".Metrologia.42(5):406–410.doi:10.1088/0026-1394/42/5/011.ISSN0026-1394.S2CID122841691.
External links
[edit]- A detailed discussion of measurements and the propagation of uncertaintyexplaining the benefits of using error propagation formulas and Monte Carlo simulations instead of simple significance arithmetic
- GUM,Guide to the Expression of Uncertainty in Measurement
- EPFL An Introduction to Error Propagation,Derivation, Meaning and Examples of Cy = Fx Cx Fx'
- uncertainties package,a program/library for transparently performing calculations with uncertainties (and error correlations).
- soerp package,a Python program/library for transparently performing *second-order* calculations with uncertainties (and error correlations).
- Joint Committee for Guides in Metrology (2011).JCGM 102: Evaluation of Measurement Data - Supplement 2 to the "Guide to the Expression of Uncertainty in Measurement" - Extension to Any Number of Output Quantities(PDF)(Technical report). JCGM.Retrieved13 February2013.
- Uncertainty CalculatorPropagate uncertainty for any expression