Sequential probability ratio test

"SPRT" redirects here. For standard platinum resistance thermometers, seeresistance thermometer.

Thesequential probability ratio test(SPRT) is a specificsequential hypothesis test,developed byAbraham Waldand later proven to be optimal by Wald andJacob Wolfowitz.Neyman and Pearson's 1933 resultinspired Wald to reformulate it as a sequential analysis problem. The Neyman-Pearson lemma, by contrast, offers arule of thumbfor when all the data is collected (and its likelihood ratio known).

While originally developed for use inquality controlstudies in the realm of manufacturing, SPRT has been formulated for use in the computerized testing of human examinees as a termination criterion.

Theory

As in classicalhypothesis testing,SPRT starts with a pair of hypotheses, say$H_{0}$and$H_{1}$for thenull hypothesisandalternative hypothesisrespectively. They must be specified as follows:

$H_{0}:p=p_{0}$

$H_{1}:p=p_{1}$

The next step is to calculate the cumulative sum of the log-likelihood ratio,$\log \Lambda _{i}$,as new data arrive: with$S_{0}=0$,then, for$i$=1,2,...,

$S_{i}=S_{i-1}+\log \Lambda _{i}$

Thestopping ruleis a simple thresholding scheme:

• $a<S_{i}<b$:continue monitoring (critical inequality)
• $S_{i}\geq b$:Accept$H_{1}$
• $S_{i}\leq a$:Accept$H_{0}$

where$a$and$b$($a<0<b<\infty$) depend on the desiredtype I and type II errors,$\ Alpha$and$\beta$.They may be chosen as follows:

$a\approx \log {\frac {\beta }{1-\ Alpha }}$and$b\approx \log {\frac {1-\beta }{\ Alpha }}$

In other words,$\ Alpha$and$\beta$must be decided beforehand in order to set the thresholds appropriately. The numerical value will depend on the application. The reason for being only an approximation is that, in the discrete case, the signal may cross the threshold between samples. Thus, depending on the penalty of making an error and thesampling frequency,one might set the thresholds more aggressively. The exact bounds are correct in the continuous case.

Example

A textbook example isparameter estimationof aprobability distribution function.Consider theexponential distribution:

$f_{\theta }(x)=\theta ^{-1}e^{-{\frac {x}{\theta }}},\qquad x,\theta >0$

The hypotheses are

${\begin{cases}H_{0}:\theta =\theta _{0}\\H_{1}:\theta =\theta _{1}\end{cases}}\qquad \theta _{1}>\theta _{0}.$

Then the log-likelihood function (LLF) for one sample is

${\begin{aligned}\log \Lambda (x)&=\log \left({\frac {\theta _{1}^{-1}e^{-{\frac {x}{\theta _{1}}}}}{\theta _{0}^{-1}e^{-{\frac {x}{\theta _{0}}}}}}\right)\\&=\log \left({\frac {\theta _{0}}{\theta _{1}}}e^{{\frac {x}{\theta _{0}}}-{\frac {x}{\theta _{1}}}}\right)\\&=\log \left({\frac {\theta _{0}}{\theta _{1}}}\right)+\log \left(e^{{\frac {x}{\theta _{0}}}-{\frac {x}{\theta _{1}}}}\right)\\&=-\log \left({\frac {\theta _{1}}{\theta _{0}}}\right)+\left({\frac {x}{\theta _{0}}}-{\frac {x}{\theta _{1}}}\right)\\&=-\log \left({\frac {\theta _{1}}{\theta _{0}}}\right)+\left({\frac {\theta _{1}-\theta _{0}}{\theta _{0}\theta _{1}}}\right)x\end{aligned}}$

The cumulative sum of the LLFs for allxis

$S_{n}=\sum _{i=1}^{n}\log \Lambda (x_{i})=-n\log \left({\frac {\theta _{1}}{\theta _{0}}}\right)+\left({\frac {\theta _{1}-\theta _{0}}{\theta _{0}\theta _{1}}}\right)\sum _{i=1}^{n}x_{i}$

Accordingly, the stopping rule is:

$a<-n\log \left({\frac {\theta _{1}}{\theta _{0}}}\right)+\left({\frac {\theta _{1}-\theta _{0}}{\theta _{0}\theta _{1}}}\right)\sum _{i=1}^{n}x_{i}<b$

After re-arranging we finally find

$a+n\log \left({\frac {\theta _{1}}{\theta _{0}}}\right)<\left({\frac {\theta _{1}-\theta _{0}}{\theta _{0}\theta _{1}}}\right)\sum _{i=1}^{n}x_{i}<b+n\log \left({\frac {\theta _{1}}{\theta _{0}}}\right)$

The thresholds are simply twoparallel lineswithslope$\log(\theta _{1}/\theta _{0})$.Sampling should stop when the sum of the samples makes an excursion outside thecontinue-sampling region.

Applications

Manufacturing

The test is done on the proportion metric, and tests that a variablepis equal to one of two desired points,p₁orp₂.The region between these two points is known as theindifference region(IR). For example, suppose you are performing a quality control study on a factory lot of widgets. Management would like the lot to have 3% or less defective widgets, but 1% or less is the ideal lot that would pass with flying colors. In this example,p₁= 0.01andp₂= 0.03and the region between them is the IR because management considers these lots to be marginal and is OK with them being classified either way. Widgets would be sampled one at a time from the lot (sequential analysis) until the test determines, within an acceptable error level, that the lot is ideal or should be rejected.

Testing of human examinees

The SPRT is currently the predominant method of classifying examinees in a variable-lengthcomputerized classification test(CCT).The two parameters arep₁andp₂are specified by determining a cutscore (threshold) for examinees on the proportion correct metric, and selecting a point above and below that cutscore. For instance, suppose the cutscore is set at 70% for a test. We could selectp₁= 0.65andp₂= 0.75.The test then evaluates the likelihood that an examinee's true score on that metric is equal to one of those two points. If the examinee is determined to be at 75%, they pass, and they fail if they are determined to be at 65%.

These points are not specified completely arbitrarily. A cutscore should always be set with a legally defensible method, such as amodified Angoff procedure.Again, the indifference region represents the region of scores that the test designer is OK with going either way (pass or fail). The upper parameterp₂is conceptually the highest level that the test designer is willing to accept for a Fail (because everyone below it has a good chance of failing), and the lower parameterp₁is the lowest level that the test designer is willing to accept for a pass (because everyone above it has a decent chance of passing). While this definition may seem to be a relatively small burden, consider thehigh-stakes case of a licensing testfor medical doctors: at just what point should we consider somebody to be at one of these two levels?

While the SPRT was first applied to testing in the days ofclassical test theory,as is applied in the previous paragraph, Reckase (1983) suggested thatitem response theorybe used to determine thep₁andp₂parameters. The cutscore and indifference region are defined on the latent ability (theta) metric, and translated onto the proportion metric for computation. Research on CCT since then has applied this methodology for several reasons:

1. Large item banks tend to be calibrated with IRT
2. This allows more accurate specification of the parameters
3. By using the item response function for each item, the parameters are easily allowed to vary between items.

Detection of anomalous medical outcomes

Spiegelhalteret al.have shown that SPRT can be used to monitor the performance of doctors, surgeons and other medical practitioners in such a way as to give early warning of potentially anomalous results. In their 2003 paper, they showed how it could have helped identifyHarold Shipmanas a murderer well before he was actually identified.

Extensions

MaxSPRT

More recently, in 2011, an extension of the SPRT method called Maximized Sequential Probability Ratio Test (MaxSPRT)was introduced. The salient feature of MaxSPRT is the allowance of a composite, one-sided alternative hypothesis, and the introduction of an upper stopping boundary. The method has been used in several medical research studies.

See also

• CUSUM
• Computerized classification test
• Wald test
• Likelihood-ratio test

Further reading

• Ghosh, Bhaskar Kumar(1970).Sequential Tests of Statistical Hypotheses.Reading:Addison-Wesley.
• Holger Wilker:Sequential-Statistik in der Praxis,BoD, Norderstedt 2012,ISBN978-3848232529.

External links

• Wald's Sequential Probability Ratio TestforRby Stéphane Bottine
• Wald's Sequential Probability Ratio TestforPythonby Zhenning Yu