Sequential analysis

Not to be confused with Sequence analysis.

Short description: Statistical analysis where the sample size is not fixed in advance

In statistics, sequential analysis or sequential hypothesis testing is statistical analysis where the sample size is not fixed in advance. Instead data is evaluated as it is collected, and further sampling is stopped in accordance with a pre-defined stopping rule as soon as significant results are observed. Thus a conclusion may sometimes be reached at a much earlier stage than would be possible with more classical hypothesis testing or estimation, at consequently lower financial and/or human cost.

## Contents

* 1 History
* 2 Alpha spending functions
* 3 Applications of sequential analysis
* 3.1 Clinical trials
* 3.2 Other applications
* 4 Bias
* 5 See also
* 6 Notes
* 7 References
* 8 External links

## History

The method of sequential analysis is first attributed to Abraham Wald[1] with Jacob Wolfowitz, W. Allen Wallis, and Milton Friedman[2][3] while at Columbia University's Statistical Research Group as a tool for more efficient industrial quality control during World War II. Its value to the war effort was immediately recognised, and led to its receiving a "restricted" classification.[4] At the same time, George Barnard led a group working on optimal stopping in Great Britain. Another early contribution to the method was made by K.J. Arrow with D. Blackwell and M.A. Girshick.[5]

A similar approach was independently developed from first principles at about the same time by Alan Turing, as part of the Banburismus technique used at Bletchley Park, to test hypotheses about whether different messages coded by German Enigma machines should be connected and analysed together. This work remained secret until the early 1980s.[6]

Peter Armitage introduced the use of sequential analysis in medical research, especially in the area of clinical trials. Sequential methods became increasingly popular in medicine following Stuart Pocock's work that provided clear recommendations on how to control Type 1 error rates in sequential designs.[7]

## Alpha spending functions

When researchers repeatedly analyze data as more observations are added, the probability of a Type 1 error increases. Therefore, it is important to adjust the alpha level at each interim analysis, such that the overall Type 1 error rate remains at the desired level. This is conceptually similar to using the Bonferroni correction, but because the repeated looks at the data are dependent, more efficient corrections for the alpha level can be used. Among the earliest proposals is the Pocock boundary. Alternative ways to control the Type 1 error rate exist, such as the Haybittle–Peto bounds, and additional work on determining the boundaries for interim analyses has been done by O'Brien & Fleming[8] and Wang & Tsiatis.[9]

A limitation of corrections such as the Pocock boundary is that the number of looks at the data must be determined before the data is collected, and that the looks at the data should be equally spaced (e.g., after 50, 100, 150, and 200 patients). The alpha spending function approach developed by Demets & Lan[10] does not have these restrictions, and depending on the parameters chosen for the spending function, can be very similar to Pocock boundaries or the corrections proposed by O'Brien and Fleming. Another approach that has no such restrictions at all is based on e-values and e-processes.

## Applications of sequential analysis

### Clinical trials

In a randomized trial with two treatment groups, group sequential testing may for example be conducted in the following manner: After n subjects in each group are available an interim analysis is conducted. A statistical test is performed to compare the two groups and if the null hypothesis is rejected the trial is terminated; otherwise, the trial continues, another n subjects per group are recruited, and the statistical test is performed again, including all subjects. If the null is rejected, the trial is terminated, and otherwise it continues with periodic evaluations until a maximum number of interim analyses have been performed, at which point the last statistical test is conducted and the trial is discontinued.[11]

### Other applications

Sequential analysis also has a connection to the problem of gambler's ruin that has been studied by, among others, Huygens in 1657.[12]

Step detection is the process of finding abrupt changes in the mean level of a time series or signal. It is usually considered as a special kind of statistical method known as change point detection. Often, the step is small and the time series is corrupted by some kind of noise, and this makes the problem challenging because the step may be hidden by the noise. Therefore, statistical and/or signal processing algorithms are often required. When the algorithms are run online as the data is coming in, especially with the aim of producing an alert, this is an application of sequential analysis.

## Bias

Trials that are terminated early because they reject the null hypothesis typically overestimate the true effect size.[13] This is because in small samples, only large effect size estimates will lead to a significant effect, and the subsequent termination of a trial. Methods to correct effect size estimates in single trials have been proposed.[14] Note that this bias is mainly problematic when interpreting single studies. In meta-analyses, overestimated effect sizes due to early stopping are balanced by underestimation in trials that stop late, leading Schou & Marschner to conclude that "early stopping of clinical trials is not a substantive source of bias in meta-analyses".[15]

The meaning of p-values in sequential analyses also changes, because when using sequential analyses, more than one analysis is performed, and the typical definition of a p-value as the data “at least as extreme” as is observed needs to be redefined. One solution is to order the p-values of a series of sequential tests based on the time of stopping and how high the test statistic was at a given look, which is known as stagewise ordering,[13] first proposed by Armitage.

## See also

* Optimal stopping
* Sequential estimation
* Sequential probability ratio test
* CUSUM

## Notes

1. ↑ Wald, Abraham (June 1945). "Sequential Tests of Statistical Hypotheses". The Annals of Mathematical Statistics 16 (2): 117-186. doi:10.1214/aoms/1177731118.
2. ↑ Berger, James (2008). "Sequential Analysis". The New Palgrave Dictionary of Economics (2nd ed.). pp. 438–439. doi:10.1057/9780230226203.1513. ISBN 978-0-333-78676-5. http://www.dictionaryofeconomics.com/article?id=pde2008_S000098.
3. ↑ "The Statistical Research Group, 1942–1945", Wallis 1980 (doi:10.1080/01621459.1980.10477469)
4. ↑ Weigl, Hans Günter (2013). Abraham Wald : a statistician as a key figure for modern econometrics (PDF) (Doctoral thesis). University of Hamburg.
5. ↑ Kenneth J. Arrow, David Blackwell and M.A. Girshick (1949). "Bayes and minimax solutions of sequential decision problems". Econometrica 17 (3/4): 213-244. doi:10.2307/1905525.
6. ↑ Randell, Brian (1980), "The Colossus", A History of Computing in the Twentieth Century, p. 30
7. ↑ Jennison, Christopher; Turnbull, Bruce W. (1999-09-15). Group Sequential Methods with Applications to Clinical Trials. Boca Raton, Fla.: Chapman and Hall/CRC. ISBN 978-0-8493-0316-6.
8. ↑ O'Brien, Peter C.; Fleming, Thomas R. (1979-01-01). "A Multiple Testing Procedure for Clinical Trials". Biometrics 35 (3): 549–556. doi:10.2307/2530245. PMID 497341.
9. ↑ Wang, Samuel K.; Tsiatis, Anastasios A. (1987-01-01). "Approximately Optimal One-Parameter Boundaries for Group Sequential Trials". Biometrics 43 (1): 193–199. doi:10.2307/2531959. PMID 3567304.
10. ↑ Demets, David L.; Lan, K. K. Gordon (1994-07-15). "Interim analysis: The alpha spending function approach" (in en). Statistics in Medicine 13 (13–14): 1341–1352. doi:10.1002/sim.4780131308. ISSN 1097-0258. PMID 7973215.
11. ↑ Clinical Statistics: Introducing Clinical Trials, Survival Analysis, and Longitudinal Data Analysis (First ed.). Jones and Bartlett Publishers. 2008. ISBN 978-0-7637-5850-9.
12. ↑ Ghosh, B. K.; Sen, P. K. (1991). Handbook of Sequential Analysis. New York: Marcel Dekker. ISBN 9780824784089.
13. ↑ 13.0 13.1 Proschan, Michael A.; Lan, K. K. Gordan; Wittes, Janet Turk (2006). Statistical monitoring of clinical trials : a unified approach. Springer. ISBN 9780387300597. OCLC 553888945.
14. ↑ Liu, A.; Hall, W. J. (1999-03-01). "Unbiased estimation following a group sequential test". Biometrika 86 (1): 71–78. doi:10.1093/biomet/86.1.71. ISSN 0006-3444. https://academic.oup.com/biomet/article-abstract/86/1/71/255103/Unbiased-estimation-following-a-group-sequential.
15. ↑ Schou, I. Manjula; Marschner, Ian C. (2013-12-10). "Meta-analysis of clinical trials with early stopping: an investigation of potential bias" (in en). Statistics in Medicine 32 (28): 4859–4874. doi:10.1002/sim.5893. ISSN 1097-0258. PMID 23824994.

## References

* Wald, Abraham (1947). Sequential Analysis. New York: John Wiley and Sons. https://archive.org/details/in.ernet.dli.2015.90255.
* Bartroff, J., Lai T.L., and Shih, M.-C. (2013) Sequential Experimentation in Clinical Trials: Design and Analysis. Springer.
* Ghosh, Bhaskar Kumar (1970). Sequential Tests of Statistical Hypotheses. Reading: Addison-Wesley. https://archive.org/details/sequentialtestso0000ghos.
* Chernoff, Herman (1972). Sequential Analysis and Optimal Design. SIAM.
* Siegmund, David (1985). Sequential Analysis. Springer Series in Statistics. New York: Springer-Verlag. ISBN 978-0-387-96134-7.
* Bakeman, R., Gottman, J.M., (1997) Observing Interaction: An Introduction to Sequential Analysis, Cambridge: Cambridge University Press
* Jennison, C. and Turnbull, B.W (2000) Group Sequential Methods With Applications to Clinical Trials. Chapman & Hall/CRC.
* Whitehead, J. (1997). The Design and Analysis of Sequential Clinical Trials, 2nd Edition. John Wiley & Sons.

## External links

* R Package: Wald's Sequential Probability Ratio Test by OnlineMarketr.com
* Software for conducting sequential analysis and applications of sequential analysis in the study of group interaction in computer-mediated communication by Dr. Allan Jeong at Florida State University
* SAMBO Optimization – a Python framework for sequential, model-based optimization.

Commercial

* PASS Sample Size Software includes features for the setup of group sequential designs.

* v
* t
* e

Design of experiments

Scientific
method|

* Scientific experiment
* Statistical design
* Control
* Internal and external validity
* Experimental unit
* Blinding

* Optimal design: Bayesian
* Random assignment
* Randomization
* Restricted randomization
* Replication versus subsampling
* Sample size

Treatment
and blocking|

* Treatment
* Effect size
* Contrast
* Interaction
* Confounding
* Orthogonality
* Blocking
* Covariate
* Nuisance variable

Models
and inference|

* Linear regression
* Ordinary least squares
* Bayesian

* Random effect
* Mixed model
* Hierarchical model: Bayesian

* Analysis of variance (Anova)
* Cochran's theorem
* Manova (multivariate)
* Ancova (covariance)

* Compare means
* Multiple comparison

Designs

Completely
randomized|

* Factorial
* Fractional factorial
* Plackett-Burman
* Taguchi

* Response surface methodology
* Polynomial and rational modeling
* Box-Behnken
* Central composite

* Block
* Generalized randomized block design (GRBD)
* Latin square
* Graeco-Latin square
* Orthogonal array
* Latin hypercube
Repeated measures design
* Crossover study

* Randomized controlled trial
* Sequential analysis
* Sequential probability ratio test

* Glossary
* Category
*   * Statistical outline
* Statistical topics

* v
* t
* e

Statistics

* Outline
* Index

Descriptive statistics

| Continuous data|

| Center|

* Mean
* arithmetic
* geometric
* harmonic
* Median
* Mode

|

Dispersion|

* Variance
* Standard deviation
* Coefficient of variation
* Percentile
* Range
* Interquartile range

Shape|

* Central limit theorem
* Moments
* Skewness
* Kurtosis
* L-moments

Count data|

* Index of dispersion

Summary tables|

* Grouped data
* Frequency distribution
* Contingency table

Dependence|

* Pearson product-moment correlation
* Rank correlation
* Spearman's rho
* Kendall's tau
* Partial correlation
* Scatter plot

Graphics|

* Bar chart
* Biplot
* Box plot
* Control chart
* Correlogram
* Fan chart
* Forest plot
* Histogram
* Pie chart
* Q–Q plot
* Run chart
* Scatter plot
* Stem-and-leaf display
* Radar chart
* Violin plot

Data collection

| Study design|

* Population
* Statistic
* Effect size
* Statistical power
* Optimal design
* Sample size determination
* Replication
* Missing data

|

Survey methodology|

* Sampling
* stratified
* cluster
* Standard error
* Opinion poll
* Questionnaire

Controlled experiments|

* Scientific control
* Randomized experiment
* Randomized controlled trial
* Random assignment
* Blocking
* Interaction
* Factorial experiment

Adaptive Designs|

* Adaptive clinical trial
* Up-and-Down Designs
* Stochastic approximation

Observational Studies|

* Cross-sectional study
* Cohort study
* Natural experiment
* Quasi-experiment

Statistical inference

| Statistical theory|

* Population
* Statistic
* Probability distribution
* Sampling distribution
* Order statistic
* Empirical distribution
* Density estimation
* Statistical model
* Model specification
* Lp space
* Parameter
* location
* scale
* shape
* Parametric family
* Likelihood (monotone)
* Location–scale family
* Exponential family
* Completeness
* Sufficiency
* Statistical functional
* Bootstrap
* U
* V
* Optimal decision
* loss function
* Efficiency
* Statistical distance
* divergence
* Asymptotics
* Robustness

|

Frequentist inference|

| Point estimation|

* Estimating equations
* Maximum likelihood
* Method of moments
* M-estimator
* Minimum distance
* Unbiased estimators
* Mean-unbiased minimum-variance
* Rao–Blackwellization
* Lehmann–Scheffé theorem
* Median unbiased
* Plug-in

|

Interval estimation|

* Confidence interval
* Pivot
* Likelihood interval
* Prediction interval
* Tolerance interval
* Resampling
* Bootstrap
* Jackknife

Testing hypotheses|

* 1- & 2-tails
* Power
* Uniformly most powerful test
* Permutation test
* Randomization test
* Multiple comparisons

Parametric tests|

* Likelihood-ratio
* Score/Lagrange multiplier
* Wald

Specific tests|

* Z-test (normal)
* Student's t-test
* F-test

Goodness of fit|

* Chi-squared
* G-test
* Kolmogorov–Smirnov
* Anderson–Darling
* Lilliefors
* Jarque–Bera
* Normality (Shapiro–Wilk)
* Likelihood-ratio test
* Model selection
* Cross validation
* AIC
* BIC

Rank statistics|

* Sign
* Sample median
* Signed rank (Wilcoxon)
* Hodges–Lehmann estimator
* Rank sum (Mann–Whitney)
* Nonparametric anova
* 1-way (Kruskal–Wallis)
* 2-way (Friedman)
* Ordered alternative (Jonckheere–Terpstra)

Bayesian inference|

* Bayesian probability
* prior
* posterior
* Credible interval
* Bayes factor
* Bayesian estimator
* Maximum posterior estimator

* Correlation
* Regression analysis

| Correlation|

* Pearson product-moment
* Partial correlation
* Confounding variable
* Coefficient of determination

|

Regression analysis|

* Errors and residuals
* Regression validation
* Mixed effects models
* Simultaneous equations models
* Multivariate adaptive regression splines (MARS)

Linear regression|

* Simple linear regression
* Ordinary least squares
* General linear model
* Bayesian regression

Non-standard predictors|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Isotonic
* Robust
* Heteroscedasticity
* Homoscedasticity

Generalized linear model|

* Exponential families
* Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance|

* Analysis of variance (ANOVA, anova)
* Analysis of covariance
* Multivariate ANOVA
* Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

| Categorical|

* Cohen's kappa
* Contingency table
* Graphical model
* Log-linear model
* McNemar's test

|

Multivariate|

* Regression
* Manova
* Principal components
* Canonical correlation
* Discriminant analysis
* Cluster analysis
* Classification
* Structural equation model
* Factor analysis
* Multivariate distributions
* Elliptical distributions
* Normal

Time-series|

| General|

* Decomposition
* Trend
* Stationarity
* Seasonal adjustment
* Exponential smoothing
* Cointegration
* Structural break
* Granger causality

|

Specific tests|

* Dickey–Fuller
* Johansen
* Q-statistic (Ljung–Box)
* Durbin–Watson
* Breusch–Godfrey

Time domain|

* Autocorrelation (ACF)
* partial (PACF)
* Cross-correlation (XCF)
* ARMA model
* ARIMA model (Box–Jenkins)
* Autoregressive conditional heteroskedasticity (ARCH)
* Vector autoregression (VAR)

Frequency domain|

* Spectral density estimation
* Fourier analysis
* Wavelet
* Whittle likelihood

Survival|

| Survival function|

* Kaplan–Meier estimator (product limit)
* Proportional hazards models
* Accelerated failure time (AFT) model
* First hitting time

|

Hazard function|

* Nelson–Aalen estimator

Test|

* Log-rank test

Applications

| Biostatistics|

* Bioinformatics
* Clinical trials / studies
* Epidemiology
* Medical statistics

|

Engineering statistics|

* Chemometrics
* Methods engineering
* Probabilistic design
* Process / quality control
* Reliability
* System identification

Social statistics|

* Actuarial science
* Census
* Crime statistics
* Demography
* Econometrics
* Jurimetrics
* National accounts
* Official statistics
* Population statistics
* Psychometrics

Spatial statistics|

* Cartography
* Environmental statistics
* Geographic information system
* Geostatistics
* Kriging

* Category
* '
* Commons
* WikiProject

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Sequential analysis. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Sequential_analysis&oldid=4157284"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template