Projection filters

Short description: Geometric algorithms for signal processing

Projection filters are a set of algorithms based on stochastic analysis and information geometry, or the differential geometric approach to statistics, used to find approximate solutions for filtering problems for nonlinear state-space systems.[1][2][3] The filtering problem consists of estimating the unobserved signal of a random dynamical system from partial noisy observations of the signal. The objective is computing the probability distribution of the signal conditional on the history of the noise-perturbed observations. This distribution allows for calculations of all statistics of the signal given the history of observations. If this distribution has a density, the density satisfies specific stochastic partial differential equations (SPDEs) called Kushner-Stratonovich equation, or Zakai equation. It is known that the nonlinear filter density evolves in an infinite dimensional function space.[4][5]

One can choose a finite dimensional family of probability densities, for example Gaussian densities, Gaussian mixtures, or exponential families, on which the infinite-dimensional filter density can be approximated. The basic idea of the projection filter is to use a geometric structure in the chosen spaces of densities to project the infinite dimensional SPDE of the optimal filter onto the chosen finite dimensional family, obtaining a finite dimensional stochastic differential equation (SDE) for the parameter of the density in the finite dimensional family that approximates the full filter evolution.[3] To do this, the chosen finite dimensional family is equipped with a manifold structure as in information geometry. The projection filter was tested against the optimal filter for the cubic sensor problem. The projection filter could track effectively bimodal densities of the optimal filter that would have been difficult to approximate with standard algorithms like the extended Kalman filter.[2][6] Projection filters are ideal for in-line estimation, as they are quick to implement and run efficiently in time, providing a finite dimensional SDE for the parameter that can be implemented efficiently.[2] Projection filters are also flexible, as they allow fine tuning the precision of the approximation by choosing richer approximating families, and some exponential families make the correction step in the projection filtering algorithm exact.[3] Some formulations coincide with heuristic based assumed density filters[3] or with Galerkin methods.[6] Projection filters can also approximate the full infinite-dimensional filter in an optimal way, beyond the optimal approximation of the SPDE coefficients alone, according to precise criteria such as mean square minimization.[7] Projection filters have been studied by the Swedish Defense Research Agency[1] and have also been successfully applied to a variety of fields including navigation, ocean dynamics, quantum optics and quantum systems, estimation of fiber diameters, estimation of chaotic time series, change point detection and other areas.[8]

## Contents

* 1 History and development
* 2 Projection filters derivation
* 2.1 Stratonovich-based projection filters
* 2.2 Optimal projection filters based on Ito vector and Ito jet projections
* 3 Applications
* 4 See also
* 5 References

## History and development

The term "projection filter" was first coined in 1987 by Bernard Hanzon,[9] and the related theory and numerical examples were fully developed, expanded and made rigorous during the Ph.D. work of Damiano Brigo, in collaboration with Bernard Hanzon and Francois LeGland.[10][2][3] These works dealt with the projection filters in Hellinger distance and Fisher information metric, that were used to project the optimal filter infinite-dimensional SPDE on a chosen exponential family. The exponential family can be chosen so as to make the prediction step of the filtering algorithm exact.[2] A different type of projection filters, based on an alternative projection metric, the direct L2 metric, was introduced in Armstrong and Brigo (2016).[6] With this metric, the projection filters on families of mixture distributions coincide with filters based on Galerkin methods. Later on, Armstrong, Brigo and Rossi Ferrucci (2021)[7] derive optimal projection filters that satisfy specific optimality criteria in approximating the infinite dimensional optimal filter. Indeed, the Stratonovich-based projection filters optimized the approximations of the SPDE separate coefficients on the chosen manifold but not the SPDE solution as a whole. This has been dealt with by introducing the optimal projection filters. The innovation here is to work directly with Ito calculus, instead of resorting to the Stratonovich calculus version of the filter equation. This is based on research on the geometry of Ito Stochastic differential equations on manifolds based on the jet bundle, the so-called 2-jet interpretation of Ito stochastic differential equations on manifolds.[11]

## Projection filters derivation

Here the derivation of the different projection filters is sketched.

### Stratonovich-based projection filters

This is a derivation of both the initial filter in Hellinger/Fisher metric sketched by Hanzon[9] and fully developed by Brigo, Hanzon and LeGland,[10][2] and the later projection filter in direct L2 metric by Armstrong and Brigo (2016).[6]

It is assumed that the unobserved random signal Xt∈ℝm is modelled by the Ito stochastic differential equation:

dXt=f(Xt,t)dt+σ(Xt,t)dWt

where f and σdW are ℝm valued and Wt is a Brownian motion. Validity of all regularity conditions necessary for the results to hold will be assumed, with details given in the references. The associated noisy observation process Yt∈ℝd is modelled by

dYt=b(Xt,t)dt+dVt

where b is ℝd valued and Vt is a Brownian motion independent of Wt. As hinted above, the full filter is the conditional distribution of Xt given a prior for X0 and the history of Y up to time t. If this distribution has a density described informally as

pt(x)dx=Prob{Xt∈dx|σ(Ys,s≤t)}

where σ(Ys,s≤t) is the sigma-field generated by the history of noisy observations Y up to time t, under suitable technical conditions the density pt satisfies the Kushner—Stratonovich SPDE:

dpt=ℒt*pt dt+pt[b(⋅,t)−Ept(b(⋅,t))]T[dYt−Ept(b(⋅,t))dt]

where Ep is the expectation Ep[h]=∫h(x)p(x)dx, and the forward diffusion operator ℒt* is

ℒt*p=−∑i=1m∂∂xi[fi(x,t)pt(x)]+12∑i,j=1m∂2∂xi∂xj[aij(x,t)pt(x)]

where a=σσT and T denotes transposition. To derive the first version of the projection filters, one needs to put the pt SPDE in Stratonovich form. One obtains

dpt=ℒt∗ptdt−12pt[|b(⋅,t)|2−Ept{|b(⋅,t)|2}]dt+pt[b(⋅,t)−Ept{b(⋅,t)}]T∘dYt .

Through the chain rule, it's immediate to derive the SPDE for dpt. To shorten notation one may rewrite this last SPDE as dp=F(p)dt+GT(p)∘dY ,

where the operators F(p) and GT(p) are defined as

F(p)=ℒt∗p−12p[|b(⋅,t)|2−Ep{|b(⋅,t)|2}],

GT(p)=p[b(⋅,t)−Ep{b(⋅,t)}]T.

The square root version is dp=12p[F(p)dt+GT(p)∘dY] .

These are Stratonovich SPDEs whose solutions evolve in infinite dimensional function spaces. For example pt may evolve in L2 (direct metric d2)

d2(p1,p2)=‖p1−p2‖ , p1,2∈L2

or pt may evolve in L2 (Hellinger metric dH)

dH(p1,p2)=‖p1−p2‖, p1,2∈L1

where ‖⋅‖ is the norm of Hilbert space L2. In any case, pt (or pt) will not evolve inside any finite dimensional family of densitities,

SΘ={p(⋅,θ), θ∈Θ⊂ℝn} (or SΘ1/2={p(⋅,θ), θ∈Θ⊂ℝn}).

The projection filter idea is approximating pt(x) (or pt(x)) via a finite dimensional density p(x,θt) (or p(x,θt)).

The fact that the filter SPDE is in Stratonovich form allows for the following. As Stratonovich SPDEs satisfy the chain rule, F and G behave as vector fields. Thus, the equation is characterized by a dt vector field F and a dYt vector field G. For this version of the projection filter one is satisfied with dealing with the two vector fields separately. One may project F and G on the tangent space of the densities in SΘ (direct metric) or of their square roots (Hellinger metric). The direct metric case yields

dp(⋅,θt)=Πp(⋅,θt)[F(p(⋅,θt))]dt+Πp(⋅,θt)[GT(p(⋅,θt))]∘dYt

where Πp(⋅,θ) is the tangent space projection at the point p(⋅,θ) for the manifold SΘ, and where, when applied to a vector such as GT, it is assumed to act component-wise by projecting each of GT's components. As a basis of this tangent space is

{∂p(⋅,θ)∂θ1,⋯,∂p(⋅,θ)∂θn},

by denoting the inner product of L2 with ⟨⋅,⋅⟩, one defines the metric

γij(θ)=⟨∂p(⋅,θ)∂θi,∂p(⋅,θ)∂θj⟩=∫∂p(x,θ)∂θi∂p(x,θ)∂θjdx

and the projection is thus

Πp(⋅,θ)γ[v]=∑i=1n[∑j=1nγij(θ)⟨v,∂p(⋅,θ)∂θj⟩]∂p(⋅,θ)∂θi

where γij is the inverse of γij. The projected equation thus reads

dp(⋅,θt)=Πp(⋅,θ)[F(p(⋅,θt))]dt+Πp(⋅,θ)[GT(p(⋅,θt))]∘dYt

which can be written as

∑i=1n∂p(⋅,θt)θi∘dθi=∑i=1n[∑j=1nγij(θ)⟨F(p(⋅,θt)),∂p(⋅,θ)∂θj⟩]∂p(⋅,θ)∂θidt+∑i=1n[∑j=1nγij(θ)⟨GT(p(⋅,θt)),∂p(⋅,θ)∂θj⟩]∂p(⋅,θ)∂θi∘dYt,

where it has been crucial that Stratonovich calculus obeys the chain rule. From the above equation, the final projection filter SDE is dθi=[∑j=1nγij(θt)∫F(p(x,θt))∂p(x,θt)∂θjdx]dt+∑k=1d[∑j=1nγij(θt)∫Gk(p(x,θt))∂p(x,θt)∂θjdx]∘dYk

with initial condition a chosen θ0.

By substituting the definition of the operators F and G we obtain the fully explicit projection filter equation in direct metric:

dθi(t)=[∑j=1mγij(θt)∫ℒt∗p(x,θt)∂p(x,θt)∂θjdx−∑j=1mγij(θt)∫12[|b(x,t)|2−∫|b(z,t)|2p(z,θt)dz]p(x,θt)∂p(x,θt)∂θjdx]dt

+∑k=1d[∑j=1mγij(θt)∫[bk(x,t)−∫bk(z,t)p(z,θt)dz]p(x,θt)∂p(x,θt)∂θjdx]∘dYtk .

If one uses the Hellinger distance instead, square roots of densities are needed. The tangent space basis is then

{∂p(⋅,θ)∂θ1,⋯,∂p(⋅,θ)∂θn},

and one defines the metric

14gij(θ)=⟨∂p∂θi,∂p∂θj⟩=14∫1p(x,θ)∂p(x,θ)∂θi∂p(x,θ)∂θjdx.

The metric g is the Fisher information metric. One follows steps completely analogous to the direct metric case and the filter equation in Hellinger/Fisher metric is

dθi=[∑j=1ngij(θt)∫F(p(x,θt))p(x,θt)∂p(x,θt)∂θjdx]dt+∑k=1d[∑j=1mgij(θt)∫Gk(p(x,θt))p(x,θt)∂p(x,θt)∂θjdx]∘dYtk ,

again with initial condition a chosen θ0.

Substituting F and G one obtains dθi(t)=[∑j=1mgij(θt)∫ℒt∗p(x,θt)p(x,θt)∂p(x,θt)∂θjdx−∑j=1mgij(θt)∫12|bt(x)|2∂p(x,θt)∂θjdx]dt

+∑k=1d[∑j=1mgij(θt)∫bk(x,t)∂p(x,θt)∂θjdx]∘dYtk .

The projection filter in direct metric, when implemented on a manifold SΘ of mixture families, leads to equivalence with a Galerkin method.[6]

The projection filter in Hellinger/Fisher metric when implemented on a manifold SΘ1/2 of square roots of an exponential family of densities is equivalent to the assumed density filters.[3]

One should note that it is also possible to project the simpler Zakai equation for an unnormalized version of the density p. This would result in the same Hellinger projection filter but in a different direct metric projection filter.[6]

Finally, if in the exponential family case one includes among the sufficient statistics of the exponential family the observation function in dYt, namely b(x)'s components and |b(x)|2, then one can see that the correction step in the filtering algorithm becomes exact. In other terms, the projection of the vector field G is exact, resulting in G itself. Writing the filtering algorithm in a setting with continuous state X and discrete time observations Y, one can see that the correction step at each new observation is exact, as the related Bayes formula entails no approximation.[3]

### Optimal projection filters based on Ito vector and Ito jet projections

Now rather than considering the exact filter SPDE in Stratonovich calculus form, one keeps it in Ito calculus form

dpt=ℒt*pt dt+pt[b(⋅,t)−Ept(b(⋅,t))]T[dYt−Ept(b(⋅,t))dt].

In the Stratonovich projection filters above, the vector fields F and G were projected separately. By definition, the projection is the optimal approximation for F and G separately, although this does not imply it provides the best approximation for the filter SPDE solution as a whole. Indeed, the Stratonovich projection, acting on the two terms F and G separately, does not guarantee optimality of the solution p(⋅,θ0+δt) as an approximation of the exact p0+δt for say small δt. One may look for a norm ‖⋅‖ to be applied to the solution, for which

θ0+δt≈argminθ ‖p0+δt−p(⋅,θ)‖.

The Ito-vector projection is obtained as follows. Let us choose a norm for the space of densities, ‖⋅‖, which might be associated with the direct metric or the Hellinger metric.

One chooses the diffusion term in the approximating Ito equation for θt by minimizing (but not zeroing) the δt term of the Taylor expansion for the mean square error

Et[‖p0+δt−p(⋅,θ0+δt)‖2],

finding the drift term in the approximating Ito equation that minimizes the (δt)2 term of the same difference. Here the δt order term is minimized, not zeroed, and one never attains (δt)2 convergence, only δt convergence.

A further benefit of the Ito vector projection is that it minimizes the order 1 Taylor expansion in δt of

‖E[p0+δt−p(⋅,θ0+δt)]‖.

To achieve (δt)2 convergence, rather than δt convergence, the Ito-jet projection is introduced. It is based on the notion of metric projection.

The metric projection of a density p∈L2 (or p∈L2) onto the manifold SΘ (or SΘ1/2) is the closest point on SΘ (or SΘ1/2) to p (or p). Denote it by π(p). The metric projection is, by definition, according to the chosen metric, the best one can ever do for approximating p in SΘ. Thus the idea is finding a projection filter that comes as close as possible to the metric projection. In other terms, one considers the criterion θ0+δt≈argminθ ‖π(p0+δt)−p(⋅,θ)‖.

The detailed calculations are lengthy and laborious,[7] but the resulting approximation achieves (δt)2 convergence. Indeed, the Ito jet projection attains the following optimality criterion. It zeroes the δt order term and it minimizes the (δt)2 order term of the Taylor expansion of the mean square distance in L2 between π(p0+δt) and p(⋅,θ0+δt).

Both the Ito vector and the Ito jet projection result in final SDEs, driven by the observations dY, for the parameter θt that best approximates the exact filter evolution for small times.[7]

## Applications

Jones and Soatto (2011) mention projection filters as possible algorithms for on-line estimation in visual-inertial navigation,[12] mapping and localization, while again on navigation Azimi-Sadjadi and Krishnaprasad (2005)[13] use projection filters algorithms. The projection filter has been also considered for applications in ocean dynamics by Lermusiaux 2006.[14] Kutschireiter, Rast, and Drugowitsch (2022)[15] refer to the projection filter in the context of continuous time circular filtering. For quantum systems applications, see for example van Handel and Mabuchi (2005),[16] who applied the quantum projection filter to quantum optics, studying a quantum model of optical phase bistability of a strongly coupled two-level atom in an optical cavity. Further applications to quantum systems are considered in Gao, Zhang and Petersen (2019).[17] Ma, Zhao, Chen and Chang (2015) refer to projection filters in the context of hazard position estimation, while Vellekoop and Clark (2006)[18] generalize the projection filter theory to deal with changepoint detection. Harel, Meir and Opper (2015)[19] apply the projection filters in assumed density form to the filtering of optimal point processes with applications to neural encoding. Broecker and Parlitz (2000)[20] study projection filter methods for noise reduction in chaotic time series. Zhang, Wang, Wu and Xu (2014) [21] apply the Gaussian projection filter as part of their estimation technique to deal with measurements of fiber diameters in melt-blown nonwovens.

## See also

* Filtering problem
* Generalized filtering
* Nonlinear filter
* Extended Kalman filter
* Recursive Bayesian estimation

## References

1. ↑ 1.0 1.1 "Swedish Defense Research Agency Scientific Report" (PDF). http://www.foi.se/ReportFiles/foir_1074.pdf.
2. ↑ 2.0 2.1 2.2 2.3 2.4 2.5 Brigo, Damiano; Hanzon, Bernard; LeGland, Francois (1998). "A differential geometric approach to nonlinear filtering: the projection filter". IEEE Transactions on Automatic Control 43 (2): 247–252.
3. ↑ 3.0 3.1 3.2 3.3 3.4 3.5 3.6 Brigo, Damiano; Hanzon, Bernard; LeGland, Francois (1999). "Approximate nonlinear filtering by projection on exponential manifolds of densities.". Bernoulli 5 (3): 407–430.
4. ↑ Chaleyat-Maurel, Mireille and Dominique Michel (1984), Des resultats de non existence de filtre de dimension finie. Stochastics, volume 13, issue 1+2, pages 83–102.
5. ↑ M. Hazewinkel, S.I. Marcus, H.J. Sussmann (1983). Nonexistence of finite-dimensional filters for conditional statistics of the cubic sensor problem. Systems & Control Letters 3(6), Pages 331-340, https://doi.org/10.1016/0167-6911(83)90074-9.
6. ↑ 6.0 6.1 6.2 6.3 6.4 6.5 Armstrong, John; Brigo, Damiano (2016). "Nonlinear filtering via stochastic PDE projection on mixture manifolds in L2 direct metric". Mathematics of Control, Signals and Systems 28 (1): 1–33.
7. ↑ 7.0 7.1 7.2 7.3 Armstrong, John; Brigo, Damiano; Rossi Ferrucci, Emilio (2019). "Optimal approximation of {SDE}s on submanifolds: the Ito-vector and Ito-jet projections". Proceedings of the London Mathematical Society 119 (1): 176–213.
8. ↑ Armstrong, J., Brigo, D., and Hanzon, B. (2023). Optimal projection filters with information geometry. Info. Geo. (2023). https://doi.org/10.1007/s41884-023-00108-x
9. ↑ 9.0 9.1 Bernard Hanzon (1987). A differential-geometric approach to approximate nonlinear filtering. In: C.T.J. Dodson, Editor, Geometrization of Statistical Theory, pages 219–223. ULMD Publications, University of Lancaster
10. ↑ 10.0 10.1 Brigo, D. (1996). Filtering by projection on the manifold of exponential densities. PhD dissertation, Free University of Amsterdam
11. ↑ John Armstrong and Damiano Brigo (2018). Intrinsic stochastic differential equations as jets. Proceedings of the Royal Society A - Mathematical physical and engineering sciences, 474(2210), 28 pages. doi: 10.1098/rspa.2017.0559.
12. ↑ Jones, Eagle S; Soatto, Massimo (2011). "Visual-inertial navigation, mapping and localization: A scalable real-time causal approach". The International Journal of Robotics Research 30 (4): 407–430.
13. ↑ Azimi-Sadjadi, Babak; Krishnaprasad, P.S. (2005). "Approximate nonlinear filtering and its application in navigation". Automatica 41 (6): 945–956.
14. ↑ Lermusiaux, Pierre F. J (2006). "Uncertainty estimation and prediction for interdisciplinary ocean dynamics". Journal of Computational Physics 217 (1): 176–199.
15. ↑ Kutschireiter, Anna; Rast, Luke; Drugowitsch, Jan (2022). "Projection filtering with observed state increments with applications in continuous-time circular filtering". IEEE Transactions on Signal Processing 70.
16. ↑ van Handel, Ramon; Mabuchi, Hideo (2005). "Quantum projection filter for a highly nonlinear model in cavity QED". Journal of Optics B: Quantum and Semiclassical Optics 7.
17. ↑ Gao, Qing; Gao, Guofeng; Petersen, Ian R (2019). "An exponential quantum projection filter for open quantum systems". Automatica 99: 59–68.
18. ↑ Vellekoop, M. H.; Clark, J. M. C. (2006). "A nonlinear filtering approach to changepoint detection problems: Direct and differential-geometric methods". SIAM Review 48 (2): 329–356.
19. ↑ Harel, Yuval; Meir, Ron; Opper, Manfred (2015). "A tractable approximation to optimal point process filtering: Application to neural encoding". Advances in Neural Information Processing Systems 28.
20. ↑ Broecker, Jochen; Parlitz, Ulrich (2000). "Noise reduction and filtering of chaotic time series". Proc. NOLTA 2000.
21. ↑ Zhang, Xian Miao; Wu Wang, Rong; Xu, Bugau (2014). "Automated measurements of fiber diameters in melt-blown nonwovens". Journal of Industrial Textiles 43 (4): 593–605.

* v
* t
* e

Stochastic processes

Discrete time|

* Bernoulli process
* Branching process
* Chinese restaurant process
* Galton–Watson process
* Independent and identically distributed random variables
* Markov chain
* Moran process
* Random walk
* Loop-erased
* Self-avoiding
* Biased
* Maximal entropy

Continuous time|

* Bessel process
* Birth–death process
* Brownian motion
* Bridge
* Excursion
* Fractional
* Geometric
* Meander
* Cauchy process
* Contact process
* Continuous-time random walk
* Cox process
* Diffusion process
* Empirical process
* Feller process
* Fleming–Viot process
* Gamma process
* Geometric process
* Hunt process
* Interacting particle systems
* Itô diffusion
* Itô process
* Jump diffusion
* Jump process
* Lévy process
* Local time
* Markov additive process
* McKean–Vlasov process
* Ornstein–Uhlenbeck process
* Poisson process
* Compound
* Non-homogeneous
* Point process
* Schramm–Loewner evolution
* Semimartingale
* Sigma-martingale
* Stable process
* Superprocess
* Telegraph process
* Variance gamma process
* Wiener process
* Wiener sausage

Both|

* Branching process
* Galves–Löcherbach model
* Gaussian process
* Hidden Markov model (HMM)
* Markov process
* Martingale
* Differences
* Local
* Sub-
* Super-
* Random dynamical system
* Regenerative process
* Renewal process
* Stochastic chains with memory of variable length
* White noise

Fields and other|

* Dirichlet process
* Gaussian random field
* Gibbs measure
* Hopfield model
* Ising model
* Potts model
* Boolean network
* Markov random field
* Percolation
* Pitman–Yor process
* Point process
* Cox
* Poisson
* Random field
* Random graph

Time series models|

* Autoregressive conditional heteroskedasticity (ARCH) model
* Autoregressive integrated moving average (ARIMA) model
* Autoregressive (AR) model
* Autoregressive–moving-average (ARMA) model
* Generalized autoregressive conditional heteroskedasticity (GARCH) model
* Moving-average (MA) model

Financial models|

* Black–Derman–Toy
* Black–Karasinski
* Black–Scholes
* Chen
* Constant elasticity of variance (CEV)
* Cox–Ingersoll–Ross (CIR)
* Garman–Kohlhagen
* Heath–Jarrow–Morton (HJM)
* Heston
* Ho–Lee
* Hull–White
* LIBOR market
* Rendleman–Bartter
* SABR volatility
* Vašíček
* Wilkie

Actuarial models|

* Bühlmann
* Cramér–Lundberg
* Risk process
* Sparre–Anderson

Queueing models|

* Bulk
* Fluid
* Generalized queueing network
* M/G/1
* M/M/1
* M/M/c

Properties|

* Càdlàg paths
* Continuous
* Continuous paths
* Ergodic
* Exchangeable
* Feller-continuous
* Gauss–Markov
* Markov
* Mixing
* Piecewise deterministic
* Predictable
* Progressively measurable
* Self-similar
* Stationary
* Time-reversible

Limit theorems|

* Central limit theorem
* Donsker's theorem
* Doob's martingale convergence theorems
* Ergodic theorem
* Fisher–Tippett–Gnedenko theorem
* Large deviation principle
* Law of large numbers (weak/strong)
* Law of the iterated logarithm
* Maximal ergodic theorem
* Sanov's theorem

Inequalities|

* Burkholder–Davis–Gundy
* Doob's martingale
* Kunita–Watanabe

Tools|

* Cameron–Martin formula
* Convergence of random variables
* Doléans-Dade exponential
* Doob decomposition theorem
* Doob–Meyer decomposition theorem
* Doob's optional stopping theorem
* Dynkin's formula
* Feynman–Kac formula
* Filtration
* Girsanov theorem
* Infinitesimal generator
* Itô integral
* Itô's lemma
* Karhunen–Loève_theorem
* Kolmogorov continuity theorem
* Kolmogorov extension theorem
* Lévy–Prokhorov metric
* Malliavin calculus
* Martingale representation theorem
* Optional stopping theorem
* Prokhorov's theorem
* Quadratic variation
* Reflection principle
* Skorokhod integral
* Skorokhod's representation theorem
* Skorokhod space
* Snell envelope
* Stochastic differential equation
* Tanaka
* Stopping time
* Stratonovich integral
* Uniform integrability
* Usual hypotheses
* Wiener space
* Classical
* Abstract

Disciplines|

* Actuarial mathematics
* Econometrics
* Ergodic theory
* Extreme value theory (EVT)
* Large deviations theory
* Mathematical finance
* Mathematical statistics
* Probability theory
* Queueing theory
* Renewal theory
* Ruin theory
* Statistics
* System on Chip design
* Stochastic analysis
* Time series analysis
* Machine learning

* List of topics
* Category

* v
* t
* e

Statistics

* Outline
* Index

Descriptive statistics

| Continuous data|

| Center|

* Mean
* arithmetic
* geometric
* harmonic
* Median
* Mode

|

Dispersion|

* Variance
* Standard deviation
* Coefficient of variation
* Percentile
* Range
* Interquartile range

Shape|

* Central limit theorem
* Moments
* Skewness
* Kurtosis
* L-moments

Count data|

* Index of dispersion

Summary tables|

* Grouped data
* Frequency distribution
* Contingency table

Dependence|

* Pearson product-moment correlation
* Rank correlation
* Spearman's rho
* Kendall's tau
* Partial correlation
* Scatter plot

Graphics|

* Bar chart
* Biplot
* Box plot
* Control chart
* Correlogram
* Fan chart
* Forest plot
* Histogram
* Pie chart
* Q–Q plot
* Run chart
* Scatter plot
* Stem-and-leaf display
* Radar chart
* Violin plot

Data collection

| Study design|

* Population
* Statistic
* Effect size
* Statistical power
* Optimal design
* Sample size determination
* Replication
* Missing data

|

Survey methodology|

* Sampling
* stratified
* cluster
* Standard error
* Opinion poll
* Questionnaire

Controlled experiments|

* Scientific control
* Randomized experiment
* Randomized controlled trial
* Random assignment
* Blocking
* Interaction
* Factorial experiment

Adaptive Designs|

* Adaptive clinical trial
* Up-and-Down Designs
* Stochastic approximation

Observational Studies|

* Cross-sectional study
* Cohort study
* Natural experiment
* Quasi-experiment

Statistical inference

| Statistical theory|

* Population
* Statistic
* Probability distribution
* Sampling distribution
* Order statistic
* Empirical distribution
* Density estimation
* Statistical model
* Model specification
* Lp space
* Parameter
* location
* scale
* shape
* Parametric family
* Likelihood (monotone)
* Location–scale family
* Exponential family
* Completeness
* Sufficiency
* Statistical functional
* Bootstrap
* U
* V
* Optimal decision
* loss function
* Efficiency
* Statistical distance
* divergence
* Asymptotics
* Robustness

|

Frequentist inference|

| Point estimation|

* Estimating equations
* Maximum likelihood
* Method of moments
* M-estimator
* Minimum distance
* Unbiased estimators
* Mean-unbiased minimum-variance
* Rao–Blackwellization
* Lehmann–Scheffé theorem
* Median unbiased
* Plug-in

|

Interval estimation|

* Confidence interval
* Pivot
* Likelihood interval
* Prediction interval
* Tolerance interval
* Resampling
* Bootstrap
* Jackknife

Testing hypotheses|

* 1- & 2-tails
* Power
* Uniformly most powerful test
* Permutation test
* Randomization test
* Multiple comparisons

Parametric tests|

* Likelihood-ratio
* Score/Lagrange multiplier
* Wald

Specific tests|

* Z-test (normal)
* Student's t-test
* F-test

Goodness of fit|

* Chi-squared
* G-test
* Kolmogorov–Smirnov
* Anderson–Darling
* Lilliefors
* Jarque–Bera
* Normality (Shapiro–Wilk)
* Likelihood-ratio test
* Model selection
* Cross validation
* AIC
* BIC

Rank statistics|

* Sign
* Sample median
* Signed rank (Wilcoxon)
* Hodges–Lehmann estimator
* Rank sum (Mann–Whitney)
* Nonparametric anova
* 1-way (Kruskal–Wallis)
* 2-way (Friedman)
* Ordered alternative (Jonckheere–Terpstra)

Bayesian inference|

* Bayesian probability
* prior
* posterior
* Credible interval
* Bayes factor
* Bayesian estimator
* Maximum posterior estimator

* Correlation
* Regression analysis

| Correlation|

* Pearson product-moment
* Partial correlation
* Confounding variable
* Coefficient of determination

|

Regression analysis|

* Errors and residuals
* Regression validation
* Mixed effects models
* Simultaneous equations models
* Multivariate adaptive regression splines (MARS)

Linear regression|

* Simple linear regression
* Ordinary least squares
* General linear model
* Bayesian regression

Non-standard predictors|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Isotonic
* Robust
* Heteroscedasticity
* Homoscedasticity

Generalized linear model|

* Exponential families
* Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance|

* Analysis of variance (ANOVA, anova)
* Analysis of covariance
* Multivariate ANOVA
* Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

| Categorical|

* Cohen's kappa
* Contingency table
* Graphical model
* Log-linear model
* McNemar's test

|

Multivariate|

* Regression
* Manova
* Principal components
* Canonical correlation
* Discriminant analysis
* Cluster analysis
* Classification
* Structural equation model
* Factor analysis
* Multivariate distributions
* Elliptical distributions
* Normal

Time-series|

| General|

* Decomposition
* Trend
* Stationarity
* Seasonal adjustment
* Exponential smoothing
* Cointegration
* Structural break
* Granger causality

|

Specific tests|

* Dickey–Fuller
* Johansen
* Q-statistic (Ljung–Box)
* Durbin–Watson
* Breusch–Godfrey

Time domain|

* Autocorrelation (ACF)
* partial (PACF)
* Cross-correlation (XCF)
* ARMA model
* ARIMA model (Box–Jenkins)
* Autoregressive conditional heteroskedasticity (ARCH)
* Vector autoregression (VAR)

Frequency domain|

* Spectral density estimation
* Fourier analysis
* Wavelet
* Whittle likelihood

Survival|

| Survival function|

* Kaplan–Meier estimator (product limit)
* Proportional hazards models
* Accelerated failure time (AFT) model
* First hitting time

|

Hazard function|

* Nelson–Aalen estimator

Test|

* Log-rank test

Applications

| Biostatistics|

* Bioinformatics
* Clinical trials / studies
* Epidemiology
* Medical statistics

|

Engineering statistics|

* Chemometrics
* Methods engineering
* Probabilistic design
* Process / quality control
* Reliability
* System identification

Social statistics|

* Actuarial science
* Census
* Crime statistics
* Demography
* Econometrics
* Jurimetrics
* National accounts
* Official statistics
* Population statistics
* Psychometrics

Spatial statistics|

* Cartography
* Environmental statistics
* Geographic information system
* Geostatistics
* Kriging

* Category
* '
* Commons
* WikiProject

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Projection filters. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Projection_filters&oldid=2684175"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template