Kuiper's test

Kuiper's test is used in statistics to test whether a data sample comes from a given distribution (one-sample Kuiper test), or whether two data samples came from the same unknown distribution (two-sample Kuiper test). It is named after Dutch mathematician Nicolaas Kuiper.

Kuiper's test is closely related to the better-known Kolmogorov–Smirnov test (or K-S test as it is often called). As with the K-S test, the discrepancy statistics D⁺ and D⁻ represent the absolute sizes of the most positive and most negative differences between the two cumulative distribution functions that are being compared. The trick with Kuiper's test is to use the quantity D⁺ + D⁻ as the test statistic. This small change makes Kuiper's test as sensitive in the tails as at the median and also makes it invariant under cyclic transformations of the independent variable. The Anderson–Darling test is another test that provides equal sensitivity at the tails as the median, but it does not provide the cyclic invariance.

This invariance under cyclic transformations makes Kuiper's test invaluable when testing for cyclic variations by time of year or day of the week or time of day, and more generally for testing the fit of, and differences between, circular probability distributions.

One-sample Kuiper test

Illustration of the two-sample Kuiper Test statistic. Red and blue lines each correspond to an empirical distribution function, and the black arrows show the points distances which sum to the Kuiper Statistic.

The one-sample test statistic, $V_{n}$, for Kuiper's test is defined as follows. Let F be the continuous cumulative distribution function which is to be the null hypothesis. Denote by F_n the empirical distribution function for n independent and identically distributed (i.i.d.) observations X_i, which is defined as

$F_{n}(x)={\frac {{\text{number of (elements in the sample}}\leq x)}{n}}={\frac {1}{n}}\sum _{i=1}^{n}1_{(-\infty ,x]}(X_{i}),$

where $1_{(-\infty ,x]}(X_{i})$ is the indicator function, equal to 1 if $X_{i}\leq x$ and equal to 0 otherwise.

Then the one-sided Kolmogorov–Smirnov statistic for the given cumulative distribution function F(x) is

$D_{n}^{+}=\sup _{x}[F_{n}(x)-F(x)],$

$D_{n}^{-}=\sup _{x}[F(x)-F_{n}(x)],$

where $\sup$ is the supremum function. And finally the one-sample Kuiper test is defined as,

$V_{n}=D_{n}^{+}+D_{n}^{-},$

or equivalently

$V_{n}=\sup _{x}[F_{n}(x)-F(x)]-\inf _{x}[F_{n}(x)-F(x)],$

where $\inf$ is the infimum function.

Tables for the critical points of the test statistic $V_{n}$ are available, and these include certain cases where the distribution being tested is not fully known, so that parameters of the family of distributions are estimated.

The asymptotic distribution of the statistic ${\sqrt {n}}V_{n}$ is given by,

${\begin{aligned}\operatorname {Pr} ({\sqrt {n}}V_{n}\leq x)=&1-2\sum _{k=1}^{\infty }(4k^{2}x^{2}-1)e^{-2k^{2}x^{2}}\\&+{\frac {8}{3{\sqrt {n}}}}x\sum _{k=1}^{\infty }k^{2}(4k^{2}x^{2}-3)e^{-2k^{2}x^{2}}+o({\frac {1}{n}}).\end{aligned}}$

For $x>{\frac {6}{5}}$, a reasonable approximation is obtained from the first term of the series as follows

$1-2(4x^{2}-1)e^{-2x^{2}}+{\frac {8x}{3{\sqrt {n}}}}(4x^{2}-3)e^{-2x^{2}}.$

Two-sample Kuiper test

The Kuiper test may also be used to test whether two random samples, either on the real line or the circle, come from a common but unknown distribution. In this case, the Kuiper statistic is

$V_{n,m}=\sup _{x}[F_{1,n}(x)-F_{2,m}(x)]-\inf _{x}[F_{1,n}(x)-F_{2,m}(x)],$

where $F_{1,n}$ and $F_{2,m}$ are the empirical distribution functions of the first and the second sample respectively, $\sup$ is the supremum function, and $\inf$ is the infimum function.

For continuous distributions and large samples, the null distribution of the normalized statistic approaches the same limiting form as in the one-sample case. As $m,n\to \infty$ with $m/(m+n)\to \lambda \in (0,1)$,

$\operatorname {Pr} \!\left({\sqrt {\tfrac {nm}{n+m}}}\,V_{n,m}\leq x\right)\to 1-2\sum _{k=1}^{\infty }(4k^{2}x^{2}-1)\,e^{-2k^{2}x^{2}},$

the two-sample analogue of the limiting distribution given by Kuiper. This large-sample approximation is distribution-free only under continuity; it can be inaccurate for small samples and does not apply when the data contain ties.

In those cases an exact p-value can instead be obtained from the permutation (conditional) distribution of $V_{n,m}$ given the pooled sample. This remains valid whether or not the data contain tied observations, so the two samples may come from a continuous, purely discrete or mixed distribution, and it can be computed by a recurrence that is valid for arbitrary (possibly large and unequal) sample sizes. For continuous data the permutation and ordinary Kuiper statistics share the same limiting null distribution, so their critical values agree asymptotically; this agreement has been shown to extend to arbitrary underlying distributions, including discrete or mixed data with ties. Approximate p-values may also be obtained by resampling.

Example

We could test the hypothesis that computers fail more during some times of the year than others. To test this, we would collect the dates on which the test set of computers had failed and build an empirical distribution function. The null hypothesis is that the failures are uniformly distributed. Kuiper's statistic does not change if we change the beginning of the year and does not require that we bin failures into months or the like. Another test statistic having this property is the Watson statistic, which is related to the Cramér–von Mises test.

However, if failures occur mostly on weekends, many uniform-distribution tests such as K-S and Kuiper would miss this, since weekends are spread throughout the year. This inability to distinguish distributions with a comb-like shape from continuous uniform distributions is a key problem with all statistics based on a variant of the K-S test. Kuiper's test, applied to the event times modulo one week, is able to detect such a pattern. Using event times that have been modulated with the K-S test can result in different results depending on how the data is phased. In this example, the K-S test may detect the non-uniformity if the data is set to start the week on Saturday, but fail to detect the non-uniformity if the week starts on Wednesday.

Two-sample example

Suppose the days of the week on which some event occurs are recorded for two groups, giving discrete data on the seven points of a weekly cycle (so tied observations are unavoidable). Writing the days as positions 1–7 and cumulating each group's empirical distribution function, the largest excess of the first group's ECDF over the second is $D^{+}=0.4$ and the largest excess in the other direction is $D^{-}=0.4$, giving a Kuiper statistic of $V_{n,m}=D^{+}+D^{-}=0.8$. Because $V_{n,m}$ is unchanged by the choice of which day starts the week, the same value is obtained under any cyclic relabelling — unlike the Kolmogorov–Smirnov statistic, whose value can depend on where the cycle is cut. An exact p-value for these discrete data, accounting for the ties, can then be obtained from the permutation recurrence.

Software implementations

Implementations of the two-sample Kuiper test differ in how the p-value is obtained:

• The R package KSgeneral provides the function Kuiper2sample, which computes exact p-values for continuous, purely discrete or mixed data, including tied observations, for arbitrary sample sizes.
• The Python library Astropy provides kuiper_two, which returns the statistic together with an approximate false-positive probability computed from an asymptotic formula.
• The R package twosamples provides kuiper_test, which obtains the p-value by permutation resampling of the pooled sample.

See also

• Kolmogorov–Smirnov test