Probability Theory Order Statistics

Consider a set X,X,,Xn of independent and identically distributed

(continuous) random variables.
Definition 1.1. Let X(k) denote the k:th smallest of X,X,,Xn. The random
Chapter 4 vector (X(1),X(2),,X(n)) is called the order statistic and X(k) the k:th order
variable, k=1,2,,n.
For the order statistic we thus have that
Order Statistics
It is of natural interest to find the joint probability distributions of these
ordered random variables, and we will begin by finding the marginal
probability distributions of the extremes, that is

Thommy Perlinger, Probability Theory 1 2

The probability distribution of X(n) Example 1

From the common distribution function F(x) it follows that Let Y,Y,,Yn be a set of independent and identically distributed U(0,).
Determine the probability distribution of Y(n).

Since

independent
it follows that
and identically
distributed

Differentiation according to the chain rule gives us the density function of X(n).

3 Thommy Perlinger, Probability Theory 4

1
 Example 2 Example 2
Let Y,Y, be a sequence of independent and identically
distributed U(0,1). Furthermore, let X be discrete with
probability function

Find the distribution of W = min(Y,Y,,YX).

We first observe that W | X=x is the first order statistic from a

sample of size x.

Thommy Perlinger, Probability Theory 5 Thommy Perlinger, Probability Theory 6

The probability distribution of X(k) Example for the median

We now generalize and look at the marginal probability distribution of X(k), Let X1,X2, and X3 be a random sample from an Exp(1) distribution. Find the
i.e. the k:th order variable, k=1,2,,n. distribution of the median and compute its mean.

Theorem 1.2. Consider a set X,X,,Xn of independent and identically According to Theorem 1.2 the density of the median, i.e. X(2), is given by
distributed (continuous) random variables with density function f(x) and
distribution function F(x). For k=1,2,,n, the density of X(k) is given by

The mean of X(2) is thus

Proof. Both a formal and a heuristic proof can be found in the textbook.

Thommy Perlinger, Probability Theory 7 Thommy Perlinger, Probability Theory 8

2
 The joint distribution of X(1) and X(n) Exercise 2.6
Theorem 2.1. Consider a set X,X,,Xn of independent and identically Exercise 2.6. Suppose n points are chosen uniformly and independently of
distributed (continuous) random variables with density function f(x) and each other on the unit disc. Compute the expected value of the area of the
distribution function F(x). The joint density of the extremes is given by annulus obtained by drawing circles through the extremes.

What to do? Let the distance of a point to the center be Rn, n=1,2,,n.
If R(1)=u and R(n)=v we get the following situation:

The area of the annulus is thus given by (v2-u2). v

Theorem 2.2. The range is defined by Rn= X(n)- X(1). Let u=x(1). The density u
of Rn is then given by Step 1. Find the distribution of R, i.e. the distance to the
center.
Step 2. Find the distribution of W=R2, i.e. the squared distance
to the center.
Step 3. Find the distribution of the range RW = W(n) -W(1) for the squared
Remark. When the domain of X has an upper bound b, then for Rn=r the distance to the center.
upper bound of U becomes b-r.
Step 4. Compute E(RW), i.e. the expected value of the area of the annulus.
Thommy Perlinger, Probability Theory 9

Exercise 2.6 Exercise 2.6

Step 1. If (X1,Y1), (X2,Y2),, (Xn,Yn) represents the coordinates of the We thus want to find the marginal distribution of R. Since
chosen points, then they are i.i.d. bivariate random variables with common
density given by

it follows from the transformation theorem that

For each such point, we are interested in the distance to the center of the
circle. We therefore switch to polar coordinates, i.e. we study the bijection

and hence that

where R is the distance to the center of the circle and the angle.

Thommy Perlinger, Probability Theory 11 Thommy Perlinger, Probability Theory 12

3
 Exercise 2.6 The joint distribution of the order statistic

Step 2. Let W=R2. It follows from the (univariate) transformation theorem that Theorem 3.1. Consider a set X,X,,Xn of independent and identically
distributed (continuous) random variables with density function f(x) and
distribution function F(x). The joint density of the order statistic is given by

Step 3. The squared distances to the center can thus be seen as n i.i.d.
U(0,1). Let RW represent the range. It then follows from Theorem 2.2 that

Step 4. We conclude that RW (n-1,2). It therefore follows that the expected

value of the area of the annulus is given by the intuitively appealing

Thommy Perlinger, Probability Theory 13 Thommy Perlinger, Probability Theory 14

Problem 4.4.14 Problem 4.4.6 a

Let X1, X2 and X3 be independent U(0,1)-distributed random variables.
Prove the intuitively reasonable result that X(1) and X(3) are conditionally
independent given X(2) and determine this (conditional) distribution.
Remark. We thus have to determine the distribution of
Let X1, X2, X3 and X4 be independent U(0,1)-distributed random variables.
Compute Pr(X(3)+X(4) 1).
The first task is to find the joint density of X(3) and X(4). Theorem 3.1 tells us

It follows from Theorem 3.1 and Theorem 1.2 that

which implies that

which means that

From here on it is standard technique using the transformation theorem.

and the statement is proven. The marginal distributions are U(0,x) and U(x,1).
Thommy Perlinger, Probability Theory 16

4
 Problem 4.4.6 a Problem 4.4.6 a
To find the density of X(3)+X(4) we introduce the auxiliary variable X(3), i.e. Hence, the marginal distribution of U is given by

that is and so

and so it follows from the transformation theorem that

and so we finally find that

A little bit of care has to be taken when
determining the marginal distribution of U.
We see that the two cases 0<u<1 and 1u<2
must be considered separately.

Thommy Perlinger, Probability Theory 17 Thommy Perlinger, Probability Theory 18