Chapter 12

Appendix

12.1 Inventory of distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

12.1.1 Discrete families . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
12.1.2 Continuous families . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
12.2 Distribution tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
12.3 Calculus review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
12.3.1 Inverse function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
12.3.2 Limits and continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
12.3.3 Sequences and series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
12.3.4 Derivatives, minimum, and maximum . . . . . . . . . . . . . . . . . . . 429
12.3.5 Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
12.4 Matrices and linear systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
12.5 Answers to selected exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439

12.1 Inventory of distributions

12.1.1 Discrete families

Bernoulli(p)

Generic description: 0 or 1, success or failure, result of one Bernoulli trial

Range of values: x = 0, 1
Parameter: p (0, 1), probability of success

Probability mass function: P (x) = px (1 p)1x

Expectation: =p
Variance: 2 = p(1 p)
Relations: Special case of Binomial(n, p) when n = 1
A sum of n independent Bernoulli(p) variables is
Binomial(n, p)

Binomial(n, p)

Generic description: Number of successes in n independent Bernoulli trials

Range of values: x = 0, 1, 2, . . . , n
Parameters: n = 1, 2, 3, . . ., number of Bernoulli trials
p (0, 1), probability of success

405
406 Probability and Statistics for Computer Scientists

n 
Probability mass function: P (x) = px (1 p)nx
x
Cdf: Table A2
Expectation: = np
Variance: 2 = np(1 p)
Relations: Binomial(n, p) is a sum of n independent Bernoulli(p)
variables
Binomial(1, p) = Bernoulli(p)
Table: Appendix, Table A2
MATLAB: binocdf, binopdf, binornd

Geometric(p)

Generic description: Number of Bernoulli trials until the rst success

Range of values: x = 1, 2, 3, . . .
Parameter: p (0, 1), probability of success

Probability mass function: P (x) = p(1 p)x1

Cdf: 1 (1 p)x
1
Expectation: =
p
1p
Variance: 2 =
p2
Relations: Special case of Negative Binomial(k, p) when k = 1
A sum of n independent Geometric(p) variables is
Negative Binomial(n, p)
MATLAB: geocdf, geopdf, geornd

Negative Binomial(k, p)

Generic description: Number of Bernoulli trials until the kth success

Range of values: x = k, k + 1, k + 2, . . .
Parameters: k = 1, 2, 3, . . ., number of successes
p (0, 1), probability of success
 
x1
Probability mass function: P (x) = (1 p)xk pk
k1
k
Expectation: =
p
2 k(1 p)
Variance: =
p2
Relations: Negative Binomial(k, p) is a sum of n independent
Geometric(p) variables
Negative Binomial(1, p) = Geometric(p)
MATLAB: nbincdf, nbinpdf, nbinrnd
 Appendix 407

Poisson()

Generic description: Number of rare events during a xed time interval

Range of values: x = 0, 1, 2, . . .
Parameter: (0, ), frequency of rare events
x
Probability mass function: P (x) = e
x!
Cdf: Table A3
Expectation: =
Variance: 2 =
Relations: Limiting case of Binomial(n, p) when n , p 0, np
Table: Appendix, Table A3
MATLAB: poisscdf, poisspdf, poissrnd

12.1.2 Continuous families

Beta(, )

Generic description: A random variable taking values in [0, 1]; distribution

of R2 under the standard regression assumptions.
Range of values: 0<x<1
Parameters: , (0, )
( + ) 1
Density: f (x) = x (1 x)1
()()
Expectation: = /( + )
Variance: 2 = /( + )2 ( + + 1)

Relations: For independent X Gamma(, ) and Y Gamma(, ),

X
the ratio X+Y is Beta(, ).
As a prior distribution, Beta family is conjugate
to the Binomial model.
Beta(1,1) distribution is Uniform(0,1).
In a sample of Uniform(0,1) variables of size n,
the kth smallest observation is Beta(k, n + 1 k).
MATLAB: betacdf, betapdf, betarnd

Chi-square 2 ()

Generic description: Sum of squares of iid Normal variables

Range of values: x>0
Parameter: (0, ), degrees of freedom
1
Density: f (x) = x/21 ex/2
2/2 (/2)
Expectation: =
Variance: 2 = 2
408 Probability and Statistics for Computer Scientists

Relations: Special case of Gamma(, ) when = /2 and = 1/2.

Ratio of two independent 2 variables, divided by their
respective degrees of freedom, has F-distribution.
MATLAB: chi2cdf, chi2pdf, chi2rnd

Exponential()

Generic description: In a Poisson process, time between consecutive events

Range of values: x>0
Parameter: (0, ), frequency of events, inverse scale parameter

Density: f (x) = ex
Cdf: F (x) = 1 ex
Expectation: = 1/
Variance: 2 = 1/2

Relations: Special case of Gamma(, ) when = 1 and chi-square

with = 2 degrees of freedom. A sum of independent
Exponential() variables is Gamma(, )
MATLAB: expcdf, exppdf, exprnd

Fisher-Snedecor F(1 , 2 )

Generic description: Ratio of independent mean squares

Range of values: x>0
Parameters: 1 > 0, numerator degrees of freedom
2 > 0, denominator degrees of freedom

s
1 +
2
2
(1 x)1 22
Density function: f (x) = 1

2


x 2 2 (1 x + 2 )1 +2
2
Expectation: for 2 > 2
2 2

2 2 (1 + 2 2)
Variance: for 2 > 4
1 (2 2)2 (2 4)
2 (1 )/1
Relations: For independent 2 (1 ) and 2 (2 ), the ratio 2 (2 )/2 is F (1 , 2 )
Special case: a square of t() variable is F (1, )
Table: Appendix, Table A7
MATLAB: fcdf, fpdf, frnd

Gamma(, )

Generic description: In a Poisson process, the time of the -th event

Range of values: x>0
Parameters: (0, ), shape parameter
(0, ), frequency of events, inverse scale parameter
 Appendix 409

1 x
Density function: f (x) = x e
()
Expectation: = /
Variance: 2 = /2
Relations: For integer , Gamma(, ) is a sum of independent
Exponential() variables
Gamma(1, ) = Exponential()
Gamma(/2, 1/2) = Chi-square()
As a prior distribution, Gamma family is conjugate to
the Poisson() model and the Exponential() model.
MATLAB: gamcdf, gampdf, gamrnd

Normal(, )

Generic description: Often used as distribution of errors, measurements, sums,

averages, etc.
Range of values: < x < +
Parameters: (, ), expectation, location parameter
(0, ), standard deviation, scale parameter
 
1 (x )2
Density function: f (x) = exp
2  2 2
x
Cdf: F (x) = , use Table A4 for (z)

Expectation:
Variance: 2
Relations: Limiting case of standardized sums of random variables,
including Binomial(n, p), Negative Binomial(k, p),
and Gamma(, ), as n, k,
Table: Appendix, Table A4
MATLAB: normcdf, normpdf, normrnd, or just randn

Pareto(, )

Generic description: A heavy-tail distribution often used to model amount of

internet trac, various nancial and sociological data
Range of values: x>
Parameters: (0, ), shape parameter
(0, ), scale parameter
Density function: f (x) = x1
 x 
Cdf: F (x) = 1


Expectation: for > 1; does not exist for 1
1
2
Variance: for > 2; does not exist for 2
( 1)2 ( 2)
Relations: If X is Exponential() then Y = eX is Pareto(, )
MATLAB: gpcdf, gppdf, gprnd
410 Probability and Statistics for Computer Scientists

These commands are for the Generalized Pareto distribution.

For Pareto(, ), use gpcdf(x,1/theta,sigma/theta/sigma)

Students T()

Generic description: Standardized statistic with an estimated standard deviation

Range of values: < x < +
Parameter: > 0, number of degrees of freedom

 +1

+12 x2 2
Density function: f (x) =
1+
2
Expectation: 0

Variance: for > 2
2

Relations: Converges to the Standard Normal distribution as

Table: Appendix, Table A5
MATLAB: tcdf, tpdf, trnd

Uniform(a, b)

Generic description: A number selected at random from a given interval

Range of values: a<x<b
Parameters: < a < b < +, ends of the interval
1
Density function: f (x) =
ba
a+b
Expectation: =
2
(b a)2
Variance: 2 =
12
Relations: Uniform(0,1) distribution is Beta(1,1)
MATLAB: unifdf, unifpdf, unifrnd, or just rand