Escolar Documentos
Profissional Documentos
Cultura Documentos
Fundamental Theorems
Arithmetic
Every integer greater than one is either prime or can be expressed as an unique product of prime
numbers
Algebra
Every polynomial function on one variable of degree n > 0 has at least one real or complex zero.
Linear Programming
If there is a solution to a linear programming problem, then it will occur at a corner point or on a
boundary between two or more corner points
Factorials
If n is a positive integer, then
n! = n (n-1) (n-2) ... (3)(2)(1)
n! = n (n-1)!
A special case is 0!
0! = 1
Permutations
A permutation is an arrangement of objects without repetition where order is important.
Pn = P(n,n) = n!
ACB
BAC
BCA
CAB
CBA
Pr = P(n,r) = n! / (n-r)!
AC
BA
BC
CA
CB
Distinguishable Permutations
Sometimes letters are repeated and all of the permutations aren't distinguishable from each other.
Example: Find all permutations of the letters "BOB"
To help you distinguish, I'll write the second "B" as "b"
BOb
BbO
OBb
ObB
bBO
bOB
BBO
OBB
OBB
BBO
BBO
BBO
OBB
If a word has N letters, k of which are unique, and you let n (n1, n2, n3, ..., nk) be the frequency
of each of the k letters, then the total number of distinguishable permutations is given by:
Combinations
A combination is an arrangement of objects without repetition where order is not important.
Note: The difference between a permutation and a combination is not whether there is repetition
or not -- there must not be repetition with either, and if there is repetition, you can not use the
formulas for permutations or combinations. The only difference in the definition of a
permutation and a combination is whether order is important.
A combination of n objects, arranged in groups of size r, without repetition, and order being
important is:
n
Cr = C(n,r) = n! / ( (n-r)! * r! )
Another way to write a combination of n things, r at a time is using the binomial notation:
Example: Find all two-letter combinations of the letters "ABC"
AB = BA
AC = CA
BC = CB
Pascal's Triangle
Combinations are used in the binomial expansion theorem from algebra to give the coefficients
of the expansion (a+b)^n. They also form a pattern known as Pascal's Triangle.
1
1
5
10
1
3
1
10 5
1
1
6 15 20 15
6
1
1
7 21 35 35 21
7
1
1
Each element in the table is the sum of the two elements directly above it. Each element is also a
combination. The n value is the number of the row (start counting at zero) and the r value is the
element in the row (start counting at zero). That would make the 20 in the next to last row C(6,3)
-- it's in the row #6 (7th row) and position #3 (4th element).
Symmetry
Pascal's Triangle illustrates the symmetric nature of a combination. C(n,r) = C(n,n-r)
Example: C(10,4) = C(10,6) or C(100,99) = C(100,1)
TI-82
You can use the TI-82 graphing calculator to find factorials, permutations, and combinations.
Tree Diagrams
Tree diagrams are a graphical way of listing all the possible
outcomes. The outcomes are listed in an orderly fashion, so listing
all of the possible outcomes is easier than just trying to make sure
that you have them all listed. It is called a tree diagram because of
the way it looks.
The first event appears on the left, and then each sequential event is
represented as branches off of the first event.
The tree diagram to the right would show the possible ways of flipping two coins. The final
outcomes are obtained by following each branch to its conclusion: They are from top to bottom:
HH
HT
TH
TT
Table of Contents
in
number of events in and in , and this can be tricky. We review here some basic ways to count
outcomes from "experiments". These approaches should be familiar from high school
mathematics.
ways.
For example, suppose a class has 30 men and 25 women. There are
can pick one student to answer a question.
ways.
For example, to ride a bike, you must have the chain on both a front sprocket and a rear sprocket.
For a 21 speed bike there are 3 ways to select the front sprocket and 7 ways to select the rear
sprocket.
This linkage of OR with addition and AND with multiplication will occur throughout the course,
so it is helpful to make this association in your mind. The only problem with applying it is that
questions do not always have an AND or an OR in them. You often have to play around with rewording the question for yourself to discover implied AND's or OR's.
Example: Suppose we pick 2 numbers at random from digits 1, 2, 3, 4, 5 with replacement.
(Note: "with replacement" means that after the first number is picked it is "replaced" in the set of
numbers, so it could be picked again as the second number.) Let us find the probability that one
number is even. This can be reworded as: "The first number is even AND the second is odd, OR,
the first is odd AND the second is even." We can then use the addition and multiplication rules to
calculate that there are
When objects are selected and replaced after each draw, the addition and multiplication rules are
generally sufficient to find probabilities. When objects are drawn without being replaced, some
special rules may simplify the solution.
Problems:
1.
1.
1. A course has 4 sections with no limit on how many can enrol in each
section. 3 students each randomly pick a section. Find the probability:
1. they all end up in the same section
2. Canadian postal codes consist of 3 letters alternated with 3 digits, starting with a
letter (e.g. N2L 3G1). For a randomly constructed postal code, what is the
probability:
1. all 3 letters are the same?
2. the digits are all even or all odd? Treat 0 as being neither even nor odd.
3. Suppose a password has to contain between six and eight digits, with each digit
either a letter or a number from 1 to 9. There must be at least one number present.
1. What is the total number of possible passwords?
2. If you started to try passwords in random order, what is the probability
you would find the correct password for a given situation within the first
1,000 passwords you tried?
Permutation Rules
Suppose that distinct objects are to be ``drawn" sequentially, or ordered from left to
right in a row.
(Order matters; objects are drawn without replacement)
4. The number of ways to arrange distinct objects in a row is
Explanation: We can fill the first position in ways. Since this object can't be
used again, there are only
ways to fill the second position. So we keep
having 1 fewer object available after each position is filled.
Statistics is important, and many games are interesting largely because of the
extraordinary rate of growth of the function
01234 5
10
in
For example
1126
which means that for many problems involving sampling from a deck of cards or a
reasonably large population, counting the number of cases is virtually impossible. There
is an approximation to
says that
is asymptotic to
asymptotically equal if
and
are called
selection,
represent
E.g.
While
to
only has a physical interpretation when and are positive integers with
, it still has a mathematical meaning when is not a positive integer, as long as is a non-
in terms of factorials.
Note that
The idea in using counting methods is to break the experiment into pieces or ``jobs'' so
that counting rules can be applied. There is usually more than one way to do this.
Example: We form a 4 digit number by randomly selecting and arranging 4 digits from 1,
2, 3,...7 without replacement. Find the probability the number formed is (a) even (b) over
3000 (c) an even number over 3000.
Solution: Let be the set of all possible 4 digit numbers using digits 1, 2, ..., 7 without
repetitions.
Then has
points. (We could calculate this but it will be easier to leave it in this form
for now and do some cancelling later.)
6. For a number to be even, the last digit must be even. We can fill this last position
with a 2, 4, or 6; i.e. in 3 ways. The first 3 positions can be filled by choosing and
arranging 3 of the 6 digits not used in the final position. i.e. in
ways. Then
there are
ways to fill the final position AND the first 3 positions to
produce an even number.
Another way to do this problem is to note that the four digit number is even if and
only if (iff) the last digit is even. The last digit is equally likely to be any one of
the numbers 1, ..., 7 so
7. To get a number over 3000, we require the first digit to be 3, 4, 5, 6, or 7; i.e. it
can be chosen in 5 ways. The remaining 3 positions can be filled in
ways.
Another way to do this problem is to note that the four digit number is over 3000
iff the first digit is one of 3, 4, 5, 6 or 7. Since each of 1, ..., 7 is equally likely to
be the first digit, we get
(number
3000) =
Note that in both (a) and (b) we dealt with positions which had restrictions first,
before considering positions with no restrictions. This is generally the best
approach to follow in applying counting techniques.
8. This part has restrictions on both the first and last positions. To illustrate the
complication this introduces, suppose we decide to fill positions in the order 1
then 4 then the middle two. We can fill position 1 in 5 ways. How many ways can
we then fill position 4? The answer is either 2 or 3 ways, depending on whether
the first position was filled with an even or odd digit. Whenever we encounter a
situation such as this, we have to break the solution into separate cases. One case
is where the first digit is even. The positions can be filled in 2 ways for the first
(i.e. with a 4 or 6), 2 ways for the last, and then
ways to arrange 2 of the
remaining 5 digits in the middle positions. This first case then occurs in
ways. The second case has an odd digit in position one. There are 3
ways to fill position one (3, 5, or 7), 3 ways to fill position four (2, 4, or 6), and
ways to fill the remaining positions. Case 2 then occurs in
We need case 1 OR case 2.
ways.
Another way to do this is to realize that we need only to consider the first and last
digit, and to find (first digit is 3 and last digit is even). There are
different choices for (first digit, last digit) and it is easy to see there are 13 choices
for which first digit
Exercise: Try to solve part (c) by filling positions in the order 4, 1, middle. You should
get the same answer.
Exercise: Can you spot the flaw in the following?
There are
There are
even and 3000. (Read the conditions in the multiplication rule carefully, if you believe
this solution.)
Here is another useful rule.
9. The number of distinct arrangements of objects when
alike of a
type, ...,
alike of a
type
in
and
times as the
of the
arrangements if all
times as the
type
Example: 5 men and 3 women sit together in a row. Find the probability that
10. the same gender is at each end
11. the women all sit together.
What are you assuming in your solution? Is it likely to be valid in real life?
and 3
points.
's and
, since
equally likely.
13. Treating
There are
's and 1
Our solution is based on the assumption that all points in are equally probable.
This would mean the people sit in a purely random order. In real life this isn't
likely, for example, since friends are more likely to sit together.
Problems:
14. Digits 1, 2, 3, ..., 7 are arranged at random to form a 7 digit number. Find the
probability that
1. the even digits occur together, in any order
2. the digits at the 2 ends are both even or both odd.
15. The letters of the word EXCELLENT are arranged in a random order. Find the
probability that
1. the same letter occurs at each end.
2.
and
Combinations
This deals with cases where order does not matter; objects are drawn without
replacement.
The number of ways to choose objects from is denoted by
For and both non-negative integers with
Proof: From result 2 earlier, the number of ways to choose objects from and arrange
them from left to right is
must have
ways, so we
Note that
Example: In the Lotto 6/49 lottery, six numbers are drawn at random, without
replacement, from the numbers 1 to 49. Find the probability that
16. the numbers drawn are 1, 2, 3, 4, 5, 6 (in some order)
17. no even number is drawn.
Solution:
18. Let the sample space consist of all combinations of 6 numbers from 1, ..., 49;
there are
= (all odd
numbers)
=
points.
points in we are
(The term
omitted since
of the 4 aces.)
can be
29. Solution 3: This solution is incorrect, but illustrates a common error. Choose
1 of the 4 aces then any 12 of the remaining 51 cards. This guarantees we have at
least 1 ace, so
The flaw in this solution is that it counts some points more than once by partially
keeping track of order. For example, we could get the ace of spades on the first
choice and happen to get the ace of clubs in the last 12 draws. We also could get
the ace of clubs on the first draw and then get the ace of spades in the last 12
draws. Though in both cases we have the same outcome, they would be counted
as 2 different outcomes.
(c)
ways.
31. The split in (c) is only 1 of several possible 6-4-2-1 splits. In fact, filling in the
numbers 6, 4, 2 and 1 in the spaces above each suit
32. This is the same as (d) except the numbers 5-4-2-2 are not all different. There are
different arrangements of 5-4-2-2 in the spaces
Problems:
33. A factory parking lot has 160 cars in it, of which 35 have faulty emission controls.
An air quality inspector does spot checks on 8 cars on the lot.
1. Give an expression for the probability that at least 3 of these 8 cars will
have faulty emission controls.
2. What assumption does your answer to (a) require? How likely is it that
this assumption holds if the inspector hopes to catch as many cars with
faulty controls as possible?
34. In a race, the 15 runners are randomly assigned the numbers
probability that
. Find the
Problems on Chapter 3
35. Six digits from 2, 3, 4, ..., 8 are chosen and arranged in a row without
replacement. Find the probability that
1. the number is divisible by 2
2. the digits 2 and 3 appear consecutively in the proper order (i.e. 23)
3. digits 2 and 3 appear in the proper order but not consecutively.
36. Suppose passengers get on an elevator at the basement floor. There are floors
above (numbered 1, 2, 3, ..., ) where passengers may get off.
1. Find the probability
1. no passenger gets off at floor 1
2. passengers all get off at different floors
39. The letters of the word STATISTICS are arranged in a random order. Find the
probability
1. they spell statistics
2. the same letter occurs at each end.
40. Three digits are chosen in order from 0, 1, 2, ..., 9. Find the probability the digits
are drawn in increasing order; (i.e., the first
the second
the third) if
4.
There are 25 deer in a certain forested area, and 6 have been caught
temporarily and tagged. Some time later, 5 deer are caught. Find the
probability that 2 of them are tagged. (What assumption did you make to
do this?)
Suppose that the total number of deer in the area was unknown to you.
Describe how you could estimate the number of deer based on the
information that 6 deer were tagged earlier, and later when 5 deer are
caught, 2 are found to be tagged. What estimate do you get?
5. Lotto 6/49. In Lotto 6/49 you purchase a lottery ticket with 6 different numbers,
selected from the set
. In the draw, six (different) numbers are
randomly selected. Find the probability that
Your ticket has the 6 numbers which are drawn. (This means you win the
main Jackpot.)
Your ticket matches exactly 5 of the 6 numbers drawn.
Your ticket matches exactly 4 of the 6 numbers drawn.
Your ticket matches exactly 3 of the 6 numbers drawn.
6. (Texas Hold-em) Texas Hold-em is a poker game in which players are each dealt
two cards face down (called your hole or pocket cards), from a standard deck of
52 cards, followed by a round of betting, and then five cards are dealt face up on
the table with various breaks to permit players to bet the farm. These are
communal cards that anyone can use in combination with their two pocket cards
to form a poker hand. Players can use any five of the face-up cards and their two
cards to form a five card poker hand. Probability calculations for this game are
not only required at the end, but also at intermediate steps and are quite
complicated so that usually simulation is used to determine the odds that you will
win given your current information, so consider a simple example. Suppose we
were dealt 2 Jacks in the first round.
1. What is the probability that the next three cards (face up) include at least
one Jack?
2. Given that there was no Jack among these next three cards, what is the
probability that there is at least one among the last two cards dealt faceup?
3. What is the probability that the 5 face-up cards show two Jacks, given that
I have two in my pocket cards?
MATHEMATICAL EXPECTATIONS
"Mathematical Expectation" is one of those few topics that is rarely discussed in details in any
curriculum, but is nevertheless very important. This tutorial attempts to throw some light on this
topic by discussing few related mathematical and programming problems.
Theory
Mathematical Expectation is an important concept in Probability Theory. Mathematically, for a
discrete variable X with probability function P(X), the expected value E[X] is given by xiP(xi)
the summation runs over all the distinct values xi that the variable can take. For example, for a
dice-throw experiment, the set of discrete outcomes is { 1,2,3,4,5,6} and each of this outcome
has the same probability 1/6. Hence, the expected value of this experiment will be
1/6*(1+2+3+4+5+6) = 21/6 = 3.5. For a continuous variable X with probability density function
P(x) , the expected value E[X] is given by xP(x)dx.
It is important to understand that "expected value" is not same as "most probable value" - rather,
it need not even be one of the probable values. For example, in a dice-throw experiment, the
expected value, viz 3.5 is not one of the possible outcomes at all.
The rule of "linearity of of the expectation" says that E[x1+x2] = E[x1] + E[x2].
Problems
1. What is the expected number of coin flips for getting a head?
Ans:
Let the expected number of coin flips be x. Then we can write an equation for it a. If the first flip is the head, then we are done. The probability of this event is 1/2 and the
number of coin flips for this event is 1.
b. If the first flip is the tails, then we have wasted one flip. Since consecutive flips are
independent events, the solution in this case can be recursively framed in terms of x - The
probability of this event is 1/2 and the expected number of coins flips now onwards is x. But we
have already wasted one flip, so the total number of flips is x+1.
The expected value x is the sum of the expected values of these two cases. Using the rule of
linerairty of the expectation and the definition of Expected value, we get
x = (1/2)(1) + (1/2) (1+x)
Solving, we get x = 2.
Thus the expected number of coin flips for getting a head is 2.
Q2. What is the expected number of coin flips for getting two consecutive heads?
Let the expected number of coin flips be x. The case analysis goes as follows:
a. If the first flip is a tails, then we have wasted one flip. The probability of this event is 1/2 and
the total number of flips required is x+1
b. If the first flip is a heads and second flip is a tails, then we have wasted two flips. The
probability of this event is 1/4 and the total number of flips required is x+2
c. If the first flip is a heads and second flip is also heads, then we are done. The probability of
this event is 1/4 and the total number of flips required is 2.
Adding, the equation that we get is x = (1/2)(x+1) + (1/4)(x+2) + (1/4)2
Solving, we get x = 6.
Thus, the expected number of coin flips for getting two consecutive heads is 6.
Q3. (Generalization) What is the expected number of coin flips for getting N consecutive
heads, given N?
Let the exepected number of coin flips be x. Based on previous exercises, we can wind up the
whole case analysis in two basic parts
a) If we get 1st, 2nd, 3rd,...,n'th tail as the first tail in the experiment, then we have to start all
over again.
b) Else we are done.
For the 1st flip as tail, the part of the equation is (1/2)(x+1)
For the 2nd flip as tail, the part of the equation is (1/4)(x+2)
...
For the k'th flip as tail, the part of the equation is (1/(2k))(x+k)
...
For the N'th flip as tail, the part of the equation is (1/(2N))(x+N)
The part of equation corrresponding to case (b) is (1/(2N))(N)
Adding,
x = (1/2)(x+1) + (1/4)(x+2) + ... + (1/(2^k))(x+k) + .. + (1/(2^N))(x+N) + (1/(2^N))(N)
Solving this equation is left as an exercise to the reader. The entire equation can be very easily
reduced to the following form:
x = 2N+1-2
Thus, the expected number of coin flips for getting N consecutive heads is (2N+1 - 2).
Q4. Candidates are appearing for interview one after other. Probability of each candidate
getting selected is 0.16. What is the expected number of candidates that you will need to
interview to make sure that you select somebody?
This is very similar to Q1, the only difference is that in this case the coin is biased. (The
probability of heads is 0.16 and we are asked to find number of coin flips for getting a heads).
Let x be the expected number of candidates to be interviewed for a selection. The probability of
first candidate getting selected is 0.16 and the total number of interviews done in this case is 1.
The other case is that the first candidate gets rejected and we start all over again. The probability
for that is (1-0.16)*(x+1). The equation thus becomes x = 0.16 + (1-0.16)*(x+1)
Solving, x = 1/0.16, i.e. x = 6.25
Q5. (Generalized version of Q4) - The queen of a honey bee nest produces offsprings oneafter-other till she produces a male offspring. The probability of produing a male offspring
is p. What is the expected number of offsprings required to be produced to produce a male
offspring?
This is same as the previous question, except that the number 0.16 has been replaced by p.
Observe that the equation now becomes x = p + (1-p)*(x+1)
Solving, x = 1/p
Thus, observe that in the problems where there are two events, where one event is desirable and
other is undesirable, and the probability of desirable event is p, then the expected number of
trials done to get the desirable event is 1/p
Generalizing on the number of events - If there are K events, where one event is desirable and all
others are undesirable, and the probability of desirable event is p, then also the expected number
of trials done to get the desirable event is 1/p
The next question uses this generalization Q6. what is the expected number of dice throws required to get a "four"?
Let the expected number of throws be x. The desirable event (getting 'four') has probability 1/6
(as each face is equiprobable). There are 5 other undesirable events (K=5). Note that the value of
the final answer doesnot depend on K. The answer is thus 1/(1/6) i.e. 6.
Q7.Candidates are appearing for interview one after other. Probability of k-th candidate
getting selected is 1/(k+1). What is the expected number of candidates that you will need to
interview to make sure that you select somebody?
The result will be the sum of infinite number of cases case-1: First candidate gets selected. The probability of this event is 1/2 and the number of
interviews is 1.
case-2. Second candidate gets selected. The probability of this event is 1/6 (= 1/2 of first
candidate not getting selected and 1/3 of second candidate getting selected, multiplied together
gives 1/6) and the number of interviews is 2
case-3. Third candidate gets selected. The probability of this event is 1/2 * 2/3 * 1/4 = 1/12 (=
first not getting selected and second not getting selected and third getting selected) and the
number of interviews is 3.
...
case-k. k'th candidate gets selected. The probability of this event is 1/2 * 2/3 * 3/4 * ... * (k-1)/k
* 1/(k+1). (The first k-1 candidates get rejected and the k'th candidate is selected). This evaluates
to 1/(k*(k+1)) and the number of interviews is k
...
[ Note that similar to problem 4, here we can't just say - if the first candidate is rejected, then we
will start the whole process again. This is not correct, because the probabilty of each candidate
depends on it's sequence number. Hence sub-experiments are not same as the parent experiment.
This means that all the cases must be explicitly considered.]
The resultant expression will be
x = 1/(1*2) + 2/(2*3) + 3/(3*4) + 4/(4*5) + ... + k/(k*(k+1)) + ...
= 1/2 + 1/3 + 1/4 + ...
This is a well-known divergent series, which means that sum doesnot converge, and hence the
expectation doesnot exist.
Q8: A random permutation P of [1...n] needs to be sorted in ascending order. To do this, at
every step you will randomly choose a pair (i,j) where i < j but P[i] > P[j], and swap P[i]
with P[j]. What is the expected number of swaps needed to sort permutation in ascending
order. (Idea: Topcoder)
This is a programming question, and the idea is simple - since each swap has same probability of
getting selected, the total number of expected swaps for a permutation P is
In the current case, "success" is defined as the experiment that chooses a single digit number.
Since all choices are equiprobable, the probability of success is 9/100. (There are 9 single digit
numbers in 1 to 100). Since there are n students, the expected number of students that would
contribute to success (i.e the expected number of successes) is n*9/100
Q11. What is the expected number of coin flips to ensure that there are atleast N heads?
The solution can easily be framed in a recursive manner N heads = if 1st flip is a head then N-1 more heads, else N more heads.
The probability of 1st head is 1/2. Thus
E[N] = (1/2)(E[N-1]+1)+ (1/2)(E[N] + 1)
Note that each term has 1 added to it to account for the first flip.
The base case is when N = 1 :
E[1] = 2 (As discussed in Q2)
Simplifying the recursive case,
E[N] = (1/2)( E[N-1] +1 + E[N] + 1)
= (1/2)( E[N-1] + E[N] + 2)
=> 2 * E[N] = ( E[N-1] + E[N] + 2)
=> E[N] = E[N-1] + 2
Since E[1] = 2, E[2] = 4, E[3] = 6,..., in general E[N] = 2N. Thus, the expected number of coin
flips to ensure that there are atleast N heads in 2N.
The next problem discusses a generalization :
Q12. What is the expected number of bernaulli trials to ensure that there are atleast N
successes, if the probability of each success is p?
The recursive equation in this case is E[N] = p(E[N-1]+1)+ (1- p)(E[N] + 1)
Solving, E[N]-E[N-1] = p. Writing a total of N-1 equations:
E[N]-E[N-1] = 1/p
E[N-1]-E[N-2] = 1/p
E[N-2]-E[N-3] = 1/p
...
E[2]-E[1] = 1/p
Adding them all, E[N] - E[1] = (n-1)/p. But E[1] is 1/p (lemma -1). Hence E[N] = n/p.
Moral: If probability of success in a bernaulli trial is p, then the expected number of trials to
guaranttee N successes is N/p.
This completes the discussion on problems on Mathematical Expectation.
Exercises:
Note: Some of these are non-trivial and require concepts not discussed in this tutorial. If you are
interested you could read about probability distribution basics, more advanced topics such as
joint and bivariate distributions and transformations and a tutorial on permutations and
combinations
1. A game involves you choosing one number (between 1 to 6 inclusive) and then throwing three
fair dice simultaneously. If none of the dice shows up the number that you have chosen, you lose
$1. If exactly one, two or three dice show up the number that you have chosen, you win $1, $3 or
$5 respectively. What is your expected gain?
2. There are 10 flowers in a garden, exactly one of which is poisonous. A dog starts eating all
these flowers one by one at random. whenever he eats the posionous flower he will die. What is
the expected number of flowers he will eat before he will die?
3. A bag contains 64 balls of eight different colours, with eight of each colour. What is the
expected number of balls you would have to pick (without looking) to select three balls of the
same color?
4. In a game of fair dice throw, what is the expected number of throws to make sure that all 6
outcomes appear atleast once?
5. What is the expected number of bernaulli trials for getting N consecutive successes, given N,
if the probability of each success is p?
http://www.biostat.umn.edu/~sudiptob/pubh8429/ExpectationsBasic04.pdf
http://www.math.wsu.edu/students/aredford/documents/Notes_Expectation.pdf
http://www.mathsisfun.com/data/standard-normal-distribution.html