Escolar Documentos
Profissional Documentos
Cultura Documentos
CS 202
Epp, section ???
Aaron Bloomfield
1
How does one measure algorithms
• We can time how long it takes a computer
– What if the computer is doing other things?
– And what happens if you get a faster computer?
• A 3 Ghz Windows machine chip will run an algorithm at a
different speed than a 3 Ghz Macintosh
2
How does one measure algorithms
• We can measure how many machine
instructions an algorithm takes
– Different CPUs will require different amount of
machine instructions for the same algorithm
3
How does one measure algorithms
• We can loosely define a “step” as a single
computer operation
– A comparison, an assignment, etc.
– Regardless of how many machine instructions it
translates into
5
Bubble sort running time
• This leaves us with:
– Intel Pentium IV CPU: 29n2
– Motorola CPU: 42.2n2
– Intel Pentium V CPU: 22n2
6
An aside: inequalities
• If you have a inequality you need to show:
x<y
• You can replace the lesser side with something greater:
x+1 < y
• If you can still show this to be true, then the original
inequality is true
7
An aside: inequalities
• If you have a inequality you need to show:
x<y
• You can replace the greater side with something lesser:
x < y-1
• If you can still show this to be true, then the original
inequality is true
8
An aside: inequalities
• What if you do such a replacement and can’t
show anything?
– Then you can’t say anything about the original
inequality
9
Review of last time
• Searches
– Linear: n steps
– Binary: log2 n steps
– Binary search is about as fast as you can get
• Sorts
– Bubble: n2 steps
– Insertion: n2 steps
– There are other, more efficient, sorting techniques
• In principle, the fastest are heap sort, quick sort, and merge sort
• These each take take n * log2 n steps
• In practice, quick sort is the fastest (although this is not
guaranteed!), followed by merge sort
10
Big-Oh notation
• Let b(x) be the bubble sort algorithm
• We say b(x) is O(n2)
– This is read as “b(x) is big-oh n2”
– This means that the input size increases, the running time of the
bubble sort will increase proportional to the square of the input
size
• In other words, by some constant times n2
11
Big-Oh notation
• Consider: b(x) is O(n2)
– That means that b(x)’s running time is less than (or
equal to) some constant times n2
• Consider: l(x) is O(n)
– That means that l(x)’s running time is less than (or
equal to) some constant times n
12
Big-Oh proofs
• Show that f(x) = x2 + 2x + 1 is O(x2)
– In other words, show that x2 + 2x + 1 ≤ c*x2
• Where c is some constant
• For input size greater than some x
13
Big-Oh proofs
14
Sample Big-Oh problems
• Show that f(x) = x2 + 1000 is O(x2)
– In other words, show that x2 + 1000 ≤ c*x2
15
Sample Big-Oh problems
• Show that f(x) = 3x+7 is O(x)
– In other words, show that 3x+7 ≤ c*x
16
A variant of the last question
• Show that f(x) = 3x+7 is O(x2)
– In other words, show that 3x+7 ≤ c*x2
17
What that means
• If a function is O(x)
– Then it is also O(x2)
– And it is also O(x3)
18
Function growth rates
• For input size n = 1000
• O(1) 1
• O(log n) ≈10
• O(n) 103
• O(n log n) ≈104
• O(n2) 106
• O(n3) 109
• O(n4) 1012
• O(nc) 103*c c is a consant
• 2n ≈10301
• n! ≈102568 Many interesting problems
• nn 103000 fall into these categories
19
Function growth rates
Logarithmic
scale!
20
Integer factorization
• Factoring a composite number into it’s
component primes is O(2n)
– Where n is the number of bits in the number
21
Formal Big-Oh definition
• Let f and g be functions. We say that f(x) is
O(g(x)) if there are constants c and k such that
|f(x)| ≤ C |g(x)|
whenever x > k
22
Formal Big-Oh definition
23
A note on Big-Oh notation
• Assume that a function f(x) is O(g(x))
24
NP Completeness
• A full discussion of NP completeness takes 3 hours for
somebody who already has a CS degree
– We are going to do the 15 minute version of it today
– More will (probably) follow on Friday
25
Satisfiability
• Consider a Boolean expression of the form:
• (x1 x2 x3) (x2 x3 x4) (x1 x4 x5)
– This is a conjunction of disjunctions
• Is such an equation satisfiable?
– In other words, can you assign truth values to all the
xi’s such that the equation is true?
– The above problem is easy (only 3 clauses of 3
variables) – set x1, x2, and x4 to true
• There are other possibilities: set x1, x2, and x5 to true, etc.
– But consider an expression with 1000 variables and
thousands of clauses
26
Satisfiability
• If given a solution, it is easy to check if such a solution
works
– Plug in the values – this can be done quickly, even by hand
• However, there is no known efficient way to find such a
solution
– The only definitive way to do so is to try all possible values for the
n Boolean variables
– That means this is O(2n)!
– Thus it is not a polynomial time function
• NP stands for “Not Polynomial”
• Cook’s theorem (1971) states that SAT is NP-complete
– There still may be an efficient way to solve it, though!
27
NP Completeness
• There are hundreds of NP complete problems
– It has been shown that if you can solve one of them
efficiently, then you can solve them all
– Example: the traveling salesman problem
• Given a number of cities and the costs of traveling from any
city to any other city, what is the cheapest round-trip route
that visits each city once and then returns to the starting city?
• Not all algorithms that are O(2n) are NP
complete
– In particular, integer factorization (also O(2n)) is not
thought to be NP complete
28
NP Completeness
• It is “widely believed” that there is no efficient solution to
NP complete problems
– In other words, everybody has that belief
• If you could solve an NP complete problem in polynomial
time, you would be showing that P = NP
– And you’d get a million dollar prize (and lots of fame!)
• If this were possible, it would be like proving that
Newton’s or Einstein’s laws of physics were wrong
• In summary:
– NP complete problems are very difficult to solve, but easy to
check the solutions of
– It is believed that there is no efficient way to solve them
29