Minggu 1 Algorithm Analysis

Design and Analysis of Algorithms
Chapter 1
RAIK 283: Data Structures & Algorithms
RAIK 283: Data Structures & Algorithms
Analysis of Algorithms
b Giving credit
Dr. Ying Lu
ylu@cse.unl.edu
August 28, 2012
http://www.cse.unl.edu/~ylu/raik283/
Design and Analysis of Algorithms Chapter 2.1
where credit is due:
Most of the lecture notes are based on the slides

from the Textbooks companion website
http://www.aw--bc.com/info/levitin
http://www.aw
Several slides are from Jeff Edmonds of the
York University
I have modified them and added new slides
The Time Complexity of an Algorithm
The Time Complexity of an Algorithm
Specifies how the running time depends

on the size of the input
Purpose
Purpose
To estimate how long a program will run.
To estimate the largest input that can reasonably be
given to the program.
b To compare the efficiency of different algorithms.
algorithms.
b To help focus on the parts of code that are executed
the largest number of times.
b To choose an algorithm for an application.
b
b

Purpose (Example)
b
Chapter 1
Purpose (Example)
Suppose a machine that performs a million floatingfloatingpoint operations per second (106 FLOPS), then how
long an algorithm will run for an input of size n=50?
1) If the algorithm requires n2 such operations:
0.0025 second
Purpose (Example)
b
Purpose (Example)
0.0025 second
2) If the algorithm requires 2n such operations:
2) If the algorithm requires 2n such operations:
0.0025 second
A) Takes a similar amount of time (t < 1 sec)

B) Takes a little bit longer time (1 sec < t < 1 year)
C) Takes a much longer time (1 year < t)
Time Complexity Is a Function
over 35 years!
10
Definition of Time?
Specifies how the running time depends on the size of the

input.
A function mapping
size of input
time T(n) executed .
11
12

Definition of Time
Chapter 1
Theoretical analysis of time efficiency
# of seconds (machine, implementation dependent).
Time efficiency is analyzed by determining the number of

repetitions of the basic operation as a function of input size
# lines of code executed.
# of times a specific operation is performed

addition).
Basic operation: the operation that contributes most

towards the running time of the algorithm.
(e.g.,
input size
T(n) copC(n)
running time
13
execution time
for basic operation
Number of times
basic operation is
executed
14
Input size and basic operation examples
Problem
Problem
Input size measure
Basic operation
Input size measure
Search for key in a list of

n items
Search for key in a list of Number of items in the

n items
list: n
Multiply two matrices of

floating point numbers

Compute an
Compute an
Graph problem
Graph problem
15
Basic operation
Key comparison
16
Problem
Basic operation
Problem
Search for key in a list of Number of items in the

n items
list: n
Key comparison
Search for key in list of n

Number of items in list n Key comparison
items

Floating point
multiplication

Dimensions of matrices
Floating point
multiplication
Compute an
Compute an
Floating point
multiplication
Graph problem
Graph problem
Input size measure
17
Input size measure
Basic operation
18

Problem
Input size measure
Chapter 1
Theoretical analysis of time efficiency
Basic operation
Time efficiency is analyzed by determining the

number of repetitions of the basic operation as a
function of input size
Search for key in list of n

Number of items in list n Key comparison
items
Floating point
multiplication
Compute an
Floating point
multiplication
Graph problem
#vertices and/or edges
Visiting a vertex or
traversing an edge
19
Which Input of size n?
Which Input of size n?
Efficiency also depends on the particular input

b For
Efficiency also depends on the particular input
instance: search a key in a list of n letters
Problem input: a list of n letters

How many different inputs?
For instance: search a key in a list of n letters

There are 26n inputs of size n.
Which do we consider
for the time efficiency C(n)?
21
Best--case, averageBest
average-case, worstworst-case
Worst case: W(n

W(n) maximum over inputs of size n
Best case:
Average case: A(n

A(n) average over inputs of size n
Problem: Given a list of n elements and a search key K, find

an element equal to K, if any.
Algorithm: Scan the list and compare its successive
elements with K until either a matching element is found
(successful search)
search) or the list is exhausted (unsuccessful
(unsuccessful
search))
search
Worst case
Best case
Average case
B(n) minimum over inputs of size n

B(n
NOT the average of worst and best case

Under some assumption about the probability distribution of all
possible inputs of size n, calculate the weighted sum of expected C(n)
(numbers of basic operation repetitions) over all possible inputs of size
n.
22
Example: Sequential search

b
20
23
24
Chapter 1
An example
b
b
b
b
b
Types of formulas for basic operation count
Compute gcd(m, n) by applying the algorithm based on

checking consecutive integers from min(m, n) down to
gcd(m, n)
Input size?
Best case?
Worst case?
Average case?
Exact formula
e.g., C(n
C(n) = n(n-1)/2
Formula indicating order of growth with specific

multiplicative constant
e.g., C(n
C(n) 0.5 n2
Formula indicating order of growth with unknown

e.g., C(n
C(n) cn2
25
26
Asymptotic growth rate
Types of formulas for basic operation count

b
Exact formula
e.g., C(n
C(n) = n(n-1)/2
Formula indicating order of growth with specific
e.g., C(n
C(n) 0.5 n2
Formula indicating order of growth with unknown
e.g., C(n
C(n) cn2
A way of comparing functions that ignores constant factors

and small input sizes
O(g
O(
g(n)):
(g(n)):
(g(n)):
Most important: Order of growth within a constant

multiple as n
27
Asymptotic growth rate
28
Table 2.1
A way of comparing functions that ignores constant factors

and small input sizes
O(gg(n)): class of functions f(n) that grow no faster than g(n)

O(
(g(n)): class of functions f(n) that grow at same rate as g(n)
(g(n)): class of functions f(n) that grow at least as fast as g(n)
29
30
Chapter 1
Which are more alike?
Classifying Functions
Giving an idea of how fast a function

grows without going into too much detail.
32
Mammals
33
34
Classifying Animals
Vertebrates
Fish
Reptiles
35
Giraffe
Dogs
Mammals
Birds
Dogs
36

n1000
n2
Chapter 1
2n
n1000
n2
2n
Polynomials
37
1000n2
3n2
38
2n3
1000n2
3n2
2n3
Quadratic
39
Classifying Functions?
Functions
Functions
Polynomial
Exponential
Factorial
5 log n
Poly Logarithmic
41
Logarithmic
Constant
5
40
(log n)5
n5
25n
n!
42
Chapter 1
Classifying Functions?
Polynomial
Cubic
Logarithmic
Quadratic
43
Linear
Polynomial
5n
5n2
5n3
5n4
44
Poly Logarithmic
b log10n =
# digits to write n
= # bits to write n
= 3.32 log10n
1000
) = 1000 log(n)
b log(n
b log2n
Differ only by a
multiplicative
constant
45
Which grows faster?
log1000 n
(log n)5 = log5 n
46
Poly Logarithmic << Polynomial
n0.001
log1000 n << n0.001

For sufficiently large n
47
48

Which grows faster?
Linear << Quadratic
0.0001 n2
10000 n
Chapter 1
10000 n << 0.0001 n2

49
Which grows faster?
n1000
50
Polynomial << Exponential
20.001 n
n1000 << 20.001 n

51
Ordering Functions
52
Which Functions are Constant?
Functions
<<
<<
25n <<
Factorial
5 << 5 log n << (log n)5 << n5
<<
Exponential
<<
Polynomial
<<
Poly Logarithmic
Logarithmic
Constant
<<
5
1,000,000,000,000
0.0000000000001
-5
0
8 + sin(n)
n!
53
54


The running time of the algorithm is a Constant
if it does not depend significantly
on the size of the input.
Yes 5
Yes 1,000,000,000,000
Yes 0.0000000000001
Yes -5
Yes 0
No 8 + sin(n)
5
1,000,000,000,000
0.0000000000001
-5
0
8 + sin(n)
55
The running time of the algorithm is a Constant

It does not depend significantly
on the size of the input.
57
Which Functions are Quadratic?
58

n2
0.001 n2
1000 n2
5n2 + 3n + 2log n
n2
0.001 n2
Lie in between
1000 n2
5n2 + 3n + 2log n
56
n2
0.001 n2
1000 n2
5n2 + 3n + 2log n
5
1,000,000,000,000
9
0.0000000000001
7
-5
0
Lie in between
8 + sin(n)
Yes
Yes
Yes
No
No
Yes
Chapter 1
Ignore low-order terms

Ignore multiplicative constants.
Ignore "small" values of n.
Write (n2).
59
60
10
Chapter 1
Examples
f(n)
g(n)
1)
ln2n
2)
nk
cn
3)
nsinn
4)
2n
2n/2
5)
nlgc
clgn
6)
O(g(n))?
(g(n))? (g(n))?
lg(n!) lg(nn)
61
11

Minggu 1 Algorithm Analysis

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Minggu 1 Algorithm Analysis

Enviado por

Direitos autorais:

Formatos disponíveis

Design and Analysis of Algorithms

RAIK 283: Data Structures & Algorithms

RAIK 283: Data Structures & Algorithms

Design and Analysis of Algorithms Chapter 2.1

where credit is due:

Most of the lecture notes are based on the slides

Design and Analysis of Algorithms Chapter 2.1

The Time Complexity of an Algorithm

The Time Complexity of an Algorithm

Specifies how the running time depends

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms

1) If the algorithm requires n2 such operations:

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms Chapter 2.1

1) If the algorithm requires n2 such operations:

1) If the algorithm requires n2 such operations:

2) If the algorithm requires 2n such operations:

A) Takes a similar amount of time (t < 1 sec)

Design and Analysis of Algorithms Chapter 2.1

Time Complexity Is a Function

Design and Analysis of Algorithms Chapter 2.1

Specifies how the running time depends on the size of the

time T(n) executed .

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms

# of seconds (machine, implementation dependent).

Time efficiency is analyzed by determining the number of

# lines of code executed.

# of times a specific operation is performed

Basic operation: the operation that contributes most

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms Chapter 2.1

Input size and basic operation examples

Input size and basic operation examples

Input size measure

Input size measure

Search for key in a list of

Search for key in a list of Number of items in the

Multiply two matrices of

Multiply two matrices of

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms Chapter 2.1

Input size and basic operation examples

Input size and basic operation examples

Search for key in a list of Number of items in the

Search for key in list of n

Multiply two matrices of

Multiply two matrices of

Input size measure

Design and Analysis of Algorithms Chapter 2.1

Input size measure

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms

Input size measure

Time efficiency is analyzed by determining the

Search for key in list of n

#vertices and/or edges

Design and Analysis of Algorithms Chapter 2.1

Which Input of size n?

Design and Analysis of Algorithms Chapter 2.1

Which Input of size n?