Algorithms and Complexity

Bioinformatics
Spring 2008
Hiram College
Algorithm definition slides taken from CPSC 171, courtesy of Obertia Slotterbeck

What is an algorithm?

An algorithm is a
well-ordered collection of
unambiguous and
effectively computable operations that,
when executed,
produces a result and
halts in a finite amount of time.

AN EXAMPLE OF A VERY SIMPLE

ALGORITHM
1. Wet your hair.
2. Lather your
hair.
3. Rinse your
hair.
4. Stop.

Observe:
Operations need not be
executed by a computer only
by an entity capable of
carrying out the operations
listed.

We assume that
The algorithm begins executing at the top of the list
of operations.
The "Stop" can be omitted if we assume the last
line is an implied "Stop" operation.

A well-ordered collection of
operations
The question that must be answered is:
At any point in the execution of the algorithm, do you
know what operation is to be performed next?
Well-ordered operations:
1. Wet your hair.

Not well-ordered
operations:

2. Lather your hair.

1. Either wet your hair or

lather your hair.

3. Rinse your hair.

2. Rinse your hair.

Choices are allowed:

Well-ordered operations:

Well-ordered operations:

1. If your hair is dirty, then If your hair is dirty, then

a. Wet your hair.
Wet your hair.
b. Lather your hair.

Lather your hair.

c. Rinse your hair.

Rinse your hair.

2. Else
a. Go to bed.

Else
Go to bed.

Note: We will often omit the numbers and the letters

and assume a "top-down" reading of the operations.

Unambiguous operations
The question that must be answered is:
Does the computing entity understand what the
operation is to do?
This implies that the knowledge of the computing
entity must be considered.
For example, is the following ambiguous?
Make the pie crusts.

To an experienced cook,
Make the pie crusts.
is not ambiguous.
But, an less experienced cook may need:
Take 1 1/3 cups of flour.
Sift the flour.
Mix the sifted flour with 1/2 cup of butter
and 1/4 cup of water to make dough.
Roll the dough into two 9-inch pie crusts.
or even more detail!

Definition: An operation that is

unambiguous is called a primitive
operation (or just a primitive)
One question we will be exploring in the course is
what are the primitives of a computer.
Note that a given collection of operations may be an
algorithm with respect to one computing agent, but
not with respect to another computing agent!!

Primitives for Computer

Algorithms (e.g. PERL)
Mathematical operations: add,
subtract, multiply, divide, log, sqrt,
String operations: append, substring,
reverse,
File operations: read, write, append
Other I/O: print, scan

Primitives for Biological

Algorithms

Bind (a molecule binds to a site)

Separate (strands of DNA)
Polymerize (add base to strand)
Repair gaps

These primitives make up an algorithm

for DNA replication (A, pp. 14-16)

Effectively computable
operations
The question that must be answered is:
Is the computing entity capable of doing the operation?
This assumes that the operation must first be unambiguousi.e. the computing agent understands what is to be done.
Not effectively computable operations:
Write all the fractions between 0 and 1.
Create matter from nothing

that, when executed, produces a

result
The question that must be answered is:
Can the user of the algorithm observe a result produced
by the algorithm?
The result need not be a number or piece of text
viewed as "an answer".
It could be an alarm, signaling something is wrong.
It could be an approximation to an answer.
It could be an error message.

halts in a finite amount of time

The question that must be answered is:

Will the computing entity complete the operations in

a finite number of steps and stop?
Do not confuse "not finite" with "very, very large". A
failure to halt usually implies there is an infinite loop in
the collection of operations:
1. Write the number 1 on a piece of paper.
2. Add 1 to the number you just wrote and write it
on a piece of paper.
3. Repeat 2.
4. Stop.

Definition of an algorithm:
An algorithm is a well-ordered collection
of unambiguous and effectively
computable operations that, when
executed, produces a result and halts in
a finite amount of time.
Note: Although I have tried to give clean cut examples to
illustrate what these new words mean, in some cases, a
collection of operations can fail for more than one reason.

A Language for Algorithms

Natural Language (English)?
Too ambiguous

Programming Language (Perl)?

Too much new syntax to learn

Pseudocode
A compromise. ( just right)

What is Pseudocode?
Structured like a programming
language, but ignores many syntactical
details (like $ and ;)
Complex operations can be written in
natural language
Still, we need to agree on some
standard operations

Our Pseudocode (pp. 8-11)

Assignment
Arithmetic
Conditional execution
Repeated execution
Array access
Functions

Assignment and Variables

A variable has a name (which can be
anything in pseudocode) and a value.
Assignment changes the value
Examples:
myName <- Ellen Walker
number <- 17
copy <- number
number <- 98765

Arithmetic
In pseudocode, mathematical symbols
are allowed
2

dist ( x 2 x1 ) + ( y 2 y1 )

But, programming language style math

is easier to type
dist <- sqrt((x[2]-x[1])^2 + (y[2]-y[1])^2)

Conditional (If Statement)

Allows a choice to be made, given:
a condition,
something to do if the condition is true
something to do if the condition is false.

Example:
if today is a weekday
go to work
else
stay home

Repeated Execution (Loops)

Need to know:
Which instructions to repeat
When to stop repeating

Two kinds of loops

While loop: stop when a condition is true
For loop: repeat a specific number of times

For Loop
Loop controlled by a variable
Executes once for each value of the
variable, from a given starting value to a
given ending value
Example:
sum <- 0
for num <- 1 to 10
sum <- sum + num

While Loop
Loop is controlled by a condition.
When the condition is false, the loop no
longer executes.
Example
while (you are cold)
raise the thermostat temperature 1 degree

Array Access
An array is a sequence of values of the
same type.
We access each item, or element by its
numeric index.
Computers start counting at 0!

Array Example
Jan Feb

Mar

Apr

May

Jun Jul

Aug Sep Oct

Nov Dec

10 11

Months <- {Jan, Feb, Mar, Apr, May, Jun,

Jul, Aug, Sep, Oct, Nov, Dec}
print(months[5])
for m<- 0 to 11
print(months[m])
go to the next line

prints Jun

Assignment to an Array
An array element is really a variable, so
you can assign to it.
Example:
for n <- 0 to 99
squares[n] = n*n

Function
A function has a name, parameters (inputs),
code to execute, and a return value.
Example:
fibonacci (n)
f[0] <- 1
f[1] <- 1
for i <- 2 to n-1
f[i] <- f[i-1]+f[i-2]
return f[n-1]

Calling a Function
Write the function name, and the actual
values for the parameters
When the function is complete, the
return value replaces the name of the
function in any expression.
Example:
print(fibonacci(8))

prints 21

Exercise
Find a set of instructions, written in English, on
the Internet, preferably for a non-mathematical
task.
There should be at least 5 steps, and a
condition or a loop, preferably both.
Rewrite the instructions in pseudocode.
Possibilities:
Instruction manuals
Government sites
Game descriptions

Comparing Algorithms
There can be many different algorithms
to solve the same problem
Better algorithms
Get the correct answer (if possible)
Get better answers (otherwise)
Use fewer resources (time and space)

Algorithm Complexity
Time
How long does the algorithm take?
Abstract, dont want answer to depend on
which machine!

Space
How much space (arrays, variables) does
the algorithm need?

Time Complexity of an Algorithm

What we want to do is relate
1. the amount of work performed by an algorithm
2. and the algorithm's input size
by a fairly simple formula.

STEPS FOR DETERMING THE TIME

COMPLEXITY OF AN ALGORITHM

1. Determine how you will measure input size. Ex:

N items in a list
N x M table (with N rows and M columns)
Two numbers of length N

2. Choose an operation (or perhaps two operations) to count as

a gauge of the amount of work performed. Ex:
Comparisons
Swaps
Copies
Additions

Normally we don't count operations in input/output.

STEPS FOR DETERMING THE TIME

COMPLEXITY OF AN ALGORITHM
3. Decide whether you wish to count operations in the

Best case? - the fewest possible operations

Worst case? - the most possible operations
Average case?
This is harder as it is not always clear what is meant
by an "average case". Normally calculating this case
requires some higher mathematics such as probability
theory.
4. For the algorithm and the chosen case (best, worst,
average), express the count as a function of the input size of
the problem.
For example, we determine by counting, statements such as ...

EXAMPLES:
For n items in a list, counting the operation
swap, we find the algorithm performs 10n +
5 swaps in the worst case.
For an n X m table, counting additions, we
find the algorithm perform nm additions in
the best case.
For two numbers of length n, there are 3n +
20 multiplications in the best case.

STEPS FOR DETERMING THE TIME

COMPLEXITY OF AN ALGORITHM
5. Given the formula that you have determined, decide the
complexity class of the algorithm.
What is the complexity class of an algorithm?
Question: Is there really much difference between
3n
5n + 20
and

6n -3

especially when n is large?

But, there is a huge difference, for n large, between

n
n2
and

n3

So we try to classify algorithm into classes, based on their

counts and simple formulas such as n, n2, n3, and others.

Why does this matter?

It is the complexity of an algorithm that most
affects its running time--not the machine or its speed

The TRS-80

ORDER WINS OUT

Main language support: BASIC - typically a slow running

language
For more details on TRS-80 see:
http://en.wikipedia.org/wiki/TRS-80
The CRAY-YMP
Language used in example: FORTRAN- a fast running language
For more details on CRAY-YMP see:
http://en.wikipedia.org/wiki/Cray_Y-MP

CRAY YMP
with FORTRAN
complexity is 3n3

TRS-80
with BASIC
complexity is 19,500,000n

n is:
10

3 microsec

100

3 millisec

200 millisec
2 sec

1000

3 sec

20 sec

2500

50 sec

50 sec

10000

49 min

3.2 min

1000000

95 years

5.4 hours

Trying to maintain an exact count for an operation isn't too useful.

Thus, we group algorithms that have counts such as
n
3n + 20
1000n - 12
0.00001n +2
together. We say algorithms with these type of counts are in the
class (n) read as the class of theta-of-n or
all algorithms of magnitude n or
all order-n algorithms

Similarly, algorithms with counts such as

n2 + 3n
1/2n2 + 4n - 5
1000n2 + 2.54n +11
are in the class (n2).
Other typical classes are those with easy formulas in n such
as
1
n3
2n
lg n

k = lg n if and only if 2k = n

lg n

k = lg n if and only if 2k = n

lg 4 = ?
lg 8 = ?
lg 16 = ?
lg 10 = ?
Note that all of these are base 2 logarithms. You don't
use any logarithm table as we don't need exact values
(except on integer powers of 2).
Look at the curves showing the growth for
algorithms in
(1), (n), (n2), (n3), (lg n), (n lg n), (2n)
These are the major ones we'll use.

Figure 3.4
Work = cn for Various Values of c

Figure 3.10
Work = cn2 for Various Values of c

Figure 3.11
A Comparison of n and n2

Figure 3.21
A Comparison of n and lg n

Figure 3.21
A Comparison of n and lg n

Figure 3.25
Comparisons of lg n, n, n2 , and 2n

Making Change
US Change Problem
Input: amount of money, M, in cents
Output: smallest number of coins (quarters,
dimes, nickels, and pennies) that add up to
M

US Change Algorithm
While M>0
c <- value of largest coin with value <=
M
give c coin to customer
M <- M-c
(See mathematical version, p. 19)

US Change Examples:
77c
Quarter (77-25 = 52)
Quarter (52-25 = 27)
Quarter (27-25 = 2)
Penny (2-1 = 1)
Penny (2-1 = 1)

Same Algorithm using

Mathematics
Calculate and give the max # of quarters
(25c):
Q = floor (M / 25)
Give customer Q quarters
M = M - 25*Q

Calculate and give the max # of dimes (10c).

Calculate and give the max # of nickels (5c).
Give the rest in pennies.

Generalizing the Algorithm

Suppose a non-US money system has
d (different) coins of denominations:
c[0], c[1], c[d-1]
And, c[0] > c[1] > c[d-1]
Then, we can generalize the algorithm:

Generalizing, continued
for i <- 0 to d-1
Calculate and give the max number of
this coin (c[i])

Generalized Algorithm is Not

Correct!
c = {25, 20, 10, 5, 1}
M = 40
Result is 3 coins, not 2

Brute Force: Correct but Slow

N <- 1
While (not done)
construct a combination of N coins
if it adds up to M
return it (done=true)
else if no more combinations of N coins
N <- N+1

Improving the Change

Algorithm
Try combinations in a useful order
Weve already done this, looking at fewer
coins first.

Generate as few combinations as

possible
Use knowledge to bound the combinations
tried.