Você está na página 1de 50

Chapter 13- Algorithm Design Techniques

Algorithm Design and


Analysis
13

13.1 Introduction to Algorithm Design and Analysis


13.2 Greedy Algorithms
13.3 Divide and Conquer
13.4 Dynamic Programming
13.5 Backtracking
13.6 Branch and bound
13.7 Comparison of various Algorithm Design Techniques
13.8 Randomized Algorithms
13.9 Introduction to Algorithm Analysis
13.10 Introduction to NP-Complete Problems
13.11 Summary
13.12 Key Terms
13.13 Review Questions

4
Chapter 13- Algorithm Design Techniques

13.1. Introduction to Algorithm Design and Analysis

An algorithm design technique is a genera approach to solve problems algorithmically. Such a


strategy can be applied to a variety of problems from different area of computing. Learning
design techniques is useful because it provide us guidance in designing algorithms for new
problems. Some of the most useful algorithm design techniques are:
o Greedy Algorithms
o Divide and Conquer
o Dynamic Programming
o Backtracking
o Branch and bound

13.2. Greedy Algorithms

13.2.1. Definition
Greedy method is an approach to find the optimized solutions for the given problem. The
solutions are constructed through a sequence of steps, each step expanding a partially constructed
solution so far, until a complete solution to the problem is reached. Examples are:
 A Simple Scheduling Problem
 Huffman Codes
 Approximate Bin Packing

13.2.2. A Simple Scheduling Problem


Scheduling refers to the way processes are assigned to run on the available CPUs, since there are
typically many more processes running than there are available CPUs. This assignment is carried
out by software known as a scheduler and dispatcher. As an example, suppose we have the four
jobs and associated running times shown below.
Job Time
-----------------
j1 15

5
Chapter 13- Algorithm Design Techniques

j2 8
j3 3
j4 10
One possible schedule is shown in Figure 13.1(a). Because j1 finishes in 15 (time units), j2 in 23,
j3 in 26, and j4 in 36. The average completion time = (15+23+26+36)/4= 25.

Figure 13.1 –Job Scheduling

A better schedule, which yields a mean completion time of 17.75, is shown in Figure 13.1(b).
The schedule given in Figure is arranged by shortest job first. We can show that this will always
yield an optimal schedule.

The Multiprocessor Case


We can extend this problem to the case of several processors. Again we have jobs j1, j2, . . . , jn,
with associated running times t1, t2, . . . , t n, and a number P of processors. We will assume
without loss of generality that the jobs are ordered, shortest running time first. As an example,
suppose P = 3, and the jobs are as shown below.
Job Time
----------------
j1 3
j2 5
j3 6
j4 10
j5 11
j6 14
j7 15
j8 18
j9 20
Figure 13.2 shows an optimal arrangement to minimize mean completion time. Jobs j1, j4, and j7
are run on Processor 1. Processor 2 handles j2, j5, and j8, and Processor 3 runs the remaining
jobs. The total time to completion is 165, for an average of 165/9=18.33

6
Chapter 13- Algorithm Design Techniques

Figure 13.2 – Multiprocessor case

The algorithm to solve the multiprocessor case is to start jobs in order, cycling through
processors. It is not hard to show that no other ordering can do better, although if the number of
processors P evenly divides the number of jobs n, there are many optimal orderings. The
following figure 13.3 shows the best optimal solution for the multiprocessor job scheduling.

Figure 13.3 – Multiprocessor case (optimal solution)

13.2.3. Huffman Codes


This problem is that of finding the minimum length bit string which can be used to encode a
string of symbols. One application is text compression.

Huffman's scheme uses a table of frequency of occurrence for each symbol (or character) in the
input. This table may be derived from the input itself or from data which is representative of the
input. For instance, the frequency of occurrence of letters in normal English might be derived
from processing a large number of text documents and then used for encoding all text
documents. We then need to assign a variable-length bit string to each character that
unambiguously represents that character.

The technique works by creating a binary tree of nodes. These can be stored in a regular array
and the size of which depends on the number of symbols. A node can be either a leaf node or an
internal node. Initially, all nodes are leaf nodes, which contain the symbol itself, the weight
(frequency of appearance) of the symbol and optionally, a link to a parent node which makes it
easy to read the code (in reverse) starting from a leaf node. Internal nodes contain symbol
7
Chapter 13- Algorithm Design Techniques

weight, links to two child nodes and the optional link to a parent node. As a common
convention, bit '0' represents following the left child and bit '1' represents following the right
child.

The process essentially begins with the leaf nodes containing the probabilities of the symbol they
represent, and then a new node whose children are the 2 nodes with smallest probability is
created, such that the new node's probability is equal to the sum of the children's probability.
With the previous 2 nodes merged into one node (thus not considering them anymore), and with
the new node being now considered, the procedure is repeated until only one node remains, the
Huffman tree. The simplest construction algorithm uses a priority queue where the node with
lowest probability is given highest priority:

Algorithm

Create a leaf node for each symbol and add it to the priority queue.
While there is more than one node in the queue:
Remove the two nodes of highest priority (lowest probability) from the queue
Create a new internal node with these two nodes as children and with probability equal to the
sum of the two nodes' probabilities.
Add the new node to the queue.
The remaining node is the root node and the tree is complete

Since efficient priority queue data structures require O(log n) time per insertion, and a tree with
n leaves has 2n−1 nodes, this algorithm operates in O(n log n) time, where n is the number of
symbols.

Example
Suppose that a file contains 13 different characters, each of one having different frequency
occurrences that are shown below. Totally we have 838 characters. Each character that occupies
4 bit length. Now, we have a total of 838 × 4 = 3352 bits. To reduce the bit length, we use
Huffman algorithm. The details are shown below.

Character E T A O I N S R H L D C U
Frequency 125 93 80 76 73 71 65 61 55 41 40 31 27

8
Chapter 13- Algorithm Design Techniques

Initially, we start with low frequency occurrence characters C with frequency 31 and U with
frequency 27 for a total of 58. Now construct a tree structure with the characters are placed in the
leaf node and the result is stored in the root node. The tree structure is shown below.

Next we take D and L with a total of 81. The tree diagram is shown below.

Next, we take H with the frequency 55 and another node will be 58(the result of C & U). Draw
the tree diagram with the two nodes that are given below.

Now, we take R & S, since the frequencies are 61 & 65. Add these nodes to the tree structure
with the result of 126.

9
Chapter 13- Algorithm Design Techniques

Repeat the process by adding the nodes the lowest frequencies and form the complete Huffman
tree structure. The final picture is shown in the following figure 13.4

Figure 13.4 – Huffman Tree Structure

Figure 13.5 – Huffman Code

10
Chapter 13- Algorithm Design Techniques

The complete bit representation for the characters which are consolidated and the details are
shown in the figure 13.5. The complete Huffman tree Structure is represented in figure 13.6.
Here, every character that occupies different number of bit patterns and the total space that
needed is reduced.

Figure 13.6 – Huffman Tree Structure (Bit Pattern)

13.2.4. Approximate Bin Packing


The goal is to schedule jobs of various lengths on a fixed number of machines while minimizing
the make span, or equivalently to pack items of various sizes into a fixed number of bins while
minimizing the largest bin size. Hence, all bins have a fixed size, and we wish to minimize the
number of bins needed.

Given items with sizes s1, . . . , s n € (0, 1), pack them into the fewest number of bins possible,
where each bin is of size 1. As an example, we have an item with the weights 0.2, 0.5, 0.4, 0.7,
0.1, 0.3, 0.8. Now, we pack the items to the bin with the total capacity of size 1 in order to use
less number of bins. The figure 13.7 shows an approximate bin packing.

There are two versions of the bin packing problem.

11
Chapter 13- Algorithm Design Techniques

 On-line Algorithms - Each item must be placed in a bin before the next item can be
processed. In online algorithm, we have Next Fit, First Fit & Best Fit algorithm
 Off-line Algorithm - do not need to do anything until all the input has been read

Figure 13.7 – Optimal Bin Packing

Next Fit Online Algorithm


When processing any item, we check to see whether it fits in the same bin as the last item. If it
does, it is placed there; otherwise, a new bin is created. This algorithm is incredibly simple to
implement and runs in linear time.

Figure 13.8 – Next Fit for 0.2, 0.5, 0.4, 0.7, 0.1, 0.3, and 0.8

Figure 13.8 shows the next fit bin packing for the given weights. First, we take the weights 0.2
and then 0.5 in the order it comes and place it in the first bin. Now the total weight is 0.7. Add
the next weight 0.4, it exceeds the total bin weight of 1.0. So add the weight 0.4 to the next bin
and continue the process. For the given weights, there are 5 bins are needed to place the weights.

First Fit Online Algorithm


The first fit strategy is to scan the bins in order and place the new item in the first bin that is large
enough to hold it. Thus, a new bin is created only when the results of previous placements have
left no other alternative. A simple method of implementing first fit would process each item by
scanning down the list of bins sequentially. This would take O(n2). Figure 13.9 shows the
packing that results from first fit on our standard input.

12
Chapter 13- Algorithm Design Techniques

Figure 13.9 – First Fit for 0.2, 0.5, 0.4, 0.7, 0.1, 0.3, and 0.8
Best Fit Online Algorithm
Instead of placing a new item in the first spot that is found, it is placed in the tightest spot among
all bins.

Figure 13.10 – Best Fit for 0.2, 0.5, 0.4, 0.7, 0.1, 0.3, and 0.8

13.3. Divide and Conquer

13.3.1. Definition
The divide-and-conquer strategy solves a problem by:
1. Breaking it into sub problems that are themselves smaller instances of the same type of
problem
2. Recursively solving these sub problems
3. Appropriately combining their answers

13.3.2. Running Time of Divide and Conquer Algorithms


The running time of Divide and Conquer Algorithms is calculated using the formula,
T(n)=a(n/b)+f(n)
Where, T(n) is the running time of the given algorithm
a is the number of sub problems
b is the sub problem size
f(n) is the time to combine the solutions of sub problems

Examples
 Closest-Points Problem
 Quick Sort

13
Chapter 13- Algorithm Design Techniques

 Merge Sort
 Binary Search

13.3.3. Closest-Points Problem


Given n points in metric space, find a pair of points with the smallest distance between them.
The problem can be solved in O(n log n) time using the recursive divide and conquer approach,
e.g., as follows:
1. Sort points along the x-coordinate
2. Split the set of points into two equal-sized subsets by a vertical line x = xmid
3. Solve the problem recursively in the left and right subsets. This will give the left-side and
right-side minimal distances dLmin and dRmin respectively.
4. Find the minimal distance dLRmin among the pair of points in which one point lies on the
left of the dividing vertical and the second point lies to the right.
The final answer is the minimum among dLmin, dRmin, and dLRmin

Figure 13.11 – Closest Points problem

The figure 13.11 shows the closest points problem, which is used to find the pair of points
which are very closer than other pair of points. This is done by dividing the given region into
two equal halves and identifies the closest points in the left region and also in the right region.
Select the points which are very closer by comparing the points in the left and right region. This

14
Chapter 13- Algorithm Design Techniques

is a recursive process by dividing the region further and further and selects the points which are
very closer.

13.3.4. Quick sort


Quicksort is a divide and conquer algorithm. Quicksort first divides a large list into two smaller
sub-lists: the low elements and the high elements. Quicksort can then recursively sort the sub-
lists. Refer the concept of Quick Sort, algorithm and examples in Chapter 12.9.

13.3.5. Merge Sort


The merge sort algorithm is based on the classical divide-and-conquer paradigm. The given
elements are divided into two arrays and then sorted and finally combine the two sorted arrays to
form the resultant array. The Merge Sort concepts, algorithm, examples and running time are
explained in Chapter 12.8.

13.3.6. Binary Search


A binary search or half-interval search algorithm finds the position of a specified value (the input
"key") within a sorted array. In each step, the algorithm compares the input key value with the
key value of the middle element of the array. If the keys match, then a matching element has
been found so its index, or position, is returned. Otherwise, if the sought key is less than the
middle element's key, then the algorithm repeats its action on the sub-array to the left of the
middle element or, if the input key is greater, on the sub-array to the right. If the remaining array
to be searched is reduced to zero, then the key cannot be found in the array and a special "Not
found" indication is returned.

Algorithm
Algorithm is quite simple. It can be done either recursively or iteratively:
1. get the middle element;
2. if the middle element equals to the searched value, the algorithm stops;
3. otherwise, two cases are possible:
o Searched value is less, than the middle element. In this case, go to the step 1 for the part
of the array, before middle element.
o Searched value is greater, than the middle element. In this case, go to the step 1 for the
part of the
intarray, after middlearr[],
binarySearch(int element.
int value, int left, int right)
while (left <= right) do
Procedure int middle = (left + right) / 2;
if (arr[middle] == value)
return middle;
else if (arr[middle] > value)
right = middle - 1;
else
15
left = middle + 1;
return -1;
Chapter 13- Algorithm Design Techniques

Example

Figure 13.12 – Binary Search Example

Analysis
The recurrence relation for binary search,
C(n)=C(n/2)+1 for n>1, C(1)=1
Substitute n=2k then we get,
C(2k)=k+1=log2n+1
C(n)= log2n +1
= log2n (n+1)
If n=2i then
C(n)= log2n +1
= log2n 2i+1
= log22+ log2i+1
=1+ log2i+1= log2i+2

Complexity
The running time of binary search is 1+logn

13.4. Dynamic Programming

13.4.1. Definition
Dynamic programming is a method for efficiently solving a broad range of search and
optimization problems which exhibit the characteristics of overlapping sub problems and optimal
substructure.

16
Chapter 13- Algorithm Design Techniques

The idea behind dynamic programming is quite simple. In general, to solve a given problem, we
need to solve different parts of the problem (sub problems), and then combine the solutions of
the sub problems to reach an overall solution. Dynamic programming algorithms are used for
optimization (for example, finding the shortest path between two points, or the fastest way to
multiply many matrices). A dynamic programming algorithm will examine all possible ways to
solve the problem and will pick the best solution.

Advantages
 Computing duplications in solutions is avoided totally
 Many decision sequences are generated and all the overlapping sub instances are
considered.
Examples
 Using a Table Instead of Recursion
 Ordering Matrix Multiplications
 Optimal Binary Search Tree
 All-Pairs Shortest Path
 Multistage graphs

13.4.2. Using a Table Instead of Recursion


The Fibonacci series are given by:
0, 1, 1, 2, 3, 5, 8…
defined by the simple recurrence
F(n)=F(n-1)+F(n-2) for n>1, where F(0)=0, F(1)=1
For computing the nth Fibonacci number using recursive method, the computation is done for the
same function many times. The figure 13.13 shows the computation of Fibonacci number using
recursive methods. In this figure, F(6) will be computed and for this computation the functions
F(2), F(3), F(4) is computed many times.

Figure 13.13 – Fibonacci Series using recursive method

17
Chapter 13- Algorithm Design Techniques

Recursive Algorithm for Fibonacci


unsigned int fib( unsigned int n )
{
if( n <= 1 )
return 1;
else
return( fib( n-1 ) + fib( n-2 ) );
}

To overcome the problem of recursive method, we are using table method. In this method, the
solution to the sub problem to be stored and reused from the first time computation itself. A one
dimensional array of n+ 1 consecutive location is used to store the sub problem values. The i th
Fibonacci value is computed by adding the last to locations. This is an advantage of dynamic
programming, since it avoids the re-computation of same function for many times. The algorithm
for table method is shown below.

Non Recursive Algorithm for Fibonacci


unsigned int fibonacci( unsigned int n )
{
unsigned int i, last, next_to_last, answer;
if( n <= 1 )
return 1;
last = next_to_last = 1;
for( i = 2; i <= n; i++ )
{
answer = last + next_to_last;
13.4.3. Ordering Matrix Multiplications
next_to_last = last;
last = answer;
}
return answer;
}

13.4.3. Ordering Matrix Multiplications


This is one of the applications of dynamic programming, in which it is used to find the ordering
of matrix multiplication. Suppose we are given four matrices, A, B, C, and D, of dimensions A =
50 X 10, B = 10 X 40, C = 40 X 30, and D = 30 X 5. Although matrix multiplication is not
commutative, it is associative, which means that the matrix product ABCD can be parenthesized,

18
Chapter 13- Algorithm Design Techniques

and thus evaluated, in any order. The obvious way to multiply two matrices of dimensions p X q
and q X r, respectively, uses pqr scalar multiplications.

In the case of four matrices, it is simple to solve the problem by exhaustive search, since there
are only five ways to order the multiplications. We evaluate each case below:
(A((BC)D)): Evaluating BC requires 10 X 40 X 30 = 12,000 multiplications. Evaluating (BC)D
requires the 12,000 multiplications to compute BC, plus an additional 10 X 30 X 5 = 1,500
multiplications, for a total of 13,500. Evaluating (A((BC)D) requires 13,500 multiplications for
(BC)D, plus an additional 50 X 10 X 5 = 2,500 multiplications, for a grand total of 16,000
multiplications.
(A(B(CD))): Evaluating CD requires 40 X 30 X 5 = 6,000 multiplications. Evaluating B(CD)
requires 6,000 multiplications to compute CD, plus an additional 10 X 40 X 5 = 2,000
multiplications, for a total of 8,000. Evaluating (A(B(CD)) requires 8,000 multiplications for B
(CD), plus an additional 50 X 10 X 5 = 2,500 multiplications, for a grand total of 10,500
multiplications.
((AB)(CD)): Evaluating CD requires 40 X 30 X 5 = 6,000 multiplications. Evaluating AB
requires 50 X 10 X 40 = 20,000 multiplications. Evaluating ((AB)(CD)) requires 6,000
multiplications for CD, 20,000 multiplications for AB, plus an additional 50 X 40 X 5 = 10,000
multiplications for a grand total of 36,000 multiplications.
(((AB)C)D): Evaluating AB requires 50 X 10 X 40 = 20,000 multiplications. Evaluating (AB)C
requires the 20,000 multiplications to compute AB, plus an additional 50 X 40 X 30 = 60,000
multiplications, for a total of 80,000. Evaluating (((AB)C)D) requires 80,000 multiplications for
(AB)C, plus an additional 50 X 30 X 5 = 7,500 multiplications, for a grand total of 87,500
multiplications.
((A(BC))D): Evaluating BC requires 10 X 40 X 30 = 12,000 multiplications. Evaluating A(BC)
requires the 12,000 multiplications to compute BC, plus an additional 50 X 10 X 30 = 15,000
multiplications, for a total of 27,000. Evaluating ((A(BC))D) requires 27,000 multiplications for
A(BC), plus an additional 50 X 30 X 5 = 7,500 multiplications, for a grand total of 34,500
multiplications.
From this different ordering, (A(B(CD))) order of matrix multiplication that produces an optimal
solutions by reducing the number of multiplications.

13.4.4. Optimal Binary Search Tree


• An optimal binary search tree is a binary search tree for which the nodes are arranged on
levels such that the tree cost is minimum.
• A binary search tree is a tree where the key values are stored in the internal nodes, the
external nodes (leaves) are null nodes, and the keys are ordered lexicographically. I.e. for
each internal node all the keys in the left sub tree are less than the keys in the node, and
all the keys in the right sub tree are greater.

19
Chapter 13- Algorithm Design Techniques

• Catalan number c(n)= C(2n,n)/(n+1) is used to find the number of different possible
binary search tree for n nodes.

Example
Suppose if we have 4 nodes 3, 7, 9, & 12 then we can construct 14 different binary search trees.
The nodes and the probability values are shown below. Some of these tree structures which are
shown in the figure 13.14.

Keys 3 7 9 12
Probabilities 0.1 0.2 0.4 0.3

Figure 13.14 – Some Binary Search Tree for 3,7,9,12

Suppose for the tree (b), the average key comparisons =0.2(1)+0.1(2)+0.3(2)+0.4(3)=2.2
Then identify the optimal binary search trees by calculating the average number of key
comparisons for all those tree structure and select the search tree with minimum values.

Figure 13.15 – Structure of an Optimal Binary Search Tree

20
Chapter 13- Algorithm Design Techniques

Let C(i, j) denote the cost of an optimal binary search tree containing a i,…,aj . The cost of the
optimal binary search tree with ak as its root :

C[i,j]=mini≤k≤j { Pk.1+ s

+ s

= mini≤k≤j { Pk + s

+ s }

= mini≤k≤j { Pk+ s s + }

= mini≤k≤j {C [i,k-1]+c[k+1,j]} +

Algorithm

OptimalBST(P[1…n])
for i  1 to n do
C[i, i-1]  0
C[i, i]  P[i]
R[i, i]  i
C[n+1, n]  0
for d  1 to n-1 do
for i  1 to n-d do
Ji+d
minval  ∞
For k  i to j do
If C[ i, k-1 ] + C[ k+1, j ] < minval
minval  C[ i, k-1 ] + C[ k+1, j ]; kmin <- k
R[ i , j]  kmin
Sum  P[i];
for s  i +1 to j do
sum  sum + P[s] 21
C[ i, j]  minval + sum
return C[ 1, n], R
Chapter 13- Algorithm Design Techniques

Time complexity
The running time of optimal binary search tree is O(n 3)

13.4.5. All-Pairs Shortest Path algorithm


The all-pairs shortest path problem is the determination of the shortest graph distances between
every pair of vertices in a given graph. The problem can be solved using applications of
Dijkstra's algorithm or all at once using the Floyd-Warshall algorithm. The latter algorithm also
works in the case of a weighted graph where the edges have negative weights. The matrix of all
distances between pairs of vertices is called the graph distance matrix, or sometimes the all-pairs
shortest path matrix.

Floyd's Algorithm
The Floyd’s algorithm compares all possible paths through the graph between each pair of
vertices. It is able to do this with only Θ(|V|3) comparisons in a graph. This is remarkable
considering that there may be up to Ω(|V|2) edges in the graph, and every combination of edges is
tested. It does so by incrementally improving an estimate on the shortest path between two
vertices, until the estimate is optimal.

Consider a graph G with vertices V numbered 1 through N. Further consider a function


shortestPath(i, j, k) that returns the shortest possible path from i to j using vertices only from the
set {1,2,...,k} as intermediate points along the way. Now, given this function, our goal is to find
the shortest path from each i to each j using only vertices 1 to k + 1.

Algorithm

Input n: Number of vertices


a[0..n-1][0..n-1] -- Adjacency matrix
Output: Transformed a that contains shortest path lengths
for ( k = 0; k < n; k++ )
for ( i = 0; i < n; i++ )
for ( j = 0; j < n; j++ )
a[i][j] = min ( a[i][j], a[i][k] + a[k][j] );

22
Chapter 13- Algorithm Design Techniques

The algorithm works by:


Step 1 : Let k = 1
Step 2 : We calculate elements dijk of the shortest path length matrix found after the k-th passage
through algorithm Dk using the following equation:
dij =min{ dijk-1 , dikk-1 + dkjk-1 }
k

Step 3: If k = n, the algorithm is finished. If k < n, increase k by 1, i.e. K = k+1 and then return
to step 2.

Example

Figure 13.16 – Weighted Graph and its weight matrix D0

All elements along the main diagonal of matrix D0 equal zero since by definition Dij0=0 for i=j.
We note element d1,20 of matrix D0. This element equals 8 since the length of the branch
connecting nodes 1 and 2 is 8. Element d3,10 equals infinity since the network has no branch
which is oriented from node 3 to node 1. Element D 5,10 of matrix D0 equals infinity as well since
there is direct branch linking nodes 5 and 1. We now go to the first algorithmic step. Let k = 1.
As an illustration of Step 2 we calculate the elements of the first two rows of matrix D1.

D1,21=min{D1,20,D1,10+D1,20}=min{8,0+8}=8

D1,31=min{D1,30,D1,10+D1,30}=min{3,0+8}=3

D1,41=min{D1,40,D1,10+D1,40}=min{5,0+5}=5

D1,51=min{D1,50,D1,10+D1,50}=min{∞,0+∞}=∞

D2,11=min{D2,10,D2,10+D1,10}=min{8,0+8}=8

D2,31=min{D2,30,D2,10+D1,30}=min{2,8+3}=2

D2,41=min{D2.40,D2,10+D1,40}=min{∞,8+5}=13

D2,51=min{D2,50,D2,10+D1,50}=min{5,8+∞}=5

23
Chapter 13- Algorithm Design Techniques

Repeat the process to calculate the values for next three rows of D(1). Now the matrix D(1) is,

After the second, third, fourth and fifth passages through the algorithm, matrices D(2), D(3), D(4),
D(5) are as follows:

The distance matrix D(5) which produces the shortest distance between every pair of vertices.
This is an output of the algorithm.

13.4.6. Multistage graphs


A multistage graph G=(V,E) which is a directed graph. In this graph, all the vertices are
partitioned into k stages where k>2. In multistage graph problem, we have to find the shortest
distance from source to sink. The cost of each path is calculated by using the weight given along
that edge. The cost of a path from source (denoted as S) to sink(denoted as T) is the sum of the
costs of edges on the path.
The multistage graph problem can be solved by the following methods:
 Forward approach
 Backward approach
The figure 13.14 shows an example of multistage graph. In this graph, there is single vertex in
stage 1, 3 vertices in stage 2, 2 vertices in stage 3 and only one target vertex in stage 4.

24
Chapter 13- Algorithm Design Techniques

Figure 13.14 – Multistage graph


Backward approach
By using backward approach, the shortest path from S to T is calculated as:
d(S,T) = min {1+d(A,T),2+d(B,T)+d(C,T)}
Find out the values of d(A,T), d(B,T) and d(C,T).
d(A,T)= min{3+d(D,T),6)+d(E,T)}
d(B,T)= min{4+d(D,T),10+d(E,T)}
d(C,T)= min{3+d(E,T),d(C,T)}
Now, compute the values of d(D,T) and d(E,T)
d(D,T)=8 & d(E,T)=2
Let us put these values in the above equation,
d(A,T)= min{3+8,6+2}=8, the path is A-E-T
d(B,T)= min{4+8,10+2}=12, the path is A-D-T
d(C,T)= min{3+2,10}=5, the path is C-E-T
Substitute these values in the given equation,
d(S,T) = min {1+d(A,T),2+d(B,T)+d(C,T)}
=min{1+8,2+12,7+5}
= min{9,14,12}
=9
By using the backward method, the shortest distance between S to T through multiple stage of
vertices of the given graph is 9 with the path S-A-E-T.
Algorithm - Backward Approach
Backward(G,stages,n,p)
Back_cost[i]  0
For(i0 to n-2) do
{
rGet_Min(i,n)
back_cost[i]back_cost[r]+c[r][i]
d[i]r
}
//finding minimum cost path
P[0]0
P[stages-1]n-1
25
For(istages-1 to 1) do
P[i]d[p[i+1]];
Chapter 13- Algorithm Design Techniques

Forward approach
d(S,A) =1
d(S,B) =2
d(S,C) =7
d(S,D) =min{1+d(A,D),2+d(B,D)}
= min{1+3,2+4}
=4
d(S,E) = min{1+d(A,E),2+d(B,E),7+d(C,E)}
= min{1+6,2+10,7+3}
=7
d(S,T) = min{d(S,D)+d(D,T),d(S,E)+d(E,T),d(S,C)+d(C,T)}
= min{4+8,7+2,7+10}
=9
Now the minimum cost of the graph using forward method is 9 with the shortest path is S-A-E-T.
Algorithm – Forward Approach

Forward(G,stages,n,p)
cost[i]  0
For(in-2 downto 0) do
{
rGet_Min(i,n)
cost[i]cost[r]+c[i][r]
d[i]r
}
//finding minimum cost path
P[0]0
P[stages-1]n-1
For(i 1 to stages-1 ) do
P[i]d[p[i+1]];

Analysis
The time complexity of Multistage Graph using both Forward and Backward method is O(|V|+|
E|)

26
Chapter 13- Algorithm Design Techniques

Comparison of Divide and Conquer & Dynamic Programming

Divide and Conquer Dynamic programming


The divide-and-conquer strategy solves a Dynamic programming is a method for
problem by breaking it into sub problems that efficiently solving a broad range of search and
are themselves smaller instances of the same optimization problems which exhibit the
type of problem, recursively solving these sub characteristics of overlapping sub problems
problems and appropriately combining their and optimal substructure
answers.
Top down approach Bottom up approach
Less efficient More efficient
Duplicate solutions may be obtained Duplicate solutions is avoided totally
It splits input at specific deterministic points It splits input at every possible split points
usually in the middle rather than at a particular point

13.5. Backtracking

13.5.1. Definition
Backtracking is a process of searching the solution to the problem by searching the solution
space (the set of all feasible solutions) for the given problem. Backtracking is a process where
steps are taken towards the final solution and the details are recorded. If these steps do not lead
to a solution some or all of them may have to be retraced and the relevant details discarded. In
these circumstances it is often necessary to search through a large number of possible situations
in search of feasible solutions.

Examples
 The Turnpike Reconstruction Problem
 Games (Eight Queen problem)
 Subset Sum Problem
 Graph Coloring

13.5.2. The Turnpike Reconstruction Problem


Suppose we are given n points, p1, p2, . . . , pn, located on the x-axis. xi is the x coordinate of pi.
Let us further assume that x1 = 0 and the points are given from left to right. These n points
determine n(n - 1)/2 (not necessarily unique) distances d1, d2, . . . , dn between every pair of
points of the form | xi - xj |. It is clear that if we are given the set of points, it is easy to construct
the set of distances in O(n2) time. The turnpike reconstruction problem is to reconstruct a point
set from the distances. Let D be the set of distances, and assume that | D | = m = n(n - 1)/2. As an
example, suppose that
D = {1, 2, 2, 2, 3, 3, 3, 4, 5, 5, 5, 6, 7, 8, 10}
27
Chapter 13- Algorithm Design Techniques

Since | D | = 15, we know that n = 6. We start the algorithm by setting x1 = 0. Clearly, x6 = 10,
since 10 is the largest element in D. We remove 10 from D. The points that we have placed and
the remaining distances are as shown in the following figure.

The largest remaining distance is 8, which means that either x2 = 2 or x5 = 8. By symmetry, we


can conclude that the choice is unimportant, since either both choices lead to a solution (which
are mirror images of each other), or neither do, so we can set x5 = 8 without affecting the
solution. We then remove the distances x6 - x5 = 2 and x5 - x1 = 8 from D, obtaining

The next step is not obvious. Since 7 is the largest value in D, either x4 = 7 or x2 = 3. If x4 = 7,
then the distances x6 - 7 = 3 and x5 - 7 = 1 must also be present in D. we can come back and try
the other.

At this point, we have x1 = 0, x4 = 7, x5 = 8, and x6 = 10. Now the largest distance is 6, so either
x3 = 6 or x2 = 4. But if x3 = 6, then x4 - x3 = 1, which is impossible, since 1 is no longer in D.
On the other hand, if x2 = 4 then x2 - x0 = 4, and x5 - x2 = 4. This is also impossible, since 4
only appear once in D. Thus, this line of reasoning leaves no solution, so we backtrack. Since x4
= 7 failed to produce a solution, we try x2 = 3. If this also fails, we give up and report no
solution. We now have

28
Chapter 13- Algorithm Design Techniques

Figure 13.15 shows a decision tree representing the actions taken to arrive at the solution.

Figure 13.15 – Tree structure for Turnpike Reconstruction Problem

13.5.3. Games (Eight Queen Problem)


The eight queen’s puzzle is the problem of placing eight chess queens on an 8×8 chessboard so
that no two queens attack each other. Thus, a solution requires that no two queens share the same
row, column, or diagonal. The eight queen’s puzzle is an example of the more general n-queens
problem of placing n queens on an n×n chessboard, where solutions exist for all natural numbers
n with the exception of 2 and 3. The figure 13.16 shows the solution to the 8 queen problem. It
finds solutions by starting with a queen in the top left corner of the chess board. It then places a
queen in the second column and moves it until it finds a place where it cannot be hit by the queen
in the first column. It then places a queen in the third column and moves it until it cannot be hit
by either of the first two queens. Then it continues this process with the remaining columns. If
there is no place for a queen in the current column the program goes back to the preceding
column and moves the queen in that column.
If the queen there is at the end of the column it removes that queen as well and goes to the
preceding column. If the current column is the last column and a safe place has been found for
the last queen, then a solution of the puzzle has been found. If the current column is the first
column and its queen is being moved off the board then all possible configurations have been
examined, all solutions have been found, and the algorithm terminates.

29
Chapter 13- Algorithm Design Techniques

Figure 13.16 – Solution of 8-Queen problem

Algorithm
void NQueens(int k, int n)
{
for (int i=1; i<=n; i++)
{
if (Place(k, i))
{
x[k] = i;
if (k==n)
{
for (int j=1;j<=n;j++)
cout << x[j] << ' '; cout << endl;
}
else
NQueens(k+1, n);
}
}
}

bool Place(int k, int i)


{
for (int j=1; j < k; j++)
if ((x[j] == i) || (abs(x[j]-i) == abs(j-k)))
return(false);
return(true);
}

30
Chapter 13- Algorithm Design Techniques

Figure 13.17 – State Space Tree of N-Queen problem

Figure 13.17 shows the state space tree of 4-queen problem. A node in the state space tree is
promising if it corresponds to the partials constructed solution that may lead to the complete
solution otherwise the nodes are called non-promising. Leaves of the tree represent either the
non- promising dead end or complete solution found by the algorithm.

13.5.4. Subset Sum Problem


An instance of the Subset Sum problem is a pair (S, t), where S = {x1, x2, ..., xn} is a set of
positive integers and t (the target) is a positive integer. The decision problem asks for a subset of
S whose sum is as large as possible, but not larger than t. This problem is NP-complete. In sum
of subset problem, there are n positive integers wi and a positive integer W. The goal is to find
the all subsets of the integers that sum to W. It is similar to the 0-1 knapsack problem, of which
the solution is only one. If the values and weights of each item are equal, we have the sum of
subsets problem.
31
Chapter 13- Algorithm Design Techniques

Example
The figure 13.18 shows the pruned state space tree when backtracking is used with n = 4, W =
13, and w1 = 3, w2 = 4, w3 = 5, w4 = 6

Figure 13.18 – State space tree of subset sum problem


void sum_of_subsets (index i, int weight, int total)
Algorithm
{
if (promising(i, weight, total))
{
if (weight == W)
cout << include[1] through include[i];
else
{
include[i+1] = true;
sum_of_subsets(i+1, weight + w[i+1], total w[i+1]);
include[i+1] = false;
sum_of_subsets(i+1, weight, total w[i+1]);
}
}
}
32
Chapter 13- Algorithm Design Techniques

void bool promising (index i, int weight, int total)


{
return ((weight + total >= W) &&(weight == W || weight + w[i+1] <= W));
}
13.5.5. Graph Coloring
A coloring of a graph is almost always a proper vertex coloring, namely a labeling of the graph’s
vertices with colors such that no two vertices sharing the same edge have the same color. A
coloring using at most k colors is called a (proper) k-coloring. The smallest number of colors
needed to color a graph G is called its chromatic number, and is often denoted χ(G). Sometimes
γ(G) is used, since χ(G) is also used to denote the Euler characteristic of a graph. A graph that
can be assigned a (proper) k-coloring is k-colorable, and it is k-chromatic if its chromatic number
is exactly k. It is used in a number of scientific and engineering applications such as scheduling,
register allocation, optimization and parallel numerical computation.

Figure 13.19 – Graph coloring


The figure 13.19 shows an example graph for the graph coloring problem. In this example, we
have 6 vertices that are connected. To color the vertices, we need a minimum of 3 colors
which can be represented as R(Red), B(Blue) and G(Green).

Algorithm
Number the solution variables [v0 v1, …, vn-1].
Number the possible values for each variable [c0 c1, …, ck-1].
Start by assigning c0 to each vi.
If we have an acceptable solution, stop.
If the current solution is not acceptable, let i = n-1.
If i < 0, stop and signal that no solution is possible.
Let j be the index such that vi = cj. If j < k-1, assign cj+1 to vi and go back to step 4.
But if j ≥ k-1, assign c0 to vi, decrement i, and go back to step 6.

13.6. Branch and bound

33
Chapter 13- Algorithm Design Techniques

13.6.1. Definition
A B&B algorithm searches the complete space of solutions for a given problem for the best
solution.
Ø Branch and bound is a systematic method for solving optimization problems
Ø B&B is a rather general optimization technique that applies where the greedy method and
dynamic programming fail.
Ø However, it is much slower.
Ø On the other hand, if applied carefully, it can lead to algorithms that run reasonably fast on
average.
Ø The general idea of B&B is a BFS-like search for the optimal solution, but not all nodes get
expanded (i.e., their children generated). Rather, a carefully selected criterion determines which
node to expand and when, and another criterion tells the algorithm when an optimal solution has
been found.

Examples
 Traveling salesman problem
 Knapsack problem

13.6.2. Traveling Salesman Problem


Given a set of cities and the distance between each possible pair, the Travelling Salesman
Problem is to find the best possible way of ‘visiting all the cities exactly once and returning to
the starting point’. More formally, TSP is “Given a graph G of vertices V connected via nonzero
cost edges E, find the lowest-cost Hamiltonian circuit”

However, this particular version of the problem is the optimization problem (“find the lowest
cost”). There is another version of the problem, the decision problem, which is “Given a graph G
and a cost k, decide if there is a Hamiltonian circuit whose cost ≤k.” The decision problem, the
one which requires a cost to be given and simply decides if there is a cheaper (or equal)
Hamiltonian circuit, is NP-Complete. But almost everyone who talks about “the Traveling
Salesperson Problem” is talking about the optimization problem, which is not NP-Complete, it is
NP-Hard.

Algorithm

Choose any starting node


Consider the arcs which join the node just chosen to nodes as yet unchosen. Pick the
Exampleone with minimum weight and add it to the cycle
Repeat step 2 until all nodes have been chosen
Then adds the arc that joins the last chosen node to the first chosen node
34
Chapter 13- Algorithm Design Techniques

For the given weighted graph, the distances between the vertices are shown. Now we find the
shortest path to visit all the vertices exactly once, and reach the source vertex. Initially, we start
with a as the source vertex. Next we find the lower bound value for a. The lower bound value lb
is calculated using the formula,
lb = ∑sum of costs of the two least cost edges adjacent to v
Now, the lower bound value for a =[(1+3)+(3+6)+(1+2)+(3+4)+(2+3)]/2 = 14
This is done by taking the average cost of all the vertices. For a, we have to take the edges 1 & 3,
since, these are the two lowest cost edges of a. Then for b, take the edges 3 & 6. For c, take 1 &
2, then for vertex d, 3 & 4 are the lowest cost edges and finally for e, take 2 & 3. Now we find
the average cost of the vertices by calculating the sum of all the cost and divided by 2. Then we
get 14 as the lower bound value.

Travelling Salesman Problem is one of the Branch and Bound technique. Here, we have to find
all possible solution and choose the optimal one. From the source vertex a, visit b or c or d or e.
Suppose if b is the next visited vertex, find the lower bound value for b.
lb for ab = [(1+3)+(3+6)+(1+2)+(3+4)+(2+3)]/2 = 14
In this case, we have to select the edge ab for calculating lower bound value for b. Suppose if c is
the next visited vertex, find the lower bound value for c by choosing the edge ac.
lb for ac = [(1+3)+(3+6)+(1+2)+(3+4)+(2+3)]/2 = 14
Suppose if d is the next visited vertex, find the lower bound value for d by choosing the edge ad.
lb for ad = [(1+5)+(3+6)+(1+2)+(3+5)+(2+3)]/2 = 16
Suppose if e is the next visited vertex, find the lower bound value for e by choosing the edge ae.
lb for ae = [(1+8)+(3+6)+(1+2)+(3+4)+(2+8)]/2 = 19
Now compare the lb value of ab, ac, ad and ae. Choose the lowest lb value of the vertex i.e., ab.
Next we choose the vertex from b. There are three ways to go for the next vertex. One way is a
bc. Another way is a bd. The next way is a b e. Find the lb value of these three ways
and choose the lowest lb value. Continue the process, and finally reach the source vertex a. The
entire process that are shown in the following figure 13.20.

35
Chapter 13- Algorithm Design Techniques

Figure 13.20 – State space tree of Travelling Salesman Problem

13.6.3. Knapsack problem


Given a set of items, each with a weight and a value, determine the count of each item to include
in a collection so that the total weight is less than or equal to a given limit and the total value is
as large as possible. In the following, we have n kinds of items, 1 through n. Each kind of item i
has a value vi and a weight wi. We usually assume that all values and weights are nonnegative. To
simplify the representation, we can also assume that the items are listed in increasing order of
weight. The maximum weight that we can carry in the bag is W.
The most common formulation of the problem is the 0-1 knapsack problem, which restricts the
number xi of copies of each kind of item to zero or one.
Given:
weight w1 w2 … wn
cost c1 c2 … cn

36
Chapter 13- Algorithm Design Techniques

Knapsack weight limit K


1. Calculate vi = ci / wi for i = 1, 2, …, n
2. Sort the item by decreasing vi
3. Find j, such that
w1 + w2 +…+ wj  k < w1 + w2 +…+ wj+1
Example
Item weight value value/weight
1 4 40 10
2 7 42 6
3 5 25 5
4 3 12 4
and the knapsack capacity W is 10. The upper bound value is calculated from the formula
Ub=v+(W-w)(vi+1/wi+1). The state space tree for the knapsack problem is shown in the following
figure 12.27.

Solution
Initially, without any item included, the total weight W=10, w=0 and v=0. Now, we find the
upper bound value for the node 0.
ub for node 1 = 0+(10-0)(10) =100
Next, find the upper bound value by including the first item 1 with their value v =40 and weight
w = 4.
ub for node 2 = 40 +(10-4)(6) = 76
Alternatively, find the upper bound value without including the first item 1. Here, we are not
including the value and weight of the first item.
ub for node 2 = 0+(10-0)(6)=60
There are two possible solutions are there. Suppose if we include the first item, the upper bound
value is 76 and 60 if we are not including the first item. Choose the highest upper bound value
i.e., 76 and include the next item.

In this algorithm, whenever w that exceeds the total weight W=10, then exclude the link for the
process. Next include the item 2, the upper bound value is calculated by add the value and
weight of item 2. In this case weight w=4+7=11 that exceeds the total weight W=10. This is not a
feasible solution. Now without including the second item,
ub for item 4 = 40+(10-4)(5)= 70
This is the only possible solution. Next, we have to include the third item and find the upper
bound value. Repeat the process until we include all the items an d find the upper bound vale.
For this example, the optimal solution is (1,0,3,0).

37
Chapter 13- Algorithm Design Techniques

Figure 13.21 – State space tree of Knapsack problem

13.7 Comparison of various Algorithm Design Techniques

(i) Comparison of Divide and Conquer and Greedy Algorithm

Divide and Conquer Greedy Algorithm


It is used to obtain a solution to give problem. It is used to obtain optimum solution.
The problem is divided into small sub problems. A set of feasible solution is generated and
These problems are solved independently. optimum solution is picked up.
Finally all the solutions of sub problems are
collected together to get the solution to the
given problem.
Duplications in sub solutions are neglected, The optimum selection is without revising
which means that duplicate solutions may be previously generated solutions.
obtained.
Less efficient because of rework on solutions Greedy method is comparatively efficient but

38
Chapter 13- Algorithm Design Techniques

there is no as such guarantee of getting


optimum solution.
Examples: Quick sort, binary search Example: Minimum spanning tree, finding
shortest path.

(ii) Comparison of Divide and Conquer and Dynamic Programming.


Divide and Conquer Dynamic programming

The divide-and-conquer strategy solves a Dynamic programming is a method for


problem by breaking it into sub problems that efficiently solving a broad range of search and
are themselves smaller instances of the same optimization problems which exhibit the
type of problem, recursively solving these sub characteristics of overlapping sub problems
problems and appropriately combining their and optimal substructure
answers.

Top down approach Bottom up approach

Less efficient More efficient

Duplicate solutions may be obtained Duplicate solutions is avoided totally

It splits input at specific deterministic points It splits input at every possible split points
usually in the middle rather than at a particular point

(iii) Comparison of Greedy algorithm and Dynamic Programming.


Greedy method Dynamic programming
It is used for getting optimal solutions. It is also used for getting optimal solutions.
A set of feasible solutions and picks up the No special set of feasible solutions
optimal solution
Solution is generated without revising It considers all possible sequences in order to
previously generated solutions obtain the optimal solution
No guarantee of getting optimal solution Guaranteed to generate optimal solution

(iv) Comparison of Greedy algorithm and Backtracking.

Greedy method Backtracking


Greedy method is an approach to find the Backtracking is a process of searching the
optimized solutions for the given problem. The solution to the problem by searching the
solutions are constructed through a sequence of solution space (the set of all feasible solutions)
steps, each step expanding a partially for the given problem. Backtracking is a
constructed solution so far, until a complete process where steps are taken towards the final

39
Chapter 13- Algorithm Design Techniques

solution to the problem is reached. solution and the details are recorded.

It is used for getting optimal solutions. It is used for getting the solutions to the given
problem that may or may not be optimal.
A set of feasible solutions and picks up the No special set of feasible solutions
optimal solution
Solution is generated without revising It considers the possible sequences in order to
previously generated solutions obtain the solution
Example: Minimum Spanning Tree, Finding Example: N-queen problem, subset sum
shortest path problem

(v) Comparison of Backtracking and Branch & Bound Algorithm.

Backtracking Branch and Bound


It is used to find all possible solutions available It is used to solve optimization problem.
to the problem.
It traverse tree by Depth First Search. It may traverse the tree in any manner, DFS or
BFS.
It realizes that it has made a bad choice and It realizes that it already has a better optimal
undoes the last choice by backing up. solution that the pre-solution leads to.
It search the state space tree until a solution is It completely searches the state space tree to
found. get optimal solution
It involves feasibility function. It involves bounding function

13.8. Randomized Algorithms

An algorithm takes a source of random numbers and makes random choices during execution in
addition to the input. Behavior can vary even on a fixed input.

13.8.1. Definition
Pseudorandom numbers
 The computer is not capable of generating truly random numbers

40
Chapter 13- Algorithm Design Techniques

o The computer can only generate pseudorandom numbers--numbers that are


generated by a formula
o Pseudorandom numbers look random, but are perfectly predictable if you know
the formula
Examples
 Random Number Generators
 Skip Lists
 Primality Testing

13.8.2. Random Number Generators


The standard method to generate random numbers is the linear congruential generator, Numbers
x1, x2, . . . are generated satisfying xi + 1 = axi mod m. To start the sequence, some value of x0
must be given. This value is known as the seed. If x0 = 0, then the sequence is far from random,
but if a and m are correctly chosen, then any other 1 x0 < m is equally valid. If m is prime, then
xi is never 0. As an example, if m = 11, a = 7, and x0 = 1, then the numbers generated are 7, 5, 2,
3, 10, 4, 6, 9, 8, 1, 7, 5, 2, . . . Notice that after m - 1 = 10 numbers, the sequence repeats.

13.8.3. Skip Lists


Our first use of randomization is a data structure that supports both searching and insertion in
O(log n) expected time. The simplest possible data structure to support searching is the linked
list. Figure shows a simple linked list. The time to perform a search is proportional to the number
of nodes that have to be examined, which is at most n. Figure 13.22 shows a linked list in which
every other node has an additional pointer to the node two ahead of it in the list.

(a)

(b)

41
Chapter 13- Algorithm Design Techniques

(c)

(d)

(e)
Figure 13.22 – (a) simple linked list (b) linked list with pointers to two cells ahead (c) linked list with
pointers to four cells ahead (d) linked list with pointers to 2i cells ahead (e ) Skip List

13.8.4. Primality Testing


In this section we examine the problem of determining whether or not a large number is prime.
In this chapter, we will give a polynomial-time algorithm that can test for primality. If the
algorithm declares that the number is not prime, we can be certain that the number is not prime.
If the algorithm declares that the number is prime, then, with high probability but not 100 percent
certainty, the number is prime.

13.9. Introduction to Algorithm Analysis

13.9.1. Definition of algorithm


Algorithm is defined as a step-by-step problem-solving procedure, especially an established,
recursive computational procedure for solving a problem in a finite number of steps. The
efficiency is depend on the following factors

42
Chapter 13- Algorithm Design Techniques

 Time: The amount of time to take executes the algorithm. If the algorithm takes less
amount of time to execute then this one will be the best one.
 Space: The amount of memory required to store the algorithm and the amount of memory
required to store the input for that algorithm.
 Simplicity: The algorithm should not have any complex instructions. Those types of
instructions are simplified to number of smaller instructions.
 Generality: The algorithm should be in general. It should be implemented in any
language. The algorithm is not fit in to a specific output.

13.9.2. Properties of the Algorithm


 Finiteness: - an algorithm terminates after a finite numbers of steps.
 Definiteness: - each step in algorithm is unambiguous. This means that the action
specified by the step cannot be interpreted (explain the meaning of) in multiple ways &
can be performed without any confusion.
 Input:- an algorithm accepts zero or more inputs
 Output:- it produces at least one output.
 Effectiveness:- it consists of basic instructions that are realizable. This means that the
instructions
 Non Ambiguity: The algorithm should not having any conflict meaning.
 Range of Input: Before design an algorithm, decide what type of input going to give,
what’s is the required output.
 Multiplicity: You can write different algorithms to solve the same problem.
 Speed: Apply some idea to speed up the execution time.

13.9.3. Analysis of Algorithm


To compute the efficiency of an algorithm, consider the following two factors.
 Space Complexity
 Time Complexity
(i) Space Complexity: The amount of memory required to store the algorithm and the amount of
memory required to store the inputs for this algorithm.
(ii) Time Complexity: The amount of time required to run the algorithm.

How to measure the running time?


• Find the basic operation of the algorithm. (The inner most loop or operation is called basic
operation)
• Compute the time required to execute the basic operation.
• Compute how many times the basic operation is executed. The Time complexity calculated by
the following formula
T(n) = Cop * C(n)

43
Chapter 13- Algorithm Design Techniques

Where,
Cop – Constant (The amount of time required to execute the basic operation)
C(n) – How many times the basic operation is executed.

13.9.4. Asymptotic Notations


A problem may have numerous algorithmic solutions. In order to choose the best algorithm for a
particular task, you need to be able to judge how long a particular solution will take to run. Or,
more accurately, you need to be able to judge how long two solutions will take to run, and
choose the better of the two. You don't need to know how many minutes and seconds they will
take, but you do need some way to compare algorithms against one another.

Asymptotic complexity is a way of expressing the main component of the cost of an algorithm,
using idealized units of computational work. Consider, for example, the algorithm for sorting a
deck of cards, which proceeds by repeatedly searching through the deck for the lowest card. The
asymptotic complexity of this algorithm is the square of the number of cards in the deck. This
quadratic behavior is the main term in the complexity formula, it says, e.g., if you double the size
of the deck, then the work is roughly quadrupled.

There are three types of notations are there:


(i) Big–Oh Notation
(ii) Big–Omega Notation
(iii) Big–Theta Notation

(i) Big–Oh (or) O( ) Notation

Figure 13.23 – Big Oh Notation


A function t(n) is said to be in O(g(n)), denote t(n) € O(g(n)), if t(n) is bounded above by some
constant multiple of g(n) for all large n, i.e., if there exists some positive constant c and some
nonnegative integer n0 such that t(n) ≤ c.g(n) for all n ≥ n0.

44
Chapter 13- Algorithm Design Techniques

Example 1
2n2 = O(n3)
0 ≤ t(n) ≤ cg(n) Definition of O(g(n))
0 ≤ 2n2 ≤ cn3 Substitute
0/n3 ≤ 2n2/n3 ≤ cn3/n3 Divide by n3
Determine c
Lim
0 ≤ 2/n ≤ c n   2/n = 0
2/n maximum when n=1
0 ≤ 2/1 ≤ c = 2 Satisfied by c=2
Determine n0
0 ≤ 2/n0 ≤ 2
0 ≤ 2/2 ≤ n0
0 ≤ 1 ≤ n0 = 1 and n0=1
0 ≤ 2n2 ≤ 2n3 ∀ n ≥ n0=1

Example 2
1000n2 + 50n = O(n2)
0 ≤ t(n) ≤ cg(n)

0 ≤ 1000n2 + 50n ≤ cn2

0/n2 ≤ 1000n2/n2 + 50n/n2 ≤ cn2/n2 Divide by n2

Note that 50/n → 0 as n → ∞


0 ≤ 1000 + 50/n ≤ c
Greatest when n = 1

0 ≤ 1000 + 50/1 = 1050 ≤ c =


With c=1050
1050

0 ≤ 1000 + 50/n0 ≤ 1050 Find n0

-1000 ≤ 50/n0 ≤ 50
-20 ≤ 1/n0 ≤ 1

-20n0 ≤ 1 ≤ n0 = 1 n0=1

0 ≤ 1000n2 + 50n ≤ 1050n2 ∀ n ≥ n0=1, c=1050

45
Chapter 13- Algorithm Design Techniques

(ii) Big–Omega (or) Ω( ) Notation

Figure 13.24 – Big Omega Notation

A function t(n) is said to be in Ω (g(n)), denote t(n) € Ω (g(n)), if t(n) is bounded below by some
constant multiple of g(n) for all large n, i.e., if there exists some positive constant c and some
nonnegative integer n0 such that t(n) ≥ c.g(n) for all n ≥ n0.

Example 1
n3 = Ω(n2)
n3 = Ω(n2) with c=1 and n0=1
0 ≤ cg(n) ≤ t(n)
0 ≤ 1*12 = 1 ≤ 1 = 13
0 ≤ cg(n) ≤ t(n)

0 ≤ cn2 ≤ n3

0/n2 ≤ cn2/n2 ≤ n3/n2

0 ≤ c≤ n

with c=1 and n0=1 since n increases.


0≤1≤1
Lim
n=∞
n  
Example 2
n ≠ Ω(n2)
0 ≤ cg(n) ≤ t(n)

0 ≤ cn2 ≤ n
(iii) Big–Theta (or) θ()
0/n2 ≤ cn2/n2 ≤ n/n2 Notation
0 ≤ c ≤ 1/n

c depends on n and there is no n0 that satisfies as n increases.


Only c = 0 would satisfy but not allowed46by definition.
Lim
1/n=0
n
Chapter 13- Algorithm Design Techniques

Figure 13.25 – Big Theta Notation


A function t(n) is said to be in θ (g(n)), denote t(n) € θ (g(n)), if t(n) is bounded both above and
below by some constant multiple of g(n) for all large n, i.e., if there exists some positive constant
c1 and c2 and some nonnegative integer n0 such that
c2.g(n) ≤ t(n) ≥ c1.g(n) for all n ≥ n0.

Example
n2/2-2n = Θ(n2)
c1g(n) ≤ t(n) ≤ c2g(n)
c1n2 ≤ n2/2-2n ≤ c2n2
c1n2/n2 ≤ n2/2n2-2n/n2 ≤ c2n2/n2 Divide by n2
c1 ≤ 1/2-2/n ≤ c2
O: Determine c2 = ½
 ½-2/n ≤ c2 = ½ ½-2/n = ½

maximum of ½-2/n
Ω: Determine c1 = 1/10
 0 < c1 ≤ 1/2-2/n 0 < c1 minimum when n=5
 0 < c1 ≤ 1/2-2/5
 0 < c1 ≤ 5/10-4/10 = 1/10
n0 : Determine n0 = 5
 1/10 ≤ 1/2-2/n0 ≤ 1/2
 1/10 ≤ 1/2-2/n0 ≤ 1/2

 2/n0 ≤ 1/2-1/10 ≤ 1/2


 2/n0 ≤ 4/10 ≤ 1/2
 2/n0 ≤ 2/5 ≤ 1/2
 n0 ≥ 5 n0 = 5 satisfies

Verify
c1n2 ≤ n2/2-2n ≤ c2n2 with c1=1/10, c2=½ and n0=5
2 2 2
1/10*5 ≤ 5 /2-2*5 ≤ ½*5
25/10 ≤ 25/2-20/2 ≤ 25/2
5/2 ≤ 5/2 ≤ 25/2 Holds
2 2 2
In general: 1/10n ≤ n /2-2n ≤ ½n for n ≥ 5

13.9.5. Recurrence Relations

47
Chapter 13- Algorithm Design Techniques

A recurrence relation is an equation that recursively defines a sequence, once one or more initial
terms are given: each further term of the sequence is defined as a function of the preceding
terms.
For example, consider the sequence
x(n)=x(n-1)+n for n>0
x(0)=0

This is an example of a recurrence relation with initial condition. A initial condition can be given
for a value on n other than 0 and for some recurrences, more than one value needs to be specified
by initial conditions.
To solve a given recurrences subject to a given initial condition means to find an explicit formula
for the generic term of the sequence that satisfies both the recurrence equation and the initial
condition or to prove that such a sequence does not exist. A general solution is a formula that
specifies all such sequences.

Methods for solving recurrence relations


 Forward substitution
 Backward substitution

(i) Forward substitution


Starting with the initial term of the sequence given by the initial condition, we can use the
recurrence equation to generate the first few terms of its solution.
For example, consider the sequence,
x(n)=2x(n-1)+1, for n>1
x(1)=1
We obtain the first few terms as follows:
x(1)=1
x(2)=2x(1)+1=2.1+1=3
x(3)=2x(2)+1=2.3+1=7
x(4)=2x(3)+1=2.7+1=15
In general,
x(n)=2n-1 for n=1,2,3 and 4

(ii) Backward substitution


This method of solving recurrence relations work exactly as its name implies: using the
recurrence relation, we express x(n-1) as a function of x(n-2) and substitute the result into the
original equation to get x(n) as a function of x(n-2). Repeating the step for x(n-2) yields an
expression as a function of x(n-3).
For example,

48
Chapter 13- Algorithm Design Techniques

X(n)=x(n-1)+n and x(0)=0


Derive this,
X(n)=[x(n-2)+n-1]+n
=x(n-2)+2n-1
=[x(n-3)+n-2]+2n-1
=x(n-3)+3n-(1+2)
=x(n-4)+4n-(1+2+3)
…………….
For i terms,
X(n)=x(n-i)+i.n-(1+2+………(i-1))
Substitute i=n
X(n)=x(0)+n.n-(1+2+……..n-1)
=n2-[n.(n-1)/2]
=n(n+1)/2

13.10. Introduction to NP-Complete Problems

The complexity class NP-complete (abbreviated NP-C or NPC) is a class of decision problems. A
decision problem L is NP-complete if it is in the set of NP problems and also in the set of NP-
hard problems. The abbreviation NP refers to "nondeterministic polynomial time."

NP-complete is a subset of NP, the set of all decision problems whose solutions can be verified
in polynomial time; NP may be equivalently defined as the set of decision problems that can be
solved in polynomial time on a nondeterministic Turing machine. A problem p in NP is also NP-
complete if every other problem in NP can be transformed into p in polynomial time. NP-
complete can also be used as an adjective: problems in the class NP-complete are known as NP-
complete problems.

NP-complete problems are studied because the ability to quickly verify solutions to a problem
(NP) seems to correlate with the ability to quickly solve that problem (P). It is not known
whether every problem in NP can be quickly solved—this is called the P = NP problem. But if
any single problem in NP-complete can be solved quickly, then every problem in NP can also be
quickly solved, because the definition of an NP-complete problem states that every problem in
NP must be quickly reducible to every problem in NP-complete (that is, it can be reduced in
polynomial time).
NP-completeness
A decision problem C is NP-complete if:
 C is in NP, and
 Every problem in NP is reducible to C n polynomial time.

49
Chapter 13- Algorithm Design Techniques

Example for NP-Completeness


 Boolean satisfiability problem (Sat.)
 Knapsack problem
 Hamiltonian path problem
 Travelling salesman problem
 Sub graph isomorphism problem
 Subset sum problem
 Clique problem
 Vertex cover problem
 Independent set problem
 Dominating set problem
 Graph coloring problem

Solving NP-complete problems


The following techniques can be applied to solve computational problems in general, and they
often give rise to substantially faster algorithms:
 Approximation: Instead of searching for an optimal solution, search for an "almost"
optimal one.
 Randomization: Use randomness to get a faster average running time, and allow the
algorithm to fail with some small probability. Note: The Monte Carlo method is not an
example of an efficient algorithm, although evolutionary approaches like Genetic
algorithms may be.
 Restriction: By restricting the structure of the input (e.g., to planar graphs), faster
algorithms are usually possible.
 Parameterization: Often there are fast algorithms if certain parameters of the input are
fixed.
 Heuristic: An algorithm that works "reasonably well" in many cases, but for which there
is no proof that it is both always fast and always produces a good result. Metaheuristic
approaches are often used.

What is NP?
NP is the set of all decision problems (question with yes-or-no answer) for which the 'yes'-
answers can be verified in polynomial time (O(n^k) where n is the problem size, and k is a
constant) by a deterministic Turing machine. Polynomial time is sometimes used as the definition
of fast or quickly.

50
Chapter 13- Algorithm Design Techniques

What is P?
P is the set of all decision problems which can be solved in polynomial time by a deterministic
Turing machine. Since it can solve in polynomial time, it can also be verified in polynomial time.
Therefore P is a subset of NP.
What is NP-Complete?
A problem x that is in NP is also in NP-Complete if and only if every other problem in NP can be
quickly (ie. in polynomial time) transformed into x. In other words:
1. x is in NP, and
2. Every problem in NP is reducible to x

So what makes NP-Complete so interesting is that if any one of the NP-Complete problems was
to be solved quickly then all NP problems can be solved quickly.

What is NP-Hard?
NP-Hard are problems that are at least as hard as the hardest problems in NP. Note that NP-
Complete problems are also NP-hard. However not all NP-hard problems are NP (or even a
decision problem), despite having 'NP' as a prefix. That is the NP in NP-hard does not mean
'non-deterministic polynomial time'. Yes this is confusing but its usage is entrenched and unlikely
to change.

13.11 Summary

This chapter illustrates the algorithm design techniques found in algorithm design. When
confronted with a problem, it is worthwhile to see if any of these methods apply. A proper choice
of algorithm, combined with judicious use of data structures, can often lead quickly to efficient
solutions.
13.12 Key Terms

Greedy, divide and conquer, backtracking, Branch and bound, NP-complete, dynamic
programming, Randomized algorithm
13.13 Review Questions ???????????

Two Mark Questions


1. What are the two factors to measure the efficiency of an algorithm?
2. How to measure the running time?
3. Define greedy algorithm.
4. What are the applications of greedy method?

51
Chapter 13- Algorithm Design Techniques

5. What do you mean by divide and conquer?


6. Write the examples of Divide and Conquer.
7. Write down the formula for finding the time complexity using divide and conquer.
8. Define dynamic programming.
9. What are the applications of dynamic programming?
10. Differentiate divide and conquer & dynamic programming.
11. Differentiate Greedy and dynamic programming.
12. Define backtracking. Name any 4 methods that are using backtracking.
13. What is NP-Hard & NP-Complete?
14. What is state space tree?
15. Define promising and non-promising node.
16. Define branch and bound algorithm.
17. Give the examples for branch and bound algorithm.
18. Name the three asymptotic Notations.
19. Define Big-Oh, Big-Omega, Big Theta Notation
20. Define randomized algorithm
21. Define skiplist.
22. Define NP-completeness.
Big Questions
1. Define greedy algorithm. What are the applications of greedy algorithm? Show the operation
of all of the bin-packing strategies (First Fit, Next Fit, Best Fit) on the input 0.42, 0.25, 0.27,
0.07, 0.72, 0.86, 0.09, 0.44, 0.50, 0.68, 0.73, 0.31, 0.78, 0.17, 0.79, 0.37, 0.73, 0.23, 0.30.
2. Define dynamic programming. Apply Floyd’s algorithm to find the all pairs shortest path for
the following graph. Write an algorithm for floyd’s algorithm.

3. A file contains only colons, spaces, newline, commas, and digits in the following frequency:
colon (100), space (605), newline (100), commas (705), 0 (431), 1 (242), 2 (176),
3 (59), 4(185), 5 (250), 6 (174), 7 (199), 8 (205), 9 (217). Construct the Huffman code.
4. Consider the problem of placing eight queens on an (eight by eight) chess board. Two queens
are said to attack each other if they are the same row, or (nor necessarily main) diagonal.
Formulate a backtracking algorithm to solve this problem.
5. Write a routine to find the maximum and minimum items in a set of ‘n’ elements using divide
and conquer.

52
Chapter 13- Algorithm Design Techniques

6. Describe the greedy algorithm used to minimize the mean completion time of multiprocessor
job scheduling.

53

Você também pode gostar