Escolar Documentos
Profissional Documentos
Cultura Documentos
4
Chapter 13- Algorithm Design Techniques
13.2.1. Definition
Greedy method is an approach to find the optimized solutions for the given problem. The
solutions are constructed through a sequence of steps, each step expanding a partially constructed
solution so far, until a complete solution to the problem is reached. Examples are:
A Simple Scheduling Problem
Huffman Codes
Approximate Bin Packing
5
Chapter 13- Algorithm Design Techniques
j2 8
j3 3
j4 10
One possible schedule is shown in Figure 13.1(a). Because j1 finishes in 15 (time units), j2 in 23,
j3 in 26, and j4 in 36. The average completion time = (15+23+26+36)/4= 25.
A better schedule, which yields a mean completion time of 17.75, is shown in Figure 13.1(b).
The schedule given in Figure is arranged by shortest job first. We can show that this will always
yield an optimal schedule.
6
Chapter 13- Algorithm Design Techniques
The algorithm to solve the multiprocessor case is to start jobs in order, cycling through
processors. It is not hard to show that no other ordering can do better, although if the number of
processors P evenly divides the number of jobs n, there are many optimal orderings. The
following figure 13.3 shows the best optimal solution for the multiprocessor job scheduling.
Huffman's scheme uses a table of frequency of occurrence for each symbol (or character) in the
input. This table may be derived from the input itself or from data which is representative of the
input. For instance, the frequency of occurrence of letters in normal English might be derived
from processing a large number of text documents and then used for encoding all text
documents. We then need to assign a variable-length bit string to each character that
unambiguously represents that character.
The technique works by creating a binary tree of nodes. These can be stored in a regular array
and the size of which depends on the number of symbols. A node can be either a leaf node or an
internal node. Initially, all nodes are leaf nodes, which contain the symbol itself, the weight
(frequency of appearance) of the symbol and optionally, a link to a parent node which makes it
easy to read the code (in reverse) starting from a leaf node. Internal nodes contain symbol
7
Chapter 13- Algorithm Design Techniques
weight, links to two child nodes and the optional link to a parent node. As a common
convention, bit '0' represents following the left child and bit '1' represents following the right
child.
The process essentially begins with the leaf nodes containing the probabilities of the symbol they
represent, and then a new node whose children are the 2 nodes with smallest probability is
created, such that the new node's probability is equal to the sum of the children's probability.
With the previous 2 nodes merged into one node (thus not considering them anymore), and with
the new node being now considered, the procedure is repeated until only one node remains, the
Huffman tree. The simplest construction algorithm uses a priority queue where the node with
lowest probability is given highest priority:
Algorithm
Create a leaf node for each symbol and add it to the priority queue.
While there is more than one node in the queue:
Remove the two nodes of highest priority (lowest probability) from the queue
Create a new internal node with these two nodes as children and with probability equal to the
sum of the two nodes' probabilities.
Add the new node to the queue.
The remaining node is the root node and the tree is complete
Since efficient priority queue data structures require O(log n) time per insertion, and a tree with
n leaves has 2n−1 nodes, this algorithm operates in O(n log n) time, where n is the number of
symbols.
Example
Suppose that a file contains 13 different characters, each of one having different frequency
occurrences that are shown below. Totally we have 838 characters. Each character that occupies
4 bit length. Now, we have a total of 838 × 4 = 3352 bits. To reduce the bit length, we use
Huffman algorithm. The details are shown below.
Character E T A O I N S R H L D C U
Frequency 125 93 80 76 73 71 65 61 55 41 40 31 27
8
Chapter 13- Algorithm Design Techniques
Initially, we start with low frequency occurrence characters C with frequency 31 and U with
frequency 27 for a total of 58. Now construct a tree structure with the characters are placed in the
leaf node and the result is stored in the root node. The tree structure is shown below.
Next we take D and L with a total of 81. The tree diagram is shown below.
Next, we take H with the frequency 55 and another node will be 58(the result of C & U). Draw
the tree diagram with the two nodes that are given below.
Now, we take R & S, since the frequencies are 61 & 65. Add these nodes to the tree structure
with the result of 126.
9
Chapter 13- Algorithm Design Techniques
Repeat the process by adding the nodes the lowest frequencies and form the complete Huffman
tree structure. The final picture is shown in the following figure 13.4
10
Chapter 13- Algorithm Design Techniques
The complete bit representation for the characters which are consolidated and the details are
shown in the figure 13.5. The complete Huffman tree Structure is represented in figure 13.6.
Here, every character that occupies different number of bit patterns and the total space that
needed is reduced.
Given items with sizes s1, . . . , s n € (0, 1), pack them into the fewest number of bins possible,
where each bin is of size 1. As an example, we have an item with the weights 0.2, 0.5, 0.4, 0.7,
0.1, 0.3, 0.8. Now, we pack the items to the bin with the total capacity of size 1 in order to use
less number of bins. The figure 13.7 shows an approximate bin packing.
11
Chapter 13- Algorithm Design Techniques
On-line Algorithms - Each item must be placed in a bin before the next item can be
processed. In online algorithm, we have Next Fit, First Fit & Best Fit algorithm
Off-line Algorithm - do not need to do anything until all the input has been read
Figure 13.8 – Next Fit for 0.2, 0.5, 0.4, 0.7, 0.1, 0.3, and 0.8
Figure 13.8 shows the next fit bin packing for the given weights. First, we take the weights 0.2
and then 0.5 in the order it comes and place it in the first bin. Now the total weight is 0.7. Add
the next weight 0.4, it exceeds the total bin weight of 1.0. So add the weight 0.4 to the next bin
and continue the process. For the given weights, there are 5 bins are needed to place the weights.
12
Chapter 13- Algorithm Design Techniques
Figure 13.9 – First Fit for 0.2, 0.5, 0.4, 0.7, 0.1, 0.3, and 0.8
Best Fit Online Algorithm
Instead of placing a new item in the first spot that is found, it is placed in the tightest spot among
all bins.
Figure 13.10 – Best Fit for 0.2, 0.5, 0.4, 0.7, 0.1, 0.3, and 0.8
13.3.1. Definition
The divide-and-conquer strategy solves a problem by:
1. Breaking it into sub problems that are themselves smaller instances of the same type of
problem
2. Recursively solving these sub problems
3. Appropriately combining their answers
Examples
Closest-Points Problem
Quick Sort
13
Chapter 13- Algorithm Design Techniques
Merge Sort
Binary Search
The figure 13.11 shows the closest points problem, which is used to find the pair of points
which are very closer than other pair of points. This is done by dividing the given region into
two equal halves and identifies the closest points in the left region and also in the right region.
Select the points which are very closer by comparing the points in the left and right region. This
14
Chapter 13- Algorithm Design Techniques
is a recursive process by dividing the region further and further and selects the points which are
very closer.
Algorithm
Algorithm is quite simple. It can be done either recursively or iteratively:
1. get the middle element;
2. if the middle element equals to the searched value, the algorithm stops;
3. otherwise, two cases are possible:
o Searched value is less, than the middle element. In this case, go to the step 1 for the part
of the array, before middle element.
o Searched value is greater, than the middle element. In this case, go to the step 1 for the
part of the
intarray, after middlearr[],
binarySearch(int element.
int value, int left, int right)
while (left <= right) do
Procedure int middle = (left + right) / 2;
if (arr[middle] == value)
return middle;
else if (arr[middle] > value)
right = middle - 1;
else
15
left = middle + 1;
return -1;
Chapter 13- Algorithm Design Techniques
Example
Analysis
The recurrence relation for binary search,
C(n)=C(n/2)+1 for n>1, C(1)=1
Substitute n=2k then we get,
C(2k)=k+1=log2n+1
C(n)= log2n +1
= log2n (n+1)
If n=2i then
C(n)= log2n +1
= log2n 2i+1
= log22+ log2i+1
=1+ log2i+1= log2i+2
Complexity
The running time of binary search is 1+logn
13.4.1. Definition
Dynamic programming is a method for efficiently solving a broad range of search and
optimization problems which exhibit the characteristics of overlapping sub problems and optimal
substructure.
16
Chapter 13- Algorithm Design Techniques
The idea behind dynamic programming is quite simple. In general, to solve a given problem, we
need to solve different parts of the problem (sub problems), and then combine the solutions of
the sub problems to reach an overall solution. Dynamic programming algorithms are used for
optimization (for example, finding the shortest path between two points, or the fastest way to
multiply many matrices). A dynamic programming algorithm will examine all possible ways to
solve the problem and will pick the best solution.
Advantages
Computing duplications in solutions is avoided totally
Many decision sequences are generated and all the overlapping sub instances are
considered.
Examples
Using a Table Instead of Recursion
Ordering Matrix Multiplications
Optimal Binary Search Tree
All-Pairs Shortest Path
Multistage graphs
17
Chapter 13- Algorithm Design Techniques
To overcome the problem of recursive method, we are using table method. In this method, the
solution to the sub problem to be stored and reused from the first time computation itself. A one
dimensional array of n+ 1 consecutive location is used to store the sub problem values. The i th
Fibonacci value is computed by adding the last to locations. This is an advantage of dynamic
programming, since it avoids the re-computation of same function for many times. The algorithm
for table method is shown below.
18
Chapter 13- Algorithm Design Techniques
and thus evaluated, in any order. The obvious way to multiply two matrices of dimensions p X q
and q X r, respectively, uses pqr scalar multiplications.
In the case of four matrices, it is simple to solve the problem by exhaustive search, since there
are only five ways to order the multiplications. We evaluate each case below:
(A((BC)D)): Evaluating BC requires 10 X 40 X 30 = 12,000 multiplications. Evaluating (BC)D
requires the 12,000 multiplications to compute BC, plus an additional 10 X 30 X 5 = 1,500
multiplications, for a total of 13,500. Evaluating (A((BC)D) requires 13,500 multiplications for
(BC)D, plus an additional 50 X 10 X 5 = 2,500 multiplications, for a grand total of 16,000
multiplications.
(A(B(CD))): Evaluating CD requires 40 X 30 X 5 = 6,000 multiplications. Evaluating B(CD)
requires 6,000 multiplications to compute CD, plus an additional 10 X 40 X 5 = 2,000
multiplications, for a total of 8,000. Evaluating (A(B(CD)) requires 8,000 multiplications for B
(CD), plus an additional 50 X 10 X 5 = 2,500 multiplications, for a grand total of 10,500
multiplications.
((AB)(CD)): Evaluating CD requires 40 X 30 X 5 = 6,000 multiplications. Evaluating AB
requires 50 X 10 X 40 = 20,000 multiplications. Evaluating ((AB)(CD)) requires 6,000
multiplications for CD, 20,000 multiplications for AB, plus an additional 50 X 40 X 5 = 10,000
multiplications for a grand total of 36,000 multiplications.
(((AB)C)D): Evaluating AB requires 50 X 10 X 40 = 20,000 multiplications. Evaluating (AB)C
requires the 20,000 multiplications to compute AB, plus an additional 50 X 40 X 30 = 60,000
multiplications, for a total of 80,000. Evaluating (((AB)C)D) requires 80,000 multiplications for
(AB)C, plus an additional 50 X 30 X 5 = 7,500 multiplications, for a grand total of 87,500
multiplications.
((A(BC))D): Evaluating BC requires 10 X 40 X 30 = 12,000 multiplications. Evaluating A(BC)
requires the 12,000 multiplications to compute BC, plus an additional 50 X 10 X 30 = 15,000
multiplications, for a total of 27,000. Evaluating ((A(BC))D) requires 27,000 multiplications for
A(BC), plus an additional 50 X 30 X 5 = 7,500 multiplications, for a grand total of 34,500
multiplications.
From this different ordering, (A(B(CD))) order of matrix multiplication that produces an optimal
solutions by reducing the number of multiplications.
19
Chapter 13- Algorithm Design Techniques
• Catalan number c(n)= C(2n,n)/(n+1) is used to find the number of different possible
binary search tree for n nodes.
Example
Suppose if we have 4 nodes 3, 7, 9, & 12 then we can construct 14 different binary search trees.
The nodes and the probability values are shown below. Some of these tree structures which are
shown in the figure 13.14.
Keys 3 7 9 12
Probabilities 0.1 0.2 0.4 0.3
Suppose for the tree (b), the average key comparisons =0.2(1)+0.1(2)+0.3(2)+0.4(3)=2.2
Then identify the optimal binary search trees by calculating the average number of key
comparisons for all those tree structure and select the search tree with minimum values.
20
Chapter 13- Algorithm Design Techniques
Let C(i, j) denote the cost of an optimal binary search tree containing a i,…,aj . The cost of the
optimal binary search tree with ak as its root :
C[i,j]=mini≤k≤j { Pk.1+ s
+ s
= mini≤k≤j { Pk + s
+ s }
= mini≤k≤j { Pk+ s s + }
= mini≤k≤j {C [i,k-1]+c[k+1,j]} +
Algorithm
OptimalBST(P[1…n])
for i 1 to n do
C[i, i-1] 0
C[i, i] P[i]
R[i, i] i
C[n+1, n] 0
for d 1 to n-1 do
for i 1 to n-d do
Ji+d
minval ∞
For k i to j do
If C[ i, k-1 ] + C[ k+1, j ] < minval
minval C[ i, k-1 ] + C[ k+1, j ]; kmin <- k
R[ i , j] kmin
Sum P[i];
for s i +1 to j do
sum sum + P[s] 21
C[ i, j] minval + sum
return C[ 1, n], R
Chapter 13- Algorithm Design Techniques
Time complexity
The running time of optimal binary search tree is O(n 3)
Floyd's Algorithm
The Floyd’s algorithm compares all possible paths through the graph between each pair of
vertices. It is able to do this with only Θ(|V|3) comparisons in a graph. This is remarkable
considering that there may be up to Ω(|V|2) edges in the graph, and every combination of edges is
tested. It does so by incrementally improving an estimate on the shortest path between two
vertices, until the estimate is optimal.
Algorithm
22
Chapter 13- Algorithm Design Techniques
Step 3: If k = n, the algorithm is finished. If k < n, increase k by 1, i.e. K = k+1 and then return
to step 2.
Example
All elements along the main diagonal of matrix D0 equal zero since by definition Dij0=0 for i=j.
We note element d1,20 of matrix D0. This element equals 8 since the length of the branch
connecting nodes 1 and 2 is 8. Element d3,10 equals infinity since the network has no branch
which is oriented from node 3 to node 1. Element D 5,10 of matrix D0 equals infinity as well since
there is direct branch linking nodes 5 and 1. We now go to the first algorithmic step. Let k = 1.
As an illustration of Step 2 we calculate the elements of the first two rows of matrix D1.
D1,21=min{D1,20,D1,10+D1,20}=min{8,0+8}=8
D1,31=min{D1,30,D1,10+D1,30}=min{3,0+8}=3
D1,41=min{D1,40,D1,10+D1,40}=min{5,0+5}=5
D1,51=min{D1,50,D1,10+D1,50}=min{∞,0+∞}=∞
D2,11=min{D2,10,D2,10+D1,10}=min{8,0+8}=8
D2,31=min{D2,30,D2,10+D1,30}=min{2,8+3}=2
D2,41=min{D2.40,D2,10+D1,40}=min{∞,8+5}=13
D2,51=min{D2,50,D2,10+D1,50}=min{5,8+∞}=5
23
Chapter 13- Algorithm Design Techniques
Repeat the process to calculate the values for next three rows of D(1). Now the matrix D(1) is,
After the second, third, fourth and fifth passages through the algorithm, matrices D(2), D(3), D(4),
D(5) are as follows:
The distance matrix D(5) which produces the shortest distance between every pair of vertices.
This is an output of the algorithm.
24
Chapter 13- Algorithm Design Techniques
Forward approach
d(S,A) =1
d(S,B) =2
d(S,C) =7
d(S,D) =min{1+d(A,D),2+d(B,D)}
= min{1+3,2+4}
=4
d(S,E) = min{1+d(A,E),2+d(B,E),7+d(C,E)}
= min{1+6,2+10,7+3}
=7
d(S,T) = min{d(S,D)+d(D,T),d(S,E)+d(E,T),d(S,C)+d(C,T)}
= min{4+8,7+2,7+10}
=9
Now the minimum cost of the graph using forward method is 9 with the shortest path is S-A-E-T.
Algorithm – Forward Approach
Forward(G,stages,n,p)
cost[i] 0
For(in-2 downto 0) do
{
rGet_Min(i,n)
cost[i]cost[r]+c[i][r]
d[i]r
}
//finding minimum cost path
P[0]0
P[stages-1]n-1
For(i 1 to stages-1 ) do
P[i]d[p[i+1]];
Analysis
The time complexity of Multistage Graph using both Forward and Backward method is O(|V|+|
E|)
26
Chapter 13- Algorithm Design Techniques
13.5. Backtracking
13.5.1. Definition
Backtracking is a process of searching the solution to the problem by searching the solution
space (the set of all feasible solutions) for the given problem. Backtracking is a process where
steps are taken towards the final solution and the details are recorded. If these steps do not lead
to a solution some or all of them may have to be retraced and the relevant details discarded. In
these circumstances it is often necessary to search through a large number of possible situations
in search of feasible solutions.
Examples
The Turnpike Reconstruction Problem
Games (Eight Queen problem)
Subset Sum Problem
Graph Coloring
Since | D | = 15, we know that n = 6. We start the algorithm by setting x1 = 0. Clearly, x6 = 10,
since 10 is the largest element in D. We remove 10 from D. The points that we have placed and
the remaining distances are as shown in the following figure.
The next step is not obvious. Since 7 is the largest value in D, either x4 = 7 or x2 = 3. If x4 = 7,
then the distances x6 - 7 = 3 and x5 - 7 = 1 must also be present in D. we can come back and try
the other.
At this point, we have x1 = 0, x4 = 7, x5 = 8, and x6 = 10. Now the largest distance is 6, so either
x3 = 6 or x2 = 4. But if x3 = 6, then x4 - x3 = 1, which is impossible, since 1 is no longer in D.
On the other hand, if x2 = 4 then x2 - x0 = 4, and x5 - x2 = 4. This is also impossible, since 4
only appear once in D. Thus, this line of reasoning leaves no solution, so we backtrack. Since x4
= 7 failed to produce a solution, we try x2 = 3. If this also fails, we give up and report no
solution. We now have
28
Chapter 13- Algorithm Design Techniques
Figure 13.15 shows a decision tree representing the actions taken to arrive at the solution.
29
Chapter 13- Algorithm Design Techniques
Algorithm
void NQueens(int k, int n)
{
for (int i=1; i<=n; i++)
{
if (Place(k, i))
{
x[k] = i;
if (k==n)
{
for (int j=1;j<=n;j++)
cout << x[j] << ' '; cout << endl;
}
else
NQueens(k+1, n);
}
}
}
30
Chapter 13- Algorithm Design Techniques
Figure 13.17 shows the state space tree of 4-queen problem. A node in the state space tree is
promising if it corresponds to the partials constructed solution that may lead to the complete
solution otherwise the nodes are called non-promising. Leaves of the tree represent either the
non- promising dead end or complete solution found by the algorithm.
Example
The figure 13.18 shows the pruned state space tree when backtracking is used with n = 4, W =
13, and w1 = 3, w2 = 4, w3 = 5, w4 = 6
Algorithm
Number the solution variables [v0 v1, …, vn-1].
Number the possible values for each variable [c0 c1, …, ck-1].
Start by assigning c0 to each vi.
If we have an acceptable solution, stop.
If the current solution is not acceptable, let i = n-1.
If i < 0, stop and signal that no solution is possible.
Let j be the index such that vi = cj. If j < k-1, assign cj+1 to vi and go back to step 4.
But if j ≥ k-1, assign c0 to vi, decrement i, and go back to step 6.
33
Chapter 13- Algorithm Design Techniques
13.6.1. Definition
A B&B algorithm searches the complete space of solutions for a given problem for the best
solution.
Ø Branch and bound is a systematic method for solving optimization problems
Ø B&B is a rather general optimization technique that applies where the greedy method and
dynamic programming fail.
Ø However, it is much slower.
Ø On the other hand, if applied carefully, it can lead to algorithms that run reasonably fast on
average.
Ø The general idea of B&B is a BFS-like search for the optimal solution, but not all nodes get
expanded (i.e., their children generated). Rather, a carefully selected criterion determines which
node to expand and when, and another criterion tells the algorithm when an optimal solution has
been found.
Examples
Traveling salesman problem
Knapsack problem
However, this particular version of the problem is the optimization problem (“find the lowest
cost”). There is another version of the problem, the decision problem, which is “Given a graph G
and a cost k, decide if there is a Hamiltonian circuit whose cost ≤k.” The decision problem, the
one which requires a cost to be given and simply decides if there is a cheaper (or equal)
Hamiltonian circuit, is NP-Complete. But almost everyone who talks about “the Traveling
Salesperson Problem” is talking about the optimization problem, which is not NP-Complete, it is
NP-Hard.
Algorithm
For the given weighted graph, the distances between the vertices are shown. Now we find the
shortest path to visit all the vertices exactly once, and reach the source vertex. Initially, we start
with a as the source vertex. Next we find the lower bound value for a. The lower bound value lb
is calculated using the formula,
lb = ∑sum of costs of the two least cost edges adjacent to v
Now, the lower bound value for a =[(1+3)+(3+6)+(1+2)+(3+4)+(2+3)]/2 = 14
This is done by taking the average cost of all the vertices. For a, we have to take the edges 1 & 3,
since, these are the two lowest cost edges of a. Then for b, take the edges 3 & 6. For c, take 1 &
2, then for vertex d, 3 & 4 are the lowest cost edges and finally for e, take 2 & 3. Now we find
the average cost of the vertices by calculating the sum of all the cost and divided by 2. Then we
get 14 as the lower bound value.
Travelling Salesman Problem is one of the Branch and Bound technique. Here, we have to find
all possible solution and choose the optimal one. From the source vertex a, visit b or c or d or e.
Suppose if b is the next visited vertex, find the lower bound value for b.
lb for ab = [(1+3)+(3+6)+(1+2)+(3+4)+(2+3)]/2 = 14
In this case, we have to select the edge ab for calculating lower bound value for b. Suppose if c is
the next visited vertex, find the lower bound value for c by choosing the edge ac.
lb for ac = [(1+3)+(3+6)+(1+2)+(3+4)+(2+3)]/2 = 14
Suppose if d is the next visited vertex, find the lower bound value for d by choosing the edge ad.
lb for ad = [(1+5)+(3+6)+(1+2)+(3+5)+(2+3)]/2 = 16
Suppose if e is the next visited vertex, find the lower bound value for e by choosing the edge ae.
lb for ae = [(1+8)+(3+6)+(1+2)+(3+4)+(2+8)]/2 = 19
Now compare the lb value of ab, ac, ad and ae. Choose the lowest lb value of the vertex i.e., ab.
Next we choose the vertex from b. There are three ways to go for the next vertex. One way is a
bc. Another way is a bd. The next way is a b e. Find the lb value of these three ways
and choose the lowest lb value. Continue the process, and finally reach the source vertex a. The
entire process that are shown in the following figure 13.20.
35
Chapter 13- Algorithm Design Techniques
36
Chapter 13- Algorithm Design Techniques
Solution
Initially, without any item included, the total weight W=10, w=0 and v=0. Now, we find the
upper bound value for the node 0.
ub for node 1 = 0+(10-0)(10) =100
Next, find the upper bound value by including the first item 1 with their value v =40 and weight
w = 4.
ub for node 2 = 40 +(10-4)(6) = 76
Alternatively, find the upper bound value without including the first item 1. Here, we are not
including the value and weight of the first item.
ub for node 2 = 0+(10-0)(6)=60
There are two possible solutions are there. Suppose if we include the first item, the upper bound
value is 76 and 60 if we are not including the first item. Choose the highest upper bound value
i.e., 76 and include the next item.
In this algorithm, whenever w that exceeds the total weight W=10, then exclude the link for the
process. Next include the item 2, the upper bound value is calculated by add the value and
weight of item 2. In this case weight w=4+7=11 that exceeds the total weight W=10. This is not a
feasible solution. Now without including the second item,
ub for item 4 = 40+(10-4)(5)= 70
This is the only possible solution. Next, we have to include the third item and find the upper
bound value. Repeat the process until we include all the items an d find the upper bound vale.
For this example, the optimal solution is (1,0,3,0).
37
Chapter 13- Algorithm Design Techniques
38
Chapter 13- Algorithm Design Techniques
It splits input at specific deterministic points It splits input at every possible split points
usually in the middle rather than at a particular point
39
Chapter 13- Algorithm Design Techniques
solution to the problem is reached. solution and the details are recorded.
It is used for getting optimal solutions. It is used for getting the solutions to the given
problem that may or may not be optimal.
A set of feasible solutions and picks up the No special set of feasible solutions
optimal solution
Solution is generated without revising It considers the possible sequences in order to
previously generated solutions obtain the solution
Example: Minimum Spanning Tree, Finding Example: N-queen problem, subset sum
shortest path problem
An algorithm takes a source of random numbers and makes random choices during execution in
addition to the input. Behavior can vary even on a fixed input.
13.8.1. Definition
Pseudorandom numbers
The computer is not capable of generating truly random numbers
40
Chapter 13- Algorithm Design Techniques
(a)
(b)
41
Chapter 13- Algorithm Design Techniques
(c)
(d)
(e)
Figure 13.22 – (a) simple linked list (b) linked list with pointers to two cells ahead (c) linked list with
pointers to four cells ahead (d) linked list with pointers to 2i cells ahead (e ) Skip List
42
Chapter 13- Algorithm Design Techniques
Time: The amount of time to take executes the algorithm. If the algorithm takes less
amount of time to execute then this one will be the best one.
Space: The amount of memory required to store the algorithm and the amount of memory
required to store the input for that algorithm.
Simplicity: The algorithm should not have any complex instructions. Those types of
instructions are simplified to number of smaller instructions.
Generality: The algorithm should be in general. It should be implemented in any
language. The algorithm is not fit in to a specific output.
43
Chapter 13- Algorithm Design Techniques
Where,
Cop – Constant (The amount of time required to execute the basic operation)
C(n) – How many times the basic operation is executed.
Asymptotic complexity is a way of expressing the main component of the cost of an algorithm,
using idealized units of computational work. Consider, for example, the algorithm for sorting a
deck of cards, which proceeds by repeatedly searching through the deck for the lowest card. The
asymptotic complexity of this algorithm is the square of the number of cards in the deck. This
quadratic behavior is the main term in the complexity formula, it says, e.g., if you double the size
of the deck, then the work is roughly quadrupled.
44
Chapter 13- Algorithm Design Techniques
Example 1
2n2 = O(n3)
0 ≤ t(n) ≤ cg(n) Definition of O(g(n))
0 ≤ 2n2 ≤ cn3 Substitute
0/n3 ≤ 2n2/n3 ≤ cn3/n3 Divide by n3
Determine c
Lim
0 ≤ 2/n ≤ c n 2/n = 0
2/n maximum when n=1
0 ≤ 2/1 ≤ c = 2 Satisfied by c=2
Determine n0
0 ≤ 2/n0 ≤ 2
0 ≤ 2/2 ≤ n0
0 ≤ 1 ≤ n0 = 1 and n0=1
0 ≤ 2n2 ≤ 2n3 ∀ n ≥ n0=1
Example 2
1000n2 + 50n = O(n2)
0 ≤ t(n) ≤ cg(n)
-1000 ≤ 50/n0 ≤ 50
-20 ≤ 1/n0 ≤ 1
-20n0 ≤ 1 ≤ n0 = 1 n0=1
45
Chapter 13- Algorithm Design Techniques
A function t(n) is said to be in Ω (g(n)), denote t(n) € Ω (g(n)), if t(n) is bounded below by some
constant multiple of g(n) for all large n, i.e., if there exists some positive constant c and some
nonnegative integer n0 such that t(n) ≥ c.g(n) for all n ≥ n0.
Example 1
n3 = Ω(n2)
n3 = Ω(n2) with c=1 and n0=1
0 ≤ cg(n) ≤ t(n)
0 ≤ 1*12 = 1 ≤ 1 = 13
0 ≤ cg(n) ≤ t(n)
0 ≤ cn2 ≤ n3
0 ≤ c≤ n
0 ≤ cn2 ≤ n
(iii) Big–Theta (or) θ()
0/n2 ≤ cn2/n2 ≤ n/n2 Notation
0 ≤ c ≤ 1/n
Example
n2/2-2n = Θ(n2)
c1g(n) ≤ t(n) ≤ c2g(n)
c1n2 ≤ n2/2-2n ≤ c2n2
c1n2/n2 ≤ n2/2n2-2n/n2 ≤ c2n2/n2 Divide by n2
c1 ≤ 1/2-2/n ≤ c2
O: Determine c2 = ½
½-2/n ≤ c2 = ½ ½-2/n = ½
maximum of ½-2/n
Ω: Determine c1 = 1/10
0 < c1 ≤ 1/2-2/n 0 < c1 minimum when n=5
0 < c1 ≤ 1/2-2/5
0 < c1 ≤ 5/10-4/10 = 1/10
n0 : Determine n0 = 5
1/10 ≤ 1/2-2/n0 ≤ 1/2
1/10 ≤ 1/2-2/n0 ≤ 1/2
Verify
c1n2 ≤ n2/2-2n ≤ c2n2 with c1=1/10, c2=½ and n0=5
2 2 2
1/10*5 ≤ 5 /2-2*5 ≤ ½*5
25/10 ≤ 25/2-20/2 ≤ 25/2
5/2 ≤ 5/2 ≤ 25/2 Holds
2 2 2
In general: 1/10n ≤ n /2-2n ≤ ½n for n ≥ 5
47
Chapter 13- Algorithm Design Techniques
A recurrence relation is an equation that recursively defines a sequence, once one or more initial
terms are given: each further term of the sequence is defined as a function of the preceding
terms.
For example, consider the sequence
x(n)=x(n-1)+n for n>0
x(0)=0
This is an example of a recurrence relation with initial condition. A initial condition can be given
for a value on n other than 0 and for some recurrences, more than one value needs to be specified
by initial conditions.
To solve a given recurrences subject to a given initial condition means to find an explicit formula
for the generic term of the sequence that satisfies both the recurrence equation and the initial
condition or to prove that such a sequence does not exist. A general solution is a formula that
specifies all such sequences.
48
Chapter 13- Algorithm Design Techniques
The complexity class NP-complete (abbreviated NP-C or NPC) is a class of decision problems. A
decision problem L is NP-complete if it is in the set of NP problems and also in the set of NP-
hard problems. The abbreviation NP refers to "nondeterministic polynomial time."
NP-complete is a subset of NP, the set of all decision problems whose solutions can be verified
in polynomial time; NP may be equivalently defined as the set of decision problems that can be
solved in polynomial time on a nondeterministic Turing machine. A problem p in NP is also NP-
complete if every other problem in NP can be transformed into p in polynomial time. NP-
complete can also be used as an adjective: problems in the class NP-complete are known as NP-
complete problems.
NP-complete problems are studied because the ability to quickly verify solutions to a problem
(NP) seems to correlate with the ability to quickly solve that problem (P). It is not known
whether every problem in NP can be quickly solved—this is called the P = NP problem. But if
any single problem in NP-complete can be solved quickly, then every problem in NP can also be
quickly solved, because the definition of an NP-complete problem states that every problem in
NP must be quickly reducible to every problem in NP-complete (that is, it can be reduced in
polynomial time).
NP-completeness
A decision problem C is NP-complete if:
C is in NP, and
Every problem in NP is reducible to C n polynomial time.
49
Chapter 13- Algorithm Design Techniques
What is NP?
NP is the set of all decision problems (question with yes-or-no answer) for which the 'yes'-
answers can be verified in polynomial time (O(n^k) where n is the problem size, and k is a
constant) by a deterministic Turing machine. Polynomial time is sometimes used as the definition
of fast or quickly.
50
Chapter 13- Algorithm Design Techniques
What is P?
P is the set of all decision problems which can be solved in polynomial time by a deterministic
Turing machine. Since it can solve in polynomial time, it can also be verified in polynomial time.
Therefore P is a subset of NP.
What is NP-Complete?
A problem x that is in NP is also in NP-Complete if and only if every other problem in NP can be
quickly (ie. in polynomial time) transformed into x. In other words:
1. x is in NP, and
2. Every problem in NP is reducible to x
So what makes NP-Complete so interesting is that if any one of the NP-Complete problems was
to be solved quickly then all NP problems can be solved quickly.
What is NP-Hard?
NP-Hard are problems that are at least as hard as the hardest problems in NP. Note that NP-
Complete problems are also NP-hard. However not all NP-hard problems are NP (or even a
decision problem), despite having 'NP' as a prefix. That is the NP in NP-hard does not mean
'non-deterministic polynomial time'. Yes this is confusing but its usage is entrenched and unlikely
to change.
13.11 Summary
This chapter illustrates the algorithm design techniques found in algorithm design. When
confronted with a problem, it is worthwhile to see if any of these methods apply. A proper choice
of algorithm, combined with judicious use of data structures, can often lead quickly to efficient
solutions.
13.12 Key Terms
Greedy, divide and conquer, backtracking, Branch and bound, NP-complete, dynamic
programming, Randomized algorithm
13.13 Review Questions ???????????
51
Chapter 13- Algorithm Design Techniques
3. A file contains only colons, spaces, newline, commas, and digits in the following frequency:
colon (100), space (605), newline (100), commas (705), 0 (431), 1 (242), 2 (176),
3 (59), 4(185), 5 (250), 6 (174), 7 (199), 8 (205), 9 (217). Construct the Huffman code.
4. Consider the problem of placing eight queens on an (eight by eight) chess board. Two queens
are said to attack each other if they are the same row, or (nor necessarily main) diagonal.
Formulate a backtracking algorithm to solve this problem.
5. Write a routine to find the maximum and minimum items in a set of ‘n’ elements using divide
and conquer.
52
Chapter 13- Algorithm Design Techniques
6. Describe the greedy algorithm used to minimize the mean completion time of multiprocessor
job scheduling.
53