Unit-V

Traversing Trees
Depth first search
Breadth first search
Back tracking
The Knapsack Problem(3)
The Eight Queens problem
The General Templates
Branch and Bound
The assignment problem
The knapsack problpme(4)
General Consideration
The Minimax principle
Introduction to NP Completeness
Classes P and NP
Polynomial reductions
NP-Complete Problems
NP-Hard Problems
Non Deterministic algorithms.
 Part-I
Exploring Graphs
Breadth First Search

(BFS)(G)
{
list L = empty
tree T = empty
choose a starting vertex x
Search (x) while (L nonempty)
remove edge (v, w) from start of L
if w not yet visited
{
add (v, w) to T
Search (w)
}
}
04/12/17 3
Depth First Search

DFS(G)
{
list L = empty
tree T = empty
choose a starting vertex x
Search (x) while (L nonempty)
remove edge (v, w) from end of L
if w not yet visited
{
add (v, w) to T
Search (w)
}
}

04/12/17 4
Relation between
Both of these BFS and
search algorithms DFS
now keep a list of edges to
explore;
the only difference between the two is that, while both
algorithms adds items to the end of L,
BFS removes them from the beginning, which results in
maintaining the list as a queue,
while DFS removes them from the end, maintaining the list
as a stack.

04/12/17 5
BackTracking

The Knapsack Problem(3)

The Eight Queens problem
The General Templates

04/12/17 6
Backtracking
Suppose you have to make a series of decisions, among
various choices, where
You dont have enough information to know what to
choose
Each decision leads to a new set of choices

Some sequence of choices (possibly more than one)

may be a solution to your problem
Backtracking is a methodical way of trying out various
sequences of decisions, until you find one that works

04/12/17 7
Backtracking (animation)
dead end
?
dead end
dead end

?
start ? ? dead end
dead end
?
success!

04/12/17 8
The backtracking algorithm

Backtracking is really quite simple--we

explore each node, as follows:
To explore node N:

1. If N is a goal node, return success

2. If N is a leaf node, return failure
3. For each child C of N,
3.1. Explore C
3.1.1. If C was successful, return success
4. Return failure

04/12/17 9
Backtracking
The principal idea is to construct solutions one component
at a time and evaluate such partially constructed candidates
as follows.

If a partially constructed solution can be developed further

without violating the problems constraints, it is done by
taking the first remaining legitimate option for the next
component. If there is no legitimate option for the next
component, no alternatives for any remaining component
need to be considered.

In this case, the algorithm backtracks to replace the last

component of the partially constructed solution with its next
option.

04/12/17 10
10
Backtracking
This kind of processing is often implemented by constructing a tree
of choices being made, called the state-space tree.

Its root represents an initial state before the search for a solution
begins.

The nodes of the first level in the tree represent the choices
made for the first component of a solution,

the nodes of the second level represent the choices for the second
component, and so on.

A node in a state-space tree is said to be promising if it

corresponds to a partially constructed solution that may still lead to a
complete solution; otherwise, it is called nonpromising.

04/12/17 11
11
Backtracking

Leaves represent either nonpromising

dead ends or complete solutions found
by the algorithm.

If the current node turns out to be

nonpromising, the algorithm backtracks
to the nodes parent to consider the next
possible option for its last component;

if there is no such option, it backtracks

one more level up the tree, and so on.

04/12/17 12
Backtracking
Construct the state-space tree
nodes: partial solutions

edges: choices in extending partial solutions

Explore the state space tree using depth-first search

Prune nonpromising nodes

DFS stops exploring sub trees rooted at nodes that
cannot lead to a solution and backtracks to such a
nodes parent to continue the search

04/12/17 13
The Knapsack Problem(3)
Consider of size K and we want to select from a set of n
objects , where the ith object has size si and value vi, a subset
of these objects to maximize the value contained in the
knapsack with the contents of the knapsack less than or equal
to K.
Suppose that K = 16 and n = 4, and we have the following set
of objects ordered by their value density.
i vi si vi/si
1 $45 3 $15
2 $30 5 $ 6
3 $45 9 $ 5
4 $10 5 $ 2

04/12/17 14
 We will construct the state space where each
node contains the total current value in the
knapsack, the total current size of the
contents of the knapsack, and maximum
potential value that the knapsack can hold. In
the algorithm, we will also keep a record of
the maximum value of any node (partially or
completely filled knapsack) found so far.
When we perform the depth-first traversal of
the state-space tree, a node is "promising" if
its maximum potential value is greater than
this current best value.
04/12/17 15
 We begin the state space tree with the root
consisting of the empty knapsack. The current
weight and value are obviously 0. To find the
maximum potential value we treat the problem as
if it were the fractional knapsack problem and we
were using the greedy algorithmic solution to that
problem. We have shown that the greedy
approach to the fractional knapsack problem
yields an optimal solution. We place each of the
remaining objects, in turn, into the knapsack until
the next selected object is too big to fit into the
knapsack. We then use the fractional amount of
that object that could be placed in the knapsack to
determine the maximum potential value.

04/12/17 16
 totalSize = currentSize + size of
remaining objects that can be fully
placed
bound (maximum potential value) =
current Value + value of remaining
objects fully placed +
(K - totalSize) * (value density of item
that is partially placed)

04/12/17 17
 In general, for a node at level i in the state space tree (the
first i items have been considered for selection) and for
the kth object as the one that will not completely fit into
the remaining space in the knapsack, these formulae can
be written:
k-1
totalSize = currentSize + sj
j=i+1

k-1
bound = currentValue + vj + (K - totalSize) *
(vk/sk)
j=i+1

04/12/17 18
 For the root node,
currentSize = 0, currentValue = 0

totalSize = 0 + s1 + s2 = 0 + 3 + 5 = 8
bound = 0 + v1 + v2 + (K - totalSize) * (v3/s3)
= 0 + $45 + $30 + (16 - 8) * ($5)
= $75 + $40
= $115

04/12/17 19
 From the root, we add two children at level 1
-- the node where the first item is included in
the knapsack and the node where it is not.
For the child where the first item is not
included in the knapsack, the calculation for
the bound proceeds as follows:

totalSize = 0 + s2 + s3 = 5 + 9 = 14

bound = 0 + v2 + v3 + (K - totalSize) * (v4/s4)

= 0 + $30 + $45 + (16 - 14) * ($2)
= $79

04/12/17 20
The state spaced traversed by the backtracking algorithm is
displayed below. When the bound of a node is less than or
equal to the current maximum value, or adding the current item
to the node causes the size of the contents to exceed the
capacity of the knapsack, the sub-trees rooted at that node are
pruned, and the traversal backtracks to the previous parent in
the state space tree.

04/12/17 21
The Eight Queens problem

04/12/17 22
04/12/17 23
04/12/17 24
 Example: the 8 queens problem

Place 8 queens on an 8x8 chessboard in

such a way that they cannot check each other

04/12/17 25
 The 8 queens problem: representation

Phenotype:
a board configuration

Genotype: Obvious mapping

a permutation of
the numbers 1 - 8 1 3 5 2 6 4 7 8

04/12/17 26
Algorithm
This heuristic solves n queens for any n n 4 or n = 1:

1) Divide n by 12.
2) Remember the remainder (n is 8 for the eight queens
puzzle).
3) Write a list of the even numbers from 2 to n in order.
4) If the remainder is 3 or 9, move 2 to the end of the list.
5) Append the odd numbers from 1 to n in order, but, if the
remainder is 8, switch pairs (i.e. 3, 1, 7, 5, 11, 9, ).
6) If the remainder is 2, switch the places of 1 and 3, then
move 5 to the end of the list.
7) If the remainder is 3 or 9, move 1 and 3 to the end of
the list.
8) Place the first-column queen in the row with the
first number in the list, place the second-column
queen in the row with the second number in the list, etc.
04/12/17 27
 Hamiltonian Circuit Problem
we make vertex a the root of the
state-space tree. The first
component of our future solution,
if it exists, is a first intermediate
vertex of a Hamiltonian cycle to
be constructed.
Using the alphabet order to
break the three-way tie among
the vertices adjacent to a, we
select vertex b.
From b, the algorithm proceeds
to c, then to d, then to e, and
finally to f , which proves to be a
dead end.

28
04/12/17 28
Hamiltonian Circuit Problem
So the algorithm backtracks from f to e, then to
d, and then to c, which provides the first
alternative for the algorithm to pursue. Going
from c to e eventually proves useless, and the
algorithm has to backtrack from e to c and then
to b.

From there, it goes to the vertices f , e, c, and d,

from which it can legitimately return to a, yielding
the Hamiltonian circuit a, b, f , e, c, d, a. If we
wanted to find another Hamiltonian circuit, we
could continue this process by backtracking from
the leaf of the solution found.

04/12/17 29
29
04/12/17 30
The General Templates

04/12/17 31
Branch and Bound

The Assignment problem

The knapsack problem(4)

General Consideration

04/12/17 32
Introduction
Branch and Bound is a general search
method.
Starting by considering the root problem (the
original problem with the complete feasible
region), the lower-bounding and upper-
bounding procedures are applied to the root
problem.
If the bounds match, then an optimal solution
has been found and the procedure
terminates.

04/12/17 33
Introduction
Otherwise, the feasible region is
divided into two or more regions,
these sub problems partition the
feasible region.
The algorithm is applied recursively

to the sub problems. If an optimal

solution is found to a sub-problem, it
is a feasible solution to the full
problem, but not necessarily globally
optimal.

04/12/17 34
Introduction
If the lower bound for a node exceeds the best
known feasible solution, no globally optimal
solution can exist in the subspace of the feasible
region represented by the node. Therefore, the
node can be removed from consideration.
The search proceeds until all nodes have been
solved or pruned, or until some specified threshold
is met between the best solution found and the
lower bounds on all unsolved sub problems.

04/12/17 35
The assignment problem
Same as Operation Research

04/12/17 36
The knapsack problem(4)
Solve
this algorithm using branch and
bound techniques.

04/12/17 37
General Consideration
Branch and bound (BB) is a general algorithm
for finding optimal solutions of various
optimization problems, especially in discrete
and combinatorial optimization.

It consists of a systematic enumeration of all

candidate solutions, where large subsets of
fruitless candidates are discarded, by using
upper and lower estimated bounds of the
quantity being optimized.

04/12/17 38
The Minimax Principle
Same as Game Theory

04/12/17 39
 Part-II
Introduction to NP-completeness
Problems
Some problems cannot be solved by any algorithms

Other problems can be solved algorithmically but not

in polynomial time

This topic deals with the question of intractability:

which problems can and cannot be solved in
polynomial time.

This well-developed area of theoretical computer

science is called computational complexity.

04/12/17 41
Problems

In the study of the computational

complexity of problems, the first concern
of both computer scientists and computing
professionals is whether a given problem
can be solved in polynomial time by
some algorithm.

04/12/17 42
42
Polynomial time
Definition:

We say that an algorithm solves a problem in

polynomial time if its worst-case time efficiency
belongs to O(p(n)) where p(n) is a polynomial of the
problems input size n.

Problems that can be solved in polynomial time are

called tractable

and ones that can not be solved in polynomial time

are all intractable.

04/12/17 43
43
P and NP Problems

Most problems discussed in this

subject can be solved in polynomial
time by some algorithms

We can think about problems that

can be solved in polynomial time as
a set that is called P.

04/12/17 44
44
Class P

Definition: Class P is a class of decision problems that can

be solved in polynomial time by (deterministic) algorithms.

This class of problems is called polynomial.

Examples:
searching

element uniqueness

graph connectivity

graph acyclicity

04/12/17 45
45
Class NP

NP (Nondeterministic Polynomial):
class of decision problems whose
proposed solutions can be verified in
polynomial time = solvable by a
nondeterministic polynomial
algorithm

04/12/17 46
46
Non-deterministic polynomial
time
Deterministic Polynomial Time: The TM
takes at most O(nc) steps to accept a
string of length n
Non-deterministic Polynomial Time: The
TM takes at most O(nc) steps on each
computation path to accept a string of
length n

04/12/17 47
Nondeterministic polynomial algorithm

A nondeterministic polynomial algorithm is an abstract two-

stage procedure that:

generates a random string purported to solve the

problem

checks whether this solution is correct in polynomial time

By definition, it solves the problem if its capable of

generating and verifying a solution on one of its tries

04/12/17 48
48
What problems are in NP?

Hamiltonian circuit existence

Traveling salesman problem

Knapsack problem

Partition problem: Is it possible to partition a

set of n integers into two disjoint subsets with
the same sum?

04/12/17 49
49
P NP

All the problems in P can also be solved in

this manner (but no guessing is necessary),
so we have:

P NP

However, we still have a big question: P

= NP ?
04/12/17 50
50
NP-Completeness
How would you define NP-Complete?
They are the hardest problems in NP

NP-Complete

P
NP

04/12/17 51
Definition of NP-Complete
Q is an NP-Complete problem if:

1) Q is in NP
2) every other NP problem polynomial
time reducible to Q

04/12/17 52
P = NP?
No one knows if this is true
How can we make progress on this
problem?

04/12/17 53
Progress
P = NP if every NP problem has a
deterministic polynomial algorithm
We could find an algorithm for every NP
problem
Seems hard

We could use polynomial time

reductions to find the hardest
problems and just work on those
04/12/17 54
Polynomial reductions

04/12/17 55