Lecture 11

BACKTRACKING
BRANCH & BOUND
Chapter 10
Design & Analysis of Algorithms

Septermber 8

SLIIT

Contents
Backtracking
- Method
- Examples
- Time & space analysis of Backtracking

Branch and Bound

- Method
- Time & space analysis of Branch & Bound
- Comparison with backtracking
Septermber 8

SLIIT

Backtracking

Backtracking is a systematic way to

search for the solution to a problem
1. Define the solution space
2. Organize the space (graph or tree)
3. Use DFS recording best answer so far

Septermber 8

SLIIT

Example 0/1 Knapsack

n = 3, c = 30
w = {20, 15, 15}, p = {40, 25, 25}
Maximize the sum of profit
pi
without violating the weight constraint
wi c
Septermber 8

SLIIT

Example (cont.)
First, define the solution space
Let the binary vector {b1, ..., bn} define a
solution
If bi = 0, then object i not in knapsack
If bi = 1, then object i is in knapsack
Possible solutions = {0,0,0}, {0,0,1},
{0,1,0}, ..., {1,1,1}
Septermber 8

SLIIT

Example (cont.)
Second organize them into a tree/graph
A binary tree will do nicely...
0

Septermber 8

First Item

SLIIT

Second Item

Third Item

Example (cont.)
Now DFS, recording best solution so far
and ignoring paths that represent infeasible
solutions
Best = {0,0,0}
Profit = 0

Path = {0,0,0}

0
0

Septermber 8

SLIIT

1
7

Example (cont.)
w = {20, 15, 15} p = {40, 25, 25} c = 30
New Path = {0,0,1}
Feasible? Yes 15 < 30

Best so far = {0,0,0}

Profit = 0

Current = {0,0,1}
Profit = 25

Best = {0,0,1}
Profit = 25

0
0

25

Septermber 8

SLIIT

Example (cont.)
w = {20, 15, 15} p = {40, 25, 25} c = 30
New Path = {0,1,_}
Feasible? Yes 15 < 30.

0
0

25

Septermber 8

SLIIT

Example (cont.)
w = {20, 15, 15} p = {40, 25, 25} c = 30
Best so far = {0,0,1}
Profit = 25

New Path = {0,1,0}

Feasible? Yes 15 < 30.

0
0

25 25

Septermber 8

Best = {0,0,1}
Profit = 25

Current = {0,1,0}
Profit = 25

SLIIT

1
10

Example (cont.)

w = {20, 15, 15} p = {40, 25, 25} c = 30

New Path = {0,1,1}
Feasible? Yes 30 30

Best So far = {0,0,1}

Profit = 25

0
0

25 25

Septermber 8

1
50

Current = {0,1,1}
Profit = 50
Best = {0,1,1}
Profit = 50

SLIIT

1
11

Example (cont.)
w = {20, 15, 15} p = {40, 25, 25} c = 30
New Path = {1,_,_}
Feasible? Yes 20 30

0
0

25 25

Septermber 8

50
SLIIT

12

Example (cont.)
w = {20, 15, 15} p = {40, 25, 25} c = 30
New Path = {1,0,_}
Feasible? Yes 20 30

0
0

25 25

Septermber 8

1
50

SLIIT

1
13

Example (cont.)
w = {20, 15, 15} p = {40, 25, 25} c = 30
New Path = {1,0,0}
Feasible? Yes 20 30

Best so far= {0,1,1}

Profit = 50

0
0

25 25

Septermber 8

1
50

Current = {1,0,0}
Profit = 40
Best = {0,1,1}
Profit = 50

40
SLIIT

14

Example (cont.)
w = {20, 15, 15} p = {40, 25, 25} c = 30
New Path = {1,0,1}
Feasible? No 35 not 30

0
0

25 25

Septermber 8

50 40

SLIIT

1
15

Example (cont.)
w = {20, 15, 15} p = {40, 25, 25} c = 30
New Path = {1,1,_}
Feasible? No 35 not 30

Best so far = {0,1,1}

Profit = 50

0
0

25 25

Septermber 8

1
50

40
SLIIT

16

Backtracking
If it found that the newly reached node is
infeasible (I.e. the current weight is more
than the allowed weight) then there is no
point of expanding its sub trees.

Septermber 8

SLIIT

17

currentSoln {0,0,0} currentW 0 currentP 0

{0,0,0} 0 0

{0,0,0} 0 0

{1,0,0}20 40

{0,1,0}15 25

{1,0,0}20 40

{0,0,1}15 25
{1,0,0}20 40

{0,0,0}0 0
{0,1,0}15 25
Septermber 8

{0,1,1}30SLIIT
50

18

Analysis
How many nodes in the tree?
If there are n items then there are O(2n)tree
leaf nodes
Total no of nodes are (2n+1 1)
Our example, 0/1 Knapsack has O(23) leaf
nodes and (23+1 1=15) nodes
Time taken is (2n)
Note that not all might be traversed
Septermber 8

SLIIT

19

Permutation tree
In other cases the solution is a permutation
of n elements.
For example: traveling salesman
As the name suggests, the travelingsalesperson problem may be used to model
the territory covered by a salesperson.
In this problem we are given an n vertex
network and are to find a cycle of minimum
cost that includes all n vertices.
Septermber 8

SLIIT

20

Traveling salesman
1

30

2
5

4
3

20

10

Select vertex 1 as the start and end vertex.

Septermber 8

SLIIT

21

Traveling salesman
1

30

2
5

4
3

20

Septermber 8

10

Tours:
{1,2,3,4,1}
{1,2,4,3,1}
{1,3,2,4,1}
{1,3,4,2,1}
{1,4,2,3,1}
{1,4,3,2,1}
No of tours = 3! = 6
(since we are always start
with the first node,
permutations are only for the
remaining 3 nodes)
SLIIT

22

We can model the solution space for this problem using a

tree
30
1

6
B

2
3

F
4
L
1

H
4

M
1

Septermber 8

3
P

O
1

3
K

20

G
3

5
4

SLIIT

2
Q

1
W

23

Traveling salesman(contd.)
The possible tours may be described by a tree
which each roottoleaf path defines a tour. The
edgelabelsonthepathfromroottoaleafdefinea
tour. For example, the path to node R represents
thetour1,2,3,4,1.
Everytourinthenetworkisrepresentedbyexactly
one roottoleaf path in the tree. As a result, the
numberofleavesinthetreeis(n1)!
Here n = 4 and we have (4-1)! Leaf nodes
Septermber 8

SLIIT

24

Traveling salesman(contd.)
A backtracking algorithm will find a minimumcost tour by searching the solution space tree in
a depth-first manner, beginning at the root.
A possible search using above Figure would
move from node A to B to C to F to L to R. At R
the tour 1,2,3,4,1 is recorded as the best tour
seen so far. Its cost is 59.
Septermber 8

SLIIT

25

A
1

F
4
L
1

H
4

M
1

3
P

O
1

3
K

20

G
3

5
4

6
B

30

Q
1

1
W

59
Septermber 8

SLIIT

26

Traveling salesman(contd.)
From L we backtrack to the node C. and we
move forward to G and then to M and to S.
We have now constructed the tour 1,2,4,3,1
whose cost is 66. Since this tour isnt
superior to the best tour we have, we
discard the new tour and backtrack to G,
then , C and then B.
Septermber 8

SLIIT

27

A
1

F
4
L
1

H
4

M
1

59

66

Septermber 8

3
P

O
1

20

10

G
3

5
4

6
B

30

Q
1

SLIIT

1
W

28

Speeding up the solution

We can speed the search for an optimal
solution by determining whether or not a
newly reached node can possibly lead to a
solution better than the best found so far.
by the time we reach node I of we have
found the tour 1,3,2,4,1 with cost 25. At
node I the partial tour is 1,3,4 whose cost is
26.
There is no point in searching the subtree
with root I.
Septermber 8

SLIIT

29

A
1

2
3
F
4
L
1

M
1
S

59

66

Septermber 8

26

2
J

3
P

1
U

59

25
SLIIT

2
Q

1
T

3
K

H
4

20

G
3

5
4

6
B

30

1
W

59
30

Contents
Backtracking
- Method
- Examples
- Time & space analysis of Backtracking

Branch and Bound

- Method
- Time & space analysis of Branch & Bound
- Comparison with backtracking
Septermber 8

SLIIT

31

Branch & Bound

searches solution tree in either a
- breadth-first or
- least-cost manner.

Septermber 8

SLIIT

32

Breadth-first

F
4

G
3

L
1

H
4

M
1

59

66

3
K

1
S

Septermber 8

20

5
4

30

3
P

O
1

2
Q

25

66

59

59

SLIIT

33

Comparison
Backtracking DFS
Branch & Bound BFS or Least cost
The memory needed by backtracking is O(length
of longest path in the solution space organization)
The memory needed by branch and bound is
O(size of solution space organization).

Branch and Bound Need more memory than

Backtracking.
Septermber 8

SLIIT

34

Summary
Backtracking
- Method
- Examples
- Time & space analysis of Backtracking

Branch and Bound

- Method
- Time & space analysis of Branch & Bound
- Comparison with backtracking
Septermber 8

SLIIT

35