.

.
...
A Survey on Crossover for TSP
.

.
M1 Koji Jakudo

Iba Lab. Rindoku

2010/11/09
 Outline

Introduction

Crossover

Conclusion
 Outline

Introduction

Crossover

Conclusion
 What’s TSP (1)
1
TSP: ja9847.tsp

1
http://www.tsp.gatech.edu/ 1 / 61
 What’s TSP (2)

2
TSP: Traveling Salesman Problems
Given a list of cities and their pairwise
distances, the task is to find a shortest
possible tour that visits each city exactly once.

2
http://en.wikipedia.org/wiki/Travelling_salesman_problem
2 / 61
 What’s TSP (2)

2
TSP: Traveling Salesman Problems
Given a list of cities and their pairwise
distances, the task is to find a shortest
possible tour that visits each city exactly once.

NP-hard
Combinatorial optimization

2
http://en.wikipedia.org/wiki/Travelling_salesman_problem
2 / 61
Path Representation and Simple Crossover (1)

One-point crossover for path representation

3 / 61
Path Representation and Simple Crossover (2)

Lethal genes (visit the same city twice!!!)

4 / 61
 Lethal Genes

Lethal genes:
Fatal for Evolutionary Computations

5 / 61
 Lethal Genes

Lethal genes:
Fatal for Evolutionary Computations

Crossover operators that don’t generate lethal genes

5 / 61
 Lethal Genes

Lethal genes:
Fatal for Evolutionary Computations

Crossover operators that don’t generate lethal genes

Ordinal representation [Dewdney, 1985]

Not effective...

5 / 61
 What’s Better Crossover for TSP (1)

Radcliffe and Surry said:

Respectful
Transmit edges

6 / 61
 What’s Better Crossover for TSP (1)

Radcliffe and Surry said:

Respectful
Transmit edges

Respectful:
Common edges must be inherited.

6 / 61
 What’s Better Crossover for TSP (1)

Radcliffe and Surry said:

Respectful
Transmit edges

Respectful:
Common edges must be inherited.

Transmit edges:
Each edge is present in at least one of its
parents.
6 / 61
 What’s Better Crossover for TSP (2)

Watson said:
Crossover should introduce new edges into the
children.

7 / 61
 What’s Better Crossover for TSP (2)

Watson said:
Crossover should introduce new edges into the
children.

Trade-off
Inherit parents’ edges

Introduce new edges

7 / 61
 What’s Better Crossover for TSP (2)

Watson said:
Crossover should introduce new edges into the
children.

Trade-off
Inherit parents’ edges
preserve parents’ traits
Introduce new edges
diversity
escape from local optima
7 / 61
 Crossover Operators for TSP

Partially Mapped Crossover (PMX) [Goldberg, 1985]

Order Crossover (OX) [Davis, 1985]
Edge Crossover (EX) [Whitley, 1989]
Subtour Exchange Crossover (SXX) [Yamamura, 1992]
Edge Exchange Crossover (EXX) [Maekawa, 1996]
Edge Assembly Crossover (EAX) [Nagata, 1997]
Partition Crossover (PX) [Whitley, 2009]

8 / 61
 Crossover Operators for TSP

Partially Mapped Crossover (PMX) [Goldberg, 1985]

Order Crossover (OX) [Davis, 1985]
Edge Crossover (EX) [Whitley, 1989]
Subtour Exchange Crossover (SXX) [Yamamura, 1992]
Edge Exchange Crossover (EXX) [Maekawa, 1996]
Edge Assembly Crossover (EAX) [Nagata, 1997]
Partition Crossover (PX) [Whitley, 2009]

8 / 61
 Crossover Operators for TSP

Partially Mapped Crossover (PMX) [Goldberg, 1985]

Order Crossover (OX) [Davis, 1985]
Edge Crossover (EX) [Whitley, 1989]
Subtour Exchange Crossover (SXX) [Yamamura, 1992]
Edge Exchange Crossover (EXX) [Maekawa, 1996]
Edge Assembly Crossover (EAX) [Nagata, 1997]
Partition Crossover (PX) [Whitley, 2009]

8 / 61
 Crossover Operators for TSP

Partially Mapped Crossover (PMX) [Goldberg, 1985]

Order Crossover (OX) [Davis, 1985]
Edge Crossover (EX) [Whitley, 1989]
Subtour Exchange Crossover (SXX) [Yamamura, 1992]
Edge Exchange Crossover (EXX) [Maekawa, 1996]
Edge Assembly Crossover (EAX) [Nagata, 1997]
Partition Crossover (PX) [Whitley, 2009]

8 / 61
 Crossover Operators for TSP

Partially Mapped Crossover (PMX) [Goldberg, 1985]

Order Crossover (OX) [Davis, 1985]
Edge Crossover (EX) [Whitley, 1989]
Subtour Exchange Crossover (SXX) [Yamamura, 1992]
Edge Exchange Crossover (EXX) [Maekawa, 1996]
Edge Assembly Crossover (EAX) [Nagata, 1997]
Partition Crossover (PX) [Whitley, 2009]

8 / 61
 Crossover Operators for TSP

Partially Mapped Crossover (PMX) [Goldberg, 1985]

Order Crossover (OX) [Davis, 1985]
Edge Crossover (EX) [Whitley, 1989]
Subtour Exchange Crossover (SXX) [Yamamura, 1992]
Edge Exchange Crossover (EXX) [Maekawa, 1996]
Edge Assembly Crossover (EAX) [Nagata, 1997]
Partition Crossover (PX) [Whitley, 2009]

8 / 61
 Crossover Operators for TSP

Partially Mapped Crossover (PMX) [Goldberg, 1985]

Order Crossover (OX) [Davis, 1985]
Edge Crossover (EX) [Whitley, 1989]
Subtour Exchange Crossover (SXX) [Yamamura, 1992]
Edge Exchange Crossover (EXX) [Maekawa, 1996]
Edge Assembly Crossover (EAX) [Nagata, 1997]
Partition Crossover (PX) [Whitley, 2009]

8 / 61
 Outline

Introduction

Crossover

Conclusion
 PMX

PMX: Partially Mapped Crossover [Goldberg, 1985]

9 / 61
 Process of PMX (1)

Decide two crossover-points

10 / 61
 Process of PMX (2)

Copy the path between the two crossover-points

11 / 61
 Process of PMX (3)

Copy the path of another parent if possible

12 / 61
 Process of PMX (4)

Copy the remaining path with correspondence

13 / 61
 Process of PMX (5)

Copy the remaining path with correspondence

14 / 61
 Process of PMX (6)

Result of PMX

15 / 61
 Properties of PMX

PMX:
Based on two-point Crossover
Inherit edges between crossover-points
Tend to introduce new edges at outside of
crossover-points
Computation complexity: O(n)

16 / 61
 SXX

SXX: Subtour Exchange Crossover [Yamamura, 1992]

17 / 61
 Process of SXX (1)

Find subtours only including common vertices

18 / 61
 Process of SXX (2)

Copy parents’ path and exchange the subtours

19 / 61
 Properties of SXX

SXX:
Tend to inherit edges
Need to find subtours only including common
vertices
Computation complexity: O(n3 )

20 / 61
 EX

EX: Edge Crossover [Whitley, 1989]

Focuse on preserving edges
Breakthrough

21 / 61
 Process of EX (1)

Edge Map (Edge Table)

Parent1: [A B C D E F]
Parent2: [B D C A E F]
edge_map:
edge_map[A] = (B, F, C, E)
edge_map[B] = (A, C, D, F)
edge_map[C] = (B, D, A)
edge_map[D] = (C, E, B)
edge_map[E] = (D, F, A)
edge_map[F] = (A, E, B)
22 / 61
 Process of EX (2)
child = []
current city = unvisited cities.get city randomly()
while 1 do
child.push back(current city)
unvisited cities.remove(current city)
edge map.remove occurrence(current city)
entries = edge map[current city]
if entries.size() 6= 0 then
current city = entries.get city with fewest entries()
else
if unvisited cities.size() 6= 0 then
current city = unvisited cities.get city randomly()
else
return child
end if
end if
end while
23 / 61
 Process of EX (2’)
1. Select the initial city randomly.

2. Remove the current city from the edge map.

3. If the current city has entries go to step 4; otherwise

go to step 5.

4. The city with the fewest entries becomes the

current city.

5. If there are no remaining unvisited cities, then

STOP; otherwise, randomly select an unvisited city
and go to step 2.
24 / 61
 Properties of EX

EX:
Tend to inherit parents’ edges
Introduce less new edges (1-5[%])
Computation complexity: O(n)

25 / 61
 EXX

EXX: Edge Exchange Crossover [Maekawa, 1996]

26 / 61
 Process of EXX (1)

Path representation → Edge representation

27 / 61
 Process of EXX (2)

Select edges and exchange them

28 / 61
 Process of EXX (3)

Reverse vertices in order to correct the path

29 / 61
 Process of EXX (4)

Repeat the process until achieving the valid path

30 / 61
 Process of EXX (5)

Result of EXX

31 / 61
 Properties of EXX

EXX:
Definitely, inherit parents’ edges
It is proved that EXX always stops.
Computation complexity: O(n) - O(n2 )???

32 / 61
 EAX

EAX: Edge Assembly Crossover [Nagata, 1997]

Focuse on preserving edges
Generate new edges to some extent

33 / 61
 Process of EAX

1. Generate GAB := (V, EA ∪ EB )

2. Partition GAB into ”AB-cycles”

3. Generate some intermediate solutions

4. Modify the intermediate solutions into valid tours

34 / 61
 AB-cycle
AB-cycle:
Subtour in GAB
Alternately select edges of tour-A and tour-B

35 / 61
 Intermediate Solutions

Intermediate solutions
D = {∀ edge ∈ AB cycle}
iA = (V, (EA ∩ D̄) ∪ (EB ∩ D))
iB = (V, (EA ∩ D) ∪ (EB ∩ D̄))

36 / 61
 Modify Intermediate Solutions

Search all combinations

Priority: the subtour which has fewest vertices

Image of modification

37 / 61
 Properties of EAX

EAX:
AB-cycle
Inherit parents’ edges
Introduce some new edges
Computation complexity: O(n2 )?

38 / 61
 Experiment

EAX vs. EX-2opt vs. EXX

Initial population: random individuals applied 2-opt

Elitist Recombination [Thierens, 1994]

39 / 61
 Elitist Recombination
Initialization:
Npop individuals

Selection:
Parents: Npop /2 pairs
Sampling without restoration

Reproduction:
Each pair reproduces Ncross children
Select 2 best individuals from parents and
children
40 / 61
 Result

EAX found the global optimum

EAX took less generations than others

41 / 61
 PX

Partition Crossover (PX) [Whitley, 2009]

GECCO 2009 Best Paper Winner

42 / 61
 Process of PX (1)
Graph G:
Merging the vertices and edges from parents

43 / 61
 Process of PX (2)
Graph G*:
Delete common edges from G

44 / 61
 Process of PX (3)

Find two or more partitions → success

children are composed of:
Common edges
A’s subtour
B’s subtour

Cannnot find them → failure

45 / 61
 Properties of PX

PX:
Need to find two or more partitions
Children are created only from parents’ edges
Computation complexity: O(n)
Fitness:

f (P 1) + f (P 2) = f (C1) + f (C2)

46 / 61
 Experiment 1: Random Local Optima

Random local optima:

Initialized randomly and applied 2-opt (local
search)
Tend to have common edges probably

Configuration:
ATT532 (best known: 27686)
10 random local optima
45 combinations

47 / 61
 Result of Experiment 1

Success: 43 / 45 (95%)

All children were also local

optima under 2-opt

7 children were better than

the best parent

48 / 61
 Experiment 2: Optima near the Global

Optima ”near” the Global:

Random walk from the global optimum
Then, apply 2-opt

49 / 61
 Result of Experiment 2

In all 34 cases, PX rediscovered the global optimum.

50 / 61
 Partition Crossover

Advantage:
Tunnel between local optima
If parents are near the global optimum, the
child can jump directly to the global optimum.

Disadvantage:
Cannot generate new edges
Need to find two or more partitions

51 / 61
 Outline

Introduction

Crossover

Conclusion
 Summary

PMX, SXX, EX, EXX, EAX, PX

EAX is the most effective?

If local optima can be found easily, PX may be more

useful.

52 / 61
 Tips: Local Search for TSP

Local search for TSP:

2-opt method (k-opt)
LK method (Lin-Kernighan)

53 / 61
 2-opt Method
cycle.initialize randomly()

while 1 do
for all i, j, k, l do
if dik + djl < dij + dkl then
cycle = cycle − dij − dkl + dik + djl
break
end if
end for
end while

return cycle
54 / 61
 LK Method

Best known heuristic for TSP

Based on k-opt

Variable k

55 / 61
 Tips: Mutation for TSP

Mutation for TSP:

2-change mutation is major?

56 / 61
 Process of 2-change Mutation (1)

Select two vertices and exchange them

57 / 61
 Process of 2-change Mutation (2)

Four edges changed (too many!!!)

58 / 61
 Process of 2-change Mutation (3)

Reverse vertices between the exchanged vertices

59 / 61
 Process of 2-change Mutation (4)

Reverse vertices between the exchanged vertices

60 / 61
 Process of 2-change Mutation (5)

Only two edges changed (valid?)

61 / 61
Thank you for your attention.