Cooperation Ant System

Cooperation Pressure Ant System
Ant-based Multilevel Unconstrained Lot Sizing

Problem (MLULSP) Approach
Marco Maniscalco
Ashley Muthiah
Robin George
Paulo Santos
Jing Wang
Beini Sun
Stuttgart, 2008
Abstract
This paper contains the result of 6 students work from the University of Applied Sciences Stuttgart
formerly known as Hochschule für Technik Stuttgart (www.hft-stuttgart.de). The mentor of this
course was Prof. Dr. J. Homberger who manifested the requirements and fundamentals of this
problem. The goal was to run a software development team where different people have different
roles. Furthermore an algorithm had to be developed with the corresponding distributed system
which is also the main result of this work.
Distribution of the project roles:
Marco Maniscalco Architecture & Implementation of the Distributed System

Implementation of the Cooperative Pressure Ant System
Development of the Cooperation Negotiation
Research on EA and Single Ant Systems and integration of this paper
Ashley Muthiah Project lead
Research on Negotiation and Parallel EA
GUI
Robin George Research of Mediator and Parallel EA
Paulo Santos Implementation of the Datastructure Reader and Visualization
Jing Wang Research on Negotiation, Voting, Update and Parallel EA
Beini Sun Research on Parallel Ant
Mentor Prof. Dr. J. Homberger
2
CONTENT
1 INTRODUCTION........................................................................................................................................ 6
1.1 THE MULTI-LEVEL UNCONSTRAINED LOT-SIZING PROBLEM ................................................................................ 6

1.2 CODING OF SOLUTION ................................................................................................................................. 8
1.3 DECODING AND EVALUATION ........................................................................................................................ 9
2 OTHER APPROACHES ............................................................................................................................. 10
2.1 SIMPLE EA .............................................................................................................................................. 10

2.2 PARALLEL EA ........................................................................................................................................... 11
3 COOPERATION PRESSURE ANT SYSTEM ................................................................................................. 13
3.1 GRAPH REPRESENTATION ........................................................................................................................... 13

3.2 NEGOTIATION .......................................................................................................................................... 15
3.2.1 Basic Agent Interaction ................................................................................................................... 15
3.2.2 Cooperation Pressure ...................................................................................................................... 16
3.2.3 Ranking regression curve ................................................................................................................ 18
3.2.4 Voting .............................................................................................................................................. 19
3.2.5 Hare Voting System ......................................................................................................................... 19
3.2.6 Additional re-voting strategies ........................................................................................................ 21
3.2.7 Update ............................................................................................................................................. 22
3.3 RESULTS ................................................................................................................................................. 22
3.3.1 Class 2 .............................................................................................................................................. 25
3.3.2 Class 3 .............................................................................................................................................. 27
3.3.3 Comparison ..................................................................................................................................... 28
4 DISTRIBUTED SYSTEM ARCHITECTURE ................................................................................................... 29
4.1 ABSTRACT COMMUNICATION ...................................................................................................................... 29

4.2 FUNDAMENTAL DATAFLOW......................................................................................................................... 30
4.3 RMI SERVICE ARCHITECTURE ...................................................................................................................... 31
4.4 IMPLEMENTING CUSTOM STRATEGIES............................................................................................................ 32
4.4.1 Terminology..................................................................................................................................... 33
4.5 BENEFITS FROM BLACK-BOXING ................................................................................................................... 34
4.6 VISUALIZATION......................................................................................................................................... 35
5 CONCLUSION ......................................................................................................................................... 38
6 FURTHER INFORMATION ....................................................................................................................... 39
6.1 SERVER SPECIFICATION .............................................................................................................................. 39

6.2 CONTENT MANAGEMENT SYSTEM ............................................................................................................... 39
6.2.1 Framework for the Website ............................................................................................................. 39
6.2.2 Section Management ...................................................................................................................... 40
6.2.3 Category Management ................................................................................................................... 40
6.2.4 Component Management ............................................................................................................... 42
6.2.5 Menu Management ........................................................................................................................ 42
6.2.6 User Management........................................................................................................................... 42
7 GLOSSARY .............................................................................................................................................. 45
8 THE TEAM .............................................................................................................................................. 46
3
4
FIGURES
FIG. 1 NETWORK OF COMPANIES..................................................................................................................................... 7
FIG. 2 GRAPH OF ITEMS ................................................................................................................................................ 7
FIG. 3 VERTICAL PARTITIONING ....................................................................................................................................... 8
FIG. 4 HORIZONTAL PARTITIONING .................................................................................................................................. 8
FIG. 6 DECODING OF A SOLUTION .................................................................................................................................... 9
FIG. 5 TERMINOLOGY OF THE SOLUTION STRUCTURE ........................................................................................................... 9
FIG. 7 SIMPLE EA SYSTEM ........................................................................................................................................... 10
FIG. 8 PARALLEL EA SYSTEM ........................................................................................................................................ 11
FIG. 9 PHEROMONE GRAPH.......................................................................................................................................... 13
FIG. 10 SIMPLIFIED UPDATE LOGIC ................................................................................................................................ 15
FIG. 11 ITEM DISTRIBUTION ......................................................................................................................................... 16
FIG. 12 DISTRIBUTION OF THE RANKINGS PER ROUND ........................................................................................................ 17
FIG. 13 DISTRIBUTION OF THE RANKINGS PER ROUND WITH REGRESSION GRAPHS ................................................................... 17
FIG. 14 LOGARITHMIC REGRESSION CURVE ...................................................................................................................... 18
FIG. 15 VOTING SYSTEM ............................................................................................................................................. 19
FIG. 16 CONVERGENCE OF CLASS 2 INSTANCE 1 (ROUND 1 - 1933) .................................................................................... 22
FIG. 17 CONVERGENCE OF CLASS 2 INSTANCE 1 (ROUND 1 - 80) ........................................................................................ 23
FIG. 18 CONVERGENCE OF CLASS 2 INSTANCE 1 (ROUND 1 - 80) WITH FINK 5 ..................................................................... 23
FIG. 19 CONVERGENCE OF CLASS 2 INSTANCE 1 (ROUND 1 - 1933) WITH FINK 5 .................................................................. 24
FIG. 20 ACCEPTANCE OF WORST COST TO ACCIEVE BETTER OVERALL RESULT .......................................................................... 25
FIG. 21 IMPACT OF THE NUMBER OF AGENTS ON THE OVERALL FITNESS (CLASS 2 INSTANCE 1) .................................................. 26
FIG. 22: ABSTRACT COMMUNICATION BETWEEN AGENT AND MEDIATOR ............................................................................... 29
FIG. 23: COMMUNICATION OF 3 AGENTS WITH MEDIATOR ................................................................................................. 29
FIG. 24: LOGICAL DATAFLOW IN A CALCULATION ROUND.................................................................................................... 30
FIG. 25 ANT ARCHITECTURE......................................................................................................................................... 31
FIG. 26: RMI SERVICE MODEL ...................................................................................................................................... 32
FIG. 27: LOGICAL EXTENSION-POINTS AT THE CUSTOMIZING INTERFACE ................................................................................ 33
FIG. 28: SOLUTION FRAGMENTS BASED ON PARTITIONS (1 AGENT HANDLES X ITEMS) ............................................................. 34
FIG. 29: LOGICAL COMPONENT STACK ............................................................................................................................ 34
FIG. 30 SUPPLY CHAIN ORDER ...................................................................................................................................... 35
FIG. 31 RADIAL SUPPLY CHAIN ORDER ............................................................................................................................ 36
FIG. 32 REALTIME SCATTER PLOT THAT SHOWS THE COST WITH THE TIME .............................................................................. 37
FIG. 33 CMS OVERVIEW............................................................................................................................................. 40
5
1 Introduction
Cooperation between enterprises are the most important way to get benefit for both of them and
the strategic supply chain cooperation is an important to compete in today’s business climate.
Enterprises strategically ally with their downstream and upstream enterprises to acquire critical
technologies and knowledge [1] to enter the market so as to complement their core capabilities and
to disperse costs and risks [2]. For example, IBM, Intel, and Microsoft formed a strategic alliance to
enter the personal computer market. In only 15 months, they seized Apple’s market share. *3] In
another example, Dell Computer, the world largest personal computer vendor, coordinated a global
production and distribution network. Dell’s PDN spans the US, Europe and Asia to manufacture its
products. Dell organizes distribution and assembly by regions, operating one or more distribution
centres and assembly plants to serve its major market [4]. Therefore, how to make the efficiently
strategy supply chain becomes a topic we are interested in.
On the way to deep in look at the capacitated lot sizing problem for a supply chain of corporate
multi-agents, to minimize the total costs of production, inventory and transportation under the
system capacity restriction are the problems we are going to solve. In this project, Ant algorithm is
discovered as the most efficiently method.
The ant algorithm was proposed by M. Dorigo. It has been applied widely in many areas because of
its robustness and fitness to distributed computing and ease of association with other methods. For
example, it has been applied in TSP successfully.
The ants in one population select the ways according to the information accumulated in the form of a
pheromone trail deposited on paths, the ants in another population search paths randomly. Update
the pheromone according to the result of the two populations and keep the good individuals. This
can improve the searching efficiency and avoid searching in a limited area.
Alongside this advanced approach, information technology is no doubt bringing a lot of benefits
to process. It experiments between shortens lead times and decreases batch sizes, which leads to
more efficient and effective order processing. At the end, it simulates how distribution ant system is
a high-efficiency approach is used. Consequently, technological advances are necessary to keep a
competitive advantage.
1.1 The Multi-Level Unconstrained Lot-Sizing Problem
Assume a scenario like: We have a big Enterprise which produces Auto for international market.
Once a complete car is produced, it may contain many parts (like engine, windows) actually not
produce by only one enterprise but from their commercial alliances. Such this kind of enterprises,
their alliances and even the end user made a supply chain.
6
Company 1 Company 2
Company 3
Company 4
Company 5
Fig. 1 Network of companies
Those five companies build up a product structure, it describes the composition of end products in
intermediate products and components, each arc represents cooperation relationship between
them. We can simplify this structure with some specific details like:
t 1 2 3 t 1 2 3
D1,t 10 0 30 D1,t 15 25 25
Level 0 1 2
Level 1 3
Level 2 4
Level 3 5
Fig. 2 Graph of Items
7
Fig. 2 shows for each end product i and each period t, t = 1... T, of a finite planning horizon T, an
external demand di, t is given. Usually the product structure can be represented by a directed graph
with N nodes (Bookbinder and Koch 1990). Each node is representing an enterprise and product
levels show upstream, downstream relationship between them.
From these scenario and structure, it can be easily obtain the optimal or we can say “the cheapest
cost” solution by mathematical functions. But in real world, enterprises are always focusing on their
profits of course; they make any decision base on their varied costs. Thus in the following, our
problem we are facing are de-central multi-criteria decision problem.
Fig. 3 Vertical partitioning Fig. 4 Horizontal partitioning
How to make whole supply chain into de-central? The best choice always considers all these self-
interested enterprises individually, but it will lead the problem very complex and never be
harmonization. Step aside, we can think about sort them into numbers of groups… we call
that AGENT.
Thus our objectives are going to minimize its costs for each agent. Those agents represent their
desires to negotiate with other agents until find the “balance”.
1.2 Coding of Solution
Each optimal solution of the MLULSP has the property that any lot must cover an integer number of
demands (Veinott 1969). In other words, demands are not split into several lots. This property allows
determination of optimal lot sizes xi,t from optimal setup periods yi,t (Dellaert and Jeunet 2000).
Accordingly, Dellaert and Jeunet (2000) use a binary coding, where each individual represent its
result solution in this abstract way and decoding afterward together with their individual details. It is
also a convenient way handle by our IT skills.
In conformity with Dellaert and Jeunet (2000), a binary coding is selected to represent problem
solutions of the MLULSP. Each coded solution (denoted as individual in the following) G consists of an
N×T bit matrix, in which exactly T bits are reserved for each product. The numbers of the lines
correspond to the product numbers, and the column numbers to the periods. In contrast to Dellaert
and Jeunet, the genes Gi,t, i = 1,..., N, and t = 1,..., T, represent a preselection of periods that can be
used when required for production, where Gi,t = 1 when period t is preselected as a “possible”
production period for product i, and Gi,t = 0 otherwise. A solution for the setup variables yi,t is
8
calculated from the preselected periods within the decoding procedure. [Casestudy.doc] The
following diagram shows some necessary definitions in coding along with agents.
P1 P2 P3 Periods
Item 1 1 0 1 Sub Contract
Agent 1 Item 2 1 1 0
Item 3 1 1 0
Item 4 1 0 1
Agent 2 Item 5 1 1 1 Contract
Item 6 1 0 1
Solution Genotype
Fig. 5 Terminology of the solution structure
1.3 Decoding and Evaluation
After we finished coding, negotiation, evaluation, we need decode change our binary code together
with individual setup cost, inventory cost, until we can easily find out the optimal solution.
In the first decoding step, the demand di,t of product i is calculated for each period t,
t = 1,..., T. If product i is an end product, demand di,t results from the external demand given for
period t. In contrast, if product i is an intermediate product or a component, the dependent demand
di,t is calculated in accordance with (3). This means that demand di,t for product i results from the sum
of the lots that are produced for successors (i) in periods t+ti. Because the line numbers are run
through in ascending order during decoding, it is guaranteed that the lots of the successors (i)
already have been calculated.[CaseStudy.doc]
Binary solution Demand lot sizes Inventory
Fig. 6 Decoding of a solution
Here is an example shows a solution calculate through decoding steps to get the exactly
costs. After this decoding, the comparisons of cost are obviously.
9
2 Other Approaches
In order to achieve good and optimised solutions the following approaches were observed and
evaluated. Each of the following approaches has its own algorithm and yielded comparatively
different results in terms of performance issues along with its restrictions.
2.1 Simple EA
The simple EA consists of just one mediator and a client. The system gets initialized by a given
genotype which we define as the central genotype, the first (e.g. random) solution. From round to
round the mediator updates his current genotype to calculate an iterative optimized solution with
the EA Strategy. The mediator is able to derive (presumably better) solutions (in fact, rather the
negotiation strategy used by the mediator). Agents evaluate the contracts by calculating its cost
through fitness function and generate the ranking information. With this ranking information, the
mediator again changes his current solution accordingly. The following diagram depicts the Simple
EA.
Start
Mediator
Random Solution / Update
Good Solution
derive
Solution Result
Contracts
Agents Agents Evaluate

& Rank
Agents
Fig. 7 Simple EA System
10
Drawback
Although it was easy to implement the system, it yielded good results for Class 1 alone and not for
Class 2 and Class 3. Due to the huge matrix sizes of these two Classes the randomness of the solution
was reduced leading to the loss of good solutions.
2.2 Parallel EA
The parallel EA consists of a mediator and many agents (e.g. 5 agents). This system works similar to
Simple EA with the difference that the EA Strategy is executed on parallel in each of the agents. The
system gets initialized by a given genotype which we define as the central genotype, the first (e.g.
random) solution. From round to round the mediator updates his current genotype with iterative
optimized solutions calculated by agents. Agents evaluate the contracts, apply EA and generate
optimized ranking information with which the mediator updates his current solution accordingly. The
following diagram depicts the parallel EA.
Start
Mediator
Random Solution / Update
Good Solution
derive
Solution Result
Contracts
Agents Agents Evaluate

EA EA & Rank
Agents
EA
Fig. 8 Parallel EA System
Drawback
11
The Parallel EA strategy was very complex to implement due to the communication pressure across
different agents. Moreover there was no precise condition under which termination of the rounds
could be achieved unlike the simple EA where the system terminates depending upon the number of
rounds.
12
3 Cooperation Pressure Ant System
3.1 Graph Representation
This approach uses a special kind of Ant System which has been introduced on this problem domain
by Prof. Dr. Homberger. The main idea is to build up a special graph which represents the periods of
the problem instances. The nodes of the graph represent the periods of the whole time span. A
transition (the black ones) mark that a production from to takes place where
, k is the number of the item. The dotted transitions (the red ones) mark that there is a possible
production. If there is an existing transition (the black one), then a 1 is set in the corresponding
genotype otherwise 0.
P4 P4
P4
P3 P3
P3
P2
P2 P2
P1
P1 P1
Item 1 Item 2 Item 3
Fig. 9 Pheromone graph
When regarding the trees, it is clear to see that we have sub trees for all production items containing
each the list of periods. The number of all transitions is and the number of
possible transitions is . Then we have in maximum

transitions per matrix where is the number of items. We have nodes in the graph. From
the above graph we can derive a genotype used by the MLULSP for calculating a specific solution.
13
1 0 1 1
1 1 1 0
1 0 0 1
Every transition has its own probability to be chosen. Starting point is random data for this data
structure, which means, all transitions have to be distributed equally in probability. When we inspect
one of the sub trees from the above figure, we can derive an adjacency matrix from it (how it in fact
is realized in the implementation). These are the starting values for the probability:
This makes sure that each transition can be chosen at the beginning with the same probability. New
genotypes can now be derived from this structure. The above matrix represents a genotype fragment
for exactly one item. The overall graph consists of such pheromone matrices. In order to derive a
genotype from this graph, the period of the (First Possible Demand) will result in a “1” in the
genotype and all periods prior to the are “0”. The probability to set a transition from to is
given with:
Now a random number within the interval is chosen and for each period it is verified if
.
After a “1” has been set, the calculation starts at this position just like it did at the FPD and continues
till the last period (S) has been reached. This gets repeated for all pheromone matrices of all items to
derive a full genotype matrix in the end.
Example:
(With big numbers for better readability)
800 400 200 100 50 25
14
(2) Agents agree on contracts

500 Solutions
Input: 500
Solutions
(1) generate
Negotiation
Output: 1
Pheromones
Solution
(3) Used to update the

pheromones
Fig. 10 Simplified Update Logic
3.2 Negotiation
3.2.1 Basic Agent Interaction
Items of the problem instance are equally distributed over the agents. There are several strategies to
distribute them, in our research, we used an equally separation of items. The number of items gets
divided by the number of agents, which leads to the following distribution:
15
Agent 1
Agent 2
Agent 3
Fig. 11 Item Distribution
In every round all agents get the solution fragments like shown in Fig. 10. We used the population
size of 500 individuals per round. Every agent owns a private, hidden fitness function whit gets used
to calculate the fitness for all 500 individuals every round. As a consequence every agent constructs a
ranking list with the ranking interval where is the number of individuals per round, in our
case . Every solution gets its own ranking, which means that there may not exists 2 solutions
with the same ranking. These ranking lists are returned to the mediator. Since the agents only return
ranking information the information hiding is not violated here, the mediator does not know/use
cost information. This is an important requirement when distinguishing between distributed and
central approaches. So the mediator owns agents which return ranking lists with the interval
.
3.2.2 Cooperation Pressure
Since the objectives of all agents are usually contrary, finding a good compromise is a non-trivial task
for the mediator. Research showed that very often some specific solutions are rated very different by
agents. The following shows the best solution per round (with centrally minimized overall cost) and
how the agents rank this “best” solution. The orange line is the overall cost; the other graphs
represent each agent.
16
600
Ranking
500
400
300
200
100
0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 Rounds
Fig. 12 Distribution of the rankings per round
It is clear to see that some agents usually agree on similar rankings whereas one agent (the red one)
always acts in opposition to all others. This is the agent which owns most of the low-level items
which seem to have a big impact on the overall cost. Also you can see is that there is convergence
within the ranking numbers of the “best” solution. This leads to the assumption that if you let the
agents fight they usually don’t find a good consensus which conforms to the social welfare. (See e.g.
round 33 in the above figure.)
600
Ranking
500
400
300
200
100
0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 Rounds
-100
-200
Fig. 13 Distribution of the rankings per round with regression graphs
A new border is introduced where initially and . The mediator

now balances all ranking lists with this border. The algorithm traverses all ranking lists and only
17
keeps these solutions which do not violate . In fact the ranking of the solution has
to fulfill .
An acceptance counter is introduced for each solution that does not violate the border.
gets incremented every time when the solution has been accepted by an agent and the
corresponding border. After traversing all ranking lists, all voted solutions that have an
are put into a pool of candidates.
At the beginning of the negotiation is , which means that all agents are equally
treated and have the freedom to vote for any solutions. Agents contribute equally to the pool of
candidate solutions.
The negotiation algorithm then self-modifies this border every round. This affects
the number of agreement candidates in the pool. If the mediator applies more pressure with the
time, the number of candidates shrinks and forces the agents to cooperate. If and only if the agents
cooperate and agree on suboptimal solutions, they are able to find a consensus.
3.2.3 Ranking regression curve
The adaption of is achieved with a logarithmic function that decreases the upper
border. Finding such functions is a trivial task since the only unknowns are the number of rounds to
scale and the value. In our research we used 500 individuals (leading to a maximum rank of
500) and 40 rounds. This results in the following logarithmic regression curve:
600
Rank
500
400
300
200
100
Round
0
0 1 10 20 30 40 50 60 70 80
Fig. 14 Logarithmic regression curve
The negotiation algorithm tries to improve the overall system convergence with good agreement
solutions. The algorithm hereby avoids bad solutions with the time by applying the cooperation
pressure on the agents. Agents can not insist on a solution which is avoided by all other agents in an
advanced round. In the beginning all agents are free with their vote whereas in the end only
solutions can be agreement candidates which satisfy all of the agents with a good ranking value.
It is a matter of fact that the number of candidates is affected by the acceptance of each agent. If the
cooperation pressure is too high, no cooperation is presumable because the favored candidates of
18
each agent resides outside of the pressure set . If the pressure is too low, too many
(contrary) candidates are found which may lead to local minima.
3.2.4 Voting
The size of the set of candidate solutions may be . The agents have to arrange an
additional ballot. An adapted hare STV (single transferable vote) system is used to eliminate the
worst candidate and let the agents deliver the freed votes to their other candidates. Then the
algorithm eliminates the next worst candidate until only one candidate solution is remaining. This
solution represents the round agreement where all agents are happy with, or at least take this as
compromise where some agents are happy.
It is a matter of fact that a specific round may have no candidate at all. This happens when all round
solutions are outside of . In case of that, the same hare system is called without
finding candidates (but regarding all solutions).
Regarding these two cases it is a fact that every round the algorithm is able to name exactly one
solution on which all agents agreed. The resulting solution is used to update the ant pheromones and
generate new individuals for the next generation.
Ranking 500 Solutions

Ranking List S 1
List lower
Supper 500
Ranking
List
Rank R S ,S
solution lower upper Candidate
Mediator
Solutions
0..500
Hare
System (STV)
0..1
Agreement Solution
Fig. 15 Voting System
3.2.5 Hare Voting System
Two cases have to be defined for the hare system. First the set of candidate solution is empty due to
the fact that all solutions are outside of which means it is .
19
The second case is the most common case when (e.g.) candidates are found. The hare system is
now used to eliminate all except one - to vote one solution out of this set. The voting works in two
steps.
First a set of solutions with accumulated votes is calculated. This means that all ranking lists of all
agents are merged into one ranking list where the corresponding rankings are transformed into
votes. The following example shows the result of agents with candidates:
Agent 1 Agent 2 Agent 3

Votes Rank Solution Rank Solution Rank Solution
4 1 A 1 B 1 E
3 2 B 2 C 2 B
2 3 D 3 A 3 D
1 4 C 4 E 4 A
0 5 E 5 D 5 C
The transformation from ranking to votes is given by . The global

number of votes is hereby:
The above table then results in the following merged table:
Solution Votes
B 3+4+3=10
A 4+2+1=8
E 0+1+4=5
D 2+0+2=4
C 1+2+0=3
Now the hare system detects the worst solution which in this example is C. The freed votes are given
back to the agents so that they can now assign them to different (remaining) solutions and the voting
starts over with the remaining solutions. Assume the following assignments:
Agent 1 assigns 3 votes to A
Agent 2 assigns 3 votes to B

Agent 3 assigns 3 votes to E
This leads to a new situation:
Solution Votes
B 10+3=13
20
A 8+3=11
E 5+3=8
D 4
The hare system now starts again with eliminating the worst solution which in this example is D.
Assume the following assignments:
Agent 3 assigns 4 votes to E
This leads to a new situation:
Solution Votes
B 13+4=17
A 11+4=15
E 8+4=12
The hare system now starts again with eliminating the worst solution which in this example is E.
Assume the following assignments:

This leads to the final situation:
Solution Votes
B 17+12+12=41
A 15+12=27
The algorithm stops here since the compromise is solution B:

Solution Votes
B 41
3.2.6 Additional re-voting strategies
In our negotiation algorithm we used the strategy to assign the freed votes completely to the next
favored solution. Also possible (but not evaluated in detail) is the usage of a roulette wheel to not
only assign the votes to the next favored solution. An agent could set the votes by a given probability
to a set of favored solutions.
21
3.2.7 Update
The corresponding genotype of the agreement solution is used to update the pheromones. The
updater traverses over all sub tree matrices of the pheromone graph (see Fig. 9). The updater
executes two steps to update the pheromones:
1. Evaporation (for all cells)
2. Accumulation (only cells where genotype has “1”)
3.3 Results
Many tests showed that the ant system in conjunction with the cooperation pressure negotiation,
good values (13 % deviation to X-Man) can be achieved with of the proposals. Within the last
of the 200.000 proposals no better (like the central) solution was found. We found out by
experiments that 20.000 to 30.000 proposals are enough to get the maximum convergence possible
with the given system. Also the hidden central solution which could be extracted from within the 500
proposals does not reach the central X-Man solution.
Convergence - Class 2 Instance 1

450000
400000
350000
300000
250000
Fitness
Ant 5
200000 Zentral
150000 Julian 5
100000
50000
0
1
85
169
253
337
421
505
589
673
757
841
925
1009
1093
1177
1261
1345
1429
1513
1597
1681
1765
1849
1933
Fig. 16 Convergence of Class 2 Instance 1 (Round 1 - 1933)
22

450000
400000
350000
300000
250000
Fitness
Ant 5
200000 Zentral
Julian 5
150000
100000
50000
0
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79
Fig. 17 Convergence of Class 2 Instance 1 (Round 1 - 80)

1200000
1000000
800000
Ant 5
Fitness
600000 Zentral
Julian 5
400000 Fink 5
200000
0
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79
Fig. 18 Convergence of Class 2 Instance 1 (Round 1 - 80) with Fink 5
23

1200000
1000000
800000
Ant 5
Fitness
600000
Zentral
Julian 5
400000
Fink 5
200000
0
1
85
841
169
253
337
421
505
589
673
757
925
1009
1093
1177
1261
1345
1429
1513
1597
1681
1765
1849
1933
Fig. 19 Convergence of Class 2 Instance 1 (Round 1 - 1933) with Fink 5
Like denoted in 3.3 the ant system shows quick convergence with less proposals compared to the
fink approach, which needs 10 times more proposals to get similar results. When regarding Fig. 18 it
is clear to see that Fink with 5 agents does not show central-like convergence in early rounds. From
round 1 to 80 the Fink 5 graph has random-access characteristic whereas in the big picture (round 1 -
1900) there is a slow but clear convergence.
24
Theoretical
social welfare
Fig. 20 Acceptance of worst cost to accieve better overall result
3.3.1 Class 2
1 Agent 5 Agents 10 Agents 15 Agents vs. X-Man
1 217.022,16 234.115,55 235.587,75 246.391,50 196.189,15

2 172.921,75 174.483,75 193.669,25 185.144,95 165.838,45
3 213.370,59 234.067,50 240.248,25 226.716,10 201.226,35
4 200.663,75 225.074,00 226.616,95 474.054,25 188.010,30
5 188.309,84 185.653,25 212.138,75 218.185,90 161.603,85
6 417.106,16 464.772,60 516.745,10 498.041,05 344.795,00
7 373.984,00 413.924,80 485.787,75 442.806,25 292.908,30
8 472.694,66 434.255,20 526.088,95 500.115,10 355.111,40
9 411.961,00 412.507,20 505.699,40 533.111,80 325.606,60
10 487.103,81 496.831,21 568.829,55 500.851,15 386.082,40
11 216.440,41 219.227,45 229.211,90 246.239,65 179.761,45
12 161.272,20 197.427,50 197.046,45 207.217,15 155.938,20
13 206.849,05 210.533,60 239.614,70 237.145,75 183.219,10
14 146.777,09 174.437,20 199.422,65 203.086,30 136.484,75
15 225.302,80 230.921,35 234.821,95 470.618,84 187.042,10
16 439.975,44 466.294,05 499.546,10 492.328,05 341.215,90
17 494.328,41 490.796,90 515.395,20 520.977 378.845,10
18 436.223,16 440.960,00 483.108,95 472.124,30 347.140,80
19 518.536,09 536.060,14 544.816,20 811.789,83 412.654,90
25
20 523.634,50 548.565,85 669.082,46 610.934,65 392.010,50

21 154.391,09 158.508,05 198.079,50 182.360,35 148.003,70
22 206.563,84 207.774,70 219.246,45 663.641,09 197.694,80
23 178.224,75 197.079,00 194.416,95 197.330,05 160.692,90
24 192.610,25 213.031,40 209.333,80 723.094,69 184.358,00
25 162.916,09 164.891,50 184.404,45 178.752,10 161.457,00
26 418.700,09 424.270,70 470.412,75 436.714,20 344.969,70
27 412.671,25 422.821,00 531.992,30 431.676,25 352.902,55
28 411.362,81 428.481,95 569.403,39 445.931,40 357.438,65
29 475.743,09 490.704,25 503.560,21 466.716,15 411.687,00
30 488.241,00 496.209,80 512.249,45 511.566,15 401.732,20
31 212.601,25 224.011,05 234.210,10 359.998,70 185.374,10
32 215.918,66 216.630,00 225.064,90 556.329,90 185.693,75
33 235.087,00 242.253,50 251.302,85 244.448,05 192.793,85
34 156.498,55 177.195,75 185.661,70 183.051,25 136.848,15
35 205.773,84 200.468 212.994,65 215.297,25 166.129,05
36 375.663,25 394.054,25 433.112,40 420.889,15 291.141,20
37 413.287,84 411.707,05 442.790,75 439.232,50 339.923,75
38 414.968,75 425.899,95 446.588,55 439.646 319.986,90
39 477.663,94 498.260,09 553.233,70 524.626,04 367.798,10
40 387.647,75 396.701,45 411.424,65 402.112,15 306.313,95
12.721.011,96 13.181.862,54 14.512.961,76 16.121.292,99 10.544.623,90
Overall fitness with different numbers of agents

16.500.000
16.000.000
+ 11.08 %
15.500.000
15.000.000
Overall fitness
14.500.000
+ 10,1 %
14.000.000
13.500.000
+ 3.62 %
13.000.000
12.500.000
12.000.000
1 Agent 5 Agents 10 Agents 15 Agents
Fitness 12.721.012 13.181.863 14.512.962 16.121.293
Fig. 21 Impact of the number of agents on the overall fitness (Class 2 Instance 1)
As Fig. 21 shows, the number of agents affects the overall fitness. This is a result of the cooperation
pressure because less candidates can be found where all agents are happy. Therefore the more
agents are involved the higher is a potential compromise.
26
3.3.2 Class 3
5 Agents 10 Agents vs. X-Man
1 1.760.135 1.477.817 649.404,90

2 1.474.474 1.936.452 901.722,00
3 2.770.508 2.448.888 967.572,80
4 2.026.614 1.690.801 1.042.986,25
5 1.903.548 1.852.828 1.259.484,80
6 10.524.619 10.387.318 8.378.639,80
7 5.519.378 5.525.563 4.446.423,80
8 4.917.822 5.036.400 3.980.492,80
9 3.943.231 4.100.261 2.832.215,85
10 3.937.209 3.952.743 2.338.951,55
11 73.888.276 72.598.601 59.866.085,05
12 16.970.360 17.414.704 14.151.135,05
13 6.333.593 6.221.779 5.554.817,75
14 5.147.306 5.100.651 4.330.812,85
15 4.014.795 4.076.122 2.766.770,05
16 563.177.489 563.939.298 474.824.288,80
17 21.129.566 21.355.730 19.208.446,10
18 8.842.739 8.868.713 7.877.041,80
19 5.428.176 5.535.165 4.884.188,85
20 4.455.019 4.475.346 3.615.695,10
21 4.142.252 3.481.320 1.361.743,25
22 4.270.157 4.588.568 1.494.471,10
23 4.229.881 4.485.302 1.539.715,65
24 4.449.733 4.514.552 1.582.423,30
25 4.836.948 4.443.349 1.855.681,00
26 17.376.206 17.352.993 15.155.218,30
27 10.243.241 10.540.241 9.245.133,00
28 7.430.807 7.449.742 6.381.105,40
29 6.357.898 6.448.121 4.613.693,65
30 5.932.028 5.812.067 3.514.517,40
31 128.126.992 129.832.369 106.285.999,50
32 23.636.635 23.900.097 20.638.228,90
33 9.469.812 9.475.765 8.731.207,30
34 7.299.323 7.239.012 6.382.225,75
35 6.231.974 6.248.859 4.282.735,90
36 861.255.265 874.780.145 780.647.327,50
37 37.237.522 38.410.208 34.831.269,70
38 12.822.524 13.001.971 11.766.675,20
39 8.116.867 8.143.566 7.556.001,15
40 7.174.304 6.988.286 5.601.051,85
1.918.805.226 1.935.131.710 1.657.343.600,75
27
3.3.3 Comparison
Class 2
X-Man 10.544.624
Ant 1 12.721.012 20,6%
Ant 5 13.181.863 25,0%
Ant 10 14.512.962 37,6%
Ant 15 16.121.293 52,9%
Class 3
X-Man 1.657.343.601
Ant 5 1.918.805.226 15,8%
Ant 10 1.935.131.710 16,8%
The class 3 shows very good behavior with 5 agents whereas with class 2 and 5 agents it is 15%
higher.
28
4 Distributed System Architecture
4.1 Abstract Communication
The abstract communication between agents and the mediator can be described as a 4-step protocol
like in shown in Fig. 22. The same applies to a system with n agents shown in Fig. 23.
(1) generate/update genotype
(2) Offer X contracts

Mediator
1 Round
Agent
(4) Return X rating/ranking/etc
(3) Evaluate X contracts
Fig. 22: Abstract communication between agent and mediator
Mediator
Agent Agent Agent
Fig. 23: Communication of 3 agents with mediator
29
4.2 Fundamental dataflow
The system gets initialized by a given genotype which we define as the central genotype, the first
(e.g. random) solution. From round to round, the mediator updates this current genotype to
calculate an iterative optimized solution with a free customizable strategy. The mediator is able to
derive (presumably better) solutions (in fact, rather the negotiation strategy used by the mediator).
Agents evaluate these solutions and generate e.g. ranking information. With this information, the
mediator again changes his current solution accordingly.
Note: This technique allows delivery of multiple contracts at the same time. This must effectively not
be used by the negotiation strategy. A simple approach could be, not to send multiple contracts at
the same time but sending the current contract and in turn update exactly this solution.
Start
Mediator
Solution update
derive
current
solution
Solution Result
Solution Solution
Solution Solution
contracts
Agent
Agent evaluate
Agent
Fig. 24: Logical dataflow in a calculation round
30
generation
Coding Mediator
Matrix
decoding
ANT
Production Negotiation Core
Information
Matrix e.g. EA
hiding
Logical decoupling
contract ranking
Agent
Agent
Agent
Agent
Fig. 25 Ant Architecture
4.3 RMI Service Architecture
The distribution of the functionality is achieved using RMI1 and a client/server architecture. The
logical server hosts the RMI naming service, the so called RMI registry. The mediator represents the
logical server which spawns many agent processes. (If using a distributed environment, agents have
to be started on the according machine.)
The agent registers an AgentService at the given registry and signs up at the mediator. Finally the
mediator starts negotiating the agents via its MediatorService.
Note: this basic communication framework is decoupled from any concrete negotiation strategy,
which has to be defined on top of this communication component (see 4.5 on page 34).
1
http://java.sun.com/javase/technologies/core/basic/rmi/
31
TCP Port: 2000 Server
Mediator
RMI Registry
publish Service
service
Spawns RMI Transport: JRMP
Agent handshake
Agent
Service
Agent
Service
Agent
Service evaluate contracts
Service
Client
Fig. 26: RMI service model
The agent and the mediator are designed to live in different VMs and different physical machines.
Each agent spawns its own process containing a corresponding worker thread. The mediator
accordingly lives in an own process with a worker thread. To simplify things, both mediator and
agents are allowed to run in the same process, spawning separate threads. This enables pragmatic
tests or development on a local machine without the hassle of multi-pc issues.
4.4 Implementing custom strategies
Own negotiation strategies can be integrated easily by customizing the base communication system
without changing it. What makes a negotiation strategy up? The negotiation strategy is contains an
agent part which evaluates contracts and generates some sort of quality information like ranking or
rating. The strategy also contains a mediator part which interprets this quality information and
manipulates the genotype to again (calculate and) send new contracts to the agents.
32
Start Mediator Strategy Part

get problem Agent Strategy Part
get partitions
data
(problem data)
e.g. random get initial genotypes

genotype
e.g. produce 1 Round

ranking of Agents
contracts evaluate X contracts
evaluate X contracts
evaluate X contracts
modification
of genotype
get next genotypes
genetic logic
goes here
stop negotiation?
End
Fig. 27: Logical extension-points at the customizing interface
As seen in Fig. 27 the whole negotiation strategy can be customized. The system has no limit for
generating quality information on the agent side. Any complex data structure which can be serialized
by Java may be used by agents to exchange with the mediator. Every node in the figure above
represents a function within ICustomNegotiationStrategy which represents such a
customizing.
4.4.1 Terminology
„Contract“ in the previous sections, has the logical meaning. Technically we have the following:
33
One Solution
Solution 1
based on a
genotype X
Based on
Partitions
Agent 1 Agent 2 Agent 3
Solution Fragment Solution Fragment Solution Fragment
Contract (item1) Contract (item4) Contract (item7)
Fig. 28: Solution Fragments based on partitions (1 Agent handles X Items)
4.5 Benefits from black-boxing
The complexity of client/server communication and protocol overhead can be encapsulated in a

black box which can be customized by negotiation strategy. The main benefit is the definition of the
logical steps of the fundamental problem solving. Also can be the focus on developing negotiation
strategies based on a framework, which reduces complexity. (Like providing matrices or vector
classes, fitness calculation, lot-sizing and so on.)
Configuration and analysis GUI
Optimization Algorithm Negotiation Strategy
Covers MLULSP Lotsize

Problem
Mediator/Agent Communication Framework &
Data
communication Fitness
Reader
and distribution Algorithm
RMI
Runtime System
Java 5
Fig. 29: Logical component stack
34
4.6 Visualization
Below the problem instance ph2in1.req in a tree format showing the supply chain order. The colors
and shapes are in respect from the different agents administering each item, while the size is
according to the setup cost of each item.
Fig. 30 Supply chain order
With a radial view from the same graph, it is easier to see the distribution/partitions of the supply
chain, with one agent (blue) dominating the top three levels of the chain, while other two agents
(green and cyan) share the middle products and the last agent (red) dominates the primary (basic)
products.
35
Fig. 31 Radial supply chain order
The graph on top represents the minimum (orange), average (green) and maximum (red) costs used
each round to evolve the negotiation. As seen the evolution of the best solution (orange) is almost
linear with a very small decreasing angle and deviation, while the worst solution (red) per round also
decreases linearly, but with a greater deviation from the axis. With the help of the average (green)
cost, it can be noticed that while the worst solution in a round varies greatly, the population itself
follows the linear distribution from the best solution, from that it can be confirmed that random bad
individuals, can help find an overall best solution, without jeopardizing the population, as seen in
rounds 8 and 28, for example.
The graph below shows the cost per agent, where it is easily seen that, to achieve the best solution, a
compromise is found in the negotiation. Most agents start with a low cost, except for one agent,
which starts with a very high cost. In order to be able to decrease the high cost from this agent, the
other agents accept to increase slightly their costs, finding then, a compromise for all.
36
Fig. 32 Realtime scatter plot that shows the cost with the time
37
5 Conclusion
The main result of this project work is a new negotiation approach called „cooperation pressure“.
This approach utilizes an ant system which has been introduced by Prof. Dr. Homberger. In our
research we evaluated different genetic algorithms like a single EA (evolutionary algorithm) or
another representation of ant. Finally we based our negotiation on Prof. Hombergers representation
oft he ant system since it showed the best convergence.
The negotiation algorithm aims at the social welfare between items and agents while optimizing the
overall cost without central information. The whole optimization process is decentral, the agents do
not exchange cost information. The cooperation pressure is a technique which increases the pressure
to cooperate with other agents with the time. In the beginning oft he negotiation all agents may
influence the overall solution very well, whereas in later times the agents have to drop some of their
favored solutions to find a compromise with other agents.
The system shows good convergence and solution quality on big problem instances like Class 2 or 3
with 15 % to 30 % deviation to a central optimizing approach.
Furthermore a Java/RMI framework for distributed negotiation algorithms has been created to
accomplish the development of decentral negotiation approaches. See the appendix where to find
more information about this project.
38
6 Further Information
In this chapter we describe how we accomplish our data management and distributed project work,
since all team members are from different regions and we worked decentralized – just like agents .
When the project is going to finish, the better way of putting all the stuffs what we have in a safe
manner is to keep the documents in a web server. And also publish the necessary information as a
website, so that this could act as a black board and storage. In our project we have stored our data in
a web server and published the contents under a domain.
Domain url : http://www.softwareproject.robin.co.in/
6.1 Server Specification
Processor : Xeon QuadCore 2.4Gigahetz.

Memory : 3 Gigabyte.
Hard Disc : 250 Gigabyte SATA
Bandwidth : 100mbit Port
Operating System : CentOS
6.2 Content Management System
The content management system [CMS] used is Joomla 1.5 . The main reasons to use this content
management system are..
1. Absolutely open source.

2. Support of more than 2000 modules.
3. Easier to maintain.
4. Can be controlled remotely using a web browser.
5. Supports Linux + Apache + MySQL + PHP [LAMP]Technology.
6.2.1 Framework for the Website
This part describes the frame work we have developed on the Extension tier of the Joomla 1.5
architecture.
39
Fig. 33 CMS Overview
The frame work for the web pages is done on the extension layer of the the Joomla Frame work.
Different Components, contents, sections and modules are mounted on the this Framework. The
main part of the Frame work is the article structure. One article is under a category, one category is
under a section and one section is mounted on the extension layer. Multiple sections and categories
are allowed. It improves the scope of the website and also extendibility of the framework.
6.2.2 Section Management
There are 6 sections in the frame work. They are Status, About the Project, Documentation,
Strategies, Solutions and Downloads. Each of these sections contains different categories.
6.2.3 Category Management
1. Status
The section status contains two categories namely ”Latest “and “News Flash” . Both of them
contains relevant articles. News flash is used to display quotations on the top of the page. Latest is
used to update latest news about the project. This category is called to the home page.
2. About the Project
The structure of the section “About the Project” had the same structure as section status. The
difference is the categories and articles. Here categories are the project ( introduction of the project),
Resource (A package for all the resources available) and Architecture, which describes the
architecture we used in our project.
3. Documentation
This section improves the flexibility of the process of documentation. This section has the structure
as follows.
40
Documentation
Ashley
Robin
Beini article
Marco Paulo
article
Jing
article article
article
article
This structure make it easy to add or edit an article for a team member. A team member has an
account in the server, which allows the member to log in to the system by a user name and a
password. Each user has a role assigned on the server, which allows the users to modify the article
or do administration tasks ( see 4.5. )
4. Strategies.
This section describes the information about different strategies used in the project, what are the
advantages, what are the disadvantages and why we use it or why we dropped it. This section had
different categories, and also each category contains relevant articles about strategies.
5. Solutions
This is also a section contains three categories. Categories are Class one, Class two and class three.
The structure of the section is same like above sections. Under each category , the solutions are
classified. This structure makes it easy to navigate through different solutions we obtained.
6. Downloads
This section act as an all in one package. Here the visitor can download all the contents about the
project, includes source code, jar files, documentation etc. But the access of this page is restricted to
public users. It requires special permission from the team members or administrator to download.
Contact page helps to get the information about team members.
41
6.2.4 Component Management
This part handles the different components like search engine, contact information , banners, web
links etc. these things are considered as secondary part of the website.
Each of these components has also categories. All components are handled by a MySQL database on
the server.
6.2.5 Menu Management
This part is considered as one of the core part of the website. Menu section helps to navigate
through the entire website. These menu items cover the navigation to all the sections and
categories. Different menu sections are
1. Main menu
2. Top menu
3. Resources
4. User menu
6.2.6 User Management
This section deals with the different registered users of the system. All the six team members are the
main registered users to the system. Each of them have assigned a role so that they can perform
different operations. All of them are publishers and editors, so that they can update the content of
the website. If a user is logged in to the system, they are allowed to do the modifications. The
navigation for the user operations will be displayed as user menu. Administrator has the power to
assign additional role to a user. Each member has a contact specification on the server. It uses PHP
mail function and “localhost” as SMTP host for emailing.
42
This is the home page of our website. Navigation to different section, categories and articles are
given in the home page.
43
This page gives the navigation to different categories under the section documentation.
44
7 Glossary
Solution Consists of a genotype, demand and production matrix. A Solution regards all
incorporated items and periods.
Contract A solution can be handeled by e.g. X agents. Therfore a solution can be divided in X
contracts.
Genotype Binary coding matrix which is used by the lotsizing algo.
45
8 The Team
Marco Maniscalco Ashley Muthiah
Paulo Santos
Robin George
Jing Wang Beini Sun
46

Cooperation Ant System

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Cooperation Ant System

Enviado por

Direitos autorais:

Formatos disponíveis

Cooperation Pressure Ant System

Ant-based Multilevel Unconstrained Lot Sizing

Distribution of the project roles:

Marco Maniscalco Architecture & Implementation of the Distributed System

Mentor Prof. Dr. J. Homberger

1.1 THE MULTI-LEVEL UNCONSTRAINED LOT-SIZING PROBLEM ................................................................................ 6

2 OTHER APPROACHES ............................................................................................................................. 10

2.1 SIMPLE EA .............................................................................................................................................. 10

3 COOPERATION PRESSURE ANT SYSTEM ................................................................................................. 13

3.1 GRAPH REPRESENTATION ........................................................................................................................... 13

4 DISTRIBUTED SYSTEM ARCHITECTURE ................................................................................................... 29

4.1 ABSTRACT COMMUNICATION ...................................................................................................................... 29

6 FURTHER INFORMATION ....................................................................................................................... 39

6.1 SERVER SPECIFICATION .............................................................................................................................. 39

8 THE TEAM .............................................................................................................................................. 46

1.1 The Multi-Level Unconstrained Lot-Sizing Problem

Fig. 1 Network of companies

Fig. 2 Graph of Items

Fig. 3 Vertical partitioning Fig. 4 Horizontal partitioning

1.2 Coding of Solution

Fig. 5 Terminology of the solution structure

1.3 Decoding and Evaluation

Binary solution Demand lot sizes Inventory

Fig. 6 Decoding of a solution

Agents Agents Evaluate

Fig. 7 Simple EA System

Agents Agents Evaluate

Fig. 8 Parallel EA System

3 Cooperation Pressure Ant System

3.1 Graph Representation

Item 1 Item 2 Item 3

Fig. 9 Pheromone graph

possible transitions is . Then we have in maximum

800 400 200 100 50 25

(2) Agents agree on contracts

(3) Used to update the

Fig. 10 Simplified Update Logic

3.2.1 Basic Agent Interaction

Fig. 11 Item Distribution

3.2.2 Cooperation Pressure

Fig. 12 Distribution of the rankings per round

A new border is introduced where initially and . The mediator

3.2.3 Ranking regression curve

Fig. 14 Logarithmic regression curve

Ranking 500 Solutions

Fig. 15 Voting System

3.2.5 Hare Voting System

Agent 1 Agent 2 Agent 3

The transformation from ranking to votes is given by . The global

The above table then results in the following merged table:

Agent 2 assigns 3 votes to B

Agent 1 assigns 12 votes to A

The algorithm stops here since the compromise is solution B:

3.2.6 Additional re-voting strategies

2. Accumulation (only cells where genotype has “1”)

Convergence - Class 2 Instance 1

Fig. 16 Convergence of Class 2 Instance 1 (Round 1 - 1933)

Convergence - Class 2 Instance 1

Fig. 17 Convergence of Class 2 Instance 1 (Round 1 - 80)

Convergence - Class 2 Instance 1

Fig. 18 Convergence of Class 2 Instance 1 (Round 1 - 80) with Fink 5

Convergence - Class 2 Instance 1