MA Wehinger

Master thesis
Agent-based modeling in electricity markets:

Introducing a new predictive agent learning
approach
Lukas A. Wehinger
PSL 1021
Department ETH: EEH - Power Systems Laboratory

Department CMU: ECE - Electrical and Computer Engineering
Advisors:
Prof. Dr. Gabriela Hug (CMU)
Matthias David Galus (ETH)
Prof. Dr. Göran Andersson (ETH)
Spring semester 2010

November 1, 2010
i
Preface
This research project is the final assignment of the Master program Energy Science
and Technology at the ETH Zurich. The project combines aspects from statistics,
control engineering, machine learning, micro economics and finance. This interdis-
ciplinary project allowed me to apply my knowledge gained from my undergraduate
and graduate studies at the ETH Zurich.
I owe many thanks to the Power Systems Laboratory at ETH Zurich and the
Electrical and Computer Engineering department at Carnegie Mellon University for
hosting the master thesis.
I am very grateful to Prof. Gabriela Hug and Matthias Galus for their supervi-
sion, support and valuable advices in many regards in elaborating this interesting
thesis. A special thanks also goes to Prof. G. Andersson from ETH Zurich, who
enabled to write the master thesis at Carnegie Mellon university in Pittsburgh,
Pennsylvania. Besides the research I had the unique opportunity to experience a
different culture, country and university. I will keep these experiences in good mem-
ory. Last but not least I thank the University staff and all the students I met here
who helped me in many different aspects within this project.
Pittsburgh, September 28, 2010 Lukas Wehinger
ii
iii
Contents
Abstract vii
Symbols viii
1 Introduction 1
1.1 The European electricity market . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Goals of the European Union . . . . . . . . . . . . . . . . . . 2
1.1.2 Implementation of the electricity market liberalization . . . . 2
1.2 Congestion management . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1 Capacity allocation methods . . . . . . . . . . . . . . . . . . 5
1.2.2 Implemented congestion management schemes and outlook . 6
1.3 Market-power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.1 Market-power in electricity generation . . . . . . . . . . . . . 8
1.3.2 Market power and congestions . . . . . . . . . . . . . . . . . 8
1.4 Contribution of the thesis . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Structure of the work . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Agent-based modeling in electricity markets 11

2.1 Multi-Agent Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Benefits of agent-based modeling . . . . . . . . . . . . . . . . . . . . 12
2.3 When are agent-based models useful? . . . . . . . . . . . . . . . . . 12
2.4 Using agent-based models for electricity markets . . . . . . . . . . . 13
2.5 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3 Specification of the model 18

3.1 Market participants and input data . . . . . . . . . . . . . . . . . . . 18
3.1.1 Market participant definition . . . . . . . . . . . . . . . . . . 18
3.1.2 Data sources . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2 Supply curves, market structure and market clearing . . . . . . . . . 21
3.2.1 Supply curves . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2.2 The market clearing . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.3 Time frame of the bidding process . . . . . . . . . . . . . . . 25
3.3 Cost of production and profit calculation . . . . . . . . . . . . . . . . 25
3.4 Simplifications of the model . . . . . . . . . . . . . . . . . . . . . . . 27
4 Learning-process of the agents: The methodology 28

4.1 Overall implemented structure . . . . . . . . . . . . . . . . . . . . . 30
4.2 The price predictor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.2.1 The multi-factor regression model . . . . . . . . . . . . . . . 31
4.2.2 Exogenous factor predictions . . . . . . . . . . . . . . . . . . 35
4.2.3 Analyzing the price predictor . . . . . . . . . . . . . . . . . . 39
4.2.4 Using the price predictor for out-of-sample data . . . . . . . . 44
iv
4.3 The price adjuster . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.4 The optimization routine . . . . . . . . . . . . . . . . . . . . . . . . 48
4.4.1 Introduction to dynamic programming . . . . . . . . . . . . . 49
4.4.2 The Bellman equation . . . . . . . . . . . . . . . . . . . . . . 49
4.4.3 Applying the dynamic programming algorithm . . . . . . . . 50
4.4.4 Discussion of the optimization routine . . . . . . . . . . . . . 62
5 Testing the agent-based model 63

5.1 Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.2 The training round . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.2.1 The spot price . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.2.2 Power outputs . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.2.3 Market-power . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.3 Second run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.3.1 The spot price . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.3.2 Power outputs and estimation error . . . . . . . . . . . . . . 68
6 Model application: The German electricity market 71

6.1 Simulation settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.2 Simulation results reference scenario . . . . . . . . . . . . . . . . . . 72
6.2.1 The training run . . . . . . . . . . . . . . . . . . . . . . . . . 72
6.2.2 The second run . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.2.3 The third run . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6.2.4 Conclusion of the simulation . . . . . . . . . . . . . . . . . . 84
6.3 Scenario analysis: Increase in wind generation . . . . . . . . . . . . . 85
6.3.1 Simulation setup . . . . . . . . . . . . . . . . . . . . . . . . . 86
6.3.2 The spot price . . . . . . . . . . . . . . . . . . . . . . . . . . 86
6.3.3 Total cost for consumers . . . . . . . . . . . . . . . . . . . . . 87
6.3.4 Power outputs and profits . . . . . . . . . . . . . . . . . . . . 88
6.3.5 Price predictor and market-power . . . . . . . . . . . . . . . 89
6.3.6 Conclusion of the wind scenarios . . . . . . . . . . . . . . . . 92
6.4 Scenario analysis: Adding storage devices . . . . . . . . . . . . . . . 92
6.4.1 The spot price . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.4.4 Conclusion of the storage scenarios . . . . . . . . . . . . . . . 100
6.5 Scenario analysis: Adding a PHEV cluster . . . . . . . . . . . . . . . 101
6.5.1 The PHEV cluster model . . . . . . . . . . . . . . . . . . . . 102
6.5.2 The spot price . . . . . . . . . . . . . . . . . . . . . . . . . . 103
7 Model application: The four-country model 106

7.1 Simulation settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
7.2 Cross-border capacity allocation . . . . . . . . . . . . . . . . . . . . 107
7.2.1 Total social welfare vs. total variable cost . . . . . . . . . . . 107
7.2.2 Formulating the optimization problem . . . . . . . . . . . . . 110
7.2.3 The optimization routine . . . . . . . . . . . . . . . . . . . . 115
7.3 Simulation results reference scenario . . . . . . . . . . . . . . . . . . 116
7.3.1 The spot price . . . . . . . . . . . . . . . . . . . . . . . . . . 116
7.3.3 Exchanges and congestions . . . . . . . . . . . . . . . . . . . 122
7.4 Scenario analysis: Building additional interconnection capacity . . . 126
7.4.1 The spot price . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7.4.2 Exchanges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.4.3 Profits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
7.4.5 Conclusion of the line scenarios . . . . . . . . . . . . . . . . . 133
8 Conclusion 134
Abstract
In this thesis, a new agent learning approach “model predictive bidding” is intro-
duced for modeling electricity markets with multi-agents. In contrast to traditional
agent learning approaches such as Q-learning or learning classifier systems this
learning approach is predictive and model based. The agents create an hourly price
forward curve based on exogenous factor predictions via a multi-factor regression
model, whereby the regression coefficients become more accurate as more market-
outputs are observed. Additionally, the agents consider their market-power for
creating the hourly price forward curve by measuring a “price adjusting value” for
every bidding curve with a reinforcement learning algorithm. The price adjusting
value describes how much the spot market price deviated from the price predictions
based on exogenous factors in the past. Based on these price forecasts the agents use
an optimization routine to find the set of actions which maximizes their expected
discounted profit over several time steps. In comparison to Q-learning and learning
classifier systems, this model based learning approach increases the learning rate,
uses less memory, optimizes over multi time steps and is able to handle unknown
environmental inputs.
An agent-based model is set up whereby the agents use the model predictive bid-
ding algorithm to model the German electricity wholesale market under reference
conditions as well as a higher wind energy contribution. Every generation technol-
ogy is aggregated and modeled by a single agent. Additionally, scenarios with a
storage agent which either has a high power output or a high storage capacity are
carried out to assess the impact of the storage agent on spot prices, price volatility,
market-power, agent bidding behavior and profitability. The storage agent thereby
uses the model predictive bidding algorithm for charging/discharging his storage.
In a last simulation scenario, a cluster of PHEVs is simulated which can be charged
or discharged via the electric grid and has a variable charging/discharging power ca-
pacity depending on the amount of PHEVs connected to the grid and their current
state of charge.
In a next step, the four countries Germany, Switzerland, France and Italy are
taken into account. These countries can exchange electricity, whereby the scarce
cross-border capacity between these countries is allocated with an implicit allocation
scheme where the total social welfare over the four countries is maximized. The
power generation technology in each country is modeled by an agent which uses the
model predictive bidding learning algorithm. The simulation is carried out under
reference conditions and under conditions where the net transfer capacities (NTC)
between certain countries is increased to assess the sensitivity of these transmission
capacities on spot prices, agent behavior, profitability and market-power. In a last
simulation scenario, the NTC values are increased that no congestions occur and
that the spot price is equal for every hour in all four countries.
vii
Symbols
Symbols
c∈C Country c in C, where C = {Germany, Switzerland, France, Italy}

λt,c Spot market price at time t in country c
bt,c
λ Spot market price prediction at time t in country c
aji,t Agent i’s bidding curve j at time t
pi,t Agent i’s power output at time t
total
V Ci,t Agent i’s total variable cost of production at time t
fc Specific fuel cost
η Generator efficiency
Λj,t Market price of fuel j at day t
δ Generator ramp cost constant
ϑ Tons CO2 emissions per M W h fuel burned
ϕt Price of one ton CO2 emission at time t
p
V Ci,t Agent i’s variable production costs at time t
r
V Ci,t Agent i’s ramping costs at time t
ea
V Ci,t Agent i’s emission allowance costs at time t
tech Generation technology tech ∈ {nuclear, hard coal, lignite, gas, oil}
πi,t Agent i’s profit at time t
x
γt,c Independent exogenous variable x in country c at time t
βcx Regression coefficient of exogenous variable x in country c
ǫt,c Disturbance term in country c at time t
w Weighting factor of the pav calculation
Υ = {µ0 , ..., µN −1 } Policy: an arbitrary set of actions
Υ∗ Optimal policy: set of actions with the highest expected profit
κ Discount factor
st State s at time t
ut Control input at time t
wt Random disturbance variable at time t
Vi,t Agent i’s value function at time t
pavi |aji Agent i’s pav value at time t given action j
pi,t |aji Agent i’s power output p at time t given action j
bt,c |aj
λ b in country c given agent i uses action j
Spot price prediction λ
i
viii
bt,c |ai
λ b in country c for all action curves of agent i
Spot price predictions λ
Vi,t |aji Agent i’s value function V at time t given action j
gi,t Agent i’s profit at time t
T Vi Terminal value i
fi,T Agent i’s filling status at time T
Σi Agent i’s maximum energy storage content
ηi,charge Agent i’s charging efficiency
ηi,discharge Agent i’s discharging efficiency
cumi,t Agent i’s filling status at time t for the optimal policy
R Return
Rf Risk free return
soc State of charge of the PHEV cluster
υ PHEV drive constant
P0 Total amount of PHEVs in the PHEV cluster
Pt PHEVs connected to the grid
Acronyms and Abbreviations
OTC Over the counter

ECC European Economic Community
TPA Third-Party-Access
UCTE Union for the Co-ordination of Transmission of Electricity
TSO Transmission system operator
ENTSO-E European Network of Transmission System Operators for Electricity
ATC Available transfer capacity
Belpex Belgian Power Exchange
APX Amsterdam Power Exchange
NordPool Nordic Power Exchange
EU European Union
EEX European Energy Exchange
LPX Leipzig Power Exchange
EET E.ON Energy Trading
RWE Rheinisch-Westfälisches Elektrizitätswerk AG
UK United Kingdom
NETA New Electricity Trading Arrangements
GA Genetic Algorithm
LCS Learning Classifier Systems
ETF Exchange Traded Fund
EIA Energy Information Administration
IEA International Energy Agency
EUA Emission Allowance Unit
DWD Deutscher Wetter Dienst
HPFC Hourly price forward curve
LF Load fraction
MSE Mean squared error
RMSE Root mean squared error
hdd Heating degree day
cdd Cooling degree day
OLS Ordinary least squares
RSS Residual sum of squares
pav Price adjusting value
VAR Value at risk
PAR Profit at risk
std Standard deviation
CMU Carnegie Mellon University
ETH Eidgenössische Technische Hochschule
Chapter 1
Introduction
“Most of the energy of political work is devoted to correcting the effects of misman-
agement of government”
Milton Friedman
The European Union tries to set rules for common and well-functioning electricity
markets and cross-border trade since the electricity market liberalization. In this
introduction, the regulations No 1228/2003 and its replacement 714/2009 which
specifies these rules are analyzed. Additionally, market-power often present in elec-
tricity generation is discussed.
1.1 The European electricity market
The European electricity industry has undergone substantial changes since the start
of the deregulation process in the early 1990’s. The electricity markets moved
away from vertically integrated monopolies to a liberalized market. The value
creation chain was unbundled into generation, transmission and distribution. This
development was driven mostly by economic reasons. It was argued that the market
forces leads to a more efficient and transparent electricity trading. Power exchanges
have been established where electricity is traded in spot and future markets [1] [2].
Cross-border acquisitions have surged as national companies try to diversify

their sources of revenue. Trading companies are operating on various European
power exchanges and are thereby providing liquidity to the various exchanges and
OTC markets. Cross-border trading activity has increased since the start of the
deregulation process in Europe and is promoted by the European Union to achieve
the goal of an internal market for electricity in Europe. These cross-border trading
activities are introducing new challenges. Some cross-border transmission lines are
congested which makes its access a scarce good. Implementing an efficient and
transparent allocation of cross-border capacity is a major challenge faced by the
European Union and subject to current debate [3] [2] [4].
1
Chapter 1. Introduction
1.1.1 Goals of the European Union
A common energy policy is a key goal since the foundation of the European Eco-
nomic Community (ECC). The European Union has set the following objectives for
a common electricity policy [2]:
• Establish of a competitive electricity market.
• Increase the efficiency in generation, transmission and distribution of electric-

ity.
• Increase the competition by liberalize markets for electricity generation and

by building high voltage transmission lines.
• Implement the Third-Party-Access (TPA) for industrial- and large-scale con-

sumers as well as distribution companies.
1.1.2 Implementation of the electricity market liberalization
The directive 2003/54/EG by the European Parliament and the regulation (EC) No
1228/2003 of the European Parliament and the Council was a first approach to set
common rules for an internal electricity market and cross-border trade in Europe.
They were a direct outgrowth of the work of the Florence Regulatory Forum. The
directive and regulation were in place from July 2004 until they were repealed by
regulation No 714/2009 [2] [5].
In the following, some central clauses of the regulation No 1228/2003 are high-
lighted. Clause no. 3 states that [6]:
“The creation of a real internal electricity market should be promoted

through an intensification of trade in electricity, which is currently un-
derdeveloped compared with other sectors of the economy.”
Furthermore, clause no. 4 in regulation No 1228/2003 states that [6]:
“Fair, cost-reflective, transparent and directly applicable rules ... should

be introduced with regard to cross-border tarification and the allocation
of available interconnection capacities, in order to ensure effective access
to transmission systems for the purpose of cross-border transactions.”
The regulation No 1228/2003 put a special emphasis on increasing competition

and cross-border trade in achieving the goal of an efficient pan-European electricity
market. The regulation highlights that although there is electricity trade between
member states, there is still no common single European market for electricity. In
a perfect market the prices for the same good should be identical for all consumers,
which is not the case for electricity in Europe. Electricity prices vary strongly from
one area to another. These variations are caused by congestions in the transmission
network [7] [1].
2
It is important to understand the underlying technical characteristics of the

transmission grid. Some facts are highlighted:
• The countries in Europe were considered as control areas, the original idea be-
hind control areas was the establishment of defined areas, in which generation
and consumption of electricity matches more or less at every point in time.
Neighboring control areas were connected to help within the UCTE area if
sudden unpredictable shortages in consumption or production occurred. The
original purpose of control areas concerning the balancing of production and
consumption is no longer conformed. With an increase in cross-border trade
since the liberalisation process the transmission lines connecting certain coun-
tries in Europe are often critically congested during the daily operation, which
means that these transmission lines are driven to their thermal or stability
limits, making these capacities scars [2] [1].
• The electricity in the network cannot be directed, rather it follows the ohmic
law. Physical electricity flow between two trading partners does not follow
the way of their economic trade. The physical flow affects other lines and
areas of the network as well. These so called ‘loop-flows’ cannot be avoided
and can spread over the whole UCTE network. As a result, network access
rules and the allocation of transmission capacity has to consider the physical
system properties [2] [1].
An increase in cross-border transactions and cross-border capacities helps devel-

oping a common single electricity market in Europe and promotes that electricity
is produced at lowest cost in Europe. The congestions in cross-border lines are
addressed in article no. 6 on “General principles of congestion management” [6]:
“Network congestion problems shall be addressed with non-discriminatory

market based solutions which give efficient economic signals to mar-
ket participants and transmission system operators involved. Network
congestion problems shall preferentially be solved with non transaction
based methods.”
Regulation No 1228/2003 specifies that congestion problems shall be addressed

with non-discriminatory and market-based approaches. A further definition of non-
discriminatory is not provided. It is also questionable which congestion manage-
ment scheme gives efficient economic signals. The regulation addresses the revenue
resulting from the allocation of interconnection capacity. The article defines that
the revenue should be used for one or more of the following purposes [6]:
• Guaranteeing the actual availability of the allocated capacity;

• Network investments maintaining or increasing interconnection capacities;
• As an income to be taken into account by regulatory authorities when ap-
proving the methodology for calculating network tariffs, and/or in assessing
whether tariffs should be modified.
Since 2009, regulation No 1228/2003 is repealed by the newer regulation No

714/2009 of the European Parliament and of the Council. The newer regulation
3
mentions that regulation No 1228/2003 made significant contributions towards the

creation of such an internal market in electricity by deliver real choice of electricity
supply for all consumers and by encourage cross-border trade. It results in efficiency
gains, more competitive prices and higher standards of service as well as contribute
to security of supply and sustainability. The new regulation also highlights some
shortcomings in developing an internal electricity market under the old regulation
[4]:
“However, at the present, there are obstacles to the sale of electricity on

equal terms, without discrimination or disadvantage in the Community.
In particular, non-discriminatory network access and an equally effective
level of regulatory supervision do not yet exist in each Member State,
and isolated markets exist. ... ‘Prospectus for the internal gas and elec-
tricity market’ and ‘Inquiry pursuant to Article 17 of Regulation (EC)
No 1/2003 into European gas and electricity sectors (Final Report)’
demonstrated that the present rules and measures neither provide the
necessary framework nor provide for the creation of interconnection ca-
pacities to achieve the objective of a well-functioning, efficient and open
internal market.”
This article attributes the source shortcomings in implementing the internal

market in electricity to the non-discriminatory network access and the creation of
interconnection capacities. The current congestion scheme does not provide the
right economic incentives. The Regulation provides a suggestion to overcome these
deficits. In clause no. 6 the Regulation notes that [4]:
“In particular, increased cooperation and coordination among the trans-

mission system operators is required to create network codes for pro-
viding and managing effective and transparent access to the transmis-
sion network across borders, and to ensure coordinated and sufficiently
forward-looking planning and sound technical evolution of the transmis-
sion system in the Community, including the creation of interconnection
capacities, with due regard to the environment.”
The transmission and distribution networks are still a natural monopoly in Eu-
rope because of their large economy of scale. They are characterized by high fixed
costs and comparably low variable costs. The European Union requires that the net-
works are owned by nationwide companies (TSOs). This federalistic implementation
of nationwide TSOs within Europe complicates the cooperation and coordination
among the TSOs. The European Network of Transmission System Operators for
Electricity (ENTSO-E) is an association of Europe’s TSOs. The association is an
attempt to ensure coordination of network operation. But congested power lines are
a source of revenue for the nationwide TSOs, an increase in cross-border capacity
could decrease the congestion revenues. Additionally, building new interconnection
capacities is oftentimes a difficult task. New power lines would have to be built in
populated areas where there is a public opposition against them [5].
4
1.2 Congestion management
In the following different capacity allocation methods are introduced. Examples of

implemented schemes in Europe are discussed to show how these methods are used
in reality.
1.2.1 Capacity allocation methods
In Europe, different capacity allocation methods are in use. They can be grouped
as follows [8]:
• First come, first served: Capacity is allocated to the order in which the trans-
mission request have been received by the TSO. Starting from the earliest
request all the requests are fully granted until the available capacity is used
up.
• Pro rata: Each applicant is granted a fixed share of his requested capacity
amount, the share being equal to the amount of available capacity divided by
the sum of all requested capacity amounts.
• Explicit auctions: In explicit auctions the seller determines available trans-

fer capacities (ATCs) considering security analysis. After the seller accepts
bids from potential buyers he allocates the capacity to the ones that value it
the most [9]. The following observations and financial implications regarding
explicit auctions can be stated [9]. Explicit auctions are
– non-discriminatory
– transparent
– often a joint co-ordinated mechanism between the concerned TSOs
– implemented with different features: uniform clearing price (in most
cases) vs. pay as bid
– reflecting the cost of using the cross-border capacity (with perfect market
assumption)
– not providing an opportunity for arbitrage. Internal and cross-border
trade present the same profit opportunity for participants
– sending efficient signals to market players for the operation and the value
of the network
• Implicit auctions: In implicit auctions an initial energy market clearing is

performed. If ATC are reached, markets ‘split’ into pre-determined price ar-
eas. These areas are then cleared individually. Therefore, implicit auctions
are often called market splitting or market coupling. The following observa-
tions and financial implications regarding implicit auctions can be stated [9].
Implicit auctions are
– non-discriminatory
– transparent
– a joint co-ordinated mechanism between the concerned TSOs
– requiring a centralized power exchange
5
– requiring financial instruments for long term price-hedging and bilateral

trade between price-areas
– not providing an opportunity for arbitrage. Internal and cross-border
trade present the same profit opportunity for participants
– sending efficient signals to market players for the operation and the value
of the network
The regulation No 1228/2003 and its repealing regulation No 714/2009 clearly

state that the implementation of market-based congestion management methods
are preferred. These methods should be non-discriminatory, market-based and give
efficient economic signals [4].
A market based allocation would imply that the price for the transmission ca-
pacity is equal to the opportunity cost of the capacity. In other words, the cost
for the marginal user of the scarce capacity should be equal to the price differential
between the electricity markets at either end of the transmission.
First come first served and pro rata methods of capacity allocation fail to meet
the criteria of being non-discriminatory and market-based. In both of the cases
the capacity allocation is made without considering the user’s value and there is
no guarantee high value users will gain access to the transmission in preference to
low value users. In contrast, the explicit and implicit auctions appear to meet the
criteria of being non-discriminatory and market-based [8]
Article [8] concludes that implicit auctions appear to be the economically most
efficient mechanism for allocating line capacity. Especially taking into account that
there is uncertain (or asymmetric) information and some players exercise market-
power (see section 1.3.2).
1.2.2 Implemented congestion management schemes and out-

look
The following table shows implemented congestion management schemes for selected
cross-border capacity allocation. The first table 1.1 represents the allocation in
the day-ahead time frame whereas table 1.2 the allocation for longer time frames
(months or years). The selected cross-border transmission lines in the tables are
the ones which are further investigated in this thesis.
In the following, other implemented energy and cross-border allocations schemes

in Europe are examined. The NordPool area consisting of Denmark, Finland, Nor-
way and Sweden use an implicit cross-border allocation scheme. The exchange
capacity allocation is an integral part of the day-ahead spot price calculation. Con-
gestion on interconnections between price areas results in price differences between
them. Congestions can occur on lines connecting neighboring countries or on lines
connecting areas within a country. If there are no congestions in the transmission
lines the spot prices in the price areas will be equal. This simultaneous allocation
of energy and capacity ensures a maximal utilization of exchange capacity.
The markets Belpex (Belgium), Apx (Netherlands) and Powernext (France)

use a trilateral market coupling and an implicit cross-border capacity allocation
6
Cross-border line Allocation method

Italy-Switzerland Implicit auction (or legal framework different from EU)
Switzerland-Italy Implicit auction(or legal framework different from EU)
France-Switzerland Priority list(or legal framework different from EU)
Switzerland-France No congestion
Italy-France Implicit auction (or no congestion)
France-Italy Explicit and implicit auctions
Germany-France Explicit auction
France-Germany Explicit auction
Switzerland-Germany Explicit auction
Germany-Switzerland Explicit auction
Table 1.1: Day-ahead allocation method, source [9]
Cross-border line Allocation method

Italy-Switzerland Auction of financial transmission rights
Switzerland-Italy Auction of financial transmission rights
France-Switzerland No long-term allocation
Switzerland-France No long-term allocation
Italy-France Auction of financial transmission rights
France-Italy Auction of financial transmission rights
Germany-France Explicit auctions of physical transmission rights
France-Germany Explicit auctions of physical transmission rights
Switzerland-Germany Explicit auctions of physical transmission rights
Germany-Switzerland Explicit auctions of physical transmission rights
Table 1.2: long-term allocation method, source [9]
mechanism. This example shows that a centralized energy market is not a nec-
essary requirement for implicit cross-border capacity allocation methods. A joint
co-ordinated mechanism between the concerned TSOs is sufficient to implement the
implicit allocation scheme [10].
There is a development toward a centralized European electricity market in Eu-

rope. Germany’s electricity exchanges, the European Energy Exchange (EEX) and
the Leipzig Power Exchange (LPX) merged in 2002. The new merged electricity
bourse is now merging with the French electricity exchange Powernext. E.ON En-
ergy Trading (EET), the E.ON group’s wholesale trading arm and major European
energy operator said it expects more consolidation and fewer exchanges in the fu-
ture. “The creation of marketplaces across national borders for different regions
is a natural step on the road to more integrated European power markets” (Tony
Cocker, chief executive of EET). This integrated electricity market will induce more
implicit capacity allocation method as they are used in the NordPool area.
1.3 Market-power
Market-power is understood as the ability of the market participants to deviate

prices from the competitive levels in a profitable way [11]. In electricity markets,
market-power describes the ability of market players to influence the spot market
price by their bidding. A higher concentration in electricity generation generally
leads to higher market-power. In contrast, in a competitive market structure, the
7
decisions on power output are unable to significantly alter the market price and the
market players act as “price takers”.
1.3.1 Market-power in electricity generation
Throughout the liberalization process in Europe there has been a movement from
a more fragmented and competitive electricity wholesale market to an oligopolistic
one where a few utilities share a major market stake in electricity production. In
the German electricity market for instance the overcapacity and the large number
of market players led to a rapid and considerable price drop at the beginning of
the liberalization process which reduced industry profitability. RWE announced a
rise in sale of 25% but 15% lower profits in the year 2000 [12]. This low profitabil-
ity prevented new market-entry and did not set incentives for new investments in
power generation capacity. The result was a consolidation phase where the Ger-
man electricity market transformed from one with eight major vertically integrated
generating companies and many smaller ones in 1999 to one where four dominant
firms have a combined market share of over 90% by the beginning of 2001 [12].
The consolidation in electricity generation in Europe has lead to a high con-

centration in power production where markets providesthe opportunity to exercise
market-power to these dominant firms. An efficient competitive market maximizes
the total social welfare, provides proper investment incentives and enhances the se-
curity of supply [13]. The concentration in power production severely impediments
this desired outcome and leads to uncompetitive price outcomes. Market-power in
the electricity wholesale industry is therefore a growing concern in the European
Union.
1.3.2 Market power and congestions
In [1] it is argued that market participants in Spain, California, the UK and Ger-
many were suspected to exploit market-power by driving the transmission system
to its limits [1]. By creating these artificially congestions in the network, the market
is split into a high and low priced area. A market participant in the high price area
has an incentive in this market split.
Additionally, under explicit auctions, market participants know the outcome of

the transmission allocation before they decide how to respond in the energy mar-
ket. An option is granted to the players which obtained transmission capacity. A
participant who can exercise market-power would have a different incentive for the
price differential between the two markets if he gains significant transmission ca-
pacity than if he has obtained no or only a small amount of transmission capacity.
This implies that a market participant influences the market outcomes in the two
energy markets depending on his position in the transmission market. This sys-
tematic dependency between transmission and energy market outcome may allow
the dominant players to captures rent from others, even under the assumption of
perfect information [8]. The allocation of line capacity is not predetermined in im-
plicit allocation schemes, but there might still be ways for market participants with
market-power to gain advantage of congested lines by observing historical corre-
lations between their bidding behavior and the price outcome under transmission
congestions. How these market-participants can capture rent is not as straightfor-
8
ward as it is in explicit auctions [8].
1.4 Contribution of the thesis
To establish a well-functioning market for electricity which sets the right incentives
and leads to an optimal outcome in terms of social welfare is not a trivial task
[14]. This thesis aims to simulate the market outcome of an integrated electricity
market in Central Europe where market-power in electricity generation is present.
In particular, the countries Germany, France, Switzerland and Italy are examined.
A multi-agent model is carried out for the simulation to incorporate the effect of
market-power. The following tasks are performed in the thesis:
• Setting up a multi-agent model for the four countries.
• Introduce a new agent learning methodology. The new learning process pro-
vides a faster learning rate than comparable classifier based learning ap-
proaches.
• Implement an implicit cross-border allocation scheme and analyze the price

outcome.
• Assess the effect of a higher wind energy contribution in Germany.
• Analyze the price outcome in Germany by introducing a higher electricity

storage capacity with an advanced charging/discharging scheme.
• Introduce an PHEV (plug-in hybrid electric vehicle) cluster which is charged/discharged

via the electric grid and assess its effect on the spot market.
• Perform a sensitivity analysis on spot prices in these four countries by changing

the cross-border capacities.
1.5 Structure of the work
The thesis is structured as follows: In chapter 2 an introduction into agent-based

modeling is given including a discussion on the advantage of agent-based models
over other modeling approaches. A special emphasis is put on introducing agent-
based models for electricity markets. In chapter 3, the agent-based model used in
this thesis is specified. The market participants and the market clearing process are
introduced and the data sources are defined. Furthermore, the simplifications of the
model are discussed. Chapter 4 presents a new learning algorithm model predictive
bidding, which is a enhancement of current learning algorithms and provides a fast
and efficient learning for the agents. The agent-based model based on the derived
learning algorithm is applied to the German electricity wholesale market. The re-
sults are discussed in chapter 6. Scenario analysis es are carried out to assess the
effect of a higher wind energy contribution, additional storage devices and a PHEV
cluster on electricity spot prices. In chapter 7, the agent-based model is applied
to the four countries Germany, Switzerland, France and Italy. The cross-border
capacity allocation scheme is introduced which maximizes the total social welfare.
The simulation results are presented under a reference scenario and scenarios with
9
increased cross-border capacities. In one simulation scenario the cross-border ca-

pacities are set to infinity to assess the outcome under a copper-plate model.
10
Chapter 2
Agent-based modeling in
electricity markets
“Experience is the teacher of all things”

Julius Caesar
Several modeling approaches have been proposed to assess and evaluate the inter-
actions between market players of electricity markets. Hobbs et al. distinguish the
following approaches in [15]: a) analysis of existing markets, b) market concentra-
tion analysis using current market data, c) equilibria analysis and d) multi-agent
modeling. The interaction between the agents in in multi-agent models thereby
ranges from intense competition to collusion.
An agent-based model is derived to model electricity markets in this thesis.

First, a definition and analysis of agent-based modeling is provided. Furthermore,
it is shown how these models can be applied to electricity markets.
2.1 Multi-Agent Models
Historically, economists and engineers have addressed questions about how decisions
are made with aggregated models, assuming perfect information and a rational
behavior. The derived equilibria were societally ‘optimal’ [16].
In recent years, a disaggregated modeling approach has advanced. They are

referred to agent-based models. These models have been motivated because of
limitations in traditional economic assumptions and approaches and were enabled
by computational advances [17]. Agent-based models use autonomous decision-
making entities called agents. The decisions are taken based on a set of rules
whereby these rules are based on heuristic decisions. The agents are characterized
by bounded rationality and are able to learn, adapt and reproduce [18].
The agent-based models are a class of computational models. They put a spe-
cial emphasis on the interactions of agents to assess their effects on the system as
a whole. A key notion is that simple behavioral rules on a micro-level generate
11
Chapter 2. Agent-based modeling in electricity markets
complex outcomes on a macro-level. A main feature of the agent-based models is

the repetitive competitive interactions between the agents [17] [18].
According to [19], the most general characterization of agents is given by the

following properties:
• autonomy: agents operate without the direct intervention of humans or others,

and have some kind of control over their actions and internal state.
• social ability: agents interact with other agents (and possibly humans) via
some kind of agent-communication language.
• reactivity: agents perceive their environment, (which may be the physical
world, a user via a graphical user interface, a collection of other agents, the
internet, or perhaps all of these combined), and respond to occurring changes
in a timely fashion.
• pro-activeness: agents do not simply act in response to their environment,
they are able to exhibit goal-directed behavior by taking the initiative.
The simulated evolution of the system can be studied both from the perspective
of the aggregate population as well as the individual agent behavior.
2.2 Benefits of agent-based modeling
Agent-based model offer three main benefits over other modeling techniques [18].
Agent-based models:
• capture emergent phenomena, these phenomenas result from the interaction of

the individual entities. “The whole is more than the sum of its parts because
of the interactions between the parts” [18]. These emergent phenomenas can
have properties which are decoupled from the properties of the single entities.
• provide a natural description of a system. If the system is composed of be-
havioral entities, agent-based models are most natural and closer to reality to
model these systems.
• are flexible. The flexibility comes in different dimensions. More agents for
instance can be added, the complexity of the agent, their behavior, degree of
rationality, ability to learn and evolve can be tuned.
2.3 When are agent-based models useful?
In [18] a summary is given about usefulness of agent-based models. The following

situations are mentioned. Agent-base models are useful when:
• the interaction between the agents is complex: For instance nonlinear, discon-
tinuous or discrete.
12
• space is crucial and the agents are able to move.
• the agents exhibit complex behavior, including learning and adaption.
2.4 Using agent-based models for electricity mar-

kets
As described in 2.1, agent-based models allow to analyze the interdependencies of

the micro level (participants) and the macro level (the overall market structure).
This bottom-up modeling approach appears suitable to assess the evolution of elec-
tricity prices. Power generating units can be modeled as agents. They fulfill the
basic requirements for agents given by [19] in section 2.1. The power generating
units are autonomous, they have a social ability in a sense that they interact on a
market place with each other and reactivity because they collect inputs about their
state and the environment to decide on their actions. Furthermore, these units use
a pro-activeness to try different actions to increase their profit.
The interaction between the agents (power generating units) is complex and the
agents are able to learn effects related to repetitive behavior and to incorporate the
effect of market-power [20], [21]. An oligopoly market is assumed. Thus, electric-
ity producers may bid strategically above their marginal cost as they realize their
possible influence on market prices [22].
No assumptions concerning the attainment of equilibrium are made in model-

ing an electricity market. There is no guarantee that the aggregated behavior of
the system is stable if every agent is trying to maximize its profit. Rather, if an
equilibrium is obtained or not is assessed from the actual model execution [17].
Most modeling studies assume that there is a bidding process in a central market
place supervised by a TSO which results in a set of market clearing prices. Most
studies neglect bilateral OTC trading and in terms of network modeling most studies
disregard transmission constraints entirely [15]. In the model used in this thesis
OTC trades are neglected as well, but transmission constraints on cross-border
transmission lines are taken into account as in the work presented in [20].
2.5 Related work
A first group of agent-based models is referred to as simulations applying model-

based adaption algorithms according to [23]. These earlier methodologies have naive
or intuitive learning formulations and are tailored for specific designs of the markets
they simulate. Most of the work in this field has been conducted at the London
Business School.
• John Bower, Derek W. Bunn et al. from the Energy Markets Group at the
London Business School presented agent-based models mainly for the Eng-
land and Wales electricity market in the year 2000. Their simulation results
in Model-based comparison of pool and bilateral markets for electricity [24]
13
and Experimental analysis of the efficiency of uniform-price versus discrim-

inatory auctions in the England and Wales electricity market [25] compared
different market mechanisms and bidding schemes. The generators applied
a simple reinforcement learning algorithm with the goal to maximize profits.
The agents adjusted their bidding strategy to their last round success. Trans-
mission costs were thereby neglected. The simulation results were validated
against classical models of monopoly, duopoly and perfect competition.
• In 2001, John Bower, Derek W. bunn and Claus Wattendrup applied the
same model to the German electricity wholesale market in A model-based
analysis of strategic consolidation in the German electricity industry [26]. The
authors analyzed the impact of mergers and concentration in the German
utility industry on spot prices. They concluded that electricity prices would
rise considerably as an effect of mergers and acquisitions.
• In Agent-Based Simulation - An Application to the New Electricity Trading
Arrangements of England and Wales [27], Derek Bunn and Fernando Oliveira
presented an application of the agent-based model to the proposed new elec-
tricity trading arrangements (NETA) in the U.K. In contrast to the models
described above, the authors incorporated an active demand side. A bilat-
eral forward market followed by a balancing mechanism was considered and
modeled. This agent-based simulation model provided insights to pricing and
strategic decisions ahead of NETA’s actual introduction.
• Marija D. Ilic and Poonsaeng Visudhiphan modeled electricity markets by
agents, which adjust their bidding strategy by using information about past
market prices and their own marginal cost curves. They showed that repeti-
tion of bidding plays a significant role in the market dynamics and that tradi-
tional static market models are not capable of showing these dynamics. In the
thesis An Agent-based Approach to Modeling Electricity Spot Markets [28] P.
Visudhiphan used and Auer et al’s softmax action selection algorithm in de-
termining the generator’s bid supply functions. Auer et al’s algorithm assigns
a probability to each possible action. The probability of choosing an action is
adjusted every round based on the rewards received from the chosen action.
An other group of researcher uses genetic algorithms (GA) for the agent learn-
ing process. GA are heuristic methods inspired by biological evolution. In agent
learning methodologies with GAs, the market participant’s strategies are coded into
bitstrings. The fittest strategies are thereby passed from one generation to the next
by the production of offsprings. By using crossover and mutations, the GA are able
to model the genetic dynamics underlying natural evolution. In electricity market
agent-based models, the GAs can give the agents the ability to search for optimal
bidding strategies [23].
• James Nicolaisen et. al. used GAs for the learning mechanism of the buy-
ers and sellers in Concentration and Capacity Effects on Electricity Market
Power [29]. The GA was used to determine the agent’s bid and ask prices.
Different ratios of buyers and sellers as well as buying and selling capacities
were investigated. The market power was measured in terms of deviation from
competitive equilibrium.
• In A Co-evolutionary Approach to Modeling the Behavior of Participants in
Competitive Electricity Markets [30] the agents observed a discrete state of
the environment. Based on this state, they chose an action in the set of
14
bidding actions. The agents co-evolved their bidding strategy by using a

genetic algorithm. The paper concluded that a state-based bidding strategy
helps facilitate the learning process.
Alvin E. Roth and Ido Erev demonstrated that reinforcement learning can make
useful predictions in experimental games. The reinforcement algorithm presented
in Predicting How People Play Games: Reinforcement Learning in Experimental
Games with Unique, Mixed Strategy Equilibria [31] is based on psychological find-
ings. It incorporates the aspects of experimentation and forgetting. A considerable
amount of agent-based electricity models apply this algorithm [23].
• A. Weidlich and D. Veit used the Erev and Roth algorithm in Bidding in
interrelated day-ahead electricity markets: Insights from an agent-based simu-
lation model. Two markets were cleared sequentially: a day-ahead market and
a day-ahead balancing market. The success was evaluated by integrating the
opportunity cost of profit that could have been obtained in the other market.
The agents learned from past trading results using a modified Erev and Roth
Algorithm. The authors showed that the timing of market clearing plays an
important role.
• In Market power and efficiency in a computational electricity market with
discriminatory double-auction pricing J. Nicolaisen used the Erev and Roth
learning algorithm for an agent-based model to study market power in electric-
ity markets. The outcomes of the simulation was compared to earlier studies
in which buyers and sellers use genetic algorithms for learning.
Another learning technique often applied is Q-learning. It is a reinforcement

technique and was first introduced by Watkins [32]. The agents thereby learn an
action-value called Q value that describes the expected reward of taking a given
action in a given state and following a fixed policy thereafter. The strength of
Q-learning is that it is able to compare the expected utility of available actions
without using a model of the environment. Q-learning is often applied to Markov
decision processes.
• T. Krause and G. Andersson compared different congestion management schemes

in Evaluating Congestion Management Schemes in Liberalized Electricity Mar-
kets Using an Agent-based Simulator [20]. The market participants used a Q-
learning algorithm in this model. However, the agents were not differentiating
the state they were in each iteration.
• The paper Agent-Based Simulation of Power Markets under Uniform and
Pay-as-Bid Pricing Rules using Reinforcement Learning [33] studied the price
outcome under uniform and pay-as-bid (discriminatory) pricing schemes. The
authors used the SA-Q-learning algorithm, a slightly changed version of Q-
learning. Agents usually face a tradeoff between exploration and exploitation
in Q-learning. The paper introduced a methodology for a good exploration
and exploitation ratio. The authors defined environmental states by the last
round’s market price.
A breakthrough in agent-based models was the introduction of learning classifier

systems (LCS). They combine reinforcement learning to increase the probability
15
of choosing a successful action and genetic algorithm to reproduce new actions

from successful ones. A main feature of LCS is that they are adaptive in a sense
that the ability to choose the best action improves with experience. The source of
improvement is reinforcement. The observed reward after a trial for a given action is
used by LCS to alter the likelihood of taking that action in similar circumstances. In
contrast to Q-learning, LCS classifies an environmental input or state. Every agent
uses a classifier population, a set of classifier, whereby every classifier proposes
a certain action. When a particular environmental input occurs, the LCS forms
a match set of classifiers whose conditions satisfy the input. The classification
introduces generality: A classifier can match two environmental inputs which are
not identical. This allows that the agents are able to classify an input they have not
observed in the past. The classification of the input should thereby be a trade-off
between accuracy and generality. Accuracy means that the agent should be able to
classify an input correctly and general that unknown environmental inputs (inputs
which have not occurred in the past) should be classified correctly [34].
In the following some papers which use the LCS algorithm for the agent learning
process are introduced.
• In A Multiagent Model of the UK Market in Electricity Generation [35], A.

Bagnall and G. Smith presented an agent-based model of the pre NETA UK
market. The agents used a hierarchical learning classifier system. Three main
research questions were addressed by the authors: First, are the agents able to
learn optimal strategies when competing against nonadaptive agents. Second,
are the agents able to learn bidding strategies observable in the real world and
third, can cooperation among the agents evolve in certain market situations
without explicit communication. The authors referred to the work done in [25]
and [27] in their paper to highlight the difference in agent learning between
learning classifier systems and other methodologies. The authors emphasized
that the main difference is the complexity of the agent architecture. As [23]
correctly states, the authors did not mention why this high complexity of
agent behavior is needed and in which way their results were more valuable
than other simulation models. As a lot of other authors, this paper has not
convincingly made it clear what “failures of simpler learning representations
they avoid with the very complex LCS applied in their model” [23].
• In “What is a Learning Classifier System?” some well known researcher in
the field of LCS were asked this question. Some researcher responded are
J. Holland, R. Smith, P. Lanzi and S. Wilson. S. Wilson and P. Lanzi did
a considerable work in the development of XCS, which is a subcategory of
LCS. This work did not focus on the mathematical fundamentals of LCS nor
did it apply LCS to a certain problem, rather, it described what generally the
strength and justifications of LCS are. P. Lanzi concluded that LCS are better
than traditional reinforcement learning techniques and are more general than
Watkin’s Q-learning [32] because Q-learning makes the assumption that the
environment must be a Markov Decision Process. Additionally, Q-learning
requires that the agent’s have a goal which LCS does not require. LCS might
be very general and not expressible in terms of an optimization problem (e.g.
survive).
A main difficulty of LCS is to measure the “goodness” of a classifier. This

measure is used by the genetic algorithm to select strong classifiers for reproduction,
mutation etc. and to delete weak classifiers. This goodness is also used to select a
16
certain classifier is several classifiers match a certain environmental input. In LCS,

a measure called fitness is introduced as measure for the goodness of a classifier,
whereby two subgroups of fitness definition exist: the strength-based (e.g. ZCS)
and accuracy based (e.g. XCS). Often, the probability of choosing a classifier over
another is proportional to its fitness. An accuracy based fitness describes that the
strength of a classifier does not depend only on the absolute reward prediction, but
also on the accuracy the reward can be estimated (e.g. standard error of reward
prediction). This implies that classifiers remain in a population and that classifiers
are chosen over other classifiers which do not predict the highest absolute reward,
but predict it more accurately [34].
• In An algorithmic description of XCS [36], a description of the XCS classifier

system’s parameters, structure and algorithm is provided. The algorithm was
presented in a pseudo code and has helped implementing the algorithm in any
programming language. The paper highlighted the strengths of XCS and the
differences to traditional LCS.
• The paper Classifier Fitness Based on Accuracy [37] addressed several ques-
tions related to XCS. It showed how the fitness of a classifier is measured
and calculated and how the genetic algorithm is applied. It highlighted the
problems of classifier systems with niches 1 and how XCS is solving them.
Additionally, the main components of XCS: performance, reinforcement and
discovery were discussed. Finally, XCS was applied to different Multiplexer
problems to analyze the performance of XCS with different parameter settings.
1 Different environmental states (niches) have different payoff levels. A problem which occurs
with classifier systems is that classifier which match in high payoff niches takeover classifiers which
only apply at low-payoff states
17
Chapter 3
Specification of the model
“It is easier to change the specification to fit the program than vice versa”
Alan Perlis
As described in chapter 2, the electricity generating units are modeled as agents and
line flows between neighboring countries are taken into account. The model at hand
covers the countries Germany, France, Switzerland and Italy and is referred to as
the four-country model later in the thesis. Every electricity generation technology
(e.g. gas, coal, wind) within a country is aggregated in one single decision making
agent.
The agents compete on a countrywide central market place by sending hourly

supply curves to the market. The supply curve specifies the agent’s electricity
production level as a function of price. The market collects all the supply curves
and aggregates them to a nationwide market supply curve. It is assumed that the
electricity markets between these four countries are coupled. Taking into account
the demand curve for every country, the four energy markets are cleared with the
possibility to trade electricity between the countries. An implicit auction method
is assumed for the congestion management scheme. A more detailed description on
the implemented market coupling can be found in chapter 7.
The resulting market prices for the four countries are fed-back to the agents,
which compute their profit based on a generator specific cost model. The objective
of the agents is to maximize their profits. The current state of the environment and
current state of power output as well as forecasts for the environment are taken into
account by the strategic decisions of the agents. The learning process of the agents
is described in chapter 4.
3.1 Market participants and input data
3.1.1 Market participant definition
Each agent represents one of the following electricity generation technologies:
18
Chapter 3. Specification of the model
• Nuclear
• Coal
• Gas
• Oil
• Wind
• Storage
• Hydro
The storage agent represents the pump-hydro storage power plants and the hydro
agent the river power plants. The pump-hydro storage power plants can buy and
sell electricity and have no natural water inlet whereas the hydro power plants have
a constant water inlet and only a little storage capacity. As a result, the hydro
power plants have to sell the constant water inlet in short time frames.
All agents except the wind agent participate actively in the market. A fixed feed-
in tariff is paid to the wind producers, which provide the entire available amount
of wind energy to the market independently of the current market price.
Table 3.1 shows the installed generation capacity for every agent and the aggre-
gated installed capacity per country.
Country Generation technology Installed capacity [MW]

Germany Nuclear 20’263
Germany Coal 37’855
Germany Gas 16’643
Germany Wind 21’139
Germany Total 95’900
France Nuclear 63’260
France Coal 14’727
France Gas 9’818
France Hydro 20’829
France Storage 4’303
France Total 112’937
Switzerland Nuclear 3’220
Switzerland Hydro 13’465
Switzerland Storage 1’636
Switzerland Total 18’321
Italy Gas 34’382
Italy Oil 22’921
Italy Coal 8’022
Italy Hydro 13’573
Italy Storage 7’544
Italy Total 86’442
Table 3.1: Installed capacities for every agent, source EIA
The model is crucial to the input data. Based on observations from electricity
traders, several inputs monitored by the traders are chosen for the model. All agents
receive the following exogenous input at hour t:
19
• Hour
• Current oil price
• Wind forecast for Germany
• Temperature forecast for all countries
• Weekend factor
• Load forecast per country
Only a wind forecast for Germany is needed, since the model assumes that only
Germany has wind power plants. This assumption is reasonable since out of these
countries only Germany has a significant amount in wind generation. The agents
receive the temperature forecast for all four countries. The reason is that in France
for instance the spot price is correlated to the temperature. Agents in countries
which are connected to France via transmission lines are able to anticipate the
higher prices in France by knowing the temperature forecast for France. Hard coal,
lignite and gas prices tend to be correlated to the oil price, as a result, the agents
receive the oil price as proxy for their fuel costs. For learning purposes, all agents
have a memory where they store historical exogenous factors, power outputs, the
corresponding market clearing prices λk,c and the action curve they used ai,k at
hours k ∈ t − 1, t − 2, ..., t − N and country c.
Each agent is parameterized and described by the following parameters:
• Generation type (nuclear, coal, gas, oil, wind, storage, hydro) of the agent.
• Maximum power output [MW].
• Country in which the agent is located (Germany, France, Switzerland or Italy).
• Generator efficiency curve η as a function of the generation technology and

output level.
• Generator ramp cost constant δ [e /MW].
• CO2 emissions per M W h [ton/MWh] (depending on the output level, the

efficiency, the fuel and the specific CO2 emissions of the fuel).
• Technical ramp capacity τ [MW/h].
• For the storage device:
– Efficiency charge ηcharge (pumping efficiency).

– Efficiency discharge ηdischarge (turbine efficiency).
– Maximum storage capacity.
3.1.2 Data sources
The data for the parameters as well as for the input vectors are taken from various
sources:
20
• Fuel costs: The coal exchange traded fund (ETF) NYSEArca: KOL, as well
as the FTSE USA - coal index (FTSE: WIUSA1771.L) from Yahoo Finance
are used to derive the daily coal price. The Amex Natural Gas Index (XNG)
serves proxy for the daily gas price. The data for the daily oil price is taken
from EIA. The fuel costs for the nuclear power stations (enriched Uranium)
is considered constant.
• CO2 costs: For the daily prices of the CO2 emissions ϕ, the prices for Emission
Allowance Units (EUA) are used. The EUAs are traded over the European
Union Emission Trading System. The EEX Carbon Index (Carbix) is taken
as reference price.
• Wind data: The wind and day-ahead wind forecast data is provided by trans-
power stromübertragungs GmbH, a subsidiary of E.ON AG. The feed-in tariff
(Einspeisevergütung) is taken from the Bundesverband WindEnergie e.V and
amounts to 75 e /MWh.
• Historical electricity prices for Germany are provided by the European Energy
Exchange (EEX).
• The historical load data as well as the values for the net transfer capacities
(NTC) are obtained from ENTSO-E.
• The hourly temperature data for Germany is provided by the Deutscher Wet-
terdienst (DWD). For the European temperature data, the source European
Climate Assessment [38] is used.
3.2 Supply curves, market structure and market

clearing
3.2.1 Supply curves
All agents except the wind agent send an hourly supply curve to the central market
place. The supply curves represent the agent’s power output level as a function of
the spot market price. Three different types of supply curves are differentiated: The
ones for the inelastic bidders, for the elastic bidders and for the storage bidders.
• Inelastic bidder: It is assumed that the hard coal, lignite and nuclear agent
have ramping constraints which means they can only ramp their production
level a certain amount from one hour to the other. As a result, they are sending
a more inelastic supply curve to the market place as the gas fired power plant
agent for instance. Figure 3.1 shows a supply curve of an inelastic bidder. It
is important to mention that the supply curve is inelastic, but not perfectly
inelastic; The agent’s power output is not constant, rather it is still a function
of the spot market price. The supply curve is characterized and parameterized
by two points x and y.
The value x stands for the power output level when the spot price is zero and
y for the power output when the spot price is equal to the defined maximum
price which is equal to 200 e /MWh in this example. The power output for
every spot price between these two values is linearly interpolated.
21
100
80
Power output [%]

60
40
y
20
0
0 50 100 150 200
Spot price [e /MWh]
Figure 3.1: Supply curve for the inelastic bidders
It always holds that x ≤ y. If x = y, than the supply curve is perfectly

inelastic and the power output is constant for every spot price.
• Elastic bidders: The supply curves for the elastic bidders is characterized by
the points x and y as shown in Figure 3.2. It is assumed that the elastic
bidders are able to ramp their production level from full load to zero or from
zero to full load between two consecutive hours. The agents representing gas
and oil fired power plants qualify as elastic bidders.
The value x represents the spot price at which the agent starts to increase
its production level and the value y the spot price at which the agent reaches
100% power output. If the spot price is smaller than x, the power output is
0% and if the spot price is greater than y, the power output is 100%. All
values between x and y are linearly interpolated.
It always holds that x ≤ y. If x = y, than the supply curve is perfectly elastic
and the power output changes from full load to zero at one specific spot price.
The agents with the elastic supply curves have higher marginal costs than
the inelastic bidders but more flexibility in choosing the power output. They
usually choose a power output level of zero if the market price is considerable
lower then their marginal cost or full load if the spot price is above their
marginal cost.
• Storage bidders: The storage bidders have the ability to buy and sell electricity
depending on the spot price. As shown in Figure 3.3, their supply curve is
parameterized by the values x and y. A negative power output represents
buying electricity while a positive represents selling electricity. The pump-
storage hydro power plants qualify as storage bidders. This type of power
plant pumps water from lower basins to higher ones with cheap electricity
and turbines it when the spot price reaches a certain level. It is assumed that
there is no natural water inflow which means that all the electricity that is
sold has to be bought at some point in time before. The current reservoir
level is the integral of all historical buyings and sellings.
Furthermore, it is assumed that the storage agents are symmetric in a sense
that the maximum turbine power level is equal to the maximum pumping
22
100 y
Power output [%] 80
60
40
20
x
0
0 50 100 150 200
Spot price [e /MWh]
Figure 3.2: Supply curve for the elastic bidders
power level.
The value x represents the spot price at which the storage agent starts to
decrease the pumping from 100% and at value y the storage agent reaches
100% turbine level and sells as much electricity as possible. If the price is
lower then x, the storage agents pumps at full load and if the spot price is
greater then y, the agent turbines at full load. The turbine or pumping power
level between spot values x and y is derived by a linear interpolation. The
point of intersection between the x-axis and the power output graph represents
the spot price at which the agent changes from pumping to turbining.
100 y
80
60
Power output [%]
40
20
Spot price [e /MWh]
0
50 100 150 200
-20
-40
-60
-80
-100
x
Figure 3.3: Supply curve for the storage bidders
It always holds that x ≤ y. If x = y, than the supply curve is totally elastic

and the power output changes from full pumping to full turbining at one
specific spot price.
23
3.2.2 The market clearing
The market supply curve is derived by aggregating the supply curves for all agents
per country. The storage agent’s supply curve is decomposed. The positive part
of the supply curve is added to the market supply curve while the negative part is
added to the market demand curve.
Starting point for deriving the market demand curve is the load per country c
for a specific hour t. At first, the demand is assumed to be perfectly inelastic at
the country load level. Then, the negative parts of the storage supply curves are
added.
Figure 3.4 shows an example of a market clearing. The market clearing price
λt,c is derived by intersecting the aggregated supply curve with the demand curve.
After the market clearing, the model calculates the electricity output pi,t of agent
i at time t based on the agent’s supply curve j (aji,t ) and the market clearing price
λt,c for every country c.
100'000 Storage agent buying
80'000
Power output [MWh]
60'000
Market demand curve
40'000
Market supply curve
20'000
Market clearing price (λt,c )
0
0 50 100 150 200
Spot price [e /MWh]
Figure 3.4: Market clearing
The market clearing is not always possible. Two unfavorable outputs can prevent
the market supply and demand curve to intersect:
• At market price zero, the supply level is higher than the demand level: This
case occurs if there are a lot of agents with tight ramping constraints or high
ramping costs (inelastic bidders) and the demand fluctuates heavily between
consecutive timesteps. The agents are not able to decrease the production
level enough to encounter the effect of a reduced load for a certain hour. The
same effect can happen if there is excess wind energy and the remaining agents
are not able to lower their power output. Some electricity spot markets such
as the EEX in Germany allow negative spot prices. In a situation with excess
power generation a negative spot price outcome is likely. In the simulation in
this thesis negative prices are not defined, in this case, the market price (λt , c)
is set to 0 e /MWh for this country and hour.
• At market price equal to the maximum price of simulation, the demand level
24
is higher than the supply level: The agents are not able to meet the demand.
In every country the installed capacity is higher than the maximum load for
every hour t. This case therefore does not occur because there is not enough
generation capacity, but because of ramping constraints or high ramping costs.
The agents are not able or willing to ramp their production up to meet the
demand. If this case occurs, the market price (λt , c) is equal to the maximum
price of simulation.
The effects described above occur with inexperienced agents but rarely with
experienced agents. The reason is that a price equal to the maximum price of
simulation is a high profit opportunity and a price equal to zero a loss for every
agent. Since the agents optimize over multiple steps (see chapter 4), the agents
ramp their production level over these steps and anticipate very high or low prices
early. A very high expected price drives the agents to increase the production over
several price steps in order to exploit the profit opportunity and a very low expected
price induces the agents to start decreasing the production level over several time
steps.
3.2.3 Time frame of the bidding process
At day d, the agents send 24 hourly supply curves for the upcoming day d + 1.
The market clearing is then performed for the 24 hours. This implies that the
agents know the prices for the upcoming day hours at day d. The implementation
of the day-ahead spot market is in accordance with the implemented structure of
the major electricity spot markets in Europe where the day-ahead auction market
of single hours is the closest to a spot market [39]. In this thesis, there is no future,
intra-day or balancing market modeled. The work focuses on the day-ahead market.
3.3 Cost of production and profit calculation
The calculation of the variable cost for the agents is crucial to the model since it
is directly related to the calculation of the agent’s reward. The following table 3.2
provides a summary of all the parameters used in the cost model to calculate the
total variable cost:
Symbol Parameter description
fc Specific fuel cost
η Generator efficiency
Λ Daily market price of fuel
δ Generator ramp cost constant
ϑ Tons CO2 emissions per MWh fuel burned
ϕ Daily price of one ton CO2 emission
Table 3.2: Generator specific parameters of the cost model
total
The quantification of the total variable cost V Ci,t for the different generators
is a modification of the model proposed by [40] and follows:
total p r ea
V Ci,t = V Ci,t + V Ci,t + V Ci,t (3.1)
25
p r
where V Ci,t represents the variable production costs, V Ci,t the ramping and
ea
V Ci,t the emission allowance costs at time t for generator i. The production
p
costs V Ci,t comprises specific fuel costs f c (techi , Λm,t ), which depend on agent’s
generation technology, the daily market price of the specific fuel m (Λm,t ), the
agent’s efficiency η (pi,t , techi ) and power output pi,t . The technologies tech ∈
{nuclear, hard coal, lignite, gas, oil} are differentiated. The efficiency depends on
the generation technology and agent i’s power output level pi,t at time t. The de-
pendency between the efficiency and the power output level is approximated by a
quadratic function. The variable cost of production can be calculated according to:
p f c (techi , Λm,t )
V Ci,t = pi,t · (3.2)
η (pi,t , techi )
The ramping costs V C r is calculated by the difference in output level from one
hour to the next multiplied by a generator specific ramp cost constant δ (techi ):
r
V Ci,t = (pi,t − pi,t−1 ) · δ (techi ) (3.3)
ea
The emission allowance costs V Ci,t is determined by a fuel specific constant
ϑ (f ueli ), which specifies the number of tons of CO2 emissions per M W h fuel
burned and the daily price for the emission allowance ϕt in price per ton CO2
produced. Therefore,
ea 1
V Ci,t = pi,t · ϑ (f ueli ) · ϕt · (3.4)
η (pi,t , techi )
The paper [40] suggests using an additional cost term to incorporate the effects
of generator shut-downs. It is argued that if an agent turns off the production, the
boiler, pipping etc. cools down. Turning the power plant on later induces additional
losses and therefore costs which depend on the length of the shut-down. These costs
are not incorporated into the model used in this thesis. The three main reasons are:
• In general, the gas turbines turn off their production because of their high
marginal costs and lax ramping constraints. Since the gas turbines do not use
a boiler to heat up pressurized water but burn the gas in the turbine directly
the losses associated to shut-downs are minor.
• The data to model the shut-downs is not publicly available. The function to
model these costs might be reasonable, but the determination of its parameters
is difficult.
• It is difficult to implement this type of costs because they depend on the length
the generator has been shut down and are highly nonlinear. They actually do
not influence the decision to turn a power plant on again or not because at
the time of the decision to go online these costs can be considered sunk costs
and should therefore not affect the decision. As a result, their influence on
the bidding strategy is most likely not significant.
Finally, agent i calculates its profit at time t (πi,t ) according to
26
total
πi,t = λt,c · pi,t − V Ci,t (3.5)
where λt,c represents the spot price in country c of time t.
3.4 Simplifications of the model
The following simplifications are made in this thesis for modeling the central Eu-
ropean electricity markets. The simplifications are made to reduce the complexity
of the model. It is assumed that the simplifications do not alter the simulation
outcome dramatically.
• There are only generators and consumers in the market, no third party trading
intermediaries.
• Implicit auctions are assumed for cross-border trades.
• The grid within a country is modeled as a copper-plate with no congestions

and losses.
• The load part of the aggregated demand function is perfectly inelastic, which
is a feasible assumption since the short run demand for electricity is quite
inelastic in reality. Only the buying of the storage agents leads to an elastic
demand curve.
• There is no forward and futures market where electricity can be traded.
• The entire volume is traded via the spot market place, i.e. there are no
bilateral OTC trades.
27
Chapter 4
Learning-process of the
agents: The methodology
“Good decisions come from experience, and experience comes from bad decisions”
Unknown author
The findings from thesis [41] are used to develop a new framework for the agent
learning process. In this thesis the agents should follow a distinct goal: To max-
imize their profit. The learning methodology should incorporate basic concepts
of Q-learning, but instead of deriving the Q-values 1 from trying different actions
in different states, the agents calculate the Q-values directly based on a model.
The model is a multi-factor regression model and allows calculating the expected
discounted rewards (profits) for every combination of action, thereby taking into
account exogenous factors and market-power.
Later in the thesis it is shown that the expected discounted profit for a bid-
ding strategy (arbitrary set of action curves) can be derived by estimating future
spot prices. The dependency is as follows: With estimated future spot prices, the
power outputs and revenue for every bidding strategy are given because every sup-
ply curve defines the agent’s power output for an estimated future spot price. With
the estimated power outputs and an appropriate cost model, the agents can also
calculate the cost for every bidding strategy according to the cost equations in 3.3.
By knowing the revenue and costs, the agents derive the profit for these strate-
gies. The price predictions also incorporate the effects of market-power. The price
predictions become more accurate as more market outcomes are observed.
The algorithm introduced in this thesis is similar to Model Predictive Control

in a way that a model generates predictions and the input (action curve) is set that
a given performance criterion (the reward) is maximized. The model predictive
bidding learning algorithm is based on and incorporates the following properties:
• The learning process is price driven. The agents learn to forecast future spot
1 In this context, the Q-values are defined as the discounted expected rewards if an agent uses
a certain action in a certain state at time t and follows in an optimal way thereafter. Following in
an optimal way thereafter means that the agent will choose the actions with the highest expected
profit at t + 1 and thereafter
28
Chapter 4. Learning-process of the agents: The methodology
prices based on exogenous variables and exogenous variables predictions by

measuring the sensitivities of these factors on spot prices. The price estimates
thereby incorporate market-power effects.
• The agents measure their market-power based on the variations in prices not
explained by the exogenous factors. The measurement of how much a cer-
tain action alters the predicted spot price outcome is based on reinforcement
learning.
• Based on the spot price predictions, the agents use an optimization routine
to choose the supply curves which maximize the expected discounted future
profits.
• The agents optimize over multiple steps in the future. In most classifier based
models the agents are only optimizing on a single step basis, which means they
maximize the profit for time t + 1. A multiple step optimization is important
to model the storage agents which have to buy electricity when the expected
spot price is low and sell when it is high. In a single-step optimization, the
storage agents would want to sell as much energy as they can for the next time
step and never buy energy. A multi step optimization is also important for
the inelastic bidders due to their ramping constraints. For them, maximizing
only the front hour can result in unfavorable outcomes.
A simple example explains the reasoning: It is assumed that the expected
spot price will be low at t + 1 where t is the current time step but very high at
t + 2 and thereafter. If an inelastic bidder is maximizing the reward for t + 1,
he would decrease the production level at t + 1. If instead the inelastic bidder
uses a multiple period optimization, he might start to increase the production
level at t + 1 to be in a high power output level at time t + 2, even if it results
in a loss at t + 1. This ramping scheduling can result in a higher discounted
profit over multiple time steps.
In the following, the advantages of predictive bidding over classifier based learn-
ing systems and Q-learning are summarized:
• As P. Lanzi highlights in [42], learning classifier systems in general do not

require that the agents have a distinct goal. LCS might be very general and
not expressible in terms of an optimization problem. In XSC for instance, the
agents choose preferably classifiers with a high fitness. The fitness increases
with a high reward prediction accuracy rather than the absolute value of re-
ward prediction. The agents are therefore not primarily trying to maximizing
their profits. What they try to maximize is hard to assess. With model pre-
dictive bidding, it is clearly defined that the ultimate goal of the agents is to
maximize their profit.
• The learning algorithm in LCS is based on trial and error and on evolution
of the classifiers. The outcome of LCS model depends highly on randomness
and is not reconstructible. It is therefore hard to draw conclusions from an
LCS model. Furthermore, the outcome from two simulations with identical
environments can be very different. In a simulation outcome with an LCS
model, it is hard to answer why an agent did a certain action in a certain
situation because the agent’s decision can be based on pure randomness. In
model predictive bidding there is much less randomness; it is therefore easier
to draw conclusions about the agent’s behavior. Additionally, a simulation
29
outcome is reconstructible since the agents are always acting the same way
under the same environmental input.
• Not all combinations of supply curves for every state/environmental input

have to be tried by the agents to experience a relationship between spot price
and state/environmental input as in classifier based systems and Q-learning.
The amount of combinations between these factors are manifold with several
exogenous factors. The computation time to discover all these combinations
requires a high computational effort. In model predictive bidding the agents
only estimate the regression coefficients of the price predictor model (the linear
sensitivities of each exogenous factor on spot prices) and what how a certain
supply curve influences the spot prices (see section 4.3 on the definition of
price adjusting values) instead of trying all state, environmental input and
action combinations. This enables the agents to learn much faster.
• Q-learning and learning classifier systems are very data intense. All the Q-
values for the different combinations of states/environmental inputs have to
be saved in lookup-tables. This very quickly looses viability with increasing
environmental inputs. For classifier systems, the classifier population becomes
very big if the agents make precise reward predictions. In contrast, only the
sensitivities of the exogenous factors on market prices and the price adjusting
values (pavs) have to be stored in model predictive bidding.
• If the agents have no Q-values or matching classifiers for an environmental

input/state combination, they do not have a reasonable basis to make a de-
cision. In these cases, a random action is proposed. Under model predictive
learning the agents are able to make reasonable predictions once they dis-
covered the regression coefficients and are able to act on any combination of
environmental input/state.
• In model predictive learning the simulation breaks the effect of spot price
variations down to the different exogenous factors. Therefore, conclusions
can be drawn on how much an exogenous factor influences the spot price.
Furthermore, the model allows indicating the market-power of each agent by
the pav values. This decomposition of spot price variation is not possible with
Q-learning or classifier based learning systems.
4.1 Overall implemented structure
Figure 4.1 shows the structure of the implemented learning methodology. Each
agent uses a price predictor to predict future spot prices based on predictions of
exogenous factors. Additionally, the spot price estimates are adjusted to incorporate
the effect of market-power. Based on this information, the agents use an optimizer
based on dynamic programming to find the optimal set of supply curves (an optimal
policy), which maximizes the agent’s expected discounted profit in order to weight
the actual profit more than the ones in the future. Historical exogenous variables,
spot prices and corresponding supply curves are saved by each agent.
The idea is that electricity prices are driven by three fundamental factors: The
load, exogenous factors and market-power. In an environment with a static merit
order 2 of electricity supply, the demand curve shifts depending on the load. There-
2 The merit order is a way of ranking available sources of energy, especially electrical generation,
30
HISTORICAL DATA
The historical data is
stored by every agent and
used by the price
predictor and price
adjuster
PRICE PREDICTOR PRICE ADJUSTER OPTIMIZATION MARKET

An optimal set of actions
An hourly priced forward The prices of the HPFC The front 24 actions of
(policy) given the
curve (HPFC) is are adjusted for every every agent are sent to the
adjusted prices is
constructed with a action to incorporate the market where a market
calculated with dynamic
regression model effect of market power clearing takes place
programming
EXOGENOUS FAC-
TORS PREDICTION
Predictions for the

exogen-ous factors are
made
Figure 4.1: Flow chart of the learning methodology
fore, deviations in the electricity prices can be explained by the load. The load
tends to be higher during peak hours, during the week and in winter month.
Other fundamental factors are exogenous factors such as wind or temperature.

Some of these factors are related to the load, e.g. a lower temperature results in a
higher heating demand which increases the load. In contrast, wind has an effect on
the supply curve: Higher wind generation shifts the supply curve which decreases
the spot price.
Every agent constructs an hourly price forward curve (HPFC) using the price
predictor, which uses the forecasted load and exogenous factors as input values.
As described, there is a third fundamental factor which is the market-power. It is
based on the strategic bidding of the agents as they learn their influence on market
prices. To model this fundamental factor, the price adjuster is introduced. With the
unexplained part in prices (the difference between observed and estimated prices)
the agents estimate their influence on market prices.
The building blocks in Figure 4.1 are explained in more detail in the following
sections.
4.2 The price predictor
4.2.1 The multi-factor regression model
The agents use a multi-factor statistical model to predict future spot prices and to
generate an hourly price forward curve. In this thesis a modification of the model
in order of their costs of production, so that the most efficient are more likely to be called to
generate, rather than less efficient plants [43].
31
introduced in [44] is used to model the price predictor. The resolution for the
forward curve is one hour because on major European energy exchanges electricity
is bought and sold on an hourly basis. A multi-factor model is introduced to make
bt,c for country c and hour t.
day-ahead hourly spot price predictions λ
The multi-factor model is given as
bt,c = βc,1 γt,c,1 + βc,2 γt,c,2 + ... + βc,N γt,c,N + ǫt,c

λ (4.1)
where the estimated spot price λ bt,c is the dependent variable and γt,c = (γt,c,1 , γt,c,2 , ..., γt,c,N )
a country and time specific vector. ǫt,c is a disturbance term; an unobserved ran-
dom variable which adds noise to the linear relationship between the dependent and
independent variables. Vector βc = (βc,1 , βc,2 , ..., βc,N ) consists of the parameters
βc,1 , βc,2 , ..., βc,N which have to be estimated with historical data. They describe
the sensitivity of the dependent variable with respect to each independent variable.
The following independent variables γt,c,1 , γt,c,2 , ..., γt,c,N in γt,c are considered
by the price predictor to predict future spot prices (the dependent variable):
• Hour: The electricity prices show a daily pattern. Prices are usually higher
during peak hours then off-peak hours. A set of 24 dummy variables for every
hour is implemented. At for instance t = 1, the first dummy variable is 1 and
all the other ones 0, at time t = 2, the second dummy variable is 1 and all the
other ones 0 and so on.
• Current oil price: It is assumed that the oil prices follow a random walk. The
current oil price is therefore the best predictor for future oil prices. Since the
agents optimize their scheduling for around one week or less, this assumption
is reasonable because oil price changes tend to be minor within this time
frame. Additionally, electricity producers do not buy their fuel on a daily
basis, they usually buy their fuel in advance and hedge themselves against
major price changes. Electricity prices are correlated with oil prices since the
oil prices are correlated to fuel prices for electricity generation (gas, oil and
coal). Higher oil prices in general result in higher electricity prices and vice
versa.
• Wind forecast: Electricity prices and wind are negatively correlated. The
dependency is due to the wind power plants. Since a fixed feed-in tariff is
paid to the wind power plants they feed the maximum amount possible into
the grid and do not consider the current spot price. Periods with high wind
result in lower spot prices because of the additional supply and vice versa.
• Weekend factor: Electricity prices tend to be lower during Saturdays and

Sundays, mostly because the demand by the industry is lower. This factor
consists of a dummy variable duple. The first variable is 1 if it is a Saturday
and the second is 1 if the specified day is a Sunday.
• Historical price fraction: A historical arithmetic mean of the spot price is

taken as a factor. The historical price fraction for hour t and country c is
calculated according to
M
X λt−i·8760,c
P Ft,c = (4.2)
i=1
mean (λt−1 , λt−2 , ..., λt−M ·8760 )
32
A historical price fraction of 0.8 at hour t for instance would indicate that in
the past M years the price has been 80% of the mean price for the same hour.
The value of M depends on the data availability. It should be chosen high
enough to remove outliers but not too high to introduce a bias by structural
changes of consumers. Figure 4.2 shows an example of an averaged historical
price fraction with M = 3. The daily price variations can nicely be observed
in the figure.
1.5
1
Price fraction
0.5
0
0 20 40 60 80 100 120 140 160
Hour
Figure 4.2: Price fraction with averaged historical data
• Spot price 24 hours ago: This factor can be introduced to reduce the serial
correlation present in the price residuals if the model incorporates only the
exogenous factors described so far. The serial correlation in the residuals be-
tween the spot price and the spot price 24 hours ago is statistically significant.
For an additional discussion about the serial correlation see section 4.2.3
• Spot price one week ago: Again, this factor is introduced to reduce the serial
correlation present in the residuals. The serial correlation in the residual be-
tween the spot price and the spot price one week ago is statistically significant.
For an additional discussion about the serial correlation see section 4.2.3
• Heating degree day (hdd): This value is calculated according to
hddt,c = max {18.3 − tempt,c , 0} (4.3)

The hdd is calculated for every country taking the temperature or temperature
forecast. The temperature is measured in ◦ C.
• Cooling degree day (cdd): This value is calculated according to
cddt,c = max {tempt,c − 18.3, 0} (4.4)

The cdd is calculated for every country taking the temperature or temperature
forecast. The temperature is measured in ◦ C. The reason for using hdd and
cdd instead of the temperature is the assumption that from a temperature
33
perspective the load is lowest for 18.3◦ C and since the electricity price is
mostly load driven so is correspondingly also the electricity price. As a result,
linear deviations from 18.3◦ C are taken as factors in the statistical model. The
electricity price dependency on temperature is most significant in France, since
a lot of heating and cooling is done with electricity in France. This is mostly
due to the historical low electricity prices in France.
• Month: This factor is introduced to capture seasonal effects. Electricity prices

are usually lower during the summer months than the winter ones. The reason
lies again in the lower load during the summer. This factor consists of 12
dummy variables, one for every of the twelve month per year.
• Constant: As a rule, the constant term is always included in the set of regres-
sors, it is called the intercept.
The agents use a storage to store past independent variables γt,c and prices λt,c
to estimate the parameters in βc . The agents use the ordinary least square (OLS)
method for estimating the unknown regression coefficients in βc .
The estimation of the regression coefficients in βc by OLS follows the following

T
procedure: for an arbitrary vector b = (b1 , b2 , ..., bN ) the quantity λt,c − b · γt,c is
3
called the residual for the t-th observation in country c . Geometrically, it is the
vertical distance between the point (γt,c , λt,c ) and the hyperplane λ = b · γ ′ . The
residual assesses the degree of fit between the actual data and the regression model.
The residual sum of squares (RSS) measures the overall fit of the model [45]. The
RSS follows
N
X
T
2
RSS(b) = λt,c − b · γt,c (4.5)
i=1
The vector b which minimizes (4.5) is called OLS estimator β. The RSS func-
tion possesses a unique global minimum, which can be calculated with the explicit
formula
n
!−1 n
1X T 1X T
βc = argminb∈RN (RSS(b)) = γ · γt,c · γ · λt,c (4.6)
n t=1 t,c n t=1 t,c
where n are the number of observations in the data storage of the agent. With
bt,c values can be predicted according to
the vector βc the λ
bt,c = βc · γ T
λ (4.7)
t,c
For unbiased future spot price predictions it is important that the agent’s price
predictor is correctly specified. The OLS estimator requires several assumptions
[45]:
3γT = γt,c transpose
t,c
34
• Strict exogeneity The errors in the regression should have conditional mean
zero:
E [ǫ|γ] = 0 (4.8)
The consequences are that the errors have a mean of zero and that the regres-
sors are uncorrelated with the errors.
• No multicollinearity The regressors in γ must all be linearly independent.

When this assumption is violated, the values for the regression coefficients
β can not be learned, but the prediction of λ values is still possible for new
values of the regressor that lie in the same linearly dependent subspace.
• Homoscedasticity The error term should have the same variance σ 2 in every
observation.
E ǫ2 |γ = σ 2 (4.9)
• Nonautocorrelation The error terms should be uncorrelated between observa-

tions:
E [ǫi ǫj |γ] = 0 for i 6= j (4.10)
• Normality The errors have a normal distribution conditional on the regressor.
In subsection 4.2.3 the multi-factor model is tested with historical EEX data to
analyze if the model is correctly specified. Furthermore, the ability of the model to
forecast spot prices with out-of-sample data is assessed.
4.2.2 Exogenous factor predictions
The hour, weekend factor and month are known for a particular hour in the future.
The price fraction is calculated based on historical prices according to (4.2). It is
assumed that the load in a particular hour in the future is equal to the arithmetic
mean of that hour in the past. As described, the current oil price serves as proxy
for future oil prices since the oil price is assumed to be a random variable which
follows a random walk. Therefore, the only predictions that have to be made are
wind and temperature for every country.
• Wind: As described in Chapter 3, the model assumes that there are only
wind power plants in Germany and not in the other three countries. Hence,
a wind prediction is needed only for Germany. For the front 24 hours, wind
forecasts are given (see 3.1.2). For horizons longer than one day no wind
forecast is provided and a simplification has to made. It is assumed that
the wind forecast for hours after the front day is equal to the mean forecast
for the front day. This is a quite drastic simplification, the reason for this
simplification is that wind forecasts longer than one day are not accurate in
general.
• Temperature: There is no day-ahead or longer horizon historical forecast avail-

able. Therefore, the agents create a temperature forecast based on historical
temperature observations by using a Fourier series approximation. The fre-
quencies with the highest amplitudes are selected to approximate the historical
temperatures. For more information regarding Fourier series approximation
with Matlab see [46].
35
A Fourier series decomposes a signal into a sum of sines and cosines of different
frequencies. The signal can be approximated by
z
X
f (t) = a0 + (an sin(2πnt) + bn cos(2πnt)) (4.11)
n=1
where f (t) is the signal in the time domain and an and bn unknown coefficients,
n corresponds to the frequency for which the unit is Hertz (Hz). The unknown
coefficients an and bn obtained by deriving the integral
Z 1
an = 2 f (t)sin(2πnt)dt (4.12)
0
and
Z 1
bn = 2 f (t)cos(2πnt)dt (4.13)
0
It is thereby assumed that the signal lasts for exactly 1 second (0 < t < 1).
The coefficient a0 corresponds to a frequency of 0 and is obtained by
Z 1
a0 = 2 f (t)dt (4.14)
0
The temperature time series for Germany are hourly values: it is a discrete
finite signal. A discrete approximation for the integrals (4.12), (4.13) and
(4.14) has to be used. One way of approximating the integral is the use of
the trapezoidal rule where the function is divided into several trapezoids. The
area of a single trapezoid is computed using
f (tj ) + f (tj+1 )
Aj = (tj+1 − tj ) (4.15)
2
The points tj are assumed to be equally spaced and tj+1 − tj = ∆t. Applying
the trapezoidal rule to the integral of the coefficients, an is given as
an =∆t (sin(2πnt1 )f (t1 )) +

 
z−1
X
∆t 2 sin(2πntj )f (tj ) + (4.16)
j=2
∆t (sin(2πntz )f (tz ))
Using the same approximation, the coefficients for bn are calculated by
bn =∆t (cos(2πnt1 )f (t1 )) +

 
z−1
X
∆t 2 cos(2πntj )f (tj ) + (4.17)
j=2
∆t (cos(2πntz )f (tz ))
36
The square of the coefficient’s absolute value for a given frequency is con-
sidered the power of the signal. A spectral analysis is performed and the z
frequencies with the highest powers in the temperature time series approxi-
mation are chosen. Then, the temperature signal is reconstructed using these
z frequencies.
The approximation is shown with data for Germany and the year 2006 for
different amounts of frequencies included.
40
Temperature Germany 2006
Approximation
35
30
25
Temperature [◦ C]
20
15
10
−5
−10
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Hour
Figure 4.3: Temperature Fourier series approximation, z = 2 (year 2006)
Figure 4.3 shows a temperature Fourier series approximation with the two
most dominant frequencies (z = 2). It captures the seasonal variation of the
temperature with higher temperatures during summer month and lower during
the winter, but it does not capture daily variations with higher temperatures
during the day than during the night.
In Figure 4.4, the approximation with z increased to 15 is shown. Now,
the approximation captures more of the temperature variations. To see how
well the approximation performs on out-of-sample data, the Fourier series is
compared to German temperature data of the year 2007.
The approximation clearly does not fit the out-of-sample data as well as the
sample data, but it captures the main temperature dynamics. The problem
is that this type of approximation fails to incorporate if a summer is very hot
compared to other summers for instance. If the temperature is very high for
a long period, nuclear power plants have to decrease the amount of cooling
water they use which reduces their power output. Together with an increase
in air conditioning demand this can result in extreme spot price outcomes as
observed in the year 2003. The temperature approximation could be expanded
to incorporate unlikely but severe temperature behaviors. For simplicity, this
is not done in this thesis.
By analyzing the temperature residuals (the actual temperature values minus
the approximated ones) it can be stated that increasing z from 2 to 15 increases
the accuracy in predicting temperature values. But adding more and more
frequencies does not improve the prediction ability since at one point only
37
40
Approximation
35
30
25
Temperature [◦ C]
20
15
10
−5
−10
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Hour
Figure 4.4: Temperature Fourier series approximation, z = 15 (year 2006)
additional noise is added. Hence, z = 15 seems to be a good trade off in

capturing the main temperature dynamics and not adding too much noise.
Consequently, the agents use a Fourier series approximation with the 15 most
dominant frequencies of the trailing year as temperature approximation for
every hour in a year.
38
40
Approximation
35
30
25
Temperature [◦ C]
20
15
10
−5
−10
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Hour
Figure 4.5: Temperature approximation for out-of-sample data, z = 15
4.2.3 Analyzing the price predictor
Once the factors are chosen the price predictor is tested on historical data. Data
for Germany (country g) and the year 2006 is taken as sample data. The model
takes on the form described in (4.1). The regression coefficients in β are related to
the independent variables γ as follows: βg,1 , ..., βg,24 to the 24 hours, βg,25 to the oil
price, βg,26 to the wind, βg,27 to a Saturday, βg,28 to a Sunday, βg,29 to the heating
degree day, βg,30 to the cooling degree day, βg,31 to the price fraction, βg,32 , ..., βg,43
to the month, βg,44 to the spot price 24 hours ago, βg,45 to the spot price one week
ago and finally, βg,46 to a constant.
Different implementations of the regression model are analyzed. Special empha-

sis is put on the descriptive power of the model described by the R-squared and the
adjusted R-squared values as well as the root mean squared error. The R-squared
value (R2 ) is often referred to as the coefficient of determination and is defined as
the ratio of explained variance to the total variance of the dependent variable λ [45].
The R2 value is calculated according to
Pn 2
bt,c − λ
λ
t=1
R 2 = Pn 2 (4.18)
t=1 λt,c − λ
where λ represents the arithmetic mean for all λ bt,c values. The R2 is always
between 0 and 1, where 0 is a poor fit and 1 a good degree of fit. Adding an
additional independent variable to the model almost always increases the R2 value,
even if the additional independent variable does not add explanatory power to the
regression. The adjusted R-squared value is a slightly modified version of the R2 ,
designed to penalize for the excess number of independent variables which do not
add explanatory power. The adjusted R-squared value is defined by
39
n−1
adjusted R2 = 1 − 1 − R2 (4.19)
n−N
where N is the number of regressors and n the sample size. The mean squared
error (MSE) is one way of quantifying the difference between an estimation and
the true value of the quantity being estimated. MSE measures the average of the
squared residual and is calculated by
2
b =E
MSE(λ) bt,c − λt,c
λ (4.20)
The difference between λbt,c − λt,c results from randomness or because the es-
timator does not account for information that could produce a more accurate es-
timate [47]. The MSE is the second moment of the error and incorporates the
variance of the estimator and its bias. For an unbiased estimator, the MSE is the
variance.
It is also analyzed if the OLS assumptions described earlier are valid. The
serial correlation present in the residuals as well as multicollinearity are discussed.
Heteroscedasticity (random variables have different variances) is not analyzed in
this thesis. The reason is that it does not cause the OLS coefficient estimates
to be biased, it only causes their variance to be biased and underestimated. The
probability of making a type 1 error is increased if heteroscedasticity is present.
The first modeling approach
A F-test 4 is first conducted to assess if all the factors as a whole have a descriptive
power on the dependent variable. If yes, t-tests 5 are conducted to analyze if the
single regression coefficients are statistically significant or not by testing if they
are significantly different than zero, but the critical t-values for the t-test could be
distorted because of heteroscedasticity, serial correlation and multicollinearity.
It is generally assumed that the time series has no unit root and is therefore
stationary. This assumption is not proven but by analyzing the time series it can
be stated that electricity prices have a mean reverting behavior and as a result do
not have a unit root.
Figure 4.6 shows the observed EEX and the day-ahead spot price predictions
for the year 2006. In Figure 4.7, the first 400 hours of the year 2006 are shown to
confirm the ability to regress intra-day variations in spot prices.
By observing the plots it can be stated that the model captures the daily patterns
in electricity prices. It also captures seasonal effects but during some longer time
periods the simulated prices are constantly above or below the observed EEX prices.
This is a sign of serial correlation in the residual. To conduct the amount of serial
4 An F-test is a statistical test in which the test statistic has an F-distribution under the null
hypothesis. It is used to identify how well the model as a whole fits the population from which
the data were sampled
5 A t-test is a statistical test in which the test statistic follows a Student’s t distribution under
the null hypothesis
40
200
Simulated
Observed
180
160
140
Spot price [e ]
120
100
80
60
40
20
0
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Hour
Figure 4.6: Simulated and regressed German spot prices (year 2006)
correlation present in the residuals a Durbin-Watson test 6 is conducted. The

value for the Durbin-Watson test statistics is 1.232, which indicates the presence of
positive serial correlation. This autocorrelation violates the ordinary least squares
(OLS) assumption that the error term is uncorrelated. This positive autocorrelation
does not bias the coefficient estimates but their standard errors are underestimated
which overstates their t-statistics. Introducing more lagged time factors in the
autoregressive part of the model could probably reduce the positive serial correlation
present in the residuals and increase the prediction ability of the model.
An F-test is conducted and indicates that the factors overall have a descriptive
power. In a second step, the p-values 7 of the regression coefficients are analyzed
to assess if they are statistically significant different than 0 on a 0.05 significance
level.
Only the regression coefficients βg,27 ,βg,30 ,βg,31 ,βg,32 ,βg,45 and βg,46 are signifi-
cantly different from zero. This implies that only the factors Saturday, the cooling
degree day, the price fraction, the month January, the spot price one week ago
(and the constant) should be used to predict prices. Again, even their t-statistics
are biased and overestimated by the presence of serial correlation, multicollinearity
and heteroscedasticity. A hypothesis can be formulated that only a few regression
coefficients are statistically different than zero because the price fraction is already
explaining a lot of the variation in the dependent variable. The hypothesis can
be tested by the multicollinearity and by removing the price fraction to assess if
the remaining regression coefficients will be significant. There is a high degree of
multicollinearity present in the factors which can be seen in the coefficient covari-
ance matrix. Especially the regression coefficients for the hour and price fraction
are highly correlated. Multicollinearity results in the fact that the regression coef-
6 The Durbin-Watson statistic is a test statistic to detect autocorrelation present in the residuals
from a regression analysis. It is named after James Durbin and Geoffrey Watson
7 The p-value is the probability of obtaining a test statistic at least as extreme as the one that
was actually observed, assuming that the null hypothesis is true
41
200
Simulated
Observed
180
160
140
Spot price [e ]
120
100
80
60
40
20
0
0 50 100 150 200 250 300 350 400
Hour
Figure 4.7: Daily variations in German spot prices (year 2006)
ficients are insignificant for the affected variables in the multiple regression, but a
rejection of the hypothesis that the coefficients are insignificant as a whole (using
the F-test).
The R-squared value for the regression is 0.2843 and the adjusted R-squared
value, is 0.2805. This is a low value and it means that only about 28% of the
variation in the dependent variable can be explained by the independent variable.
The mean squared error of the regression is 1761 e /MWh2 and the root mean
squared error (for an unbiased estimator the root mean squared error is equal to
the standard error) is 41.96 e /MWh. This value for the root mean squared error
is high. It indicates that the agents would have a standard error in predicting spot
prices of 41.96 e /MWh based on the factors chosen; their price predicting ability
is very low. To specify the model in a better way, two measures are taken.
Removing outliers
By analyzing the price data it can be observed that there are several outliers: The
maximum observed price is 2436 e /MWh for instance. These outliers distort the
regression coefficients estimates. In a first step, a Matlab tool is used to diagnose
outliers. An outlier is suggested if the corresponding residual is larger than expected
in 95% of new observations. These outliers are removed and another OLS regression
is conducted.
The Durbin-Watson test statistics is now lowered to 0.577 which indicates a very
high degree of serial correlation. The R2 value increases to 0.811 and the adjusted
R-squared to 0.810, which indicates that now 81% of the variation in the dependent
variable can be explained by the independent variables. The root mean squared
error decreases to 10.65 e /MWh, which is an acceptable value. By removing the
42
outliers the regression coefficients βg,28 and βg,29 for Sundays and the heating degree
days are additionally statistically significantly different from zero.
Dealing with multicollinearity
Since there are a lot of regression coefficients which are not statistically significant
different from zero and a lot of factors are highly correlated, the price fraction is
removed. Now, all the remaining regression coefficients are statistically significant!
Hence, the hypothesis that the historical price fraction explains most of the variation
of the dependent variable is verified. After removing the outliers, the Durbin-
Watson test statistic is as low as 0.5331, the adjusted R-squared value 0.716 and
the root mean squared error 12.96 e /MWh. Therefore, the descriptive power of
the model decreases after removing the price fraction.
Dealing with serial correlation
As a least measure, the autoregressive part of the model is modified. Not only the
weekly and daily lagged price values are added but also λt−1 , λt−2 , λt−3 . Their
regression coefficients are statistically different from zero on a very low level of signif-
icance (p-values: 2.25e − 180 for λt−1 , 6.62e − 41 for λt−2 and 2.32e − 06 for λt−3 ).
The decay in the level of significance is also an indicator that an autoregressive
model is preferred over a moving average time series model.
With these additional factors the Durbin-Watson test statistic is 2.0012, which
is very close to 2 and indicates that there is no serial correlation left in the residuals.
The adjusted R-squared value is 0.409, which is an increase compared to previous
value of 0.2805. The outliers are removed and the regression is performed again.
The Durbin-Watson test statistic after the removal is 0.97, which indicates still high
serial correlation in the residuals, but the serial correlation is significantly lowered.
To remove more of the serial correlation more lagged terms have to be added, which
is not done in this thesis because there is no plausible rational for it. The adjusted
R-squared value is now increased to 0.843 and the root mean squared error reduced
to 10.4 e /MWh.
With the statistical work done so far the price predictor for the agents can be
set up. The following factors are finally chosen:
• Lagged λ values at λt−1 , λt−2 , λt−3 , λt−24 , λt−7·24 .
• Factors for Saturdays and Sundays.
• Cooling degree days
• Heating degree days
• Price fraction
• Oil price (although the regression coefficient was statistically not different
from zero on a 5% significance level it can still be significant in different
time periods with high oil prices for instance. This factor is chosen based on
rational reasoning and not on the basis of statistical analysis ).
43
• Wind (which is not significant on a 5% level, but on a 10% level of confidence).
These factors have a good descriptive power to estimate the day-ahead hourly
spot prices. Clearly, making longer time horizon spot price predictions decreases
the prediction accuracy. The agents will incorporate the higher uncertainty in price
prediction for longer time horizons by using a discount factor (see section 4.4).
4.2.4 Using the price predictor for out-of-sample data
To test the effectiveness of the price predictor a set of out-of-sample data for Ger-
many is taken to test the price predictor. The regression coefficients were chosen
based on 2006 data and the price predictor is used to predict 2007 Germany hourly
day-ahead spot prices.
200
Simulated
Observed
180
160
140
Spot price [e ]
120
100
80
60
40
20
0
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Hour
Figure 4.8: Out-of-sample predictions of German spot prices (year 2007)
Figure 4.8 shows the regressed λ values for the out-of-sample set of data against
observed EEX prices. Clearly, the out-of-sample data is not fitted as well as the
sample data. Figure 4.9 shows the first 400 days in the year 2007 to highlight the
ability to predict intra-day price variations. The fit is dependent on the chosen
interval; In Figure 4.10 a different time interval is chosen (hour 1200 to 1600), the
fit is worse in this time interval. The simulated prices are generally too high in this
time interval. In other intervals however, the simulated prices can be too low as
well.
The R2 values are 0.715 and 0.667 for the models without and with removing
outliers. Interestingly, the model which did not remove outliers can predict more
accurately the hourly day-ahead spot price variation with out-of-sample data. This
value states that around 71% of the hourly day-ahead spot price variation can be
explained by the independent variables.
44
100
Simulated
Observed
90
80
70
Spot price [e ]
60
50
40
30
20
10
0
0 50 100 150 200 250 300 350 400
Hour
Figure 4.9: Daily variation of predicted German spot prices (year 2007)
The root mean squared error is the most important measure to assess the pre-
diction ability of the price predictor. The values are 16.29 e /MWh for the model
which did not remove outliers and 18.00 e /MWh for the one which did. Again, the
model which did not remove the outliers has a higher performance regarding the
root mean sqaurd error with out-of-sample data.
The regression model is able to make hourly day-ahead spot price predictions
with out-of-sample data with a standard deviation of 16.29 e /MWh. It is important
to state that in the agent-based model this value can differ significantly since the
model generates spot prices internally and does not use EEX prices. The assumption
is that a price predictor which has a good predicting ability on historical EEX prices
will have a high predicting ability in the spot prices generated by the multi-agent
model as well. This relationship must not be true; therefore, a close look will be
taken at the prediction ability of the agents in the agent-based model. Possibly, the
price predictor has to be modified further to give the agents a good price predicting
ability.
45
100
Simulated
Observed
90
80
70
Spot price [e ]
60
50
40
30
20
10
0
1200 1250 1300 1350 1400 1450 1500 1550 1600
Hour
Figure 4.10: Daily variation of spot prices with a poor fit (year 2007)
4.3 The price adjuster
Another building block in Figure 4.1 is the price adjuster. Its purpose is to cap-
ture effects related to market-power. Ideally, the agents could use an additional
factor in the multi-factor model (4.1) for the supply curve they choose. This factor
would describe the influence which a certain action curve has on spot prices. If
the corresponding regression coefficient would be significantly different from zero
the agents would have the ability to influence the market price and as a result have
market-power. The supply curve is represented by multiple data points, whereas the
multi-factor model (4.1) requires a single number measure as independent variable.
Therefore, it is not possible to include the supply curves as additional factor. One
could argue that the supply curves could be numbered and these numbers could be
used in the multi-factor model. This is not possible because there is no ascending
or descending order in the supply curves: This means that it is not defined before
the simulation which supply curves will cause the spot market price λ to increase,
decrease or not to affect it. Therefore, it is not clear which supply curve has to
have a low and which a high number in the numbering process. Additionally, the
multi-factor model measures linear dependencies, but the effect of different supply
curves on spot prices can be highly nonlinear.
Therefore, the price adjuster is introduced. The price adjuster measures dynam-
ically the dependency between spot price λ and the corresponding supply curve. If
the multi-factor model in (4.1) is correctly specified, the error term ǫ should be
pure white noise. If it is assumed that the bidding of the agents affects the spot
prices, ǫ is not a pure white noise anymore. In the residuals there is information
about the effect of the agent’s bidding on spot prices. The price adjuster filters this
information out by using a reinforcement algorithm.
The implemented structure for the price adjuster is simple and straightforward.
There is potential for improving the price adjuster, but it is outside of the scope of
46
this thesis. The price adjuster measures the residuals λbt,c − λt,c , the estimated spot
price minus the observed price for time t in country c given agent i used action curve
j=f (t)
j (ai ) at time t and calculates dynamically the price adjusting value (pavi |aji )
for action j with the reinforcement algorithm:

bt,c
w · pavi |aji + λt,c − λ
pavi |aji = (4.21)
w+1
where w is a weighting factor. This weighting factor can be interpreted as a

learning rate and defines how fast the agent learns with new information. A low
w value thereby specifies a high learning rate, but w should not be chosen to low
because there is also noise present in the residuals. This implemented structure
should converge in expectation value (the measured pav value in (4.21) should
converge towards the true pav value if the action aji is played infinitely times),
because errors should average out. The agents calculate a pav value for all their
action curves. If the bidding of an agent would have no influence on market price,
all his pav values should be zero in the long-run.
The pavi |aji value indicates the agent that when he used action aji in the past,
the observed spot price λ used to be higher or smaller by the amount of pavi |aji than
the value he estimated λ. b The agent assumes he will have the same market-power in
the future, so he can adjust his price predictions for the future by the pav values for
every action and he will get a price prediction which takes into account his action:
bt,c |aj . This price predictions are calculated according to (4.22) for every action aj :
λ i i
bt,c |aj = λ
λ bt,c + pavi |aj (4.22)
i i
80
70
60
Price estimation [e ]
50
40
30
20
10
0 10 20 30 40 50 60 70 80
Hour
Figure 4.11: HPFC for three days
Figure 4.11 and Figure 4.12 show an example for applying the price adjuster. It
is assumed that agent i’s price predictor predicts prices for the front 3 days and he
47
150
Price estimation [e ]
100
50
−50
150
80
100
60
50 40
20
0 0
Action curve Hour
Figure 4.12: Future prices depending on actions after using the price adjuster
constructs the 3 day HPFC. This HPFC is given in Figure 4.11. For every hourly
price agent i adds his pavi |aji values for every available action aji . The result is a
matrix which gives a price estimate for every hour in the front three days and for
every action aji . The price prediction is now not only depending on time t, but also
on action aji . In Figure 4.12, the price predictions for the front 72 hours and agent
i’s set of action curves (aji ) are plotted. The more market-power agent i has, the
higher will be the gradient along the action curve axis in Figure 4.12.
The spot prices in the past used to be higher than the agent’s predictions for the
action curves with low numbering as seen in the example in Figure 4.12. In contrast,
the actions numbered with a higher value have lower or negative pav values. The
action curves with a low numbering propose little power output by agent i while
the ones with a high number propose a higher output.
4.4 The optimization routine
Another building block in Figure 4.1 is the optimization routine. It is a mapping

of the price prediction matrix to the optimal set of actions µ1 , ..., µN (the optimal
policy), which maximizes agent i’s expected total discounted profit over N time
steps.
Dynamic programming is used as optimization routine, mainly because it is

fast in computation and can guarantee a global maximum. The routine has to be
calculated for every agent and every day of simulation. Therefore, the computation
time for this optimization is crucial.
48
4.4.1 Introduction to dynamic programming
In this subsection, a short introduction to dynamic programming is provided. For a

more in depth discussion about dynamic programming the reader is referred to [48].
The basic problem
The following basic problem is formulated. A discrete-time dynamic system is given:
st+1 = ft (st , ut , wt ) (4.23)
where st is a state, ut a control and wt a random disturbance variable. The

control ut takes values in a nonempty set U (st ) which depends on the current state
st . The disturbance wt is characterized by a probability distribution Pt (·|st , ut )
which depends on st and ut but not on prior disturbances wt−1 , ..., w0 .
The control laws (policies) that consist of a sequence of functions
{µ0 , ..., µN −1 } (4.24)
where µt maps the states st into controls ut = µt (st ) that µt (st ) ∈ Ut (st ). Given
an initial state s0 and a policy {µ0 , ..., µN −1 }, the states st are defined through the
system equation
st+1 = ft (st , µt , wt ) (4.25)
Thus, for a given valuing function gt , t = 0, 1, ..., N , the expected reward of a

policy Υ {µ0 , ..., µN −1 } starting at s0 is
( N −1
)
X
N t
JΥ (s0 ) = E κ gN (sN ) + κ gt (st , µt , wt ) (4.26)
t=0
where the expectation is taken over the random variables wt and st . κ represents
a discount factor (0 < κ ≤ 1). An optimal policy Υ∗ is one that maximizes this
reward:
JΥ∗ (s0 ) = max JΥ (s0 ) (4.27)
4.4.2 The Bellman equation
To find the optimal policy Υ∗ (s0 ), Bellman expressed the well known optimality
principle [49]:
49
“An optimal policy has the property that whatever the initial state and
initial decision are, the remaining decisions must constitute an optimal
policy with regard to the state resulting from the first decision.”
In this thesis it is assumed that there is no random disturbance wk and s is

deterministic and not a random variable. The value function V (st ) is introduced
and defined as
N −1
!
X
N t
V (s0 ) = JΥ∗ (s0 ) = max κ gN (sN ) + κ gt (st , µt ) (4.28)
t=0
For an infinite horizon problem the basic formulation takes the form:
∞
X
V (s0 ) = max κt gt (st , µt ) (4.29)
t=0
subject to the constraint
st+1 = ft (st , µt ) ∀ t = 0, 1, ..., ∞ (4.30)
The optimality principle can be formulated as a recursive definition described

as
V (s0 ) = max (g(st , µt ) + κV (st+1 )) (4.31)
subject to (4.30). The formulation in (4.31) can be simplified even further by

plugging in the value of the next state:
V (s0 ) = max (g(st , µt ) + κV (ft (st , µt ))) (4.32)
The Bellman equation can be solved by backward induction either analytically

or numerically.
4.4.3 Applying the dynamic programming algorithm
Defining the expected power outputs and profits
The future spot price is a random variable with a certain distribution, as a result,
the agents are not able to accurately predict the future spot prices and an estimation
error with a certain distribution occurs. The distribution of the estimation error is
different for every agent since the agents incorporate different market-power. The
historical estimation errors for the N historical hours k = t − 1, t − 2, ..., t − N and
agent i are defined as λk,c − λbk,c |aj=f (k) , which describe the observed spot prices
i
50
minus the estimated spot prices taking into account that agent i used action curve
j at time k (the spot price prediction after the price adjuster). The spot price
predictions after adjusting for market-power are probably more accurate for agents
with major market-power, therefore, their historical estimation errors are lower.
This induces asymmetric information among the agents. Based on the historical
estimation errors, every agent calculates a historical estimation error distribution,
which gives a probability pr(∆λ) for every estimation error from -200 e /MWh to
200 e /MWh (because 200 e /MWh is the maximum price of simulation). If an
agent has a very high estimation capability, the probability for an estimation error
of 0 e /MWh would be 1 and every other probability 0, which implies that the
agent predicted the observed spot price almost surely. Two assumptions are made:
first, the agents assume that these probabilities do not change when making price
forecasts and second, the agents assume that these probabilities are independent
from the price level, action curve, exogenous factors etc.
Agent i’s state h is defined as his expected power output E(pi,t |ahi ) if he uses
action curve h. As a result, agent i’s state is only a function of action curve h since
bt,c is given by the price predictor and pavi |aj from the price adjuster. Agent i’s
λ i
set of actions ai is assumed to be finite and equal to M , the number of states for
agent i for every hour t is therefore equal to M as well.
sh,t = E(pi,t |ahi ) = f (ahi ) (4.33)
To be able to calculate the value of the total discounted profit (reward) for an
arbitrary policy, agent i has to know the spot prices (λ bt,c |ai ) and states (pi,t |ai )
for every future hourly time step. The future spot price prediction is a random
variable, to calculate the expected power output for a given action curve j and at
a future time t, agent i uses the following formula:
200
X
E(pi,t |aji ) = bt,c |aj )
pri,T (∆λ) · aji (∆λ + λ (4.34)
i
∆λ=−200
where aji (λ) defines the power output for price λ through agent i’s supply curve
j. The values for pr(∆λ) are taken from the probability distribution of historical
estimation errors by agent i. These probabilities depend on the current time T ,
since these probabilities are updated after every market outcome.
Figure 4.13 is an example for calculating the expected states matrix. The exam-
ple uses three future hourly time steps and takes three action curves into account.
For every time step, agent i can choose one of three action curves ai1,2,3 . Further-
more, it is assumed that agent i estimated historical spot prices almost surely, which
implies that pri,T (0) = 1. The agent first constructs the HPFC. In the example,
the values for the HPFC are λ bt=1 = 125, λbt=2 = 100 and λ bt=3 = 150. In Figure
4.13, these price forecasts are indicated by the dotted lines.
The pav values for the three action curves ai1,2,3 are: pavi |a1i = 25, pavi |a2i = −25
and pavi |a3i = 50. These pav value adjustments made by the price adjuster are the
solid shifted lines in Figure 4.13.
The pavi |ai values result in the following spot price forecasts given agent i
51
HPFC 125 100 150

0 50 100 150 200 0 50 100 150 200 0 50 100 150 200
100 +25 +25 +25

100 100
Output [%]
Output [%]
Output [%]
Action 1 50 50 50
pav: 25e
0 0 0
0 50 100 150 200 0 50 100 150 200 0 50 100 150 200
Price [€/MWh] Price [€/MWh] Price [€/MWh]
-25 -25 -25

100 100 100
Output [%]
Output [%]
Output [%]
Action 2 50 50 50
pav: -25e
0 0 0
0 50 100 150 200 0 50 100 150 200 0 50 100 150 200
+50
100 +50 +50
100 100
Output [%]
Output [%]
Output [%]
Action 3 50 50 50
pav: 50e
0 0 0
0 50 100 150 200 0 50 100 150 200 0 50 100 150 200
Time 1 Time 2 Time 3
Figure 4.13: Construction of the state matrix
chooses action j: λ bt=1,c |aj=1 = 150, λ bt=1,c |aj=2 = 100, λ bt=1,c |aj=3 = 175 for time
i i i
t = 1, λbt=2,c |aj=1 = 125, λ bt=2,c |aj=2 = 75, λ bt=2,c |aj=3 = 150 for time t = 2 and
i i i
bt=3,c |aj=1 = 175, λ
finally λ bt=3,c |aj=2 = 125, λ bt=3,c |aj=3 = 200 for time t = 3.
i i i
With the price forecast, agent i can calculate the estimated power outputs
E(pi,t |ai ) according to Formula (4.34). In Figure 4.13, the intersections of the
solid shifted lines of the price predictor and the supply curves define the estimated
power outputs since the agent predicted spot prices almost surely.
The result is a price matrix λ bt,c |ai and an expected state matrix pi,t |ai for every
time step and action curve. For simplicity, E(pi,t |ai ) is written as pi,t |ai . Figure
4.14 shows an example with three future time steps and 6 action curves. The values
for the value function V are initially all set to zero. With dynamic programming
the actions aµi 0 ,...,µN is found whereby the action µt at time t is chosen in order to
maximize the expected discounted profit.
Formulating the dynamic programming problem
The reward function gi,t corresponds to agent i’s profit at time t. The state variable
st is defined by agent i’s power output pi,t |aji given agent i uses action j. The reward
function gi,t can be calculated by
bt,c |aj · pi,t |aj ) − (V C p + V C r + V C ea )

gi,t = E(λ (4.35)
i i i,t i,t i,t
52
b1,c , pi,1 , Vi,1 )|a6i

(λ b2,c , pi,2 , Vi,2 )|a6i
(λ b3,c , pi,3 , Vi,3 )|a6i
(λ
b1,c , pi,1 , Vi,1 )|a5i

(λ b2,c , pi,2 , Vi,2 )|a5i
(λ b3,c , pi,3 , Vi,3 )|a5i
(λ
b1,c , pi,1 , Vi,1 )|a4i

(λ b2,c , pi,2 , Vi,2 )|a4i
(λ b3,c , pi,3 , Vi,3 )|a4i
(λ
b1,c , pi,1 , Vi,1 )|a3i

(λ b2,c , pi,2 , Vi,2 )|a3i
(λ b3,c , pi,3 , Vi,3 )|a3i
(λ
Current state b1,c , pi,1 , Vi,1 )|a2i

(λ b2,c , pi,2 , Vi,2 )|a2i
(λ b3,c , pi,3 , Vi,3 )|a2i
(λ
λ0,c , pi,3
b1,c , pi,1 , Vi,1 )|a1i
(λ b2,c , pi,2 , Vi,2 )|a1i
(λ b3,c , pi,3 , Vi,3 )|a1i
(λ
Time 0 Time 1 Time 2 Time 3
Figure 4.14: Price and state matrix
and takes the expected revenue as well as production (V C p ), ramping (V C r )

and emission allowance (V C ea ) costs into account (see chapter 3 section 3.3 for
calculating the costs). The expected revenue is calculated according to the following
formula:
200
X
bt,c |aj · pi,t |aj ) =
E(λ bt,c |aj ) · (∆λ + λ
pri,T (∆λ) · aji (∆λ + λ bt,c |aj ) (4.36)
i i i i
∆λ=−200
p
V Ci,t and V C ea are only functions of pi,t |aji , whereas V C r depends on the
j=f (t+1) j=f (t)
production difference pi,t+1 |ai and pi,t |ai (the state at time t + 1 and
state at time t). A simplification is made since it is assumed that V C r << V C p .
V C r is calculated according to
j=f (t+1) j=f (t)

V C r = (E(pi,t+1 |ai ) − E(pi,t |ai )) · δ (techi ) (4.37)
instead of calculating every probability weighted production difference. As a

j=f (t) j=f (t+1)
result, gt can be written as gt = gt (pi,t |ai , pi,t+1 |ai ). Agent i’s problem
to find the set of actions (optimal policy) which maximizes his expected discounted
profit is now formulated according to
N −1
!
X j=f (t) j=f (t+1)
N t
V (pi,0 ) = max κ gN (pi,N ) + κ gt (pi,t |ai , pi,t+1 |ai ) (4.38)
t=0
The problem can be reformulated to find an action curve µt for every hour t in
agent i’s set of actions which maximizes V (pi,0 ). This problem is solved using the
Bellman equation and numerical backward induction as
53
V (pi,t ) = max (gt (pi,t , pi,t+1 ) + κV (pi,t+1 )) (4.39)
An algorithmic description for the inelastic and elastic bidders
In this section, an algorithmic description is provided to demonstrate the backward

induction and application of the Bellman equation. The agents in the model try
to maximize their profit over an infinite time period. This would result in infinite
backward induction steps, which is not feasible because this would result in infinite
computations. Therefore, the horizon is capped, which means that the agents try
to maximize their profit over a finite rolling horizon. The agents are discounting
the future profits because the uncertainty in predicting future spot prices increases
with increasing estimation horizon. As a result, the agents value profit more in the
near future than in the far future. A discount factor of 0.95 is used in the model.
This implies that for instance the profit one week in the future (at t = 7 · 24) is
valued only 0.957·24 ≈ 0.02% compared to a profit at time t = 0.
The concept of rolling horizon defines that agent i chooses his optimal policy
at day d for the next week (aµi 1 , ..., aµi N ). He sends the first 24 supply curves
(aµi 1 , ..., aµi 24 ) to the market where the market clearing takes place and discards the
supply curves aµi 25 , ..., aµi N . At day d + 1, agent i optimizes again his production
µ
over the upcoming week (aµi 25 , ..., ai N +24 ), whereby the action curve aµi 24 serves as
initial position for this optimization. Agent i then sends the front 24 supply curves
to the market and so on.
The optimization horizon in the model is chosen finite and equal to one week.
The first step in deriving the optimal policy of actions is to specify the terminal value
T V after one week, which is equal to the term κN gN (pi,N ) in (4.38). In the model
used in this thesis the terminal value is set to E(λ bt,c |aj · pi,t |aj ) − (V C p + V C ea ),
i i i,t i,t
which implies that agent i assumes that he produces one additional hour at the
current power output at the very end of the horizon (no ramping). The effect of the
terminal value on the first 24 supply curves (the relevant supply curves since only
the 24 front supply curves are actually used) is minor because of the discounting.
Figure 4.15 visualizes that all the value functions V for all states at t = N are
set equal to the corresponding terminal value. The next step is to move one hour
backwards to t − 1 (which is t = 2 in Figure 4.15).
At time t = N −1 state y is chosen: pi,t |ayi . Then, all the combinations pi,t |ayi →
pi,t+1 |axi for all actions x at t + 1 are analyzed. Agent i has ramping constraints if
he is an inelastic bidder. The power output difference between t and t + 1 must be
smaller or equal to the technical ramp capacity
|pi,t+1 − pi,t | ≤ τi (4.40)
The spot price prediction at time t and t + 1 is a random variable and can take
any value between 0 e /MWh and 200 e /MWh (these are the limits set by the
simulation). Therefore, from the analyzed state y (pi,t |ayi ) at time t it has to be
ensured that the ramping constraints are satisfied for t + 1. Figure 4.16 serves as
an example. The supply curve ayi at time t as well as a supply curve at time t + 1
are shown. The most extreme power outputs occur at prices 0 e /MWh and 200
54
b2,c |a6i · pi,2 |a6i ) − T C(qi,2 , qi,3 |axi ) + κ · Vi,3 |axi )

Vi,2 |a6i = max(E(λ V C r = f (qi,2 , qi,3 |a6i )
Vi,6 |a6i = T V6
V C = f (qi,2 , qi,3 |a5i )
r
Vi,6 |a5i = T V5
V C r = f (qi,2 , qi,3 |a4i )
Vi,6 |a4i = T V4
Vi,6 |a3i = T V3
Vi,6 |a2i = T V2
Current state
Vi,6 |a1i = T V1
Figure 4.15: Backward induction
e /MWh. Their corresponding hypothetical power outputs are indicated as pex1,t

and pex2,t for the supply curve at time t and pex1,t+1 and pex2,t+1 for the supply
curve at time t + 1. If the following holds
|pex1,t+1 − pex1,t | ≤ τi and |pex2,t+1 − pex1,t | ≤ τi (4.41)
|pex1,t+1 − pex2,t | ≤ τi and |pex2,t+1 − pex2,t | ≤ τi (4.42)
it is ensured that the ramping constraints are not violated.
100
80
Power output [%]
pex2,t
60
Supply curve at t
pex1,t pex2,t+1
40
Supply curve at t + 1
20 pex1,t+1
0
0 50 100 150 200
Spot price [e /MWh]
Figure 4.16: Example of ramping constraints
55
Only if the ramping constraint is satisfied for the action curve x (axi ) at time
t + 1, x is considered feasible and is further analyzed. The elastic bidders and the
storage bidders have no ramping constraints, hence, all supply curves at time t + 1
are analyzed. The next step is to calculate the discounted profits from the state y
to all the feasible states x at time t + 1 by calculating the following equation:
V (pi,t+1 |axi , pi,t |ayi ) = (gt (pi,t |axi , pi,t+1 |ayi ) + κV (pi,t+1 |axi )) (4.43)
The action curve x which maximizes V (pi,t+1 |axi , pi,t |ayi ) is chosen. This x value
as well as its corresponding V value
V = max {(gt (pi,t |axi , pi,t+1 |ayi ) + κV (pi,t+1 |axi )} (4.44)
which is referred to as Vi,t |ayi , the value function V for action curve y at time
t, are saved in state y (pi,t |ayi ). If agent i is in state y (pi,t |aij=y ) at time t he
knows that he has to chose the action curve x which is saved in state y for t + 1
to maximize his expected discounted profit. The V value saved in state y at time t
(Vi,t |ayi ) indicates him the expected discounted profit if he chooses action x at time
t + 1 and follows in an “optimal way” thereafter.
As indicated in Figure 4.15 for state pi,2 |a6i the states x = 4, 5 and 6 for t = 3 are
feasible states. The value E(λ b2,c |a6 ·pi,2 |a6 )−(V C p +V C ea ) is state y specific.
i i i,t=2 i,t=2
The ramping costs V Ci,t depend on the state x (pi,3 |axi ) at t + 1. There are three
r
V values if agent i is in state 6 at time t = 2 in this example. The maximum value

for V (pi,3 |ai4,5,6 , pi,2 |a6i ) is chosen. This V value and the corresponding x ∈ 4, 5, 6
value are saved in the state pi,2 |a6i .
This procedure is repeated for all states and all times in a backward manner.
The result is that in every state for every time step an x and V value are saved.
This allows that agent i knows which action curve he has to send to the market at
t + 1 if he is in state y at time t and what his expected maximum discounted profit
is if he uses this action curve and follows in an “optimal way” thereafter. In Figure
4.17, the arrows show what action curves x agent i has to choose at time t + 1 if he
is in a certain state y at time t. The arrows also indicate the optimal actions (the
“optimal way”) after t + 1.
Once this procedure is finished, the current state of the agent becomes relevant.
The current state pi,0 of agent i is equal to the power output at hour t = 24 and
day d. In words, this relationship is defining that the current state for tomorrow
is equal to the power output of the last hour (t = 24) today. Depending on the
current state an optimal supply curve for t + 1 is found by choosing action x for
hour t = 1 which maximizes the V value according to
V (pi,t+1 |axi , pi,0 ) = max {(g0 (pi,0 , pi,1 |axi ) + κV (pi,1 |axi )} (4.45)
Since agent i is in a distinct current state pi,0 , x is exactly one value. This x
value for t = 1 is saved under the name µ1 , the first value of the optimal policy. By
observing the stored x value in state µ1 at time t = 1, agent i knows the optimal
policy value (µ2 ) for t = 2 and so on. This process is called forward induction. In
56
Vi,6 |a6i = T V6
Vi,6 |a5i = T V5
Vi,6 |a4i = T V4
Vi,6 |a3i = T V3
Vi,6 |a2i = T V2
Current state
Vi,6 |a1i = T V1
Figure 4.17: Backward induction
figure 4.18, this process of forward induction is shown. Agent i follows the arrows
which are indicators for the µ values at time t + 1.
Vi,6 |a6i = T V6
Vi,6 |a5i = T V5
Vi,6 |a4i = T V4
Vi,6 |a3i = T V3
Vi,6 |a2i = T V2
Current state
Vi,6 |a1i = T V1
Figure 4.18: Forward induction
In the vector µ ∈ (µ1 , ..., µN ) the optimal policy is saved. The first 24 supply
curves (aµi 1 ,...,µ24 ) are sent to the market, the following supply curves (aµi 25 ,...,µN )
are discarded.
In the following, a compact formulation of the algorithm is provided (see algo-

rithm 1). M is the number of possible supply curves of agent i.
57
Algorithm 1 The optimization routine for the elastic and inelastic bidder
1: Backward induction:
2: Set all terminal values at time t = N
3: for time t = N − 1 to 1 do
4: for state y = 1 to M at time t do
5: for state x = 1 to M at time t + 1 do
6: Check if action x is feasible from state y
7: if It is feasible then
8: Calculate the V value for state x
9: else
10: Do not take state x into account
11: end if
12: end for
13: Pick x with max(V ) and save x in state y
14: end for
15: end for
16: Forward induction:
17: for state x = 1 to M at time t = 1 do
18: Check if action x is feasible from the current state
20: Calculate the V value for state x
21: else
22: Do not take state x into account
23: end if
24: end for
25: Pick x with max(V ) and µ1 ← x
26: for time t = 1 to N do
27: Go to state µt and set µt ← xt−1 , where xt−1 is the x value saved in state µt
28: end for
µ ,...,µ24
29: Send action curves ai 1 to the market
58
An algorithmic description for the storage bidders
The storage bidders can buy and sell electricity and they incorporate a storage to
store a certain amount of energy. As seen, their supply curves or part of it can
be negative which indicates buying the corresponding amount of energy whereas
positive means selling energy. If these agents sell or buy electricity and how much
j=f (t)
they sell or buy depends on the supply curve ai they send to the market and
the spot price λt,c at time t.
There is a main difference between the storage bidders and the elastic and in-
elastic bidders. The elastic and inelastic bidders use the power output pi,t at time
t as their state. In contrast, the storage bidders use their filling status fi,T , which
RT
is equal to t=1 pi,t dt, as their state at time T . In contrast to the power output pi,t ,
the filling status is a continuous measure. The DP algorithm cannot be applied in
this case. The solution used in this thesis is that the filling status fi,T is discretized
in M states to enable applying the dynamic programming algorithm.
A constraint for the storage bidder is

Z T
fi,T = pi,t dt ≥ 0 and fi,T ≤ Σi ∀ T = 1, ..., N (4.46)
t=1
which describes that the storage agent cannot be discharged below zero and also
cannot be charged higher than a maximum value Σi at any time. Note that Σi is
an energy content and given in MWh.
Starting point for the DP algorithm is again setting the terminal value T V . A
rule for the storage bidders is that at t = N , the reservoir of agent i should be
filled with 21 · ǫ. Therefore, all terminal values are chosen to be 0 except a very high
value is assigned to the middle filling level state fi,N = M 2 at time t = N , where M
describes the amount of states Σi is discretized in. This implies that agent i tries
to reach that value at time t = N because he expects a “high virtual reward” by
attaining this state.
The reward function for the storage bidders looks different than for the elastic
and inelastic bidders. It is divided into two cases: charging and discharging. It
follows
bt,c |aj · pi,t |aj )

E(λ bt,c |aj · (E(f (i, t)) − E(f (i, t − 1)))
λ
i i i
gt = − ≈− (4.47)
ηi,charge ηi,charge
for charging (if E(f (i, t)) ≥ E(f (i, t − 1))) and
gt = ηi,discharge · E(λ bt,c |aj · pi,t |aj )

i i
(4.48)
bt,c |aj · (E(f (i, t)) − E(f (i, t − 1)))
≈ ηi,discharge λ i
for discharging (if E(f (i, t)) < E(f (i, t − 1))).
59
For the storage agent, the Bellman equation can be rewritten as

V (fi,T +1 , fi,T = y) = max (gT (fi,T +1 , fi,T ) + κV (fi,T +1 )) (4.49)
From the terminal value at T = N , one time step is moved back. From there, a
state y (f (i, T = N − 1) = y) at time T − 1 is chosen. Now, the production levels
x (pi,N |axi ) at T = N are calculated. From the analyzed state y it is checked that
Σi
0≤ · (E(f (i, T = N − 1)) − 1) + E(pi,N |axi ) ≤ Σi (4.50)
M −1
is satisfied which specifies that the current reservoir level at time T plus any
charging/discharging flows x at time T + 1 should stay within the limits specified
in (4.46). If there is no feasible charging/discharging flow at time T + 1 which
satisfies (4.46) for state y, the value function for this particular state y is set to a
very high negative value. With this setting, it is ensured that storage agent i does
not approach this state y at time T . The next step is to specify if pi,T +1 |axi charges
or discharges and calculate the value function for all charging/discharging flows x
at time T + 1 according to
V (fi,T +1 , fi,T = y) = gT (fi,T +1 , fi,T ) + κV (fi,T +1 ) (4.51)
with the reward function g calculated by (4.47) if x charges or (4.48) if x dis-

charges. The value x which maximizes V (fi,T +1 , fi,T = y) is saved in state y.
Thereby, this x value specifies the action curve at time T + 1 that leads to the
filling status fi,T +1 at T + 1. Agent i chooses this action x if he is in state y at
time T to maximize his discounted expected profit. This procedure is repeated for
every state y and time T in a backward manner.
After this backward induction the current state of storage agent i is taken into
account. From the current state (fi,0 ) the action curve x is chosen which satisfies
the filling constraints and has the highest value function V . There is only one x
value which maximizes V (fi,1 , fi,0 ) because agent i is in one distinct current state.
This x value is saved in µ1 .
The next step is the forward induction. For the forward induction, a value
cumi,0 is introduced and set to the current state fi,0 . The value cum is the filling
level if agent i follows the optimal policy; the cum level at time t = 1 is calculated
according to
cumi,1 = cumi,0 + pi,1 |axi · ∆t (4.52)
In state cumi,1 at time T = 1 agent i reads the x value saved, which gives the
optimal action for time T = 2 and saves this x value under µ2 . This procedure is
continued until a terminal value is reached. Under µ the optimal policy is saved
given agent i’s current state. The storage agent sends the first 24 supply curves
(aiµ1 ,...,µ24 ) to the market.
In the following, a compact formulation of the algorithm for the storage agents
is provided (see algorithm 2).
60
Algorithm 2 The optimization routine for the storage bidder

1: Backward induction:
2: Set all terminal values at time t = N
3: for time t = N − 1 to 1 do
4: for state y = 1 to storage states at time t do
5: for state x = 1 to storage states at time t + 1 do
6: Check if the filling status y plus the flow x (pi,t |axi ) fulfill the constraints
8: Calculate the V value for flow x
9: else
10: Do not take the flow x into account
11: end if
12: end for
13: if x is empty then
14: Set V equal to a very negative value
15: else
16: Pick x with max(V ) and save x in state y
17: end if
18: end for
19: end for
20: Forward induction:
21: for state x = 1 to storage states at time t = 1 do
22: Check if the current filling status plus the flow x (pi,1 |axi ) fulfill the constraints

24: Calculate the V value for flow x
25: else
26: Do not take the flow x into account
27: end if
28: end for
29: Pick x with max(V ) and µ1 ← x
30: Set cumi,0 ← fi,0
31: for time t = 1 to N do
32: Set cumi,t ← cumi,t−1 + x, where x = pi,t |axi
33: Go to state cumi,t and set µt ← xt , where xt is the x value saved in state
cumi,t
34: end for
35: Send action curves aµi 1 ,...,µ24 to the market
61
The fact that the spot price estimation is a random variable affects the storage
bidders. The storage agent is choosing a set of actions to buy and sell electricity.
These actions have to satisfy the constraints that the storage is never below 0 and
higher than Σi . Since the actual spot prices are different than the estimated ones,
it has to be checked that this constraint is fulfilled also during the actual market
clearing. If an agent for instance has no water in his storage and his supply curve for
the next hour results in selling electricity, he has to be restricted to sell electricity.
4.4.4 Discussion of the optimization routine
The agents only maximize their profit and do not take risk into account. Since
the spot price prediction is a random variable, different bidding strategies can have
different risks (measured by standard deviation of profit, value at risk (VAR) or
profit at risk (PAR)). Profit at risk is a measure for risk often used by electricity
trading companies. A more realistic implementation is one where the optimal policy
is one which maximizes the risk adjusted profit. A measure often taken for the risk
adjusted return is the Sharp ratio. The Sharp ratio is defined by the following
formula:
R − Rf E (R − Rf )
S= =p (4.53)
σ var (R − Rf )
where R is an asset return, Rf is the return on a benchmark asset (such as the

risk free rate of return) and σ the standard deviation of the excess return R−Rf [50].
With a given spot price prediction distribution, a Monte Carlo simulation could be
used to assess the risk present in a bidding strategy. With different profits and
risks for different bidding strategies the one with the highest Sharp ratio could be
chosen. Instead of returns, profits could be used in (4.53), but finding a value for
Rf is difficult since a “risk free profit” in electricity markets is not defined. Other
measures for risk adjusted returns are possible; e.g divide the excess profit by the
profit at risk. The implementation of risk adjusted returns is outside the scope of
this thesis and does not serve its primarily purpose. The purpose of the model is to
show the spot market outcome by using several profit maximizing agents. It can be
concluded that by not taking risk into account an agent’s bidding strategy in the
model can be very different than the strategy a utility would choose in reality.
62
Chapter 5
Testing the agent-based

model
“Testing leads to failure, and failure leads to understanding”

Burt Rutan
In this section, the model is tested in a very simple scenarios to analyze the
outcome under little complexity and to observe if the simulated prices converge or
diverge. The learning algorithm model predictive bidding is used by the agents as
learning algorithm. The model is implemented in Matlab R
. This simulation setting
is referred to as test scenario later on in the thesis.
5.1 Settings
It is assumed that there is a nuclear, hard coal lignite, gas turbine, oil, storage and
wind agent. Their maximum power outputs correspond to the real ones in Germany
(Table 6.1). The ramping capacities are 20% of the total capacity per hour for the
nuclear agent and 100% for the lignite and hard coal agent. The ramping costs are
100 e /MWh for the nuclear, hard coal and lignite agent. The values for the price
predictor are set to following constant values: The wind output and wind forecast
is 0 MW and the temperature is set to 18.3◦ C for all hours. There is no weekend
factor and the load is equal to 50’000 MW, the oil price 50 e /MWh.
The values for the fuel cost are constant as well. The oil price is 50 e /MWh,
hard coal and lignite 6 e /MWh, natural gas 23 e /MWh and Uranium 0.045
e /MWh. The emission cost are 12 e /ton CO2 .
The price predictor and price adjuster have no historical values x to make price
predictions and adjustments for the agents. As a result, an initial dataset has to
be given to the agents. In this dataset all prices are set to 1 e /MWh and all pav
values are set to 0 e /MWh. The prediction of the agents will be wrong and equal
to 1 e /MWh at the beginning of the simulation. But once they start competing on
the market and market clearings take place, the price predictor and adjuster will be
63
Chapter 5. Testing the agent-based model
fed by internally generated data. The agents have a 3 month memory, which means
that the trailing 3 month data is stored in the memory to derive the regression
coefficients for the price predictor and estimation error distribution. Older data is
deleted, which implies that the agents “forget” older market outcomes.
Because the price predictions are so inaccurate at the beginning of the simula-
tion, the agents cannot choose a meaningful optimal policy. The agents therefore
choose the optimal policy only with a given probability and a randomly generated
set of actions otherwise. The exponential function in Figure 5.1 shows the probabil-
ity of choosing a random action set. At the beginning of the simulation, the agents
choose a random set of actions about 50% of the time. This value decreases towards
zero over time to accommodate the effect of more accurate price predictions. For
the inelastic bidders, the random generated set of actions has to fullfill the ramping
constraints.
0.5
0.45
0.4
0.35
Probability
0.3
0.25
0.2
0.15
0.1
0.05
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Hour
Figure 5.1: Probability of choosing a random action
5.2 The training round
In this first run, the agents use random actions as discussed and update the regres-
sion coefficients of the price predictor as well as the pav values of the price adjuster.
To calculate the regression coefficients, the agents use historical environmental data
and price values of the trailing three months. Furthermore, the agents update the
estimation error distribution taking into account market-power after every market
outcome.
5.2.1 The spot price
Figure 5.2 shows the simulated base load price of the test scenario and training
round. A high spot price volatility is observable at the beginning of the simulation
64
180
160
140
Spot price [e /MWh]
120
100
80
60
40
20
0
0 50 100 150 200 250 300 350 400
Day
Figure 5.2: Simulated base load prices test scenario, training round
when the probability of choosing a random action is higher. This high price volatility
steadily decreases throughout the simulation as the agents learn by observing market
outcomes.
Towards the end of the simulation trial the price shows a cyclical price behavior.
It almost looks like a regime switching behavior with a high and low price level.
5.2.2 Power outputs
Figure 5.3 shows the mean power outputs for the agents. The ramping of the nuclear
agent is not as high as the lignite, hard coal or gas agent because he has a lower
ramping capacity. His power output is generally high since his marginal production
costs are very low and it is profitable to produce at almost every positive spot price
level. He tries to stabilize his production on a constant level to avoid ramping costs
by submitting perfectly inelastic supply curves. The lignite and hard coal agent are
also successfully trying to stabilize their production. The gas and oil agent show a
higher ramping behavior but are producing on higher output levels as the simulation
continues. Their marginal cost are too high to produce all the time, depending on
the spot price prediction they shut down their production to reduce the losses.
65
4
x 10
2.5
Nuclear
Lignite
Hard coal
2 Gas
Oil
Storage
Wind
Power output [MW]
1.5
0.5
−0.5
0 50 100 150 200 250 300 350 400
Day
Figure 5.3: Mean power outputs test scenario, training run
5.2.3 Market-power
The pav values for the action curves are an indicator of market-power because the
pav values measure how much the spot market price changed on average from the
estimated price for a particular action curve. Figure 5.4 shows the pav values for all
action curves and agents. The agents have different amounts of action curves in their
population. The reason lies in the different ramping constraints. The power output
difference for the prices 0 e /MWh and 200 e /MWh for one particular supply
curve has to be smaller than the ramp capacity. This limits the price elasticity for a
certain supply curve which an agent with ramping constraints can use. Therefore,
the nuclear agent has a fewer action curves than the hard coal or lignite agent.
For supply curves which propose a high power output, the nuclear agent has
supply curves with large negative pav values. He can therefore alter the outcome of
the spot price dramatically by using these supply curves. Interestingly, the nuclear
agent has action curves with very negative pav values but only a few with highly
positive pav values. Hence, the nuclear agent can influence the spot prices downward
but barely upward. It can be concluded that the inelastic bidders generally are able
to push the spot price to lower levels.
The absolute pav values for the gas agent are smaller than the ones for the
nuclear agent. The reason is that the gas agent has a lower installed capacity. He
is therefore not able to influence the spot price outcome as much. The market-
power generally increases with increased installed capacity per agent. In contrast
to the nuclear agent, the gas and storage agent have a more symmetrical pav value
distribution. They can change the spot price outcome to lower and higher price
levels.
66
80
Nuclear
Hard coal
Lignite
60 Gas
pav value [e /MWh] Storage
40
20
−20
−40
0 50 100 150 200 250
Action nr.
Figure 5.4: pav values test scenario, training run
5.3 Second run
The agents save the data from the training run in their memory. In a second
simulation trial, the agents’s price predictors and price adjusters have past data
values. From the beginning of the simulation the agents are able to predict spot
prices reasonably well. The agents therefore do not choose random actions, they
follow the optimal policy for every day which maximizes their expected profit. The
regression coefficients of the price predictor and the estimation error distribution are
still updated while the pav values are not, the following simplified example explains
the reason:
It is assumed that there are only two agents, both have two supply curves, one
which describes a low power output (supply curve bid low ) and one with a high
power output (bid high). It is assumed that the agents have 10 e /MWh market-
power, which means that if they use bid low, the spot price will on average be
a mean value plus 10 e /MWh and if they use bid high, a mean value minus 10
e /MWh. The spot price will be equal to the mean value if one agent uses bid low
and the other one bid high.
The pav values are initially set to 0 e /MWh and both agents use random
actions. If one agent (called agent stat) uses bid high, two different outcomes are
possible: The other agent (called agent var ) uses bid high as well, which results
in a spot price of a mean value minus 20 e /MWh or the agent var uses bid low,
which results in a spot price equal to the mean value. If agent var uses these two
supply curves with the same probability, the pav value of agent stat will eventually
converge toward -10 e /MWh, which is an unbiased pav measurement.
It is now assumed that both agents follow the optimal policy and the price
predictor predicts a low spot price; therefore, both agents use supply curve bid low.
The spot price is now equal to a mean value plus 20 e /MWh. The case in which
67
one uses bid low and the other one bid high does not occur anymore. The price
adjuster measures +20 e /MWh, which is the overall effect if both agents use bid
low and is therefore a biased measurement. The measurement is not independent
of the other agents bidding if both agents follow the optimal policy. Therefore,
updating the pav values is only meaningful when the agents use random actions.
In the following, the results of the simulation trial are discussed.
120
100
Spot price [e /MWh]
80
60
40
20
0
0 50 100 150 200 250 300 350 400
Day
Figure 5.5: Simulated base load price test scenario, second trial
Figure 5.5 shows the simulated spot price for the second simulation trial. The
spot price does not converge to an equilibrium, rather, the spot price shows a cyclical
behavior with high and low price outcomes. It also shows some regime switching
behavior with high and low price levels, sometimes the spot price even drops to 0
e /MWh. The spot price shows hardly any mean reverting behavior which is often
observed in real electricity prices.
5.3.2 Power outputs and estimation error
In Figure 5.6, the power outputs for the different agents are shown. It illustrates
the competition among the agents as they try to gain a higher market-share. When
the gas agent has a high power output the inelastic bidders have to reduce their
outputs and vice versa. The inelastic bidders among each other follow similar power
output strategies.
The nuclear, lignite and hard coal agent are generally producing on high power
output levels because of their low marginal production costs. The oil and gas agent
on the other hand have higher marginal costs. If the predicted spot price is above
68
20000
Nuclear
Lignite
Hard coal
Gas
15000 Oil
Storage
Wind
Power output [MW]
10000
5000
−5000
0 50 100 150 200 250 300 350 400
Day
Figure 5.6: Mean power outputs test scenario, second trial
their marginal costs, they are willing to supply at full load, if it is lower than their
marginal costs, they try to shut down the production. The oil agent has long periods
where he does not produce electricity at all. The storage agent is buying and selling
electricity during times with a spot price volatility.
In the following, the estimation error distributions of the agents are analyzed.
Figure 5.7 shows the distributions of the estimation errors for all agents taken into
account their market-power. The distributions look different for all agents. It can
be concluded that the agents have asymmetric information. The root mean squared
error of the estimations is 26 e /MWh for the nuclear, 31 e /MWh for the lignite,
29 e /MWh for the hard coal, 35 e /MWh for the gas, 36 e /MWh for the oil and
finally 35 e /MWh for the storage agent. It indicates that the spot price prediction
ability increases with increasing market-power. This observation is not surprising.
If there are for instance only two agents of which one has market-power and the
other does not, then the agent without market-power has to estimate spot prices
only based on exogenous factors, because he does not know how the other agent is
going to influence the spot price based on his bidding. The agent with market-power
on the other hand can take his bidding into account as an additional factor to make
spot price predictions.
In general, the estimation distribution is wide and the root mean squared errors
are high which implies that the agents cannot predict the spot prices accurately.
The reason is that the spot price variations are not driven by exogenous factors but
by the intense competition among the agents.
In the next section, the model is applied to the German electricity wholesale
market. The changing load and exogenous factors can lead to a very different
spot price behavior and different market-powers. It can be assumed that more of
the spot price deviations are driven by exogenous factors which would lead to a
higher spot price predictability and lower estimation errors. It is possible that in
an environment with changing loads and exogenous factors a mean reverting spot
69
0.06
Nuclear
Lignite
Hard coal
0.05 Gas
Oil
Storage
0.04
Probability
0.03
0.02
0.01
0
−200 −150 −100 −50 0 50 100 150 200
Est. error [e /MWh]
Figure 5.7: Estimation error distribution test scenario, second trial
price behavior is attainable.
70
Chapter 6
Model application: The

German electricity market
6.1 Simulation settings
The German electricity market is modeled to test the model predictive bidding
learning algorithm described in chapter 4 in a realistic environment. Additionally,
different scenarios are considered to assess the effect of an additional wind energy
contribution, storage devices and a PHEV cluster on German spot prices. Seven
agents are taken into account. Each agent represents one of the seven generation
technologies described in table 6.1. Additionally, table 6.1 shows the installed gen-
eration capacity for the implemented agents.
Generation technology Installed capacity [MW]

Nuclear 20’263
Hard coal 19’947
Lignite 17’908
Gas turbine 16’643
Oil 2’578
Storage 2’210
Wind 21’139
Total 100’688
Table 6.1: Installed capacities for the German agents, source EIA
According to EIA, 2’210 MW hydro power capacity was installed in 2006. Some
of the installed capacity represents river, some storage hydro and some pump-
storage hydro power plants. For simplicity, they are all assumed to be pump-
storage hydro power plants and are aggregated to one storage agent. The effect
of this simplification is minor since this storage agent accounts for less than 3% of
total installed capacity in Germany.
The fact that every generation technology is aggregated and modeled by one
agent does not reflect reality. As mentioned, the four biggest utilities in Germany
have over 90% market share in electricity production. These utilities have different
power plants in their portfolios. It is more reasonable to model each generation
71
Chapter 6. Model application: The German electricity market
technology within a utility as one agent as it is done in [41] for instance. But the
goal of the thesis is not finding a model that optimally resembles the German elec-
tricity market but rather gaining insights into the bidding strategy of the different
generation technologies. Additionally, the model is also used to model the interac-
tion between countries and for simplicity, the generation technologies within each
country are aggregated.
As discussed in Section 4.2, the price estimator was set up to predict historical
EEX prices. Thereby, it was assumed that the regression model that is capable of
predicting EEX prices is also capable of predicting internally generated prices in
the agent-based model. It turns out that for the agent-based model, the estimation
error of the price predictor is smaller if the auto regressive part of the price predictor
is neglected.
The year 2007 serves as a simulation basis. The first run is the training run, as
in chapter 5, the agents have only 1 e /MWh values in their memories for making
price predictions and the pav values are all set to 0 e /MWh. The agents choose
random actions with the probability specified in Figure 5.1. The simulated prices
are compared to observed 2007 EEX price data and the behavior of the agents
is analyzed. It is important to highlight that the prices generated by the model
cannot be compared to the prices of the regression model in chapter 4. The fit of
the regressed prices and the observed EEX prices is higher because the regression
model takes historical EEX prices as input to derive the regression coefficients. In
contrast, the agent-based model does not take any historical EEX prices as input.
All the prices are internally generated by the model.
6.2 Simulation results reference scenario
6.2.1 The training run
Figure 6.1 shows the simulated and observed EEX prices for the training run and
the reference year 2007. The simulated prices tend to be much higher than the
observed EEX prices at the beginning of the simulation. The price predictor of the
agents predicts a low price because the price predictor uses a dataset consisting of
spot prices equal to 1 e /MWh. To minimize losses, the agents send supply curves
with low power outputs to the market. As a result, the market supply curve shows
little power output for all prices. In this case, it is likely that the demanded quantity
exceeds the maximum power output of the market supply curve. A market clearing
is not possible and the case occurs where the price is set equal to the maximum
price of simulation, which is 200 e /MWh.
The 200 e /MWh data points are stored in the agents memory. The price
predictor will eventually predict higher prices and more agents start to increase
their production. At one point, the price predictions are too high and very low
prices result because the agents choose action curves with high power outputs.
This behavior and the fact that the agents choose a random set of actions results
in the high price volatility. In contrast to the test simulation scenario explained
in chapter 5, a price convergence is reached over simulation time. A reason for
the convergence could be that the agents can use different strategies. The inelastic
bidders can provide a constant power output while the elastic bidders are ramping
72
200
Simulated base load price
Observed EEX base load price
180
Spot price [e /MWh] 160
140
120
100
80
60
40
20
0
0 50 100 150 200 250 300 350 400
Day
Figure 6.1: Base load prices reference scenario, training run
their production to make profits during the high load hours. In the test scenario
with a constant load, this strategy splitting was not possible.
The volatility of the simulated prices decreases throughout the training run since
the probability of choosing a random action decreases and the price predictor is
able to predict more accurate spot prices. The estimation error shown in Figure 6.2
verifies that in contrast to the estimation error in the test scenario (chapter 5), the
price prediction accuracy increases throughout simulation time. It can therefore be
concluded that changing environmental inputs and loads leads to a higher predicting
accuracy by the agents.
The estimation error still contains information about market-power which is

filtered out by each agent individually. If no agent can exercise market-power and
the price predictor is correctly specified then the estimation error should be pure
white noise.
Figure 6.3 shows the power outputs for the different generators. At the beginning
of the simulation the inelastic bidders are reducing their power output due to low
predicted spot prices. Since the observed spot prices are very high, the gas agent is
producing at almost full load. As the price predictions become more accurate and
the probability of choosing a random action decreases, the inelastic bidders raise
and stabilize their power output. The storage agent benefits from the high price
volatility; his output heavily oscillates around zero indicating buying and selling
electricity.
The profits for the agents are shown in Figure 6.4. The nuclear agent has high
losses at the beginning of the simulation because of his ramping costs. His profits
at the end of the simulation are the highest ones his price predictions become more
accurate and the probability of choosing a random action is decreased. The high
profits result from the lowest production costs. However, even at the end of the
training run, the nuclear agent has some days with losses. This is due to the
73
400
200
Est. error [e /MWh] 0
−200
−400
−600
−800
−1000
−1200
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Hour
Figure 6.2: Estimation error reference scenario, training run
fact that he still chooses a random action with a little probability. If this random
action forces him to ramp the production heavily, a loss results. In contrast to the
nuclear agent, the gas agent benefits from the high spot prices at the beginning
of the simulation and with the high power output he can generate high profits.
These profit opportunities fade away once the nuclear, lignite and hard coal agent
increase their power outputs. In a next step, the pav values for the supply curves
are analyzed in Figure 6.5.
As in the test scenario, the oil, storage and gas agent have only minor market-
power described by their pav values. The nuclear, lignite and hard coal agent on
the other hand can exercise major market-power. Some pav values are as high as
200 e /MWh and as low as -80 e /MWh. The pav values of 200 e /MWh for the
nuclear, lignite and hard coal agent occur for action number 1, which is specifies
the action curve with 0% power output for all prices. If one inelastic bidder shuts
down production, the aggregated supply is not sufficient to serve the load and the
spot price jumps to 200 e /MWh.
The market-power indicated by the pav values is not describing the reality cor-
rectly. The nuclear power plants for instance are owned by several utilities. There-
fore, there is not one supply curve for all the nuclear power plants in Germany. It
is highly unlikely in reality that all utilities decide to shut down the production of
their nuclear power plants at the same time. Furthermore, the electricity output of
the nuclear power plants is often sold on forward and future markets and not on
the spot market. The spot market is only used to balance shorter time variations.
Simulated [e /MWh] Observed EEX [e /MWh]

Mean base load price 70.85 37.98
Mean peak load price 83.61 48.15
Price standard deviation 60.60 30.35
Table 6.2: Simulation summary, training run
74
20000
Nuclear
Lignite
Hard coal
Gas
15000 Oil
Storage
Wind
Power Output [MW]
10000
5000
−5000
0 50 100 150 200 250 300 350 400
Day
Figure 6.3: Mean power outputs reference scenario, training run
The simulation summary in Table 6.2 confirms the higher standard deviation
and the higher price level of the simulation compared to the observed EEX prices.
In the next section, it is analyzed if the increased competition among the more
experienced agents leads to a lower price level.
75
6
x 10
2.5
Nuclear
Lignite
2 Hard coal
Gas
Oil
1.5 Storage
Wind
1
Profit [e ]
0.5
−0.5
−1
−1.5
−2
0 50 100 150 200 250 300 350 400
Day
Figure 6.4: Mean daily profits reference scenario, training run
200
Nuclear
Lignite
Hard coal
Gas
150 Oil
Storage
100
pav value [e ]
50
−50
−100
0 20 40 60 80 100 120 140 160
Action nr.
Figure 6.5: pav values reference scenario, training run
76
6.2.2 The second run
After the training run, a second simulation trial for the same year is performed. The
price predictor and price adjuster have past data values and from the beginning of
the simulation the agents are able to predict reasonable spot prices and are therefore
not using random actions.
180
Simulated base load price
Observed EEX base load price
160
140
Spot price [e /MWh]
120
100
80
60
40
20
0
0 50 100 150 200 250 300 350 400
Day
Figure 6.6: Base load prices reference scenario, second run
Figure 6.6 shows the simulated and observed EEX base load prices. The two
price series coincide significantly better compared to the training run. In contrast
to the test scenario, the prices of the simulation do not show a regime switching
behavior in the second run. Rather, they show the same pattern than the observed
EEX prices. It can therefore be concluded that the observed EEX spot prices are
based on fundamental factors such as environmental conditions. It is important to
mention that the EEX spot market is with physical delivery, which means that the
buyers and sellers exchange electricity as specified by their bids. This results that
there is less speculation in the spot market. Future markets however can be settled
with financial settlement, which attracts traders that hold positions for speculation
reasons.
Interestingly, there are some high EEX prices at the end of the year 2007 of more
than 200 e /MWh. The simulated prices do not capture these values. There is no
unusual high demand during that time or few wind or very unusual temperature
behavior. The reason for the high prices during that time could be one of the
following:
• Power plant outages in Germany during the end of the year 2007.
1
• The future market for oil and other commodities was in contango at the
1 Contango is a term used in the futures market to describe a situation in which the price of a
commodity for future delivery is higher than the spot price or a far future delivery price higher
than a nearer future delivery [51]
77
end of 2007. Traders may predicted higher electricity prices since they are
correlated to oil prices and thereby pushed electricity prices out of their fun-
damental values.
• The high prices might be caused by cross-border trade. One or more sur-
rounding country of Germany might have suffered from power plant outages
or unusual exogenous factors.
180
Simulated base load prices
Observed EEX base load prices
160
140
Spot price [e /MWh]
120
100
80
60
40
20
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Figure 6.7: Cumulative price distributions reference scenario, second run
The cumulative price distribution for the simulated and observed EEX prices
and the year 2007 is plotted in Figure 6.7. The figure shows that there are more high
price outcomes observed at the EEX than simulated. The simulation in contrast
shows higher price outcomes in lower levels; In the middle price range, the simulated
and observed EEX prices coincide.
An analysis at a micro level (single agent level) is taken where Figure 6.8 shows
the power outputs for the agents. The power outputs for the inelastic agents are
often in a defined range and there is less volatility in power production compared
to the training run (see Figure 6.8). The lignite, hard coal and nuclear agent are
producing on a high and steady level since they have lower production costs than
the gas and oil agent. The spot price on a intra-day level is sometimes below
and sometimes above the marginal costs of the gas agent; therefore, he ramps his
production heavily to capture these intra-day price variations.
The evolution of the profits over time provides insights into the learning behavior
of the agents. Figure 6.9 shows the profits for the second simulation run and the
year 2007. Several observations can be made by comparing the profits of the second
run to the profits from the training run:
• The nuclear, lignite and hard coal agent always make profits and never losses.
• The nuclear agent generates the highest profits because he has the lowest
marginal production costs and learns to avoid the expensive ramping costs.
78
20000
Nuclear
Lignite
Hard coal
Gas
15000 Oil
Storage
Wind
Power output [MW]
10000
5000
−5000
0 50 100 150 200 250 300 350 400
Day
Figure 6.8: Mean power outputs reference scenario, second run
• The volatility of the profits is much lower compared to the training run. This
is mainly caused by a lower spot price volatility compared to the training run
and lower ramping rates of the agents.
• The gas agent generates no overall profit.
The gas agent in general profits from a high price level. If prices do not reach a
level above his marginal costs, he cannot generate a profit. As seen in the simulation,
there are only a few high price outcomes. The gas agent should not incur losses
since he should shut down production if the predicted spot price is lower than his
marginal costs. The losses can have two causes: Either, his pav values are biased
or his price predictions are wrong. It can be assumed that the pav values are not
heavily biased since the pav values of the gas agent are small for all action curves
as seen in Figure 6.5. Hence, the losses are a result of wrong price predictions.
Therefore, the estimation error is analyzed.
Figure 6.10 shows histograms of the price estimation error for the training and
second run. The distribution of the estimation error is narrower for the second run
which implies that there are more observations around 0 e /MWh. The histograms
of both runs do not resemble a normal distribution: The distributions are skewed
and show a high excess kurtosis. Partially, this is due to the fact that there is
information about market-power in the residuals.
For the gas agent, the estimation error in figure 6.10 does not change much
after adjusting for market-power since the gas agent has low absolute pav values.
The marginal costs for the gas agent are high. The first of January 2007 and at
80% load for instance the gas agent has marginal production costs of 60.93e /MWh
according to the cost model. The following cases can occure: If the price estimate
is above this 60.93e /MWh, the gas agent is likely to produce at a high power
output. The estimation error can cause that the price estimate is too high and the
actual spot price is lower than 60.93e /MWh. In this case losses for the gas agent
79
5
x 10
10
Nuclear
Lignite
Hard coal
Gas
Oil
Storage
Wind
5
Profit [e ]
−5
0 50 100 150 200 250 300 350 400
Day
Figure 6.9: Mean daily profits reference scenario, second run
occur. The losses can be dramatic since the excess kurtosis in the estimation error
suggest that some prediction errors are very far of from 0 e /MWh. On the other
hand, if the price estimation is below 60.93 e /MWh, the gas agent shuts down his
production even if the price prediction is wrong and a very high spot price occures.
The probability to attain a very high profit is therefore low whereas the probability
for a high loss is high. This asymmetric probability results in the fact that the gas
agent realizes a loss overall.

Table 6.3: Simulation summary, second run
Table 6.3 visualizes the simulation summary for the second run and confirms
the lower simulated base and peak load prices than observed EEX prices. The
price standard deviation for the simulation is considerably lower than the observed
standard deviation. This is due to some very high observed EEX price outcomes
which heavily inflate the price standard deviation.
Throughout the second simulation trial the regression coefficients of the price
predictor are updated. This further increases the prediction accuracy of the price
predictor. Another simulation trial is performed to analyze the effect of a longer
learning time. The simulated prices should be higher than the observed EEX prices
with experienced agents since the agents in the model have more market-power than
in reality. The higher market-power is due to a higher production aggregation than
in reality.
80
500
Second run
Training run
450
400
350
Observations
300
250
200
150
100
50
0
−200 −150 −100 −50 0 50 100 150 200
Estimation error [e /MWh]
Figure 6.10: Histograms of the estimation errors reference scenario
6.2.3 The third run
The simulated base load price for the second and third simulation run is shown in
Figure 6.11. The agents further increase the price level, but not by a significant
amount.
By comparing the simulated base load prices to the observed EEX prices in
Figure 6.12 it can be stated that the observed EEX prices still show price outcomes
on a high level. But in the middle price range, the agents in the model are able to
push the price level above the observed EEX prices.
The simulation summary in Table 6.4 confirms the observation that the simu-
lated base and peak load prices are higher in the third run compared to the second
run. The mean base load price is in the same order of magnitude as the observed
EEX prices while the mean simulated peak load price is still considerably lower.
This again is a consequence of some very high observed EEX prices in 2007, which
inflate the mean peak load price. The simulated price standard deviation decreased
from the second to the third simulation trail.

Table 6.4: Simulation summary, third run
A closer look is taken at the price predictor and output quantities to analyze how
the agents are able to increase the price level from the second to third simulation
trial.
The price predictor shows slightly more observations around 0 e /MWh in the
81
180
Simulated base load price second run
Simulated base load price third run
160
140
Spot price [e /MWh]
120
100
80
60
40
20
0
0 50 100 150 200 250 300 350 400
Day
Figure 6.11: Base load prices reference scenario, second and third run
third run than in the second run. The pav values do not change from the second
to the third simulation trial, it can therefore be concluded that the price predicting
capability is increased from the second to the third simulation trial. This increase
in prediction accuracy results from more accurate regression coefficients of the price
predictor.
In the third run the regression coefficients for the oil price, wind forecast, heating
degree day and the historical price fraction are statistically significant on a 5%
level. The cooling degree day is significant on a 10% level of significance while the
regression coefficients for the hour, Saturdays and Sundays are insignificant on a
10% significance level. The root mean squared error between the simulated and
predicted spot prices is 11.7 e /MWh in the model, which is a low value. The root
mean squared error of the price predictions in the model is in the same order of
magnitude as the price predictor for historical EEX prices. Therefore, it can be
concluded that the multi-factor model used by the price predictor is also able to
forecast spot prices which are internally generated by the agent-based model.
In Figure 6.13, the power outputs for the different agents are shown. Interest-
ingly, the nuclear, lignite and hard coal agent do not produce at a constant power
output. Instead, their output shows a regime switching behavior by producing on
different output levels. This behavior results in higher price outcomes. The mecha-
nism for choosing different output levels is further analyzed and discussed in section
6.4, where it is shown that these regime switches are related to Edgeworth price
cycles which emerge as the agents experience their impact on spot market prices.
In the next section, the simulation results of the three simulation trials are
discussed and an additional fourth simulation trial is included.
82
180
Simulated base load prices
Observed EEX base load prices
160
140
Spot price [e /MWh]
120
100
80
60
40
20
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Figure 6.12: Cumulative price distributions reference scenario, third run
16000
Nuclear
Lignite
14000 Hard coal
Gas
Oil
12000
Storage
Wind
Power output [MW]
10000
8000
6000
4000
2000
−2000
0 50 100 150 200 250 300 350 400
Day
Figure 6.13: Mean power outputs reference scenario, third run
83
6.2.4 Conclusion of the simulation
In this section, a close look is taken at the evolution of profits for the individual
agents and simulation runs. The yearly profits for all agents and every simulation
trial is shown in Table 6.5.
sim 1 [109 e ] sim 2 [109 e ] sim 3 [109 e ] sim 4 [109 e ]

Nuclear 3.05 2.94 3.27 3.48
Lignite 0.59 1.04 1.40 1.82
Hard coal 0.45 0.99 1.34 1.51
Gas -0.10 -0.82 -0.75 -0.76
Oil -0.42 -0.44 -0.47 -0.37
Storage 0.20 0.12 0.10 0.10
Wind 1.21 1.21 1.21 1.21
Table 6.5: Yearly profits of the simulation trials
The nuclear agent experiences a drop in profits from the first simulation trial to
the second one, but starts to increase his profits thereafter. In the first simulation
trial the simulated spot prices where high. This allowed the nuclear agent to achieve
a high overall profit, even if he suffered losses from ramping during some time
intervals. The hard coal and lignite agent increase their profit from one simulation
trial to the next one even in an environment of decreasing average spot prices. This
is a direct result of a more efficient production scheduling which decreases ramping
costs. More accurate spot price predictions and the availability of accurate pav
values allow for better production scheduling.
In contrast, the gas and storage agent suffer from lower profits from one simu-
lation trial to the next one. The gas and storage agent can be viewed as a spread
option where the gas agent is long electricity and short fuel. He profits from a high
fuel/electricity spread and a high spread volatility. The volatility for the fuel is the
same for the different simulation trials, but the electricity price volatility decreases.
In contrast, the storage agent is long peak and short off-peak hour electricity. He
benefits from a high peak/off-peak spread and high spread volatility. Since the
electricity price volatility decreases, their value expressed in option terms decreases
from one simulation trial to the next one. An efficient production scheduling is
less important to these agents since they have low ramping costs. The wind agent
receives a fixed paid-in tariff, so his profits stays the same from one trial to the next
one.
sim 1 [105 e ] sim 2 [105 e ] sim 3 [105 e ] sim 4 [105 e ]

Nuclear 4.84 1.96 1.80 1.75
Lignite 4.04 1.64 1.54 1.64
Hard coal 3.64 1.55 1.42 1.43
Gas 3.70 1.86 1.21 1.07
Oil 0.70 0.60 0.57 0.55
Storage 0.77 0.53 0.51 0.53
Wind 1.26 1.26 1.26 1.26
Table 6.6: Standard deviations of the yearly profits
The profit standard deviations for the four simulation trials and all agents are
plotted in Table 6.6. The standard deviations indicate the risks of the agents as
discussed in section 4.4.4. The standard deviation is proportional to the profit at
84
risk, a measure for risk often used in the electricity utility industry. On a risk
adjusted basis, the agents are able to increase their profit measured by profits
divided by standard deviation of profits with more experience.
The lower standard deviation in profits is mostly driven by a lower simulated

standard deviation in spot prices. This lower price standard deviation results from
a better price prediction anticipation by the agents, which in turn is a direct result
from better spot price predictions. Table 6.7 shows a summary of the estimation
error made by the price predictor for the four simulation trials.
sim 1 sim 2 sim 3 sim 4

Mean pred. error [e /MWh] -2.36 0.032 -0.59 -0.36
Std pred. error [e /MWh] 38.08 19.78 15.50 13.59
Skewness pred.error [e /MWh3 ] -0.87 -2.74 1.45 2.38
Table 6.7: Summary of the price predictions
The standard deviation of the estimation error is a measure for the ability of
the agents to predict future spot prices. In terms of predicting spot prices, it can
be concluded that there is a learning effect present. The agents learn fast from the
first to the second, but slower from the third to the fourth simulation trial as shown
in Table 6.7. Additionally, the agents learn to better schedule their production and
to bid strategically by restricting power output under certain circumstances. The
learning process is divided into three phases:
• The agents learn their influence on market prices and measure their pav values.
• The agents try to stabilize their power outputs to decrease ramping costs.
• The agents start to restrict their power outputs to drive the spot price to
higher levels and enable higher profits.
After the four simulation runs under reference conditions the model is carried
out to simulate wind scenarios in which the wind energy contribution is increased
and storage scenarios where additional storage capacity is introduced.
6.3 Scenario analysis: Increase in wind generation
In Germany, wind power has benefited from renewable promotion policies. A fixed
feed-in tariff is paid and green certificates are granted to the wind power producers.
This provides low risk to investors in renewable technologies and has promoted the
development of wind energy [52] [53]. The support policies have kept the wind
generation aside from spot and forward markets. They act as an isolated device
and are thereby ignoring market signals. This fact was accepted at the initial
state when the wind power capacity was low since the associated wind integration
costs were low and absorbed by other market participants. The wind integration
costs result from a high variability in wind power production and a low production
predictability [54].
However, the EU Renewables Directive and the Green Package 20:20:20 promote
renewable energies in Europe and target a 20% share in renewable energy sources
85
by 2020 [55]. Wind energy thereby can contribute a major share of this increased
renewable energy demand. Inevitably, increasing the share of wind energy rises the
wind integration costs. Consequently, their impact on electricity markets become of
increasing importance. Electricity market designs need to be assessed and adjusted
to give the right incentive to handle the increase in wind energy integration costs.
6.3.1 Simulation setup
In this section of the thesis, it is investigated how a higher wind energy contribu-
tion effects the outcome in the agent-based model. Three factors are particularly
analyzed: The price predictor, the spot price and the total costs for the consumers.
In the current agent-based model, 21.1 GW of wind power are installed. Three
scenarios are considered: one in which the wind power is doubled, one in which it is
tripled and a nuclear replacement scenario where the nuclear power plants are taken
offline and replaced by wind power generators. In Germany and the year 2007, the
mean power the wind generators were feeding to the grid was around 1.78 GW.
According to EEX, the nuclear power plants provided a mean power of around 15.8
GW the same year. This implies, that the wind generators have to be expanded
by 8.9 times the current installed wind capacity to replace the power output of the
nuclear power plants.
For each of these three scenarios, the simulation is run for three trials, as per-
formed in the reference scenario. At the beginning of the simulation, the agents
are inexperienced. In the first trial, the agents choose random actions with a given
probability in accordance to the reference scenario. The three simulation trials al-
low the agents to adapt and learn in the environment. The outcome after the third
trial are compared.
Table 6.8 shows the mean base and peak load prices as well as price standard
deviation under the different scenarios. The information is misleading since the
wind power bypasses the electricity market. The energy the wind agent produces is
subtracted from the demand and the spot price is thereby lowered. The spot price
does not reflect the true price of electricity. It is more meaningful to analyze the total
costs for the consumers as performed in the next subsection, which incorporates the
feed-in tariff paid to the wind producer.
Ref. scenario Double wind Triple wind Nuclear rep.

[e /MWh] [e /MWh] [e /MWh] [e /MWh]
Mean base load 36.27 33.13 34.12 24.35
Mean peak load 40.50 41.88 40.76 28.52
Price std 16.02 20.99 23.29 27.35
Table 6.8: Summary mean spot prices wind scenarios
Table 6.8 also highlights the increase in price standard deviation the additional
wind introduces. The wind forecast deviates from the actual wind. Therefore, the
price estimation uncertainty increases and the agents are not able to act perfectly
to wind level deviations which increases the price volatility.
86
On an hourly price level, the nuclear replacement scenario shows outcomes in

which the production is not capable of serving the load or the production is higher
than the load. As described, in these situations a redispatching is necessary, which is
not implemented in the agent-based model. In the reference scenario, in 6.3% of the
simulated hours a redispatiching is necessary whereas in the double wind scenario
in 10.6%, in the triple wind scenario in 11.4% and finally in the nuclear replacement
scenario in 32.8% of the simulated hours. The value for the nuclear replacement
scenario is high. Consequently, the well-functioning of the day-ahead spot market
is negatively influenced by an increased wind energy contribution. From a technical
standpoint these price outcomes also indicate a higher instability and a greater risk
of blackouts. A major problem is that the wind energy producing agents do not
act on market signals. They feed in the maximum amount of electricity available
at a fixed price. A more market based approach in paying wind energy producers
as proppsed in [54] could help reduce these unfavorable price outcomes.
Another issue is not analyzed in this thesis, but becomes of importance with a
higher wind energy contribution. The wind energy is produced mainly offshore and
away from populated areas; therefore, the electricity produced has to be transported
over long distances. Congestions in the network within Germany could occur, which
results in different price areas within Germany.
A comparison between the outcome of the model predictive bidding and the
classifier agent-based model in [41] shows that the simulated price patterns under
these wind scenarios are different. The classifier based model does not incorporate
the ability for a multiple period production scheduling. The price volatility is there-
fore higher in the classifier based models. Additionally, unfavorable price outcomes
where a redespatching is necessary do not arise as often in model predictive bidding
model. A multi period production planning is therefore crucial to handle a higher
wind energy contribution.
6.3.3 Total cost for consumers
The total cost T C for the consumers are calculated according to
8760
XX N

TC = pi∈wind,t
/ λt,c · t + pi∈wind,t F T · t (6.1)
t=1 i=1
where F T represents the fixed feed-in tariff paid to the wind producers which
amounts to 75 e /MWh and N the number of agents. The total costs for the
consumers are 11.1 · 109 e in the reference, 11.1 · 109 e in the double wind, 11.9 · 109
e in the triple wind and finally 15.8 · 109 e in the nuclear replacement scenario.
The high wind energy contribution in the nuclear replacement scenario consid-
erably increases the total cost for the consumers. Surprisingly, the double and triple
wind scenarios do not significantly increase the total costs for the consumers, al-
though the fixed feed-in tariff is much higher than the average spot price level and
wind power replaces conventional produced electricity. The reason that the total
costs are not significantly higher is that a higher wind energy contribution decreases
the average spot price level. This effect offsets the higher price paid to the wind
generator.
87
The higher wind energy contribution in the wind scenarios do effect the profits
of the agent. In the next section, the profits and power outputs of the agents are
analyzed.
6.3.4 Power outputs and profits
The increase in wind energy increases the uncertainty in the demand, since the
wind energy is subtracted from the demand. In this subsection, it is analyzed
how the agent’s power outputs behave in an environment with increased demand
uncertainty. Additionally, the effect of an increased production ramping demand
on the agents’s profitability is assessed.
4
x 10
5
Lignite
Hard coal
Gas
4 Oil
Storage
Wind
Power output [MW]
−1
0 50 100 150 200 250 300 350 400
Day
Figure 6.14: Mean power outputs nuclear replacement scenario
Figure 6.14 shows the mean power outputs for the different agents under the
nuclear replacement scenario. For simplicity, only the nuclear replacement scenario
is analyzed to show the most extreme wind scenario. The ramping demand by the
coal and gas agent is increased compared to the reference scenario. The lignite
and hard coal agent do not have the ability to stabilize their production to reduce
ramping costs. This affects their profitability, the profits are lowered by a higher
wind energy contribution.
If the wind contribution in Figure 6.14 is small, the lignite and hard coal agent
immediately start to stabilize their production until the wind contribution increases
again. Figure 6.14 also shows why so many unfavorable price outcomes occur under
the nuclear replacement scenario. The range for the wind power infeed is 0 GW
to over 50 GW. There is not enough backup power available if there is no wind
blowing to fully supply the load and during some hours the load is lower than the
power fed in by the wind generators.
88
Ref. scenario Double wind Triple wind Nuclear rep.

[109 e ] [109 e ] [109 e ] [109 e ]
Nuclear 3.27 2.80 2.56 0
Lignite 1.40 0.86 0.71 0.61
Hard coal 1.34 0.75 0.71 0.56
Gas -0.75 -0.78 -0.62 -0.42
Oil -0.46 -0.42 -0.42 -0.44
Storage 0.10 0.13 0.15 0.18
Wind 1.21 2.42 3.63 10.77
Total 6.11 5.76 6.72 11.26
Table 6.9: Summary profits wind scenarios
Table 6.9 shows the profits of each agent and wind scenario. The profits of the
nuclear, lignite and hard coal agent are highly reduced by a higher wind energy
contribution since they cannot produce on an efficient and steady level. The gas
and oil agent profit from a higher price volatility and are therefore able to reduce
their losses. The storage agent also benefits from the higher price volatility and is
able to considerably increase his profits.
Furthermore, the volatility of the profits is higher in the nuclear replacement

scenario compared to the reference scenario. The profit at risk is thereby increased,
which indicates a higher risk level under the wind scenarios compared to the refer-
ence scenario.
It can be concluded that from a profit perspective, utilities with a high nuclear or
coal portfolio are opposing a higher wind energy contribution because it erodes the
profitability and increases the risk of their portfolio. In contrast, utilities with gas,
oil or storage portfolios are on one hand favoring a higher wind energy contribution
because it increases their profitability, but on the other hand, they are opposing it
because it increases the risk.
6.3.5 Price predictor and market-power
Two effects related to a higher wind energy contribution and market-power are
likely: The price predictor is affected by a higher wind energy contribution because
the wind forecast error induces an additional error to the price predictions. The
higher uncertainty in price predictions lowers as a consequence the market-power of
the agents measured by the pav values. It is likely that the backup power needed
in a higher wind energy environment provides an additional source of gaming. The
agents learn to exploit this gaming opportunity which results in higher market-
power.
The histograms of the price prediction errors in Figure 6.15 show an unexpected
outcome. The histogram for the double wind scenario shows more observations
around 0 e /MWh than the reference scenario. For the nuclear replacement and the
triple wind scenario, the histograms of the estimation error show fewer observations
around 0 e /MWh.
The price predictor under the reference scenario has a residual mean squared er-
ror (MSE) of 136.85 e /MWh2 . The double wind scenario has a MSE of 186.4e /MWh2 ,
89
600
Reference scenario
Double wind
Triple wind
500 Nuclear replacement
400
Observations
300
200
100
0
−200 −150 −100 −50 0 50 100 150 200
Figure 6.15: Histograms estimation error for the wind scenarios
the triple 169.2e /MWh2 and finally, the nuclear replacement 502.9 e /MWh2 .
The price predictor in the double wind scenario has more observations around
0 e /MWh but a higher mean squared error than in the reference scenario. It
results from the fact that the price predictor in the double wind scenario has a
higher sample excess kurtosis 2 than in the reference scenario. The kurtosis for
the reference scenario’s price predictor is equal to 28.54 e /MWh4 (which indicates
also that the price prediction error is far from being normal distributed), 59.11
e /MWh4 for the double and 52.96 e /MWh4 for the triple wind scenario. This
implies that the double wind scenario shows a lot of estimation error observations
around 0 e /MWh and a few very extreme estimation errors. A higher wind energy
contribution therefore results in overall less accurate price predictions.
The pav values in Figure 6.16 suggest that the market-power increases in the
double and triple wind scenario compared to the reference scenario for the lignite
agent. For simplicity, only the pav values for the lignite agent are plotted. It is
assumed that the pav values for the hard coal agent show the same behavior. This
observation proves that the lignite agent exploits the backup power needed in a
higher wind energy environment, which provides an additional source for gaming.
The market-power is reduced again for a very high wind energy contribution as
described in the nuclear replacement scenario. In this scenario, the effect of inaccu-
rate price predictions dominates. As mentioned, the price predictor’s MSE in the
nuclear replacement scenario is around 502.9 e /MWh2 , which indicates a standard
deviation in predictions of around 22.4 e /MWh, which is a high uncertainty.
The pav values for the gas agent change only minor between the wind scenarios
as shown in Figure 6.17. There are two explanations: First, the gas agent has a
lower installed capacity, as a result, he is in general not able to alter the spot price
2 Kurtosis is a measure of the peakedness of a probability distribution. Higher kurtosis means
more of the variance is the result of infrequent extreme deviations, as opposed to frequent modestly
sized deviations [56]
90
30
Reference scenario
Double wind
20 Triple wind
Nuclear replacement
10
0
pav value [e ]
−10
−20
−30
−40
−50
−60
0 50 100 150 200 250
Action nr.
Figure 6.16: PAV values for the lignite agent and different wind scenarios
as much as the lignite agent. Second, the gas agent does not depend on accurate
price predictions the way the coal agent does. The gas agent has virtually zero
ramping costs in the model. He is therefore mostly ramping his production to full
load or zero load. All states in between are not chosen as often. The gas agent
requires from the price predictor only the information if he needs to shut down
production or ramp to full load. Therefore, the lower accuracy in price predictions
in the nuclear replacement scenario do not affect the gas agent’s pav values.
91
30
Reference scenario
Double wind scenario
25
Triple wind scenario
Nuclear replacement
20
15
pav value [e ]
10
−5
−10
−15
−20
0 50 100 150 200 250
Action nr.
Figure 6.17: PAV values for the gas agent and different wind scenarios
6.3.6 Conclusion of the wind scenarios
A higher wind energy contribution lowers the mean spot price level but increases
the total costs for consumers due to the fact that the wind power bypasses the spot
market. It also increases the spot price volatility which results in higher risk for
the power generating companies. The profits of the inelastic bidders are lowered
but their market-power can increase. In contrast, the elastic bidders can increase
their profits, but are not able to gain market-power with a higher wind energy
contribution. From a technical perspective, the model shows that a redispatching
is more often necessary. It can be concluded that the current market design does
not set the right incentives. It is often cheaper to ramp a wind power plant than to
ramp a gas, oil or even coal fired power plant. Therefore, the wind producers should
react on market signals by being paid the current spot price plus a markup instead
of the fixed feed-in tariff as proposed in [54]. This would result in a lower spot price
volatility, higher profits for the inelastic bidders and fewer redispatchings.
6.4 Scenario analysis: Adding storage devices
In the year 2003 in Germany, the pump-storage hydro power plant Goldisthal was
finished. The power plant is located in the Thuringian Highland. It has an installed
capacity of 1’060 MW and is one of the biggest hydro power plants in Germany and
Europe. The lower and upper water reservoirs are artificially created. The upper
reservoir contains around 12 million m3 of water on an area of 55 ha. The storable
amount of electrical energy corresponds to around 8.5 GWh. The turbines can
be operated for 8 hours under full load. The hight difference between the lower
an upper reservoir is around 300 m [57]. More of these artificial types of hydro
power plants could be introduced in Germany to handle the increasing uncertainty
in power production induced by additional renewable sources, mainly wind energy.
92
Additionally, the Vehicle-to-grid (V2G) concept, where plug-in electric vehicles

(PHEV) communicate with the power grid to sell or throttle their charging rate
on demand could contribute to the storage capacity in the future. For additional
information regarding the integration of PHEVs into energy networks the reader is
referred to [58] [59] [60] [61].
In this section, the effect of a storage device with a high power output and a low
electrical energy storage such as the pump-storage hydro power plant Goldisthal on
spot prices is assessed. For this simulation, it is assumed that an additional 5 GW
of this type of power plant is installed in Germany. With the existing storage power
of 2’210 MW in Germany the maximum power of the storage agent is increased
to 7’210 MW. Further, it is assumed that the storage agent can store a maximum
of 57.7 GWh. This implies that the storage agent can provide the full maximum
power output for 8 hours if the reservoir is fully filled. This simulation is referred
to as high power scenario later in the thesis.
In a second scenario, the effect of a storage device with a high electric energy
content but a lower power output is assessed. The PHEVs connected to the grid can
be viewed as this type of storage device. The rate by which the PHEVs are charged
and discharged is limited, but as a substantial amount of PHEVs is connected, the
electrical energy content is high. It is assumed that the storage agent increased the
installed power output by 2 GW to a total of 4.21 GW. The storable energy content
is increased to 600’000 MWh. This would theoretically imply that the storage
agent can deliver the full power output for 143 hours if the storage is fully charged.
The DP optimization is set that the storage agent tries a bidding strategy which
maximizes his profit but at the same time has a filling level which corresponds to
50% of total available storage content after the next front 3 days. It is therefore
highly unlikely that the storage agent ever reaches a state that the reservoir is totally
empty or full throughout the simulation. This scenario is a strong simplification
since the number of PHEVs changes throughout the day which results in a varialbe
storage capacity and power output. Furthermore, the PHEVs are an additional
load since they have to be charged to be able to drive with electricity. Nevertheless,
the simulation can give insights into the effect of additional storage capacity with
a high storage capacity and low power output in general. The simulation setting is
refered as high energy scenario later in the thesis
Compared to [41], the storage agent does not use a simple price depending
charging/discharging scheme. Rather, it uses the model predictive bidding algo-
rithm introduced in this thesis. The storage agent in [41] led to a price lock-in
effect at the switching price between charging and discharging. It is analyzed if the
more advanced learning algorithm by the storage agent is able to avoid this effect.
Additional storage capacity is technically not an additional generation capacity,

it is only a device to buy electricity when prices are low and sell when prices are
high. Lower prices occur mainly during off-peak hours, when the demand is low and
higher prices during peak hours. A storage device would therefore buy energy during
off-peak hours, thereby pushing the off-peak hour prices up and pulling peak hour
prices down by selling during these hours. The peak/off-peak price spread should
narrow while the mean base load price should stay in the same order of magnitude.
There is a possibility that the mean base load price might decrease since additional
93
storage devices allow to use the existing generation capacity more efficiently and
reduce ramping inefficiencies. Additionally, by narrowing the peak/off-peak spread,
the additional storage capacity should lower the price volatility. This lower volatility
reduces the risk for the power producers and consumers. It also lowers the cost for
sellers of electricity to hedge against low electricity prices and for buyers to hedge
against high prices. It furthermore lowers the margins for future contracts and
therefore reduces the capital requirements for electricity buyers and sellers.
Additionally, the storage devices can help to increase the security of supply by
providing electricity in high demand hours and by providing ancillary services since
pump-storage power plants have a high ramping capacity and a fast response time.
180
Reference scenario
High power
160 High energy
140
Spot price [e /MWh]
120
100
80
60
40
20
0
0 50 100 150 200 250 300 350 400
Day
Figure 6.18: Base load prices storage and reference scenarios
Figure 6.18 shows the simulated base load prices for the high power scenario, the
high energy scenario as well as the reference scenario. The base load prices after
three simulation trials are compared. In the first year, the agents use a random
set of actions with a given probability and adapt their pav value. In the following
simulation trials, they only adjust their regression coefficients.
Ref. scenario High power High energy

[e /MWh] [e /MWh] [e /MWh]
Mean base load 36.27 34.28 36.72
Mean peak load 40.50 38.22 39.95
Price std 16.02 16.85 14.74
Table 6.10: Summary spot price storage scenarios
According to Figure 6.18 and Table 6.10, the following observations can be
stated:
• The price level of the two storage scenarios is in the same order of magnitude
as the reference scenario.
• The price volatility only decreases for the high energy scenario.
94
100
High energy
90
Spot price [e /MWh] 80
70
60
50
40
30
20
10
0
0 50 100 150 200 250 300 350 400
Day
Figure 6.19: Base load price high energy scenario
• The peak/off-peak spread is not considerably decreased with the additional

storage capacity.
• The high energy scenario shows a cyclical price pattern with a period time of
around two months.
• The price lock-in effect described in [41] is not observable in this model.
The predicted outcome with a considerable lower price standard deviation under
an increased storage contribution is not observable. Figure 6.19 shows the simulated
base load price for the high energy scenario, it demonstrates the cyclical price
pattern. The prices increase steadily for about 2 month and drop sharply thereafter.
Other authors have observed similar price pattern. In [62], this price pattern is
observed in an environment with capacity constraints on a transmission line, a
similar effect has also been observed in [63]. In [62] it is stated that this type of
cyclic behavior is related to Edgeworth Cycles. Edgeworth analyzed static Bertrand
price equilibriums and showed that “when firms face capacity constraints, there
exists generally a cyclic price behavior rather than an equilibrium point” [62]. Two
phases can be distinguished: a price war phase and a relenting phase. In the price
war phase the market participants undercut each other to increase market share. At
some point the war becomes too costly, the second phase starts. Some participants
start to increase their prices and the other ones follow. Thereafter, a new cycle
of price war starts. In this thesis, the constraint on the capacity of the strategic
agents induces the dynamic, as described by Edgeworth. In the following, some
characteristics of Edgeworth price cycles are outlined:
• Edgeworth price cycles emerge if the good is homogeneous and the customer
are extremely price sensitive. If a producer undercuts one other, he will cap-
ture all or a very large portion of the market [64]. This is the case in the
electricity market. The good electricity is homogeneous and if a vendor sup-
95
plies it a little bit cheaper than its competitors then he will be able to sell his
supplied portion.
• Smaller competitors have a greater incentive to initiate cutting, larger com-

petitors are generally the initiator of price restorations [64]. According to this
definition, the gas, oil or storage agent should start undercutting while the
nuclear, lignite or hard coal agent start to restore prices.
In section 6.4.3, it is analyzed who initiates price cuttings and who the price
restaurations. Furthermore, it is analyzed why this phenomena only occurs with a
higher storage capacity, especially if the storage capacity has a high storage capacity.
From a spot price perspective, the additional storage capacity does not help
bringing the spot price to more competitive levels or considerably decrease the
peak/off-peak spread and price volatility. The reason is that gaming activity is
increased and Edgeworth price cycles occur. These cycles inflate the spot price
volatility and mean base and peak load price level.
In this section, only the power outputs and profits for the high energy scenario
are analyzed. The reason is that the Edgeworth price cycles are more distinct in
this scenario. The high power scenario shows a spot price behavior in between the
reference and high energy scenario.
20000
Nuclear
Lignite
Hard coal
Gas
15000 Oil
Storage
Wind
Power output [MW]
10000
5000
−5000
0 50 100 150 200 250 300 350 400
Day
Figure 6.20: Power outputs high energy scenario
Figure 6.20 shows the power outputs for the different agents and the high energy
scenario. There are two regimes: In one, the gas agent has a high volatility in power
output and the nuclear, storage, lignite and hard coal agent produce on a constant
level. In the other regime, the gas agent reduces his output and the storage, nuclear,
96
lignite and hard coal agent are ramping their production heavily. These regimes
are switching one after each other, but no regime is stable for a long time.
Figure 6.21 visualizes the spot price on the right axis and the power output of
the nuclear, gas and storage agent on the other axis to assess how the power output
behavior of these agents affect the spot price. For simplicity, the power outputs for
the lignite and hard coal agent are not plotted because it follows the power output
behavior of the nuclear agent.
4
x 10
2 100
Nuclear
Gas
1.5 Storage
Base load price
1
Spot price [e /MWh]

Power output [MW]
0.5
0 50
−0.5
−1
−1.5
−2 0
0 50 100 150 200 250 300 350 400
Day
Figure 6.21: Power outputs and base load price high energy scenario
Figure 6.21 shows that the price rises during times when the nuclear agent has
a constant power output. The nuclear agent stabilizes his power output at a level
where he can produce efficiently. The storage and gas agent cause the increase in
spot prices by selling their electricity at higher prices. At a certain spot price level
the nuclear as well as the lignite and hard coal agent are trying to undercut each
other to increase their market-share and as a consequence their expected discounted
profits. The undercuttings result in a sharp spot price drop and an increased spot
price volatility. The spot price reaches a low level at which the gas agent shuts down
his production. The storage agent starts charging at this low spot price, but is not
able to shift the demand curve up enough to restore the previous spot price level.
With the high spot price volatility the nuclear agent incurs ramping losses. He tries
to stabilize the power output again. Once he has stabilized his power output, the
cycle starts again in which the gas and storage agent start to sell their electricity
at higher price levels.
During times when the spot price steadily increases, the nuclear, lignite and
hard coal agent increase their profits as seen in Figure 6.22. The reason is that their
power output is constant, but the spot price rises which results in a higher revenue
with the same costs. The raising spot prices are caused by collusion among the
agents. Collusion on the other hand is the result of a higher gaming activity among
the agents. This gaming activity is increased if the electricity storage capacity is
raised. The high price levels are not stable though, as mentioned, they trigger the
undercuttings which results in Edgeworth price cycles.
97
5
x 10
8
Nuclear
Lignite
7
Hard coal
Gas
6 Oil
Storage
5 Wind
4
Profit [e ]
−1
−2
0 50 100 150 200 250 300 350 400
Day
Figure 6.22: Daily profit, high energy scenario
The higher gaming activity among the agents in the high energy scenario is a
direct result of better spot price predictions. The storage agent buys electricity if
prices are low and sells them when they are high. This makes the spot prices more
predictable more of the time. In the reference scenario, there is a regime switching
behavior observable as described in section 6.2.3. There, the nuclear agent has times
with a constant power output and times with a high ramping behavior. It can be
assumed that in the reference simulation a very mild form of Edgeworth price cycles
is present. The gaming activity and the degree of collusion is lower in the reference
scenario because the spot prices are less predictable.
Reference scenario [109 e ] High power [109 e ] High energy [109 e ]

Nuclear 3.27 3.11 3.26
Lignite 1.40 1.07 1.52
Hard coal 1.34 0.91 1.49
Gas -0.75 -0.77 -0.64
Oil -0.46 -0.39 -0.38
Storage 0.10 0.05 0.13
Wind 1.21 1.21 1.21
Total 6.11 5.20 6.58
Table 6.11: Summary profits storage scenarios
Table 6.11 summarizes the profits of the agents in the two storage scenarios
and the reference scenario. The power generating companies generate the highest
profits in the high energy scenario. Therefore, there is an incentive by the power
generating companies to promote storage devices with a high storage capacity.
98
100
Estimation error
Spot price
80
Spot price/est. error [e /MWh]
60
40
20
−20
0 50 100 150 200 250 300 350 400

Day
Figure 6.23: Price estimation error, high energy scenario
The mean daily price estimation error and base load price for the high energy
scenario is plotted in figure 6.23. The price predictions steadily decrease during
the time intervals of increasing spot prices. It indicates that the price predictions
for these time intervals are too low and the increase in spot prices is not driven by
exogenous factors, but by collusion instead. The decrease in estimation error is less
than the increase in spot price. This is due to the fact that the regression coefficients
are updated. As a result, the price predictor will eventually predict higher prices
which triggers the nuclear, lignite and hard coal agent to undercut each other. At
this point, the spot price drops and the price prediction error is highly positive,
indicating overestimated price predictions. These very low spot prices induce the
regression coefficients to lower the price predictions, the estimation error decreases
rapidly. The power output by the nuclear, lignite and hard coal agent is reduced
again.
The time interval for raising prices in the Edgeworth price cycles therefore de-
pends on how fast the regression coefficients are updated. The agents use the
trailing 3 month data for calculating the regression coefficients, which explains that
the price rising intervals are around 2 month.
The histograms of the estimation errors in Figure 6.24 indicates that the refer-
ence scenario has more observations around 0 e /MWh than the high power and
high energy scenarios. The kurtosis of the high power scenario’s estimation error is
57.4 e /MWh4 , which implies that some observations are far away from 0 e /MWh.
The high energy scenario shows a much smaller kurtosis of only 5.62 e /MWh4 .
The price predictor in the high energy scenario therefore only rarely predicts very
wrong prices, but it does not predict as many prices around 0 e /MWh because
during periods of raising spot prices the price predictor underestimates the spot
prices and overestimates them in times of rapid falling spot prices. But the fact
99
600
Reference scenario
High power
High energy
500
400
Observations
300
200
100
0
−200 −150 −100 −50 0 50 100 150 200
Figure 6.24: Histograms estimation error for the storage scenarios
that spot price predictions are wrong only rarely allows the agents to collude.
The pav values for the gas and storage agent and the reference, high power and
high energy scenarios are shown in Figure 6.25. The nuclear, lignite and hard coal
agent’s pav values are neglected for simplicity.
For some action curves, the storage and gas agent have higher market-power
measured by the absolute pav value. The charging/discharging capacity of the
storage agent is increased in the storage scenarios; consequently, his market-power
increases. The power output of the gas agent is not increased in the storage sce-
narios, but his market-power increases nevertheless. The reason is collusion: By
colluding the agents can influence the spot prices to a higher degree even if their
installed capacity is not increased.
6.4.4 Conclusion of the storage scenarios
Increasing the power output and storable amount of electricity by the storage agent
does not result in the predicted outcome with a lower spot price volatility and a
decrease in the peak/off-peak spread. Instead, it leads to higher gaming activity
and collusion among the agents which destabilizes the spot prices and results in
price cycles. The degree of collusion and gaming is related to the predictability
of future spot prices. The storage agent buys electricity when prices are low and
sells when prices are high. This leads to a higher spot price predictability with
few extreme spot price outcomes. The collusion in the high energy and high power
scenarios is therefore mainly caused by more accurate spot price predictions.
The degree of collusion is higher in the high energy scenario than in the high
power scenario. The reason is that in the high power scenario the storage agent has
a constraint in the length he can supply or buy electricity. This constraint can lead
100
25
Gas reference
Storage reference
Gas high power
20
Storage high power
Gas high energy
Storage high energy
15
pav value [e ]
10
−5
−10
0 50 100 150 200 250
Action nr.
Figure 6.25: Prediction adjustment values
to the fact that spot price predictions become less predictable which reduces the
gaming activity.
The market-power of the gas agent increases if the storage agent has a higher
power and storage capacity. The reason for this emergence is again collusion. By
colluding the agents can exercise more market-power even if their installed capacity
is not increased.
6.5 Scenario analysis: Adding a PHEV cluster
In this scenario, it is analyzed what effect a more realistic modeling approach of

clustered PHEVs has on spot prices. The available charging/discharging capacity
of a PHEV cluster depends on the amount of PHEVs connected to the grid and
the state of charge of the PHEVs. This variable charging/discharging capacity
could reduce the market-power of a clustered PHEV agent which could result in the
avoidance of the cyclical spot price behavior observed in section 6.4.
The output of the single PHEV model introduced in [65] can be used to simulate
PHEV clusters where individual cars have different temporal behaviors. In [65], the
single vehicle model utilizes 84 different drive cycles, whereby these drive cycles are
uniformly distributed among the PHEVs in the cluster. The cycles are composed
from the Urban Dynamic Drive Cycle with 7.9 miles sampled, Highway Fuel Econ-
omy Cycle with 10.26 miles sampled, New York City Cycle with 1.18 miles sampled
and Federal Test Procedure 75 miles sampled [65].
Two simulation scenarios with PHEVs are carried out, in one the cluster consists
of 1’000’000 PHEVs (the 1 m PHEV scenario), whereas in a second scenario, the
cluster consists of 8’000’000 PHEVs (the 8 m PHEV scenario). The cluster is
101
0.9
0.8
0.7
PHEV fraction
0.6
0.5
0.4
0.3
0.2
0.1
0
0 5 10 15 20 25
Hour
Figure 6.26: Fraction of PHEVs connected to the grid
modeled as a single PHEV agent whereby the PHEVs in the cluster connected
to the grid are charged with 1.75 kW/PHEV. The average storage capacity per
PHEV is 12 kWh. Figure 6.26 visualizes the fraction of the PHEV cluster which is
connected to the grid throughout a day.
6.5.1 The PHEV cluster model
In contrast to [65], the energy purchases and sales of energy to the market do have a
major impact on the state of charge of the PHEV cluster. As a result, the available
charging/discharging capacities depend on the fraction of PHEVs connected to the
gird, the electricity they use to drive and the historical electricity purchases and
sales over the spot market. As a result, the state of charge (soc) of the PHEV
cluster can be calculated on a recursive basis:
∆qt − drt
soc(t + 1) = soc(t) + (6.2)
Q
where drt describes the amount of electricity needed to drive the PHEVs at time
t, ∆qt the electricity exchanges over the spot market and Q the maximum storage
capacity of the PHEV cluster. The electricity needed to drive the PHEVs follows
the formula
drt = (P0 − Pt ) · υ (6.3)
where P0 is the total number of PHEVs, Pt the number of PHEVs connected to

the grid and υ a constant which specifies the amount of electricity a PHEV needs
to drive. The idea is that the all PHEVs not connected to the grid are used to drive
102
around and use a certain amount of electricity. The amount of Pt can be derived
from the PHEV fraction connected to the grid shown in Figure 6.26.
The available charging/discharging power for the PHEV cluster can then be
calculated according to
charge Pt
∆qt+1 < · P0 · 1.75 kW · (1 − soc) (6.4)
P0
for charging and
discharge Pt
∆qt+1 < · P0 · 1.75 kW · soc (6.5)
P0
for discharging. A constraint of the problem is that 0.2 ≤ soc ≤ 1 which specifies
that the PHEV cluster cannot be discharged below 20% of Q. In the 1 m PHEV
scenario, the PHEV agent has a maximum storage amount of 12’000 MWh and a
maximum charge and discharge capacity of 1’750 MW. In the 8 m PHEV scenario,
the maximum storage capacity is as high as 96’000 MWh and the maximum charge
and discharge capacity 14’000 MW. The terminal value is set such that the PHEV
cluster is trying to be fully charged after three front days.
120
Reference scenario
1 m PHEV
8 m PHEV
100
Spot price [e /MWh]
80
60
40
20
0
0 50 100 150 200 250 300 350 400
Day
Figure 6.27: Base load prices PHEV and reference scenarios
Figure 6.27 shows the spot prices of the reference, the 1 m PHEV and 8 m
PHEV scenarios. The PHEVs can be considered an additional load with a time
depending charging/discharging capacity. The additional load results in a overall
higher price level, but although the price level is higher in the 8 m PHEV scenario,
103
the spot price volatility decreases. Most of the spot market outcomes are between
40 e /MWh and 60 e /MWh, a lock-in effect as in [41] is observable with a high
PHEV cluster. It can be assumed that the PHEV agent has lower market-power
due to the time dependent charging/discharging capacity which in turn does not
lead to the price cycles observed in 6.4.
Ref. scenario 1 m PHEV 8 m PHEV

[e /MWh] [e /MWh] [e /MWh]
Mean base load 36.27 36.53 48.32
Mean peak load 40.50 42.85 50.32
Price std 16.02 22.42 10.12
Table 6.12: Summary PHEV scenarios
Table 6.12 summarizes the simulations. The 1 m PHEV scenario shows that
the mean base and peak load prices do not change significantly compared to the
reference scenario, but the PHEV cluster does increase the spot price volatility.
Interestingly, the spot price volatility decreases in the 8 m PHEV scenario due to a
higher and more steady power generation utilization discussed in 6.5.3. The mean
base load prices rises around 33% in the 8 m PHEV scenario and the peak/off-peak
spread is narrowed.
16000
Nuclear
Lignite
14000 Hard coal
Gas
Oil
12000
Storage
Wind
Power output [MW]
10000
8000
6000
4000
2000
−2000
0 50 100 150 200 250 300 350 400
Day
Figure 6.28: Power outputs nuclear agent reference and PHEV scenarios
Figure 6.28 shows the power outputs of the agents in the 8 m PHEV scenario.
The PHEV cluster allows to use the nuclear, lignite and hard coal power plants very
efficiently. They produce on a high and steady level throughout the year which leads
to the very low spot price volatility. The PHEV agent allows to provide backup
power and stores excess energy if needed. In contrast to the storage scenarios in
6.4, the PHEV agent results in less predictable spot prices due to the time varying
104
charging/discharging constraints, which in turn decreases the gaming activity and

degree of collusion.
Reference scenario [109 e ] 1 m PHEV [109 e ] 8 m PHEV [109 e ]

Nuclear 3.27 3.32 5.98
Lignite 1.40 1.18 3.25
Hard coal 1.34 1.11 3.30
Gas -0.75 -0.62 -0.42
Oil -0.46 -0.42 -0.23
Storage 0.10 0.11 0.06
Wind 1.21 1.21 1.21
PHEV 0 0.01 0.01
Total 6.11 5.88 13.2
Table 6.13: Summary profits PHEV scenarios
Table 6.13 shows the profits of the agents in the reference and PHEV scenarios.
The effect on profits in the 1 m PHEV scenario is minor but in the 8 m PHEV
scenario, the inelastic bidders can dramatically increase their profits. The inelastic
bidders can operate their production on a more efficient level and avoid ramping
costs. Overall, the total profits increase by 116% from the reference scenario to
the 8 m PHEV scenario. It can therefore be concluded that the power generating
companies have a high incentive to promote charging/discharging PHEVs via the
electric grid. The PHEV agent attains a profit of roughly 10 million e in the 1 m
PHEV and 11 million e in the 8 m PHEV scenario. The reason that the profit in
the 8 m PHEV scenario is not higher is due to the much lower spot price volatility.
In the storage scenarios, the higher storage contribution generated higher profits
by exercising market-power, which is not the case in the PHEV scenarios. It is
remarkable that the profit of the PHEV agent is positive, it implies that people
could drive around with electricity and at the end of the year receive a profit if
their PHEVs are charged/discharged by the model predictive bidding algorithm.
The associated profits are 10 e per PHEV if there are 1 m PHEVs and 1.375 e per
PHEV if there are 8 m PHEVs.
105
Chapter 7
Model application: The

four-country model
“The most important single central fact about a free market is that no exchange
takes place unless both parties benefit.”
Milton Friedman
7.1 Simulation settings
The countries Germany, Switzerland, France and Italy provide a very interesting
test case due to their generator portfolios and the cross-border coupling. This test
case is studied in this chapter. The set of the four countries is referred to as C. It
is assumed that the electricity markets between these four countries are integrated.
The agents in all countries send their supply curves to nationwide market places,
where they are collected and aggregated to market supply curves. The scarce cross-
border capacity is then allocated to maximize the total social welfare. The cross-
border allocation method is therefore implicit.
The maximum amount of electricity that can be exchanged between two neigh-
boring countries is specified by the net transfer capacity (NTC values), which are
further discussed in section 7.2.2. The countries have the following agents:
• Germany has a nuclear, coal, gas and wind agent. Hard coal and lignite are
aggregated to one agent. The storage and oil agent are neglected, since their
contribution in power production is minor. The installed capacities of the
agents taken into account are stated in Table 3.1.
• France is represented by a nuclear, coal, gas, hydro and storage agent. The
nuclear agent has the highest installed capacity. The storage agent accounts
for the pump-storage power plants present mostly in the French Alps while
the hydro agent accounts for the river power plants.
• Switzerland is modeled by a nuclear, hydro and storage agent. An adjustment

is made to the installed capacities in Table 3.1 for Switzerland: EIA classifies
106
Chapter 7. Model application: The four-country model
a hydro power plant as pump-storage only if it has a pumping capacity. In

Switzerland, there are several hydro power plants which do not have pumps,
but can store the natural water inflow. To account for this storage capacity,
a fraction of the hydro power plants is reclassified as storage power plants.
• Italy has five agents: gas, oil, coal, hydro and storage. The power production
in Italy depends heavily on fossil fuels and has higher variable production
costs than France and Switzerland.
Only cross-border trades between these four countries are considered in this
simulation, trades with other trading partners outside the four countries are ne-
glected. In the next section, the implicit cross-border capacity allocation method is
discussed.
As discussed in chapter 3, the agents receive the hour, current oil price, wind
forecast for Germany, temperature for all countries, weekend factor and load forecast
per country to make spot price predictions. The year 2007 serves as basis for the
simulation.
The electricity trade between the four countries affects the producer and con-
sumer surpluses. In the model (and in reality), there is only one spot price for
every country. As a result, the consumer surplus rises and producer surplus drops if
the spot price is lower in an environment with cross-border trade compared to one
without trade. On the other hand, the consumer surplus drops and the producer
surplus rises if the spot price is higher in an environment with cross-border trade.
Other price implementations are possible; [2] for instance introduces alternative
paying schemes. In these schemes, the consumers do not necessarily pay the same
price as the generators receive within the same country. The argument for these
schemes is that in a low price country for instance, it is not justified why consumers
have to pay a higher price if electricity trade is introduced. These alternative paying
schemes are not implemented since they are not used in reality.
7.2 Cross-border capacity allocation
There are several approaches in allocating cross-border capacity as discussed in

chapter 1. An implicit approach is taken in this model. There are different allocation
schemes for the implicit method which differ in the optimization routine. One way
of allocating cross-border capacity is by minimizing total generation costs, another
way is to maximize total social welfare. The outcome in cross-border capacity
between these two optimization routines must not be the same.
7.2.1 Total social welfare vs. total variable cost
Figure 7.1 shows the calculation of the total social welfare. It is the sum of the
producer and the consumer surplus. The producer surplus ps corresponds to area 1
in Figure 7.1. The market supply curve can be viewed as an ordering of willingness
to supply. Someone is willing to supply the first amount of energy for 0 e /MWh,
but he receives the market clearing price λt,c . His surplus is therefore equal to λt,c .
107
100'000
80'000
Power output [MWh]

60'000
Market demand curve
40'000
Market supply curve 2
20'000
1
0
0 50 100 150 200
Spot price [e /MWh]
Figure 7.1: Calculation of the social welfare
Someone is willing to supply the second amount for energy for a little bit more than
0 e /MWh. He gets the market clearing price λt,c as well, his surplus is therefore a
little bit less than λt,c and so on. Someone is willing to supply the xth amount of
energy for the market clearing price λt,c . He receives the market clearing price, his
surplus is therefore 0 e /MWh. The producer surplus can therefore be calculated
according to
Z λt,c
pst,c = St,c (λ)dλ (7.1)
λ=0
and
Z p(λt,c )
pst,c = (λt,c − St,c (p)) dp (7.2)
p=0
In equation (7.1), the producer surplus is derived by integrating the supply curve
St,c (λ) over the price range whereas in (7.2), the integration of St,c (p) is performed
over the production range. St,c (λ) specifies the power output level p as a function
of the variable λ. In contrast, St,c (p) specifies the spot price λ for a given power
output p. λt,c is the market clearing price, the intersection of the market supply and
demand curve. Since St,c is monotonically increasing, the two integrations should
result in the same area.
The same concept applies for the consumer surplus cs. The market demand
curve can be viewed as a willingness to pay. In the example shown in Figure 7.1,
the demand curve is inelastic. The consumers theoretically would be willing to pay
an infinite amount, which makes the consumer surplus equal to infinity. In reality,
the short term demand curve for electricity is very inelastic because the balance
between consumption and production has to be maintained and the consumers are
usually not aware of the current spot price. As a result, they do not adjust their
consumption behavior depending on the current spot price.
108
However, an infinite consumer surplus is not realistic; at some point, a black-

out is cheaper than an extremely high spot price. As a result, the utilities who
buy electricity on the wholesale level are not willing to pay an infinite price. This
implies that the consumer surplus is finite. In the model, the maximum spot price
is set to 200 e /MWh, the consumer surplus is therefore capped at this price. The
area in figure 7.1 shows the capped consumer surplus at 200 e /MWh. The price at
which the consumer surplus is capped does not influence the cross-border capacity
allocations if the capping price is the same in all countries and the spot prices in
all countries are below the capped price.
The consumer surplus is calculated according to
Z 200
cst,c = Dt,c (λ)dλ (7.3)
λ=λt,c
and
Z p(λt,c )
cst,c = (min(200, Dt,c (p)) − λt,c ) dp (7.4)
p=0
Again, the integration of the demand curve Dt,c is once performed over the
power output p and once over the spot price λ.
The total social surplus tss is equal to the consumer and producer surplus and
follows
tsst,c =cst,c + pst,c

Z λt,c Z 200
= St,c (λ)dλ + Dt,c (λ)dλ
λ=0 λ=λt,c (7.5)
Z p(λt,c ) Z p(λt,c )
= (λt,c − St,c (p)) dp + (min(200, Dt,c (p)) − λt,c ) dp
p=0 p=0
The supply and demand curves are defined as the power output p for every
whole-numbered price. The integral in (7.5) can therefore be rewritten into a finite
sum according to
tsst,c =cst,c + pst,c

λt,c 200
X X
= St,c (λ) + Dt,c (λ)
λ=0 λ=λt,c (7.6)
p(λt,c ) p(λt,c )
X X
= (λt,c − St,c (p)) + (min(200, Dt,c (p)) − λt,c )
p=0 p=0
In Figure 7.2, the shaded area 3 represents the total variable production costs tvc
if the producers bid at marginal costs. But the agents will not bid at marginal costs
109
if they exercise their market-power. As a consequence, area 3 must not conform to

the total variable costs. Area 3 is calculated according to
λt,c
X
tvct,c = p(λt,c ) − St,c (λ) (7.7)
λ=0
Some references such as [2] and [1] propose to minimize area 3 for allocating
cross-border capacity.
100'000
Market demand curve

80'000
Power output [MWh]
60'000
3
40'000
Market supply curve

20'000
0
0 50 100 150 200
Spot price [e /MWh]
Figure 7.2: Calculation of total production cost
Minimizing tvct,c and maximizing the total social surplus tsst,c over more than
one market for cross-border capacity allocation must not result in the same outcome.
The approach to maximize the total social surplus is taken in this thesis. The main
reason for this choice is that the agents are not bidding at marginal costs and area
3 in Figure 7.2 therefore does not necessarily correspond to the total variable costs.
Furthermore, maximizing the total social surplus takes the consumer surplus into
account whereas minimizing the total variable production costs neglects it.
7.2.2 Formulating the optimization problem
In the four countries, the demand and supply curves are collected and aggregated
on an hourly basis on a nationwide market. Next, the cross-border capacity is
allocated such that the sum of the four total social surpluses is maximized, taking
into account the constraints discussed in Section 7.2.2. The price for country c ∈ C
after the optimization routine is referred to as λopt
t,c . For a high initial spot price
λin
t,c in country c without trade, the corresponding λopt
t,c with trade will probably be
in opt
lower and for low λt,c , the corresponding λt,c will be higher after the optimization
since more generation is called in the low price country and sold in the high price
country.
The trade mechanism clears first each market individually. After the clearing,
the optimization routine shifts the demand curves in the four countries vertically to
110
100'000
80'000
Power output [MWh]
60'000
40'000 Additional demand
20'000
λt,1 λopt
t,1
0
0 50 100 150 200
Spot price [e /MWh]
Figure 7.3: Demand is increased in country 1
maximize the sum of the total social surpluses taking into account the constraints.
The idea is that after the individual market clearings, the consumers are allowed to
buy electricity in another country. The consumers in a high priced country c = 2
are buying electricity in a low priced country c = 1, thereby shifting the demand
curve in country 1 upwards as shown in Figure 7.3. The change in market price in
opt
the low priced country 1 (λint,c − λ t,c ) depends on the price elasticity of the supply
curve. A high price elasticity of supply results in a lower price differential whereas
a low price elasticity results in a high differential.
100'000
80'000
Power output [MWh]
60'000
Reduced demand
40'000
20'000
λopt
t,2 λt,2
0
0 50 100 150 200
Spot price [e /MWh]
Figure 7.4: Demand is reduced in country 2
In contrast, the demand in country 2 is decreased by the same amount as it is

increased in country 1, thereby shifting the demand curve downwards as shown in
Figure 7.4. The change in market price in the high priced country 2 depends on
the price elasticity of supply in country 2.
Besides the price elasticities in supply, the price differentials in the two coun-
111
tries depends on the transfered amount of electricity. If there is no line capacity

constraint between these two countries, the consumers in the high priced country 2
buy electricity in the low priced country 1 until the two prices are equal. If there is
a constraint in the amount of capacity that can be transferred between two coun-
tries, the demand in country 1 is increased until a limit is reached. In this case,
the transmission line connecting these two countries generates a congestion revenue
which is calculated according to
opt
crt,2→1 = λopt
t,2 − λt,1 Et,2→1 (7.8)
where Et,2→1 stands for the transferred electricity from country 2 to 1.
Instead of shifting the demand curves in the four countries it is also possible to
shift the supply curves. The resulting spot prices, power outputs and exchanges
after the optimization are the same in both cases.
Before stating the objective function and constraints, the variables of the opti-
mization problem are introduced and initialized. The following variables are used
by the optimization routine:
Germany Switzerland France Italy

Market price λt,G λt,S λt,F λt,I
Demand dt,G dt,S dt,F dt,I
Supply st,G st,S st,F st,I
Table 7.1: Variables of the optimization routine
as well as the exchanges between these countries:
• Exchange France-Italy: Et,F →I
• Exchange France-Switzerland: Et,F →S
• Exchange Switzerland-Germany: Et,S→G
• Exchange Germany-France: Et,G→F
• Exchange Italy-Switzerland: Et,I→S
The variables are initialized by clearing each market separately. The initial price
λin
t,c , demand dt,c and supply dt,c ∀c ∈ C correspond to the values if every market
is cleared without trade. Each exchange value E is set to 0 MWh.
Dt,c (λ) and St,c (λ) are a function of the spot price and are defined for every whole
numbered spot price from 0 e /MWh to 200 e /MWh. For a given arbitrary value
of λ ∈ 0, ..., 200 e /MWh in country c referred to as λt,c , the supplied quantity
is defined as st,c = St,c (λt,c ) and the demanded quantity as dt,c = Dt,c (λt,c ). If
λt,c is not a whole number, the supplied and demanded quantities are derived by
interpolating between the two neighboring whole numbered spot prices.
112
The objective function
The market clearing price for every country λt,c has to be at the intersection of the
supply curve St,c and demand curve Dt,c . The supplied value has to be equal to
the demanded value for every market, which results in
dt,c = Dt,c (λt,c ) = st,c = St,c (λt,c ) ∀c ∈ C (7.9)
′
A new shifted demand curve Dt,c is introduced in every country. The properties
of Dt,c are such that it intersects St,c at the market price λ′t,c . Dt,c
′ ′
is calculated
according to
′
Dt,c = Dt,c + St,c (λ′t,c ) − Dt,c (λ′t,c ) = Dt,c + s′t,c − d′t,c (7.10)
′
Figure 7.5 shows graphically how the shifted demand curve Dt,c is derived.
100'000
80'000
Power output [MWh]
s′t,c
60'000
40'000 d′t,c
20'000
λ′t,c
0
0 50 100 150 200
Spot price [e /MWh]
′
Figure 7.5: Graphical interpretation of Dt,c
With this shifted demand curve the total social surplus can be calculated ac-
cording to
N
X
tsst,tot = (pst,c + cst,c )
c=1
  (7.11)
N λ′ 200
X X
t,c
X
=  St,c (λ) + ′
Dt,c (λ)
c=1 λ=0 λ=λ′t,c
The total social surplus tsst,tot is maximized to allocate cross-border capacity.
113
Constraints
There are several constraints in the optimization problem. The first constraint
specifies that the exchange of two neighboring countries has to stay within a limit.
This limit is referred to as NTC (net transfer capacity). In Figure 7.6, the trading
partners are indicated. Germany cannot trade electricity with Italy directly, since
there is no interconnecting line between these two countries. The NTC values are
published by ENTSO-E (European Network of Transmission System Operators) for
every half year. The local TSOs are able to change them on an intra-day level, but
unless there is a severe event, they do not deviate much from the given ENTSO-E
NTC values.
Germany
NTCG F → NTCG S
→
NTCF G → NTCS G
→
NTCF S →
France Switzerland
NTCS F →
NTCF I
→ NTCS I →
NTCI F→ NTCI S
→
Italy
Figure 7.6: Trading partners and NTC values
Table 7.2 shows the published NTC values for the chosen four countries which
are valid for half a year. The winter half year is indicated by W whereas the summer
half year by S.
Period F→I I→F F→S S→F S→D D→S D→F F→D I→S S→I
W 2006 2’650 1’800 3’200 3’500 4’400 3’400 5’600 2’850 1’460 3’850
S 2006 2’400 1’800 3’000 1’400 4’000 2’075 2’700 2’400 1’460 3’120
W 2007 2’650 995 3’200 2’300 4’000 2’400 3’300 2’850 1’460 3’890
S 2007 2’400 870 3’000 1’400 4’400 2’080 2’700 2’400 1’140 3’180
W 2008 2’650 995 3’200 2’300 4’000 2’100 2’750 2’850 1’460 3’890
S 2008 2’400 870 3’000 1’400 4’400 1’700 2’400 2’500 1’140 3’160
W 2009 2’860 886 3’200 2’300 3’200 1’600 2’760 2’800 1’810 4’240
S 2009 2’400 870 3’000 1’400 4’400 2’060 2’700 2’400 1’290 3’310
Table 7.2: NTC values, source ENTO-E
It is assumed that the local TSOs do not adjust the NTC values for irregular-
ities in the gird, generation or consumption. The NTC values depend on time t,
since they change every half year. The traded amount of electricity between two
neighboring countries cannot exceed its corresponding NTC value, i.e.
114
Et,1→2 ≤ N T Ct,1→2 and Et,2→1 ≤ N T Ct,2→1 (7.12)
Another constraint is the power balance for all four countries. It is an equality
constraint and specifies that the supply plus import minus consumption and export
has to be equal for all countries as described by
X X
st,c − dt,c + Et,x→c − Et,c→x = 0 (7.13)
x∈Ωc x∈Ωc
where Ωc specifies all countries to which country c is connected.
7.2.3 The optimization routine
The optimization routine maximizes the objective function, which specifies the total
social surplus subject to the equality and inequality constraints discussed in section
7.2.2. The interior-point method is chosen as optimizing algorithm. The algorithm
opt
takes the initial x vector xin
t as input and returns the optimized vector xt , which
specifies the optimized prices λt,c , the supplied and demanded quantities sopt
opt opt
t,c , dt,c
opt
and exchanges Et for all countries. The optimization routine is stated as follows:
  ′ 
N λt,c 200
X X X
max   St,c (λ) + ′
Dt,c (λ) (7.14)
c=1 λ=0 λ=λ′t,c
Power balances:
s.t st,G − dt,G + Et,S→D − Et,D→F = 0

st,S − dt,S − Et,S→D + Et,I→S + Et,F →S = 0
(7.15)
st,F − dt,F + Et,D→F − Et,F →S − Et,F →I = 0
st,I − dt,I − Et,I→S + Et,F →I = 0
Flow limits:
s.t − N T Ct,I→F ≤ Et,F →I ≤ N T Ct,F →I

N T Ct,S→F ≤ Et,F →S ≤ N T Ct,F →S
N T Ct,G→S ≤ Et,S→G ≤ N T Ct,S→G (7.16)
N T Ct,F →G ≤ Et,G→F ≤ N T Ct,G→G
N T Ct,S→I ≤ Et,I→S ≤ N T Ct,I→S
Price limits:
115
s.t 0 ≤ λt,G ≤ 200

0 ≤ λt,S ≤ 200
(7.17)
0 ≤ λt,F ≤ 200
0 ≤ λt,I ≤ 200
7.3 Simulation results reference scenario
In the following, the model predictive bidding algorithm is applied to model the
power generating agents in each of the four countries. The scarce cross-border
capacity is allocated according to the implicit scheme discussed and the effect of such
an implicit allocation scheme between these four countries is analyzed. Additionally,
different scenarios are simulated in which specific net transfer capacities are changed
to assess the sensitivity of certain interconnection lines on spot prices.
In this scenario, the model follows the settings described in 7.1. The NTC values
for cross-border exchanges between the four countries are set to the value published
by ENTSO-E in Table 7.2. As in the model application for the German market in
chapter 6, the simulation is run for three trials. The agents are inexperienced in
the beginning, so they choose random actions with a given probability in the first
trial. The three simulation trials allow the agents to adapt and learn in the new
environment. The outcomes after the third trial are compared.
First, the spot price is analyzed to assess the effect of an implicit cross-border allo-
cation scheme on base load prices for the four countries under reference conditions.
The base load prices for the four countries with and without exchange are shown
in Figure 7.7. The prices without exchange result if each country is cleared indi-
vidually and without cross-border exchanges. In the following, some observations
regarding the spot price in the four countries are stated:
• In France, the spot price without exchange is very low. It results from a high
installed nuclear power generation and storage capacity. With exchanges, the
base load price rises to much higher levels.
• In Germany, the opposite effect is observable. The price level is lowered by

introducing exchanges.
• In Italy, the price level does not change much by introducing cross-border
exchanges despite the fact that the Italian electricity production costs are
high and the installed generation capacity is low. The reason is that Italy has
a lot of elastic bidders (gas power plants are the major production source in
Italy). If the demand curve is shifted by introducing cross-border exchanges,
the spot price changes only minor.
• In Switzerland, the price level rises a little bit with cross-border exchanges.
116
200
Germany
Germany no exchange
180
Switzerland
Switzerland no exchange
160 France
France no exchange
Italy
Spot price [e /MWh]
140
Italy no exchange
120
100
80
60
40
20
0
0 50 100 150 200 250 300 350 400
Day
Figure 7.7: Simulated base load prices reference scenario
It can be concluded that by introducing cross-border exchanges the spot price

rises to higher levels in France and to lower levels in Germany since these countries
have a high amount of inelastic bidders. In contrast, the spot price level in Italy
does not change much because of its high elastic bidder share.
Germany [e ] Switzerland [e ] France [e ] Italy [e ]

Base load 46.85 43.68 36.64 50.71
Peak load 53.36 46.81 40.77 51.46
Price std 16.81 8.18 22.88 4.73
Table 7.3: Summary spot prices reference scenario
Table 7.3 provides a simulation summary of the mean base and peak load prices.
The mean base load price is lowest in France due to the low production costs and
highest in Italy. The mean base load price in Switzerland is in between the price
of France and Italy, since a lot of electricity is transported from France to Italy
via Switzerland as shown in 7.3.3. Italy has a very low peak/off-peak price spread
because of a high ramping capacity in power production.
In the following, the power outputs of the agents in the four countries are discussed.
Figure 7.8 visualizes the power outputs of the German agents. The nuclear
and coal agent produce on a high and steady level. They show a much lower
ramping behavior than in the case without cross-border trade discussed in chapter
6. Furthermore, these agents do not show the regime switching behavior anymore,
which resulted in the Edgeworth price cycles. The gas agent still shows a high
ramping behavior because of his high marginal production cost. It can be concluded
117
4
x 10
2
Germany Nuclear
Germany Coal
1.8
Germany Gas
Germany Wind
1.6
Power Output [MW]
1.4
1.2
0.8
0.6
0.4
0.2
0
0 50 100 150 200 250 300 350 400
Day
Figure 7.8: Power outputs reference scenario, Germany
that the cross-border exchanges with implicit capacity allocation allow the German
inelastic bidding agents to produce more efficiently by avoiding ramping.
In Figure 7.9, the power outputs of the Swiss agents are shown. The German
nuclear agent uses a steady power output level with some intra-day variations while
the Swiss nuclear agent produces on a constant power output. The reason for the
constant power output could be the high amount of storage capacity in Switzerland.
The Swiss storage agent shows a high buying/selling activity by exploiting price
differentials.
The power outputs of the agents in France in Figure 7.10 show the high pro-
duction share of the nuclear agent. Since there are only minor elastic bidders, the
nuclear agent has to ramp the production and cannot produce on a steady level
throughout the year. There are time intervals however in which the agent is able to
stabilize his production. Around day 100, the nuclear agent shows a heavy ramp-
ing behavior. This behavior strongly effects the spot prices in all four countries as
shown in figure 7.7. The coal agent produces on a steady power output in France,
while the gas agent ramps his production due to his low ramping costs and his high
marginal production costs.
In Italy, the gas agent has a major share in power production as shown in Figure
7.11. The gas agent shows a high ramping behavior while the coal agent produces
almost on a constant level. Although Italy has almost 23 GW installed oil capacity,
the oil agent has only a minor power output. His marginal costs are very high,
most of the time the spot price is not high enough to enable a profitable power
production.
In the following, the profits of the agents are analyzed. Table 7.4 shows the total
profits of the agents in Germany over the year. Additionally, the Table indicates
the specific profits, which are the total profits per installed MW power production
capacity per agent. For better comparison among the agents, this measure is more
118
7000
Switzerland Nuclear
Switzerland Hydro
6000 Switzerland Storage
5000
Power Output [MW]
4000
3000
2000
1000
−1000
−2000
0 50 100 150 200 250 300 350 400
Day
Figure 7.9: Power outputs reference scenario, Switzerland
meaningful.
Country Gen. technology Profits [109 e ] Spec. profits [104 e /MW]

Germany Nuclear 6.04 29.8
Germany Coal 3.58 9.46
Germany Gas -0.09 -0.54
Germany Wind 1.21 5.72
Germany Total 10.74 11.2
Table 7.4: Profits and specific profits Germany, reference scenario
Interestingly, the nuclear and the coal agent in Germany generate higher profits
in the four country model than in the German model described in chapter 6, al-
though the cross-border exchange results in lower spot prices in Germany. In the
German model, the nuclear agent generates a profit of 3.27 · 109 e and the lignite
and hard coal agent a combined profit of 2.74 · 109 e . The higher profits in the four
country model are due to reduced ramping costs and the production on more effi-
cient output levels. The nuclear agent in Germany is very profitable on a per MW
installed capacity basis as described by the specific profits. Overall, the German
electricity market is more profitable than the French market on a specific basis.
This is surprising since France can produce electricity at very low costs.

Switzerland Nuclear 1.18 36.6
Switzerland Hydro 1.21 16.2
Switzerland Storage 0.05 0.65
Switzerland Total 2.44 13.3
Table 7.5: Profits and specific profits Switzerland, reference scenario
Table 7.5 shows that the nuclear agent in Switzerland is much more profitable
per installed MW than the storage agent. The hydro agent is more profitable than
119
4
x 10
7
France Nuclear
France Coal
6 France Gas
France Hydro
France Storage
Power Output [MW] 5
−1
0 50 100 150 200 250 300 350 400
Day
Figure 7.10: Power outputs reference scenario, France
the storage agent because the hydro agent has a natural water inflow at zero costs
while the storage agent has to buy electricity in order to pump water into the higher
reservoir. Overall, the Swiss power production is the most profitable one measured
by the specific profits.

France Nuclear 9.15 14.5
France Coal 0.88 5.98
France Gas -0.27 -2.75
France Hydro 1.21 5.81
France Storage 0.16 3.72
France Total 11.13 9.86
Table 7.6: Profits and specific profits France, reference scenario
The French nuclear agent is not as profitable as the Swiss or German nuclear
agent. The reason is that France has not enough elastic bidders to operate the
nuclear agent on a high output level throughout the year. The ramping by the
nuclear agent induces inefficiencies and ramping costs, which lowers his profitability.
The storage agent in France is more profitable than the one in Switzerland due to
a higher spot price volatility in France.
The coal agent in Italy is very profitable due to a high and steady power output
level, which is enabled by the high amount of elastic bidders. The oil agent is
highly unprofitable due to his high production costs. The storage agent has a low
profitability because the spot price volatility is low in Italy. Overall, the Italian
electricity market is not profitable. However, this does not imply that building
an additional power plant in Italy is not profitable. It can be very profitable due
to the high spot price level. To analyze the profit of an additional power plant,
the marginal profit instead of the profits or specific profits have to be taken into
account. The marginal profits describe how much the profit change by changing
120
4
x 10
2.5
Italy Gas
Italy Oil
Italy Coal
2 Italy Hydro
Italy Storage
Power Output [MW]
1.5
0.5
−0.5
0 50 100 150 200 250 300 350 400
Day
Figure 7.11: Power outputs reference scenario, Italy

Italy Gas -1.35 -3.93
Italy Oil -2.41 -10.5
Italy Coal 1.96 24.4
Italy Hydro 1.21 8.91
Italy Storage 0.05 0.66
Italy Total -0.50 -0.58
Table 7.7: Profits and specific profits Italy, reference scenario
one unit of installed capacity.
121
7.3.3 Exchanges and congestions
5
x 10
1
France−Italy
France−Switzerland
0.8
Switzerland−Germany
Germany−France
0.6 Italy−Switzerland
0.4
Exchange [MWh]
0.2
−0.2
−0.4
−0.6
−0.8
−1
0 50 100 150 200 250 300 350 400
Day
Figure 7.12: Electricity exchanges reference scenario
Figure 7.12 shows the electricity exchanges between the four countries. The
electrical current flows according to the Kirchhoff’s laws, which means that the
electric flow is determined by the transmission line resistances. By exchanging elec-
tricity from France to Italy, a high amount of electricity flows through Switzerland
as loop-flows. The Figure therefore shows the contractual electricity exchanges and
not the actual physical flows.
The exchanges between Italy and Switzerland indicate a high electricity flow
from Switzerland to Italy. The Figure also shows a high exchange from France to
Switzerland and from France to Italy. The exchanges between these countries often
reach the maximum NTC values. France with its low production costs produces a
surplus which is sold to Italy directly or via Switzerland; Switzerland thereby acts
as a transit country. The electricity exchanges between Switzerland and Germany
are more balanced, whereas the exchanges between Germany and France are always
negative.
Figure 7.13 visualizes the net position of each country, which is defined as do-
mestic production minus domestic consumption. France is a high net exporter due
to the low production costs and the high installed capacity. Switzerland is a net
exporter as well whereas Germany and Italy are net importers throughout the year.
The implicit cross-border allocation scheme results in the fact the more electricity
is produced in the low price countries France and Switzerland and sold in the high
price countries Germany and Italy.
In Figure 7.14, the electricity exports and imports are shown relative to the
domestic load. It highlights the importance of Switzerland as an electricity transit
country with high imports and exports. During some hours, the exports and imports
even reach levels which are higher than the domestic load. France never imports
and Italy never exports electricity throughout the year.
122
5
x 10
10
Germany
Switzerland
8 France
Italy
6
Net position [MWh]
−2
−4
−6
0 50 100 150 200 250 300 350 400
Day
Figure 7.13: Net position of each country reference scenario
F→I F→S S→G G→F I→S

[106 e ] [106 e ] [106 e ] [106 e ] [106 e ]
Con. rent 564 672 605 876 1’064
Table 7.8: Summary congestion revenues reference scenario
The congestion revenues defined in (7.8) for every transmission line are shown in
Table 7.8. The line connecting Switzerland and Italy generates the highest conges-
tion revenue due to the high power flow from Switzerland to Italy. The congestion
revenues for the other interconnection lines are in the same order of magnitude. In-
terestingly, the congestion revenue for the transmission line connecting France and
Italy is lowest, although the price differential between these two countries is highest.
The lower congestion revenue is due to the low NTC values for the transmission
line connecting these two countries.
123
500
Export Germany
Import Germany
450
Export Switzerland
Import Switzerland
400 Export France
Import France
Export Italy
Exports/imports [%]
350
Import Italy
300
250
200
150
100
50
0
0 50 100 150 200 250 300 350 400
Day
Figure 7.14: Exports/imports as percentage of domestic load reference scenario
The histograms of the estimation errors are different for each country as shown
in Figure 7.15. The reasons are that the agents in the different countries have
different amounts of market-power. Therefore, the price variations explained by
exogenous factors change between the countries. Furthermore, the different elec-
tricity markets’s price variations have different dependencies on exogenous factors.
In France for instance, the temperature has a high effect on spot prices because
a high amount of the heating demand is covered by electricity. In contrast, the
temperature dependency on spot prices is much lower in Switzerland.
In the following, some interesting properties of the price predictors are high-
lighted:
• The t-values of the German price predictor show that the heating degree day
for France is statistically significant on a 0.8% significance level and the heat-
ing degree day for Italy on a 3.21 · 10−7 % significance level while the heating
degree day for Germany itself is insignificant even on a 10% significance level.
This implies that the German spot price variations can be better explained by
heating degree day variations in France and Italy than by German variations.
The reason is that the German historical price fraction is correlated to the
German heating degree day; as a result, the German historical price fraction
already explains a lot of the German spot price variations. The R-squared
value for the German price predictor is as high as 0.83, which indicates that
83% of spot price variations can be explained by exogenous factors. The mean
squared error for the German price predictor is as low as 45.9 e /MWh2 , which
indicates a very high spot price predictability by the price predictor.
• The Swiss price predictor indicates that the German wind forecast is statis-
tically significant on a very low significance level. Furthermore, the heating
124
600
Germany
Switzerland
France
500 Italy
400
Observations
300
200
100
0
−200 −150 −100 −50 0 50 100 150 200
Figure 7.15: Histograms of the estimation errors reference scenario
degree days for France and Italy are highly significant while the Swiss heating
degree day is only significant on a 1.6% level. This is due to the high linkage
of the Swiss electricity market to its surrounding markets through the high
electricity exchanges. The R-squared value of the Swiss price predictor is 0.62
and the MSE 77.8 e /MWh2 , which indicates a high price predictability.
• For the French price predictor, the German wind forecast is insignificant, but
the heating degree days for France itself and Italy are highly significant. This
is due to the high electricity exchanges between France and Italy as shown in
Figure 7.12. The R-squared value of the French price predictor is 0.44 while
the MSE is 158.6 e /MWh2
• The Italian price predictor shows that all the heating degree days for the
four countries are less significant, but interestingly, the cooling degree day for
France is highly significant. This is surprising because for the other countries’s
price predictors the cooling degree days are insignificant on a 5% significance
level. It is probably due to the fact that France has a high amount of ex-
cess and cheap electricity during the summer which it can sell to Italy and
thereby influence the Italian spot price. The R-squared value for Italy is 0.12,
which indicates that only 12% of the spot price variations can be explained by
exogenous factors. The MSE for the Italian price predictor is 161.1 e /MWh2 .
In the following, the pav values of the agents are analyzed to assess which agents
in every country have the ability to exercise market-power.
• In Germany, the nuclear agent has the highest market-power measured by the
absolute pav values. His pav values range from -148 e /MWh to 155 e /MWh.
In contrast, the gas agent has almost no market-power, his pav values range
from -5.9 e /MWh to 3.3 e /MWh.
125
• In Switzerland, the agents in general have much less market-power. The

nuclear agent’s pav values range from -20.7 e /MWh to 12.4 e /MWh, while
the storage agent’s pav values range from -3.9 e /MWh to 9.8 e /MWh.
• In France, the nuclear agent has very high market-power. His pav values
range from -244 e /MWh to 155 e /MWh. The gas agent has the lowest
market-power. His pav values only range from -10 e /MWh to 10 e /MWh.
• In Italy, the gas and coal agent have the highest market-power. Their pav
vales range from -9.8 e /MWh to 10.2 e /MWh and -11.5 e /MWh to 12.5
e /MWh respectively. All the other agents in Italy have almost no market-
power. Because of the low absolute pav values in Italy, it can be assumed that
no agent has significant market-power.
7.4 Scenario analysis: Building additional inter-

connection capacity
As outlined in Chapter 1, regulation No 714/2009 put a special emphasis on cross-

border trade and the creation of interconnection capacities to achieve the goal of
an internal electricity market. In the following, different scenarios are analyzed to
assess the effect of higher interconnection capacities on spot prices, exchanges and
congestions. Furthermore, market-power is analyzed. As highlighted in Chapter
1, it is argued that market participants are suspected to exploit market-power by
creating artificially congestions. The scenarios which are analyzed have the following
properties:
• As [66] outlines, flexible alternating current transmission systems (FACTS)

1
are meant to increase power transfer capabilities of the network to avoid
congestions. In a first scenario, the NTC values of the interconnection line
between Italy and Switzerland (the line with the highest congestion revenue)
is increased by 30%, which is a value attainable with FACTS devices. This
scenario is referred to as line S-I
• In a second scenario, the interconnection line between France and Italy (the
line which connects the electricity markets with the highest spot price differ-
ential) is increased by 30%. The scenario is described as line F-I later in the
thesis.
• Finally, in a third scenario, all the interconnection lines’s NTC values are
set to infinity, which results in a copper-plate model for these four countries
without any congestions. In the following, this scenario is called no NTC.
In the following, the mean base and peak load prices as well as the price standard
deviation are shown for the reference, the line S-I, the line F-I and the no NTC
scenarios.
1 FACTS is defined by IEEE as “a power electronic based system and other static equipment
that provide control of one or more AC transmission system parameters to enhance controllability
and increase power transfer capability” [67]
126
Ref. scenario line S-I Line F-I No NTC

Mean base load 46.85 45.39 47.85 20.42
Mean peak load 53.36 52.36 47.99 23.81
Price std 16.81 14.96 12.97 12.07
Table 7.9: Summary spot prices Germany transmission line scenarios
Table 7.9 shows the spot prices for Germany and the four scenarios. The line
S-I and line F-I scenarios barely affect the German base and peak load prices.
However, the German spot price standard deviation is reduced even though the two
interconnection lines in the scenarios are not connected to Germany directly. The no
NTC scenario shows that the mean base and peak load prices could be dramatically
reduced if there were no electricity transfer limitations in the interconnection lines.
The prices would be less than half as under reference conditions.

Mean base load 43.68 46.31 44.99 20.42
Mean peak load 46.81 49.00 47.99 23.81
Price std 8.18 5.53 7.14 12.07
Table 7.10: Summary spot prices Switzerland transmission line scenarios
In the line S-I and F-I scenario the mean base and peak load prices in Switzer-
land increase a little bit while the spot price volatility decreases as shown in Table
7.10. The increase in the spot prices is due to the fact that Switzerland generates
more electricity which is sold to Italy if the interconnection line capacities between
Switzerland and Italy or France and Italy are increased. By increasing these ca-
pacities, Switzerland can reduce its price volatility by having more “access” to the
Italian electricity generators, which are mainly elastic bidders with a high ramping
capacity. In the no NTC scenario however, the price standard deviation increases
in Switzerland.

Mean base load 36.64 29.36 30.20 20.42
Mean peak load 40.77 33.28 33.92 23.81
Price std 22.88 16.28 17.28 12.07
Table 7.11: Summary spot prices France transmission line scenarios
The French consumers largely benefit from an increase in the line S-I and line
F-I scenarios since the spot prices drop considerably as shown in Table 7.11. Fur-
thermore, the price standard deviation decreases by increasing the line capacities.
Interestingly, the spot prices are lower if the transmission line capacity between
Switzerland and Italy is increased by 30% than if the line between France and Italy
is increased by 30%. This is due to the fact that in absolute terms, increasing
the line between Switzerland and Italy by 30% results in a higher amount than
increasing the French-Italian line.
In Italy, the mean peak and base load prices do almost not change in the line S-I
and line F-I scenario despite the higher electricity imports. The reason is the high
price elasticity in the aggregated supply curve. A decrease in the Italian demand
127

Mean base load 50.71 50.68 50.95 20.42
Mean peak load 51.46 51.61 51.85 23.81
Price std 4.73 4.76 4.99 12.07
Table 7.12: Summary spot prices Italy transmission line scenarios
results in small price differentials. The mean spot price eventually drops as seen
in the no NTC scenario when there are no congestions. The lower spot price level
comes with a increase in spot price volatility.
160
Simulated German base load price
Observed German EEX base load price
140
120
Spot price [e /MWh]
100
80
60
40
20
0
0 50 100 150 200 250 300 350 400
Day
Figure 7.16: Simulated base load prices Germany no NTC scenario
Figure 7.16 shows the simulated German spot prices in the no NTC scenario
and the observed EEX prices. The simulated prices in the no NTC are similar in
all four countries. The Figure confirms the findings from Table 7.9 of lower base
and peak load prices as well as a reduced price standard deviation in the no NTC
scenario. This scenario allows a very efficient use of generation capacities in the
four countries.
7.4.2 Exchanges
In the following, the no NTC scenario is analyzed since it represents the transmission
line scenario with the highest impact on spot prices.
The electricity exchanges between the four countries and the no NTC scenario
are shown in Figure 7.17. It shows the very high electricity exchanges from France
to Italy; From the country with the lowest production costs to the country with the
highest production costs. As mentioned, this figure does not represent the actual
physical electricity flows, it shows the contractual electricity exchanges between the
128
5
x 10
6
France−Italy
France−Switzerland
Switzerland−Germany
4 Germany−France
Italy−Switzerland
Exchange [MWh]
−2
−4
−6
0 50 100 150 200 250 300 350 400
Day
Figure 7.17: Electricity exchanges no NTC scenario
countries. A high amount of electricity is also sold from France to Switzerland, from
France to Germany and from Switzerland to Italy.
The net positions of each country in Figure 7.18 show that France is exporting
much more electricity and Italy is importing much higher amounts than in the
reference scenario. The reason is that a certain amount of expensive electricity
production in Italy is replaced by cheaper electricity from France and exported to
Italy. The electricity market in Switzerland stays balanced in the no NTC scenario
whereas the German electricity market is importing additional electricity, mostly
from France.
129
5
x 10
12
Germany
Switzerland
10
France
Italy
8
Net position [MWh]
6
−2
−4
−6
−8
0 50 100 150 200 250 300 350 400
Day
Figure 7.18: Net position of each country no NTC scenario
7.4.3 Profits
In the following, the profits and specific profits of the agents in the no NTC scenario
are analyzed to assess if which generators benefit from an increase in cross-border
trade.
Germany Nuclear 3.69 18.2
Germany Coal 0.92 2.43
Germany Gas 0.16 0.96
Germany Wind 1.21 5.72
Germany Total 5.98 6.24
Table 7.13: Profits and specific profits Germany, no NTC scenario
Table 7.13 shows the profits and specific profits of the German agents in the no
NTC scenario. The German nuclear and coal agent have lower profits in the no
NTC scenario compared to the reference scenario. The lower profits result mostly
from a lower spot price level in the no NTC scenario. Interestingly, the German
gas agent generates higher profits in the no NTC scenario despite the lower spot
price level and a lower price standard deviation. Overall, the German agents are
less profitable in the no NTC scenario than in the reference scenario.

Switzerland Nuclear 0.42 13.0
Switzerland Hydro 1.21 16.2
Switzerland Storage 0.09 1.18
Switzerland Total 1.72 9.39
Table 7.14: Profits and specific profits Switzerland, no NTC scenario
The profits of the Swiss nuclear agent are lower in the no NTC scenario compared
130
to the reference scenario as shown in Table 7.14. The Swiss storage agent benefits
from a higher spot price volatility in the no NTC scenario and is able to increase his
profitability. Overall, the Swiss agents are less profitable in the no NTC scenario
compared to the reference scenario.

France Nuclear 8.24 13.0
France Coal 0.26 1.76
France Gas -0.51 -5.19
France Hydro 1.21 5.81
France Storage 0.16 3.72
France Total 9.36 8.29
Table 7.15: Profits and specific profits France, no NTC scenario
Table 7.15 shows the profits and specific profits for the French agents in the
no NTC scenario. The French nuclear agent’s profitability is in the same order of
magnitude in the no NTC and reference scenario. His specific profits are equal to
the Swiss nuclear agent’s specific profits. The French coal and gas agent have much
lower profits in the no NTC scenario. Despite the lower price standard deviation
the storage agent has the same profits in the no NTC scenario as in the reference
scenario. Overall, the French agents are a little bit less profitable in the no NTC
scenario.
Italy Gas -2.12 -6.17
Italy Oil -4.26 -18.6
Italy Coal 0.25 3.12
Italy Hydro 1.18 8.69
Italy Storage 0.27 3.58
Italy Total -5.18 -5.99
Table 7.16: Profits and specific profits Italy, no NTC scenario
In Italy, only the storage and hydro agent increase their profits in the no NTC
scenario as shown in Table 7.16. They both benefit from a high price standard
deviation in Italy in the no NTC scenario. All other agents suffer from much lower
profits due to the lower average Italian price level. Overall, the Italian agents incur
much higher losses in the no NTC scenario than in the reference scenario.
The histograms of the estimation errors for the four countries and the no NTC
scnario in Figure 7.19 show that they are close together but not similar. The reason
is that the agents in the four countries do not have the same factors in the multi-
factor price predictor model. The different factor is the historical price fraction,
which is taken only per country.
In the following, only the price predictor for Germany is analyzed since the other
price predictors have similar characteristics. The kurtosis of the price prediction
error is 29.5 e /MWh4 , indicating a high excess kurtosis and a peaked distribution.
It implies that a lot of observations are around 0 e /MWh and some are very far
away from 0 e /MWh. The R-squared value of the price predictor is 0.16, which
131
350
Germany
Switzerland
France
300
Italy
250
Observations
200
150
100
50
0
−200 −150 −100 −50 0 50 100 150 200
Figure 7.19: Histograms of the estimation errors no NTC scenario
implies that only 16% of the spot price variations is explained by exogenous factors.
This small R-squared value is due to the fact that a lot of the spot price variation is
not caused by exogenous factors but by the optimization routine instead. The mean
squared error is 265 e /MWh, which is a much higher value than in the reference
scenario.
On a 5% significance level the factors oil price, wind forecast Germany, heating
degree days France and Italy and the German historical price fraction are significant.
In the following, the market-power of the agents in the no NTC is analyzed

to assess if the agents gain or lose market-power if the cross-border capacities are
increased.
• In Germany, the nuclear agent’s pav values range from -74.2 e /MWh to 63.4
e /MWh, which indicates that the market-power of the German nuclear agent
is highly reduced in the no NTC scenario.
• The Swiss nuclear agent’s pav values range from -48.8 e /MWh to 4.58 e /MWh
and the Swiss storage agent’s from 3.54 e /MWh to 3.55 e /MWh. It indi-
cates that despite a high installed capacity the Swiss storage agent has almost
no market-power.
• The French nuclear agent’s pav value range is reduced to -51.05 e /MWh to
14.82 e /MWh. Interestingly, the German nuclear agent has a higher pav
range than the French one, although the French nuclear agent has a much
higher installed generation capacity. The French gas agent’s values range
from -5.04 e /MWh to 9.36 e /MWh, the storage agent’s from -2.89 e /MWh
to 3.59 e /MWh.
• The Italian gas agent’s pav values range from -9.65 e /MWh to 4.13 e /MWh,
the oil agent’s from -10.85 e /MWh to 5.09 e /MWh and the storage agent’s
132
from -2.11 e /MWh to 3.51 e /MWh. It indicates that no agent in Italy has
significant market-power.
Overall, the pav ranges for the agents which had major market-power in the
reference scenario have much lower market-power in the no NTC scenario. The
reason is that if the agents decide for instance to cut supply to drive prices up, the
optimization will cause that the consumers will buy electricity in other markets and
therefore lower the domestic spot price.
7.4.5 Conclusion of the line scenarios
It can be concluded that overall, no country’s power producing agents have an

incentive to build additional transmission lines since the profits are lower in an
environment with higher cross-border capacities. But building these capacities is
important since it allows to use the power plants more efficiently, drive the spot
price to more competitive levels, lowers the total costs for consumers and increases
the security of supply.
The no NTC scenario shows that market-power of the power producing agents is
decreased in an environment with higher cross-border capacities. The agents have
less gaming opportunity since the optimization routine stabilizes the prices.
133
Chapter 8
Conclusion
The forward-looking and predictive model approach of the model predictive bid-
ding learning algorithm allows the agents to schedule the production efficiently and
thereby increase their profits. The predictive scheduling is especially important for
the inelastic and storage bidders since they have to anticipate future spot price
movements. Model predictive bidding also allows the agents to learn and exercise
their market-power. The resulting pav values thereby provide insights in the amount
of each agent’s market-power. The learning rate of model predictive bidding is faster
than Q-learning and classifier based learning systems. Model predictive bidding is
also more stable because it is able to handle unknown environmental inputs. Ad-
ditionally, the multi-factor model of the price predictor allows to draw conclusions
on which factors the agents base their spot price predictions.
The model application to the German electricity wholesale market shows that
the nuclear agent has a high market-power, but does not exercise it extensively.
He rather stabilizes his power output on a high and efficient level. In contrast,
the gas and storage agents exercise their market-power, but the absolute value of
market-power is much smaller for these agents since their installed capacity is lower.
A higher wind energy contribution decreases the mean spot price level but rises
the total costs for consumers since the wind power bypasses the spot market. The
price predictions are less accurate with an increased wind energy because of the
additional wind forecast error. The market-power of the inelastic bidders initially
increases with increasing wind due to a higher back-up power demand but at one
point declines because of inaccurate spot price predictions. The market-power of
the elastic bidders changes only minor by changing the amount of wind power.
Increasing the power output and storage capacity of the storage agent results
in gaming among the agents and in a higher likelihood of collusion. By colluding,
the agents are able to push the spot prices to higher levels, for which the agents
can increase their expected discounted spot prices. At some point the higher spot
prices trigger the agents to undercut each other which results in a sharp drop in
spot prices. The inelastic bidders thereafter act as restorable forces which bring the
spot price back on a more stable level again. This behavior is repeated resulting in
price cycles which are referred as Edgeworth price cycles. The effects of the price
cycles on the agent’s profitability are manifold. The emergence and duration of
these price cycles can be sensitive to the chosen prediction horizon and the chosen
134
Chapter 8. Conclusion
discount factor.
Aggregating PHEVs to a cluster and charging/discharging them via the electric

grid does not result in price cycles. The available charing/discharging capacity
depends on the amount of PHEVs connected to the grid and their state of charge.
This time variability results in the fact that the PHEV cluster cannot exercise
market-power extensively. The PHEV cluster increases the mean spot price level
since it represents an additional load. A high PHEV cluster allows to use the
nuclear, hard coal and lignite power plants very efficiently by providing backup-
power and storing excess electricity during night times. Overall, the profits of the
power generating units increases with a numerous PHEV cluster. It can therefore
be concluded that the electricity producers have an incentive to promote the large
scale adaption of PHEVs. The PHEV cluster generates a profit throughout the year,
which is remarkable. The model predictive bidding algorithm allows the PHEV
cluster to generate a profit which is high enough to cover the costs associated to
driving the PHEVs around with electricity.
Little changes in how the storage agent is modeled show a very different market
outcome. It is remarkable that the storage and PHEV scenarios show a very different
outcome, whereby market-power plays a crucial role. A well-functioning electricity
market requires that no agent is able to significantly alter the spot price by the
bidding behavior. Only if this requirement is fulfilled the storage agent can help to
decrease the peak/off-peak spread, lower the spot price volatility and allow to use
the power plants more efficiently.
The four country model consisting of the countries Germany, Switzerland, France
and Italy with an implicit cross-border allocation scheme which maximizes the total
social surplus shows that the mean spot price level in France highly increases while
the mean price level in Germany decreases by introducing electricity exchanges.
The price level in Italy does not change as much although the mean price level is
high in Italy. The reason is that the aggregated supply curve is very elastic in Italy
due to a high amount of elastic bidders. Therefore, changing the Italian demand by
the cross-border capacity allocation routine results in a much lower price deviation
in Italy.
The price predictions of the agents show a high variation in estimation accura-
cies. The agents in Germany have the highest spot price prediction ability and the
Italian the lowest. The price predictor also shows that the agents take exogenous
variables of other countries into account in making price predictions. The nuclear
agents in all countries usually have the highest market-power whereby the French
nuclear agent has the highest market-power due to the high installed nuclear ca-
pacity. In Italy, no agent has significant market-power. Switzerland has overall the
highest profitable electricity generation measured by a per installed MW basis, the
Italian generation is not profitable at all. The high storable amount of electricity
in Switzerland allows to use the nuclear power plants very efficiently.
The exchanges show that France is selling a high fraction of its production to
Italy either directly or by using Switzerland as a transit country. The exchanges
from France to Italy, from France to Switzerland and from Switzerland to Italy
as a result often reach the maximum NTC values. All interconnection lines gen-
erate high congestion revenues whereby the line connecting Italy and Switzerland
generates the highest revenues. Overall, France is a high exporter of electricity
while Italy and Germany are net importers. Switzerland is more or less balanced.
Switzerland shows by far the highest imports/exports per domestic load due to the
135
Chapter 8. Conclusion
high interconnection capacities and a small domestic load.
Increasing the interconnection lines between Switzerland and Italy and France
and Italy by 30% does not decrease the mean spot price level in Italy and increases
the Italian spot price standard deviation. Instead, it decreases the French mean spot
price level and price volatility. The reason is that with an increased interconnection
capacity the French electricity market gains access to more elastic and storage
bidders which can store excess electricity and provide backup-power. This allows
to use the nuclear power plants in France more efficiently.
The mean spot price level in all four countries can be significantly decreased
if the interconnection lines are increased in order that no congestions occur. The
profitability of the power producing agents is thereby eroded and their market-
power reduced. It can therefore be concluded that the power generators do not
have an incentive to invest in additional interconnection capacities. The congestion
revenue for the TSOs diminishes which implies that the TSOs have no incentive
either. But building these capacities is important because it drives the spot price to
more competitive levels, allows to use the power plants more efficiently, increases the
total social welfare over all four countries, decreases the total costs for consumers
and increases the security of supply.
136
List of Figures
3.1 Supply curve for the inelastic bidders . . . . . . . . . . . . . . . . . . 22
3.2 Supply curve for the elastic bidders . . . . . . . . . . . . . . . . . . . 23
3.3 Supply curve for the storage bidders . . . . . . . . . . . . . . . . . . 23
3.4 Market clearing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.1 Flow chart of the learning methodology . . . . . . . . . . . . . . . . 31
4.2 Price fraction with averaged historical data . . . . . . . . . . . . . . 33
4.3 Temperature Fourier series approximation, z = 2 (year 2006) . . . . 37
4.4 Temperature Fourier series approximation, z = 15 (year 2006) . . . . 38
4.5 Temperature approximation for out-of-sample data, z = 15 . . . . . 39
4.6 Simulated and regressed German spot prices (year 2006) . . . . . . . 41
4.7 Daily variations in German spot prices (year 2006) . . . . . . . . . . 42
4.8 Out-of-sample predictions of German spot prices (year 2007) . . . . 44
4.9 Daily variation of predicted German spot prices (year 2007) . . . . . 45
4.10 Daily variation of spot prices with a poor fit (year 2007) . . . . . . . 46
4.11 HPFC for three days . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.12 Future prices depending on actions after using the price adjuster . . 48
4.13 Construction of the state matrix . . . . . . . . . . . . . . . . . . . . 52
4.14 Price and state matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.15 Backward induction . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.16 Example of ramping constraints . . . . . . . . . . . . . . . . . . . . . 55
137
List of Figures
4.17 Backward induction . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.18 Forward induction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.1 Probability of choosing a random action . . . . . . . . . . . . . . . . 64
5.2 Simulated base load prices test scenario, training round . . . . . . . 65
5.3 Mean power outputs test scenario, training run . . . . . . . . . . . . 66
5.4 pav values test scenario, training run . . . . . . . . . . . . . . . . . . 67
5.5 Simulated base load price test scenario, second trial . . . . . . . . . 68
5.6 Mean power outputs test scenario, second trial . . . . . . . . . . . . 69
5.7 Estimation error distribution test scenario, second trial . . . . . . . . 70
6.1 Base load prices reference scenario, training run . . . . . . . . . . . . 73
6.2 Estimation error reference scenario, training run . . . . . . . . . . . 74
6.3 Mean power outputs reference scenario, training run . . . . . . . . . 75
6.4 Mean daily profits reference scenario, training run . . . . . . . . . . 76
6.5 pav values reference scenario, training run . . . . . . . . . . . . . . . 76
6.6 Base load prices reference scenario, second run . . . . . . . . . . . . 77
6.7 Cumulative price distributions reference scenario, second run . . . . 78
6.8 Mean power outputs reference scenario, second run . . . . . . . . . . 79
6.9 Mean daily profits reference scenario, second run . . . . . . . . . . . 80
6.10 Histograms of the estimation errors reference scenario . . . . . . . . 81
6.11 Base load prices reference scenario, second and third run . . . . . . . 82
6.12 Cumulative price distributions reference scenario, third run . . . . . 83
6.13 Mean power outputs reference scenario, third run . . . . . . . . . . . 83
6.14 Mean power outputs nuclear replacement scenario . . . . . . . . . . 88
6.15 Histograms estimation error for the wind scenarios . . . . . . . . . . 90
6.16 PAV values for the lignite agent and different wind scenarios . . . . 91
6.17 PAV values for the gas agent and different wind scenarios . . . . . . 92
138
List of Figures
6.18 Base load prices storage and reference scenarios . . . . . . . . . . . . 94
6.19 Base load price high energy scenario . . . . . . . . . . . . . . . . . . 95
6.20 Power outputs high energy scenario . . . . . . . . . . . . . . . . . . . 96
6.21 Power outputs and base load price high energy scenario . . . . . . . 97
6.22 Daily profit, high energy scenario . . . . . . . . . . . . . . . . . . . . 98
6.23 Price estimation error, high energy scenario . . . . . . . . . . . . . . 99
6.24 Histograms estimation error for the storage scenarios . . . . . . . . . 100
6.25 Prediction adjustment values . . . . . . . . . . . . . . . . . . . . . . 101
6.26 Fraction of PHEVs connected to the grid . . . . . . . . . . . . . . . 102
6.27 Base load prices PHEV and reference scenarios . . . . . . . . . . . . 103
6.28 Power outputs nuclear agent reference and PHEV scenarios . . . . . 104
7.1 Calculation of the social welfare . . . . . . . . . . . . . . . . . . . . . 108
7.2 Calculation of total production cost . . . . . . . . . . . . . . . . . . 110
7.3 Demand is increased in country 1 . . . . . . . . . . . . . . . . . . . . 111
7.4 Demand is reduced in country 2 . . . . . . . . . . . . . . . . . . . . . 111
′
7.5 Graphical interpretation of Dt,c . . . . . . . . . . . . . . . . . . . . . 113
7.6 Trading partners and NTC values . . . . . . . . . . . . . . . . . . . . 114
7.7 Simulated base load prices reference scenario . . . . . . . . . . . . . 117
7.8 Power outputs reference scenario, Germany . . . . . . . . . . . . . . 118
7.9 Power outputs reference scenario, Switzerland . . . . . . . . . . . . . 119
7.10 Power outputs reference scenario, France . . . . . . . . . . . . . . . . 120
7.11 Power outputs reference scenario, Italy . . . . . . . . . . . . . . . . . 121
7.12 Electricity exchanges reference scenario . . . . . . . . . . . . . . . . 122
7.13 Net position of each country reference scenario . . . . . . . . . . . . 123
7.14 Exports/imports as percentage of domestic load reference scenario . 124
7.15 Histograms of the estimation errors reference scenario . . . . . . . . 125
7.16 Simulated base load prices Germany no NTC scenario . . . . . . . . 128
139
List of Figures
7.17 Electricity exchanges no NTC scenario . . . . . . . . . . . . . . . . . 129
7.18 Net position of each country no NTC scenario . . . . . . . . . . . . . 130
7.19 Histograms of the estimation errors no NTC scenario . . . . . . . . . 132
140
List of Tables
1.1 Day-ahead allocation method, source [9] . . . . . . . . . . . . . . . . 7
1.2 long-term allocation method, source [9] . . . . . . . . . . . . . . . . . 7
3.1 Installed capacities for every agent, source EIA . . . . . . . . . . . . 19
3.2 Generator specific parameters of the cost model . . . . . . . . . . . . 25
6.1 Installed capacities for the German agents, source EIA . . . . . . . . 71
6.2 Simulation summary, training run . . . . . . . . . . . . . . . . . . . 74
6.3 Simulation summary, second run . . . . . . . . . . . . . . . . . . . . 80
6.4 Simulation summary, third run . . . . . . . . . . . . . . . . . . . . . 81
6.5 Yearly profits of the simulation trials . . . . . . . . . . . . . . . . . . 84
6.6 Standard deviations of the yearly profits . . . . . . . . . . . . . . . . 84
6.7 Summary of the price predictions . . . . . . . . . . . . . . . . . . . . 85
6.8 Summary mean spot prices wind scenarios . . . . . . . . . . . . . . . 86
6.9 Summary profits wind scenarios . . . . . . . . . . . . . . . . . . . . . 89
6.10 Summary spot price storage scenarios . . . . . . . . . . . . . . . . . 94
6.11 Summary profits storage scenarios . . . . . . . . . . . . . . . . . . . 98
6.12 Summary PHEV scenarios . . . . . . . . . . . . . . . . . . . . . . . . 104
6.13 Summary profits PHEV scenarios . . . . . . . . . . . . . . . . . . . . 105
7.1 Variables of the optimization routine . . . . . . . . . . . . . . . . . . 112
7.2 NTC values, source ENTO-E . . . . . . . . . . . . . . . . . . . . . . 114
141
List of Tables
7.3 Summary spot prices reference scenario . . . . . . . . . . . . . . . . 117
7.4 Profits and specific profits Germany, reference scenario . . . . . . . . 119
7.5 Profits and specific profits Switzerland, reference scenario . . . . . . 119
7.6 Profits and specific profits France, reference scenario . . . . . . . . . 120
7.7 Profits and specific profits Italy, reference scenario . . . . . . . . . . 121
7.8 Summary congestion revenues reference scenario . . . . . . . . . . . 123
7.9 Summary spot prices Germany transmission line scenarios . . . . . . 127
7.10 Summary spot prices Switzerland transmission line scenarios . . . . 127
7.11 Summary spot prices France transmission line scenarios . . . . . . . 127
7.12 Summary spot prices Italy transmission line scenarios . . . . . . . . 128
7.13 Profits and specific profits Germany, no NTC scenario . . . . . . . . 130
7.14 Profits and specific profits Switzerland, no NTC scenario . . . . . . . 130
7.15 Profits and specific profits France, no NTC scenario . . . . . . . . . 131
7.16 Profits and specific profits Italy, no NTC scenario . . . . . . . . . . . 131
142
Bibliography
[1] T. Krause, “Evaluating congestion schemes in liberalized electricity markets ap-

plying agent-based computational economics,” Ph.D. dissertation, Swiss Fed-
eral Institute of Technology Zurich, 2007.
[2] R. Bacher, Optimization of liberalized electrical energy systems. Department

Informationstechnologie und Elektrotechnik (ITET), Vorlesungsnummer ETH
Zurich 227-0529-00, May 2009.
[3] F. H. Boisseleau and P. Dauphine, “The role of electricity trading and power
exchanges for the construction of a common european electricity market,” in
Power System Management and Control, vol. 488, April 2002.
[4] European Parliament and Council of the European Union, “Regulation (ec) no
714/2009 of the european parliament and of the council on conditions for access
to the network for cross-border exchanges in electricity,” L211, July 2009.
[5] O. Daxhelet and Y. Smeers, “The eu regulation on cross-boarder trade of elec-

tricity: A two-stage equilibrium model,” European Journal of Operational Re-
search, no. 181, pp. 1396–1412, 2007.
[6] European Parliament and Council of the European Union, “Regulation (ec)
no 1228/2003 of the european parliament and of the council on conditions for
access to the network for cross-boarder exchanges in electricity,” L176, June
2003.
[7] L. E. Ruff, “Stop wheeling and start dealing: Resolving the transmission
dilemma,” in Electricity Transmission Pricing and Technology. Boston:
Kluwer Academics Publisher Group, 1996.
[8] Consentec, Frontier Economics Limited, “Analysis of cross-border congestion

management methods for the eu internal electricity market,” Study commis-
sioned by the European Commission Directorate, June 2004.
[9] ETSO (Europen Transmission System Operators), “An overview of current

cross-border congestion management methods in europe,” pp. 1–31, May 2006.
[10] Apx, Powernext, Belpex. (2006, March) Trilateral market coupling algorithmic
appendix. [Online]. Available: http://www.belpex.be/index.php?id=4
[11] T. Schwarz and A. Szarkmy, “A price discovery in petroleum markets: arbi-

trage, cointegration, and the timing interval of analysis,” Journal of Futures
Markets, vol. 14, pp. 147–167, 1994.
143
Bibliography
[12] J. Bower, D. W. Bunn, and C. Wattendrup, “A model-based analysis of strate-

gic consolidation in the german electricity industry,” Energy Policy 29, pp.
987–1005, 2001.
[13] C. Growitsch and R. Nepal, “Efficiency of the german electricity wholesale mar-
ket,” European transactions on electrical power, vol. 19, pp. 553–568, February
2009.
[14] D. Veit, A. Weidlich, and J. A. Krafft, “An agent-based analysis of the ger-
man electricity market with transmission capacity constraints,” Energy Policy,
vol. 37, pp. 4132–4144, 2009.
[15] B. F. Hobbs, C. B. Metzler, and J. S. Pang, “Strategic gaming analysis for
electric power systems: an mpec approach,” IEEE Transactions on Power
Systems, vol. 15, no. 2, pp. 638–645, May 2000.
[16] W. D. Nordhaus, “The ecology of markets,” Proceedings of the National
Academy of Sciences, vol. 89, pp. 843–850, February 1992.
[17] R. L. Axtell, C. J. Andrews, and M. J. Small. (2003, Octo-
ber) Agent-based models of industrial ecosystems. [Online]. Available:
http://policy.rutgers.edu/andrews/projects/abm/abmarticle.htm
[18] E. Bonabeau, “Agent-based modeling: Methods and techniques for simulating
human systems,” in National Academy of Science, May 2002.
[19] M. Wooldridge and N. Jennings, “Intelligent agents: Theory and practice,”
Knowledge Engineering Review, vol. 10, no. 2, 1995.
[20] T. Krause and G. Andersson, “Evaluating congestion management schemes in
liberalized electricity markets using an agent-based simulator,” Presented at
the IEEE PES General Meeting, Montreal, pp. 377–384, 2006.
[21] D. W. Bunn and F. S. Oliveira, “Agent-based simulation - an application to the
new electricity trading arrangements of england and wales,” IEEE transactions
on evolutionary computation, vol. 5, no. 5, pp. 493–503, October 2001.
[22] D. M. Newbery, “Power markets and market power,” Energy J, vol. 16, pp.
39–66, 1995.
[23] A. Weidlich and D. Veit, “A critical survey of agent-based wholesale electricity
market models,” Energy Economics, vol. 30, pp. 1728–1759, 2008.
[24] J. Bower and D. Bunn, “Model-based comparison of pool and bilateral markets
for electricity,” Energy Journal, vol. 21, no. 3, pp. 1–21, 2000.
[25] J. Bower and D. Bunn, “Experimental analysis of the efficiency of uniform-price
versus discriminatory auctions in the england and wales electricity market,”
Journal of Economic Dynamics and Control, vol. 25, pp. 561–592, 2001.
[26] J. Bower, D. Bunn, and C. Wattendrup, “A model-based analysis of strategic
consolidation in the german electricity industry,” Energy Policy, vol. 29, pp.
987–1005, 2001.
[27] D. Bunn and F. S. Oliveira, “Agent-based simulation - an application to the
new electricity trading arrangements of england and wales,” IEEE Transactions
on Evolutionary Computation, vol. 5, no. 5, pp. 493–503, 2001.
[28] P. Visudhiphan, “An agent-based approach to modeling electricity spot mar-
kets,” Ph.D. dissertation, Massachusetts Institute of Technology, 2003.
144
Bibliography
[29] J. Nicolaisen, M. Smith, V. Petrov, and L. Tesfatsion, “Concentration and

capacity effects on electricity market power,” Proceeding of the 2000 Congress
on Evolutionary Computation, La Jolla, USA, pp. 1041–1047, 2000.
[30] T. D. H. Cau and E. J. Anderson, “A co-evolutionary approach to modelling

the behavior of participants in competitive electricity markets,” IEEE Power
Engineering Society Summer Meeting, vol. 3, pp. 1534–1540, 2002.
[31] I. Erev and A. E. Roth, “Predicting how people play games: Reinforcement
learning in experimental games with unique, mixed strategy equilibria,” The
American Economic Review, vol. 88, no. 4, pp. 848–881, 1998.
[32] C. J. C. H. Watkins, “Learning from delayed rewards,” Ph.D. dissertation,

Cambridge Universtiy, 1989.
[33] A. G. Bakirtzis and A. C. Tellidou, “Agent-based simulation of power mar-

kets under uniform and pay-as-bid pricing rules using reinforcement learning,”
Proceeding of the IEEE Power System Conference and Exposition, 2006.
[34] A. Bernauer. (2007) Das learning classifier system xcs. University of Tuebingen.
[Online]. Available: http://www-ti.informatik.uni-tuebingen.de/∼bernauer/
[35] A. J. Bagnall and G. D. Smith, “A muliagent model of the uk market in elec-

tricity generation,” IEEE Transactions on Evolutionary Computation, vol. 9,
no. 5, pp. 522–536, 2005.
[36] M. V. Butz and S. W. Wilson, “An algorithmic description of xcs,” Soft Com-
puting, vol. 6, pp. 144–153, 2002.
[37] S. W. Wilson, “Classifier fitness based on accuracy,” Evolutionary Computa-

tion, vol. 3, no. 2, pp. 149–175, 1995.
[38] A. M. G. Klein Tank and Coauthors, “Daily dataset of 20th-century surface

air temperature and precipitation series for the european climate assessment,”
Int. J. of Climatol., vol. 22, pp. 1441–1453, 2002.
[39] F. Müsgen, “Market power in the german wholesale electricity market,” EWI
Working Paper, no. 04.03, pp. 1–29, 2004.
[40] F. Müsgens, “Quantifying market power in the german wholesale electricity

market using a dynamic multi-regional dispatch model,” The Journal of In-
dustrial Economics, pp. 471–498, December 2006.
[41] L. A. Wehinger, M. D. Galus, and G. Andersson, “Agent-based simulator for

the german electricity wholesale market including wind power generation and
widescale phev adoption,” EEM 10, European Energy Markets, June 2010.
[42] J. H. Holland, L. B. Booker, M. Colombetti, M. Dorigo, D. E. Goldberg, S. For-

rest, R. L. Riolo, R. E. Smith, P. L. Lanzi, W. Stolzmann, and S. W. Wilson,
What Is a Learning Classifier System? Springer Berlin / Heidelberg, 2000.
[43] D. Helm and P. Andrew, “Pool prices, contracts and regulation in the british
electricity supply industry,” Fiscal Studies, vol. 13, no. 1, pp. 89–105, 1992.
[44] L. Blöchlinger, “Power prices - a regime-switching spot/forward price model

with kim filter estimation,” Ph.D. dissertation, 2008.
[45] F. Hayashi, Econometrics. Princeton University Press, 2000.
145
Bibliography
[46] B. D. Storey. Computing fourier series and power spectrum with matlab.
[Online]. Available: http://faculty.olin.edu/bstorey/Notes/Fourier.pdf
[47] E. L. Lehmann and G. Casella, Theory of Point Estiamtion. Springer, 1998.
[48] D. P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed. Athena
Scientific, 2005, vol. 1.
[49] R. Bellman, Dynamic Programming. Princeton University Press, 1957.
[50] W. F. Sharpe, “The sharpe ratio,” Journal of Portfolio Management, pp. 49–
58, 1994.
[51] B. Morrissey, Derivatives: The Official Learning and Reference Manual. Lon-
don, United Kingdom: The Securities and Investment Institute, 2009.
[52] L. Butler and K. Neuhoff, “Comparison of feed-in tariff, quota and auction
mechanisms to support wind power development,” Renewable Energy, vol. 33,
pp. 1854–1867, 2008.
[53] P. D. Rio and M. Gual, “An integrated assessment of the feed-in tariff system
in spain,” Energy Policy, vol. 35, pp. 994–1012, 2007.
[54] C. Hiroux and M. Saguan, “Large-scale wind power in european electricity

markets: Time for revisiting support schemes and market designs?” Energy
Policy, vol. 38, pp. 3135–3145, 2010.
[55] Council of European Energy Regulators ASBL, “The green package - proposed
eu ets and renewables directives,” Ref: C08-SDE-02-06, pp. 2–11, May 2008.
[56] D. N. Joanes and C. A. Gill, “Comparing measures of sample skewness and

kurtosis,” Journal of the Royal Statistical Society (Series D): The Statistician,
vol. 47, no. 1, pp. 183–189, 1998.
[57] Vattenfall Europe Generation AG and Co. KG. (2007) Pumpspeicher-kraftwerk

goldisthal. [Online]. Available: www.vattenfall.de
[58] J. Tomic and W. Kempton, “Using fleets of electric-driven vehicles for grid
support,” Journal of Power Sources, vol. 168, no. 2, pp. 459–469, 2007.
[59] M. D. Galus and G. Andersson, “Demand management of grid connected plug-

in hybrid electric vehicles (phev),” 2008 IEEE Energy 2030 Conference, p. 8
pp., 2008.
[60] M. D. Galus and G. Andersson, “Integration of plug-in hybrid electric vehicles

into energy networks,” in 2009 IEEE Bucharest PowerTech (POWERTECH),
2009, p. 8 pp.
[61] W. Kempton and S. Letendre, “Electric vehicles as a new power source for
electric utilities,” Trans. Res., vol. 2, 1997.
[62] D. Ernst, A. Minoia, and M. Ilic, “Market dynamics driven by the decision-
making power producers,” Proceedings of 2004 IREP Symposium - Bulk Power
System Dynamics and Control - VI, Cortina D’Ampezzo, Italy, 2004.
[63] S. Borenstein, J. Bushnell, and S. Stoft, “The competitive effects of transmis-

sion capacity in a deregulated electricity industry,” RAND Journal of Eco-
nomics, vol. 31, no. 2, pp. 294–325, 2000.
146
Bibliography
[64] M. Noel, “Edgeworth price cycles, cost-based pricing and sticky pricing in retail
gasoline markets,” University of California at San Diego, Economics Working
Paper Series, 2004.
[65] A. Ulbig, M. D. Galus, S. Chatzivasileiadis, and G. Andersson, “General fre-

quency control with aggregated control reserve capacity from time-varying
sources: The case of phevs,” Presented at the 2010 IREP Symposium, Bulk
Power System Dynamics and Control VIII (IREP), 2010.
[66] G. Hug and G. Andersson, “Using facts devices to resolve congestions in trans-
mission grids,” Presented at the CIGRE/IEEE PES International Symposium,
San Antonio, USA, 2005.
[67] FACTS Terms and Definitions Task Force, “Proposed terms and definitions for
flexible ac transmission systems (facts),” IEEE Transactions on Power Deliev-
ery, vol. 12, no. 4, pp. 1848–1853, 1997.
147

MA Wehinger

Enviado por

Dados do documento

Descrição original:

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

MA Wehinger

Enviado por

Direitos autorais:

Formatos disponíveis

Master thesis

Agent-based modeling in electricity markets:

Department ETH: EEH - Power Systems Laboratory

Spring semester 2010

Pittsburgh, September 28, 2010 Lukas Wehinger

2 Agent-based modeling in electricity markets 11

3 Specification of the model 18

4 Learning-process of the agents: The methodology 28

5 Testing the agent-based model 63

6 Model application: The German electricity market 71

7 Model application: The four-country model 106

c∈C Country c in C, where C = {Germany, Switzerland, France, Italy}

Acronyms and Abbreviations

OTC Over the counter

1.1 The European electricity market

Cross-border acquisitions have surged as national companies try to diversify

1.1.1 Goals of the European Union

• Establish of a competitive electricity market.

• Increase the efficiency in generation, transmission and distribution of electric-

• Increase the competition by liberalize markets for electricity generation and

• Implement the Third-Party-Access (TPA) for industrial- and large-scale con-

1.1.2 Implementation of the electricity market liberalization

“The creation of a real internal electricity market should be promoted

Furthermore, clause no. 4 in regulation No 1228/2003 states that [6]:

“Fair, cost-reflective, transparent and directly applicable rules ... should

The regulation No 1228/2003 put a special emphasis on increasing competition

It is important to understand the underlying technical characteristics of the

An increase in cross-border transactions and cross-border capacities helps devel-

“Network congestion problems shall be addressed with non-discriminatory

Regulation No 1228/2003 specifies that congestion problems shall be addressed

• Guaranteeing the actual availability of the allocated capacity;

Since 2009, regulation No 1228/2003 is repealed by the newer regulation No

mentions that regulation No 1228/2003 made significant contributions towards the

“However, at the present, there are obstacles to the sale of electricity on

This article attributes the source shortcomings in implementing the internal

“In particular, increased cooperation and coordination among the trans-

1.2 Congestion management

In the following different capacity allocation methods are introduced. Examples of

1.2.1 Capacity allocation methods

• Explicit auctions: In explicit auctions the seller determines available trans-

• Implicit auctions: In implicit auctions an initial energy market clearing is

– requiring financial instruments for long term price-hedging and bilateral

The regulation No 1228/2003 and its repealing regulation No 714/2009 clearly

1.2.2 Implemented congestion management schemes and out-

In the following, other implemented energy and cross-border allocations schemes

The markets Belpex (Belgium), Apx (Netherlands) and Powernext (France)

Cross-border line Allocation method

Table 1.1: Day-ahead allocation method, source [9]

Cross-border line Allocation method

Table 1.2: long-term allocation method, source [9]

There is a development toward a centralized European electricity market in Eu-

Market-power is understood as the ability of the market participants to deviate

1.3.1 Market-power in electricity generation

The consolidation in electricity generation in Europe has lead to a high con-

1.3.2 Market power and congestions

Additionally, under explicit auctions, market participants know the outcome of

ward as it is in explicit auctions [8].

1.4 Contribution of the thesis

• Setting up a multi-agent model for the four countries.

• Implement an implicit cross-border allocation scheme and analyze the price

• Assess the effect of a higher wind energy contribution in Germany.

• Analyze the price outcome in Germany by introducing a higher electricity

• Introduce an PHEV (plug-in hybrid electric vehicle) cluster which is charged/discharged