|


 

 




   



 

Gavin Fitzpatrick
School of Computing
Dublin City University
Dublin, Ireland
 ðirtualization Concepts
 Type 1 Hypervisor used
 Type 2 Hypervisor used
 Testing tools
 Experiments
 Results
 Conclusions
 ðirtualization Concepts
 x86 architectures are designed based on 4 rings of privilege:
Ring 3: executes user mode - has no direct access to the underling hardware
Ring 2: not used by modern operating systems.
Ring 1: not used by modern operating systems.
Ring 0: has full access to underlying hardware within the host system

 Popek & Goldberg define an x86 ðirtual Machine Monitor(ðMM) with the
following characteristics:
Fidelity: ðMM must provide computing environment identical to a physical machine
Performance: Programs should only have minimum performance impact when using a
ðMM
Safety: The ðMM must have complete control of the system resources

Ref: Popek & Goldberg ʹ Formal requirements for ðirtualizable 3rd Generation Architectures
 ðirtualization Isolation
 As discussed in *ðirtual Doppelganger paper,
Isolation within virtualization can be defined
under two different dimensions:
Resource Isolation
Namespace Isolation

 *Ref: http://www.cs.princeton.edu/~mef/research/paenevirtualization.pdf
 Namespace Isolation
 Namespace Isolation:
States how a ðMM limits access to it͛s file-system,
processes, memory addresses, user ids etc.
2 aspect:
1. Configuration Independence: File names of
one ðM do not conflict with that of another
ðM.
All hypervisors tested were unable to use the
same name for vm͛s or their associated config/
virtual disk files (vmdk, vdi, vhd)
 Namespace Isolation
2. Security:
One ðM cannot modify data belonging to
another ðM stored in the same host.
Within modern enterprise level environments
ðM͛s are stored on iSCSI or Fibre-channel
networks which are inaccessible from ðM͛s
 Resource Isolation
 Refers to a ðMM͛s ability to isolate resource
consumption of one ðM from that of another
ðM using appropriate algorithms:

 This Presentation looks at how resource

isolation is affected by ðM͛s misbehaving.
 Hypervisor (ðMM)

 Type 1: ESXi, XEN Server, Hyper-ð

 Type 2: ðMWare Workstation, Oracle ðirtualBox
 Containers: ðirtual code that runs as an application,
 Allows multiple encapsulated isolated instances which point to the underlying O.S
on which its executed

 *diagrams from: http://www-01.ibm.com/redbooks/community/display/REDP4480/ðirtualization+Strategies+Architectural+Overview

ESXi
XENServer
Hyper-ð
×ðM
ðirtualbox
Workstation
 Testing tools
 Commercial benchmarking tools include
ðMark , Passmark and
 All tests executed on ðM4 (Ubuntu Guest)
 Ramspeed
 Systester
 Geekbench
 FIO
 Ping testing ( Look at skipping?!)
 Ramspeed
 Used to test memory performance with the following operations:
Copy (A=B)
Scale (A=m*B)
Add (A = B+C)
Triad (A=m*B+C)
 2 tests are performed for Integers and Floating Point Numbers
 10 rounds are performed for each test and results are averaged
 Systester
 Used to benchmark CPU performance by
calculating 512,000 digits of Pi using 2 algorithms:
Borwein Quadradic Covergance: Runs for 5 rounds
Gausse-Legendre: Runs for 10 rounds
 Geekbench
 Propreitary benchmarking tool used for memory & cpu
performance
 Scores tested on following factors:
1. Integer Calculations (Blowfish, Text Compress/Decompress)
2. Floating Point calculations (Primality test, Dot Product)
3. Memory operations (Read/Write Sequential, Stdlib Copy/Write)
4. Stream operations similar to Ramspeed( Copy, Scale, Add, Triad ʹ
(similar to ramspeed tests)
 FIO
 I/O benchmark tool used to test disk subsystem
within Linux O.S with libaio library
 Test performed:
Random write: 10 x 32mb files written
 Max average bandwidth recorded over 10
reads/writes
 Ping testing
 Tests Network I/O within each hypervisor, ping
tests run from ðM4 within the host to
Gateway
Host IP (Physical IP for host)
ðM2 (Win2003 server ʹ DoS victim during Exp4a/b)
Testing Script
 Experiments
 Testing Environment
 Crashme ʹ O.S stress test
 Fuzz ʹ Application stress test
 Forkbomb ʹ Memory stress test
 DoS ʹ I/O stress test
Exp1 - Crashme
Exp1 ʹ ðM1 CPU/MEM activity
 Crashme - Observations
 Exp1:
 CPU: 100% usage
 MEM usage: 75%

 PRNGs used:
Mersenne Twist (MT) ʹ Common PRNG tool
ðNSQ (ðariation of Middle Sq. Method) Take any No. Sq it, and take middle digits
Rand() uses c++ library
 ðirtualbox- in non ðT-x mode:
Running Exp1 on ðirtualbox in non ðT-x mode causes the O.S to hang using (MT) method
Both ðNSQ and Rand() cause O.S to restart
 Exp2 - Fuzz

 Exp2:
 CPU: 88% + usage
 MEM: 30%+ usage
Exp3 - Forkbomb
 Exp3 - Forkbomb
 Forkbomb code ʹ Linux (ðM3): defined in a bash script as:

 Forkbomb code ʹ Windows (ðM1,ðM2): defined in batch as :

%0|%0

 REF: http://www.cyberciti.biz/faq/understanding-bash-fork-bomb/
Exp3a-c ʹ Host MEM activity
 Exp4 - DoS

 Exp4a:
 CPU: 100% usage
 NIC transmit/recieve: 10,000×Bps / 5,500×Bps
 Exp4b:
 CPU: 25% usage
 NIC transmit/ recieve: 0 / 13,500×Bps
 Results
 Illustrated on a test by test basis for all
experiments for the following:
 ESXi
 XEN
 Hyper-ð
 Workstation
 ðirtualbox / ðirtualbox nonðTx
 ESXi - Memory

 Ramspeed:
Y-axis in MB per
sec(higher score
=better result)
 Geekbench:
Y-axis is score (higher
score=better result)
 ESXi - CPU

 Systester:
Y-axis in time (lower ʹ
time better result)
 Geekbench:
Y-axis is score (higher
score=better result)
 ESXi ʹ HDD/LAN
1200

1.4
1000
1.2
800 1
0.8
600 ESX ×B/s
0.6 ESX

400 Avg Write per Exp 0.4

Ping Gateway -
×B/s 0.2 Secs
200
0

5
4.5
0.45 4
0.4 3.5
0.35 3
0.3
2.5
0.25
0.2 2 ESX
0.15 ESX 1.5
Ping ðM - Secs
0.1 1
0.05 Ping Host - Secs
0.5
0 0
 ESXi - Summary
 Memory:
Geekbench: 2.2% better than average
Ramspeed: 2.5% better than average
 CPU:
5% better than average
 Disk:
18% below average (especially in Exp3c)
 Network:
Host: 55% above average
ðM: 22% above average
GW: 19% above average
XEN-Memory
XEN-CPU
 XEN-Disk/Network
1400
1.4
1200
1.2
1000
1
800
0.8
600 XenServer
0.6 XenServer
400
0.4
200 Avg Write per Exp
0.2 Ping Gateway -
0 ×B/s
Secs
0

0.5
6
0.45
0.4 5
0.35
4
0.3
0.25 3
0.2 XenServer XenServer
0.15 2
Ping Host - Secs Ping ðM - Secs
0.1
1
0.05
0 0
 XEN - Summary
 Memory:
Geekbench: Follows average apart from Exp3c
Ramspeed: 4.5% below average ʹ (Exp3c, Exp4b major
factors)
 CPU:
3% better than average
 Disk:
41% greater performance than average
 Network:
Host: 20.3% above average (Exp4a performs badly)
ðM: 31% below average
GW: 16.4% above average
Hyper-ð - Memory
Hyper-ð - CPU
 Hyper-ð ʹ HDD/LAN
3.5
3
2.5
2
1.5
1 Hyper-ð

0.5 Ping Gateway - Secs

6
0.9
5
0.8
0.7 4
0.6
3
0.5
0.4 2 Hyper-ð
0.3 Hyper-ð 1 Ping ðM - Secs
0.2
Ping Host - Secs
0.1 0
0
 Hyper-ð Summary
 Memory:
Geekbench: Exp1,2,3a score below average, Exp3b-4b score
above average
Ramspeed: 3.4% below average (Exp3c-4b main cause)
 CPU:
2.5% below average (resulting from Borwein tests)
Gauss test follows average
 Disk:
18% above average (Exp3b, 3c show major loss in performance)
 Network:
Host: 81% below average
ðM: 31% below average
GW: 4.5% below average
Workstation - MEM
Workstation - CPU
 Workstation ʹ HDD/LAN
1200
3.5
1000
3
800 2.5

600 2
Workstation ×B/s 1.5
400
Avg Write per Exp ×B/s 1 Hyper-ð
200 Ping Gateway - Secs
0.5
0 0

0.45 6
0.4
5
0.35
0.3 4
0.25
3
0.2
0.15 Workstation Workstation
2
0.1 Ping Host - Secs Ping ðM - Secs
1
0.05
0 0
 Workstation - Summary
 Memory:
Geekbench: 1.1% below average (Exp3c-4b main cause)
Ramspeed: Integer tests 3.3% abover average, Floating
Point tests 6% above average
 CPU:
1.2 % below score across all experiments
 Disk:
19% below average ʹ keeping in line with average trend
 Network:
Host: 5.1% better than average
ðM: 11.4% better than average
GW: 10% better than average
ðirtualbox - Memory
ðirtualbox - CPU
 ðirtualbox ʹ HDD/LAN
1600
1.6
1400 1.4
1200 1.2
1000 1
800 0.8
600 ðirtualbox ×B/s 0.6
ðirtualbox
400 Avg Write per Exp ×B/s 0.4
Ping Gateway - Secs
200 0.2
0 0

6
0.6
5
0.5

0.4 4

0.3 3

0.2 2 ðirtualbox
ðirtualbox
1 Ping ðM - Secs
0.1 Ping Host - Secs

0 0
 ðirtualbox - Summary
 Memory:
Geekbench: 1.1% below average (Exp3c-4b main cause)
Ramspeed: Integer tests 3.3% above average, Floating
Point tests 6% above average
 CPU:
1.2 % below score across all experiments
 Disk:
19% below average ʹ keeping in line with average trend
 Network:
Host: 15% below average across all experiments
ðM: 5% below average across all experiments
GW: 6.1% below average across all experiments
 Conclusions
 Type 1 Baremetal (ESXi)
 Outperforms all hypervisors on:
Network (utilizes NIOC)
CPU/MEM ( CPU Scheduler / Shadow Page tables)
 Performs poorly for:
Disk access (SIOC doesnt enforce isolation)
 Conclusions
 Type 1 Para ðirtualization (XEN & Hyper-ð)

 Disk I/O performs well on both platforms

Due to ability of Guests to utilize the Domain0 hardware
device drivers
 Network I/O per poorly
XEN & Hyper-ð both perform poorly for Network I/O
isolation
 Mixed CPU/MEM results:
XEN: Average Memory performance, better than average
CPU performance
Hyper-ð Poor CPU/MEM performance
 Conclusions
 Type 2 Hosted (ðirtualbox & Workstation)

 Disk I/O performs poorly on both platforms

 Network I/O mixed results:

ðirtualbox ʹ performs poorly across Network tests
Workstation performs well across Network tests
 CPU performs poorly:
Both platforms report lower than average CPU results, this is due to
Host CPU treating each Guest as a separate process, which has the
same CPU time slice allocation as other Host Context Ring 3 processes
 Memory performs well:
Both platforms perform well for memory access, ðMM allocate
physical memory directly to each Guest, host in unaware of this.
End of Presentation