Architecting

a vCloud
Version 1.0
TEC H N I C A L W H ITE PA P E R
 Architecting a vCloud

Table of Contents

1. What is a VMware vCloud?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.1 Document Purpose and Assumptions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Cloud Computing and vCloud Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
1.3 vCloud Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2. Assembling a vCloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 vCloud Logical Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 vCloud Management Cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3 vCloud Resource Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3. Creating Services with vCloud Director. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.1 vCloud Director Constructs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.2 Establish Provider Virtual Datacenters (Prov vDCs) . . . . . . . . . . . . . . . . . . . . . . . 13
3.3 Establish Organizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.4 Establish Networking Options – Public vCloud. . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.5 Establish Networking Options – Private vCloud. . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.6 Establish Organization Virtual Datacenters (Org vDCs). . . . . . . . . . . . . . . . . . . . 18
3.7 Create vApp Templates and Media Catalogs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.8 Establish Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.9 Accessing your vCloud. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4. Managing the vCloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.1 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.2 Logging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.3 Security Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.4 Workload Availability Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5. Sizing the vCloud. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.1 Sizing Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.2 Sizing the management cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.3 Sizing the workload resource group clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

TECH N I C AL WH ITE PAPE R / 2

 Architecting a vCloud

List of Figures

Figure 1 – vCloud Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Figure 2 – vCloud Logical Architecture Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Figure 3 – vCloud Resource Group Mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Figure 4 – vCloud Director Construct to vSphere Mapping. . . . . . . . . . . . . . . . . . . . . . . 12
Figure 5 – Example Diagram of Provider Networking for a Public vCloud. . . . . . . . . . 16
Figure 6 – Configure External IPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Figure 7 – Example Diagram of Provider Networking for a Private vCloud. . . . . . . . . 17
Figure 8 – Configure Firewall Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Figure 9 - vShield Manager’s Administrator UI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Figure 10 - vCloud Director Manage and Monitor UI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Figure 11 - Configure Firewall Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

TECH N I C AL WH ITE PAPE R / 3

 Architecting a vCloud

1. What is a VMware vCloud?

1.1 Document Purpose and Assumptions
Architecting a vCloud is intended to serve as a reference for cloud architects. The target audience is VMware
Certified Professionals (VCP) familiar with VMware products, particularly VMware vSphere (vCenter Server, ESXi,
vShield Manager), vCenter Chargeback, and vCloud Director.
Before proceeding with the rest of this document you should have read the vCloud service definition for the
type of cloud you are building (private or public). This document is not intended as a substitute for detailed
product documentation, nor is it a step-by-step guide for installing a vCloud. Also, you should have access to
the following documentation referred to throughout this document for step-by-step instructions on installing
and configuring various components.

C AT E G O R Y REFERENCED DOCUMENT

Service Definitions Service Definition for Public Cloud

Service Definition for Private Cloud

vCloud vCloud Installation Guide

VMware vCloud Director Security Hardening Guide

vCloud Director VMware vCloud Director Administration Guide

vCloud Director Administrator’s Guide

vSphere vSphere Administrator Guide

vSphere Resource Management Guide

vShield vShield Manager Administrator Guide

Chargeback VMware vCenter Chargeback User’s Guide

vCloud Chargeback Models Implementation Guide

For further information, refer to the set of documentation for the appropriate product. For additional guidance
and best practices, refer to the Knowledge Base on vmware.com.

TECH N I C AL WH ITE PAPE R / 4

 Architecting a vCloud

This document is organized into these sections:

SECTION DESCRIPTION

What is a VMware vCloud? Components and definitions comprising the cloud

solution:
• Document Purpose and Assumptions
• vCloud Components

Assembling a vCloud Logical architecture of VMware product components:

• vCloud Logical Architecture
• vCloud Management Cluster
• vCloud Resource Groups

Creating Services with vCloud Director
Resource abstraction and the consumption model:
• vCloud Director Constructs
• Establish Provider Virtual Datacenters (Prov vDCs)
• Establish Organizations
• Establish Networking Options – Public vCloud
• Establish Networking Options – Private vCloud
• Create vApp Templates and Media Catalogs
• Establish Policies
• Accessing your vCloud

Managing the vCloud Administrative tasks and considerations:

• Monitoring
• Logging
• Security Considerations
• Workload Availability Considerations

Sizing the vCloud Sizing your vCloud environment:

• Sizing Considerations
• Sizing the Management Cluster

TECH N I C AL WH ITE PAPE R / 5

 Architecting a vCloud

1.2 Cloud Computing and vCloud Introduction

VMware’s vCloud leverages VMware technologies and solutions to deliver cloud computing. Cloud computing is
a new approach to computing that leverages the efficient pooling of on-demand, self-managed virtual
infrastructure to provide resources consumable as a service.
Cloud computing can be delivered as three layers of service delivery:
• Infrastructure as a Service (IaaS)
• Platform as a Service (PaaS)
• Software as a Service (SaaS)
This iteration of a vCloud focuses strictly on the IaaS layer.
The vCloud will build upon VMware vSphere by extending the robust virtual infrastructure capabilities to
facilitate delivery of infrastructure service via cloud computing.

1.3 vCloud Components

The VMware vCloud is comprised of the following components:

vCloud API
vCenter Chargeback

VMware vCloud Director

vShield Edge

VMware Sphere

Figure 1 – vCloud Overview

TECH N I C AL WH ITE PAPE R / 6

 Architecting a vCloud

VC LO U D C O M P O N E N T DESCRIPTION

VMware vCloud Director (vCD) Cloud Coordinator and UI. Abstracts vSphere
resources.
Includes:
• vCloud Director Server(s) (also known as “cell”)
• Cloud Director Database
• vCloud API, used to manage cloud objects

VMware vSphere Underlying foundation of virtualized resources.

The vSphere family of products includes:
• vCenter Server and vCenter Server Database
• ESXi hosts, clustered by vCenter Server
• Management Assistant

VMware vShield Provides network security services

Includes:
• vShield Manager (VSM) virtual appliance
• vShield Edge virtual appliances, automatically
deployed by vCloud Director

VMware vCenter Chargeback Optional component that provides resource metering

and reporting to facilitate resource showback/
chargeback
Includes:
• vCenter Chargeback Server
• Chargeback Data Collector
• vCloud Data Collector
• VSM Data Collector

Other VMware or third-party products or solutions such as orchestration are not addressed in this iteration of a
vCloud.

2. Assembling a vCloud
2.1 vCloud Logical Architecture
In building a vCloud, assume that all management components such as vCenter Server and vCenter Chargeback
Server will run as virtual machines.
As a best practice of separating resources allocated for management functions from pure user-requested
workloads, the underlying vSphere clusters will be split into two logical groups,
• A single management cluster running all core components and services needed to run the cloud.
• One or more vCloud resource groups that represent dedicated resources for cloud consumption. Each
resource group is a cluster of ESXi hosts managed by a vCenter Server, and is under the control of VMware
vCloud Director. Multiple resource groups can be managed by the same vCenter Server.
Reasons for organizing and separating vSphere resources along these lines are:
• Facilitating quicker troubleshooting and problem resolution. Management components are strictly contained
in a relatively small and manageable management cluster. They do not run on a large set of host clusters; this
could lead to situations where it is time-consuming to track down and manage such workloads.

TECH N I C AL WH ITE PAPE R / 7

 Architecting a vCloud

• Management components are separate from the resources they are managing.
• Resources allocated for cloud use have little overhead reserved. For example, cloud resource groups would
not host vCenter VMs.
• Resource groups can be consistently and transparently managed and carved up, and scaled horizontally.
The logical architecture with vSphere resource separation is depicted as follows.

Management Cluster vCloud Resource Groups

VM VM VM VM VM VM
VM VM VM VM VM VM
VM VM VM VM VM VM
VM VM VM VM VM VM
VM VM VM VM VM VM
VM VM VM VM VM VM
VM VM VM VM VM VM
wa wa wa wa wa wa
re re re re re re

vCloud Infrastructure
VM VM VM VM
• vCenter Server VMs VM
VM
VM
VM
VM
VM
VM
VM
VM VM VM VM
VM VM VM VM
• vCloud Director Cell VMs VM
VM
VM
VM
VM
VM
VM
VM
wa wa wa wa
re re re re
• vCenter Chargeback Server VMs

• vShield Manager (VSM) virtual appliance

• vCenter Database VMs

• Cloud Director Database VM

• vCenter Chargeback Database VM VM VM VM VM

VM VM VM VM
VM VM VM VM
• Load balancer VMs for VMware VM
VM
VM
VM
VM
VM
VM
VM

VM VM VM VM
VM VM VM VM
Cloud Director Cells wa wa wa wa
re re re re

• vCenter Update Manager VMs

• Data Recovery VMs

• vSphere Management Assistant (vMA) VM

vCloud infrastructure VMs
No user workloads
• vShieldEdge virtual appliances

User workloads only

Figure 2 – vCloud Logical Architecture Overview

The management cluster resides in a single physical site. vCloud resource groups also reside within the same
physical site. This ensures a consistent level of service. Otherwise, latency issues might arise if workloads need to
be moved from one site to another, over a slower or less reliable network.
Neither secondary nor disaster recovery (DR) sites are in the scope of this document. Certain limitations apply
when using VMware and 3rd party tools for disaster recovery and secondary or federated sites. Consult your
local VMware representative for assistance in understanding these limitations and possible alternatives. You can
also consult the Knowledge Base on vmware.com for additional information.

2.2 vCloud Management Cluster

To enable VMware High Availability (HA), a cluster of 3 VMware ESXi hosts will be used. While additional hosts
can be added, 3 hosts supporting just vCloud management components should be sufficient for typical vCloud
environments. For detailed sizing of the management cluster see Sizing the vCloud in this document.
A VMware HA percentage-based policy and a N+1 host architecture will be used instead of dedicating a single
host for host failures. This will allow the management workloads to run evenly across the hosts in the cluster
without the need to dedicate a host strictly for host failure situations. Additional hosts can be added to the
management cluster for N+2 or more redundancy but this is not required by the current vCloud service
definitions.

TECH N I C AL WH ITE PAPE R / 8

 Architecting a vCloud

Host networking in the management cluster will be configured per vSphere best practices, including (but not
limited to) the following:
• Separation of network traffic for security and load considerations by type (management, VM, vMotion/Fault
Tolerance (FT), storage.
• Network path redundancy.
• Use of vNetwork Distributed Switches where possible for network management simplification. The
architecture calls for the use of vNetwork Distributed Switches in the user workload resource group, so it is a
best practice to use the vNetwork Distributed Switch across all of your clusters, including the management
cluster.
• Increasing the MTU size of the physical switches as well as the vNetwork Distributed Switches to at least 1524
to accommodate the additional MAC header information used by vCloud Director Network Isolation links.
vCD-NI is called for by the service definition and the architecture found later in this document. Failure to
increase the MTU size could adversely affect performance of the network throughput to VMs hosted on the
vCloud infrastructure.
Shared storage in the management cluster will be configured per vSphere best practices, including (but not
limited to) the following:
• Storage paths will be redundant at the host (connector), switch, and storage array levels.
• All hosts in a cluster will have access to the same datastores.
• The use of RDMs in the vCloud Director infrastructure is currently not supported and should be avoided.
Management components running as VMs in the management cluster include the following:
• vCenter Server(s) and vCenter Database
• vCloud Director Cell(s) and vCloud Director Database
• vCenter Chargeback Server(s)
• vShield Manager (one per vCenter Server)
Optional management functions, deployed as VMs include:
• vCenter Update Manager
• VMware Data Recovery
• VMware Management Assistant (vMA)
For more information on the resources needed by the VMs in the management cluster refer to Sizing the vCloud
in this document.
The optional management VMs are not required by the service definition but they are highly recommended to
increase the operational efficiency of the solution.
All of the management VMs can be protected by VMware HA and FT, unless the vCenter Server VM has 2 vCPUs,
in which case it cannot use FT and a solution such as vCenter Heartbeat should be considered. vCenter Site
Recovery Manager (SRM) can be used to protect some components of the management cluster. At this time,
vCenter Site Recovery Manager will not be used to protect vCloud Director cells because a secondary (DR) site
is out of scope of the vCloud, and changes to IP addresses and schemas in recovered vCloud Director cells can
result in problems.
Unlike a traditional vSphere environment where vCenter Server is used by administrators to provision VMs,
vCenter Server plays an integral role in end-user self-service provisioning by handling all VM deployment
requests by vCloud Director. Therefore, ensuring the availability of vCenter Servers with a solution such as
vCenter Heartbeat is highly recommended.

TECH N I C AL WH ITE PAPE R / 9

 Architecting a vCloud

vShield Edge appliances are deployed automatically by vCloud Director as needed and will reside in the vCloud
resource groups, not in the management cluster. They will be placed in a separate resource pool by vCloud
Director and vCenter. For additional information on the vShield Edge appliance and its functions, refer to the
vShield Manager Administrator guides.

2.3 vCloud Resource Groups

Each resource group represents a cluster of VMware ESXi hosts under the management of a vCenter Server and
associated with a single vSphere Cluster.

vCenter
vCloud Resource Group = Host Cluster

vCenter
Resource Pool

Figure 3 – vCloud Resource Group Mapping

While it is possible to create multiple vCenter resource pools per host cluster, it is best to dedicate the cluster for
use by vCloud Director. vCloud Director will automatically allocate resources to cloud organizations by creating
resource pools with appropriate reservations and limits within the cluster. Since vCloud Director manages
vSphere resources by proxy through a vCenter Server and automatically creates resource pools within vCenter
as needed, using vCenter Server to create resource pools or nested pools can go against the efficient allocation
of resources by vCloud Director. Multiple parent-level resource pools can also add unnecessary complexity and
lead to unpredictable results or inefficient use of resources, if the reservations are not set appropriately.
To summarize, it is a best practice to use a 1-to-1mapping with vCloud Resource Group to vCenter host cluster.
Resource pools will be automatically created by vCloud Director.
Compute Resources
All hosts in the vCloud resource groups will be configured per vSphere best practices, similar to the
management cluster. VMware HA will also be used to protect against host and VM failures.
Resource groups can be of different compute capacity sizes (number of hosts, number of cores, performance of
hosts) to support differentiation of compute resources by capacity or performance for service level tiering
purposes.
For a detailed look at how to size the vCloud resource groups, refer to Sizing the vCloud in this document.
Storage
Shared storage in the vCloud resource groups will be configured per vSphere best practices, similar to the
management cluster. Storage types supported by vSphere will be used. The use of RDMs in the vCloud Director
infrastructure is currently not supported and should be avoided.
Creation of datastores will need to take into consideration Service Definition requirements and workload use
cases, which will affect the number and size of datastores to be created. vCloud Director will assign datastores
for use through provider virtual datacenters ( provider vDCs), and only existing vSphere datastores can be
assigned.

TECH N I C AL WH ITE PAPE R / 1 0

 Architecting a vCloud

Datastores in the vCloud resource groups will be used for vCloud workloads, known as vApps. vSphere best
practices apply for datastore sizing in terms of number and size. Vary datastore size or shared storage
characteristic if providing differentiated or tiered levels of service. Sizing considerations include:
• Datastore size:
–– What is the average vApp size x number of vApps x spare capacity?
For example: Avg VM size * # VMs * (1+ % headroom)
–– What is the average VM disk size?
–– How many VMs are in a vApp?
–– How many VMs are to be expected?
–– How much spare capacity do you want to allocate for room for growth (express in a percentage)?
• Datastore use:
–– Will expected workloads be transient or static?
–– Will expected workloads be disk-intensive?
The public cloud service definition calls for a capacity of 1,500 VMs initially and specifies 60 GB of storage per
VM. You should consider these numbers when sizing your datastores.
Additionally, an NFS share must be set up and made visible to all cells for use by vCloud Director for transferring
files in a vCloud Director multi-cell environment. NFS is the required protocol for the transfer volume. Refer to
the vCloud Installation Guide for more information on where to mount this volume.
Networking
Host networking for hosts within a vCloud resource group will be configured per vSphere best practices in the
same manner as the vCloud management cluster. In addition, the value of the number of vNetwork Distributed
Switch ports per host should be increased from the default value of 128 to the maximum of 4096. Increasing the
ports will allow for vCloud Director to dynamically create port groups as necessary for the private organization
networks created later in this document. Refer to the vSphere Administrator Guide for more information on
increasing this value.
Networking requirements specific to the vCloud resource groups that facilitate cloud networking include:
• Increasing the MTU size of the physical switches as well as the vNetwork Distributed Switches to at least 1524
to accommodate the additional MAC header information used by vCloud Director Network Isolation links.
vCD-NI is called for by the service definition and the architecture found later in this document. Failure to
increase the MTU size could adversely affect performance of the network throughput to VMs hosted on the
vCloud infrastructure.
• Pre-configured vSphere port groups for use in connecting to external networks:
–– These can be using standard vSwitch port groups, vNetwork Distributed Switch port groups, or the Cisco
Nexus 1000V.
–– In a vCloud for service providers, these pre-configured port groups will provide access to the internet.
–– Make sure to have sufficient vSphere port groups created and made available for VM access in the vCloud.
• VLANs to support private networks:
–– Private networks are private with respect to an organization.
–– Hosts must be connected to VLAN trunk ports.
–– Private networks are backed by VLAN IDs or network pools, which use fewer VLAN IDs.
–– vNetwork Distributed Switches are required.
–– MTU size should be increased to a minimum of 1524 bytes, # of vCD-NI networks per VLAN.
–– Note that vCloud Director creates port groups automatically as needed.

TECH N I C AL WH ITE PAPE R / 11

 Architecting a vCloud

3. Creating Services with vCloud Director

3.1 vCloud Director Constructs
VMware vCloud Director introduces logical constructs, such as provider virtual datacenters (vDCs), and security
boundaries, such as organizations, to facilitate multi-tenancy consumption of resources.
The following diagram depicts the logical constructs within vCloud Director that abstract underlying vSphere
resources.

Admin Organization Organization A

Users Access Control Users Access Control

Catalogs Provisioning Policies Catalogs Provisioning Policies

User Clouds User Clouds

vApp vApp
(VMs with vApp Network) (VMs with vApp Network)
Organization vDCs Organization vDCs

vSphere
vApp Network
Port Groups or
Organization Network Organization Network dvPort Groups
External Networks

Resource Pools
Organization vDCs Organization vDCs Organization vDC

Provider vDC: Gold Provider vDC: Silver Provider vDC: Bronze Datastores

Figure 4 – vCloud Director Construct to vSphere Mapping

VC LO U D D I R E C TO R C O N S T R U C T DESCRIPTION

Provider Virtual Datacenter (vDC) Logical grouping of vSphere compute resources

(backed by a vCenter resource pool automatically
created by vCloud Director when attaching a vSphere
cluster) and assigned datastores for the purposes of
providing cloud resources to consumers.

Organization A unit of administration that represents a logical

collection of users, groups, and computing resources,
and also serves as a security boundary from which
only users of a particular organization can deploy
workloads and have visibility into such workloads in
the cloud.

In the simplest term, an organization = an association

of related end consumers.

TECH N I C AL WH ITE PAPE R / 12

 Architecting a vCloud

VC LO U D D I R E C TO R C O N S T R U C T DESCRIPTION

Organization Virtual Datacenter (vDC) Subset allocation of a provider vDC resources

assigned to an organization. An organization vDC
allocates resources using one of three models:
• Pay as you go
• Reservation
• Allocation

vApp Templates and Media Catalogs A collection of available services for consumption.
Catalogs contain vApp templates (preconfigured
containers of one or more virtual machines) and/or
media (ISO images of operating systems).

External Network A network that connects to the outside using an

existing vSphere network port group.

Organization Network A network visible within an organization. It can be an

external organization network with connectivity to an
external network, and use a direct or routed
connection, or it can be an internal network visible
only to vApps within the organization.

vApp Network A network visible within a vApp. It can be connected

to other vApp networks within an organization and
use a direct or routed connection, or it can be an
internal network visible only to VMs within the vApp.

3.2 Establish Provider Virtual Datacenters (Prov vDCs)

A provider vDC is backed by a vCenter resource pool that is automatically created by vCloud Director when
attaching a vSphere cluster that will back the provider vDC. When creating a provider vDC, take the following
rules and guidelines into consideration:
• At least one provider vDC is required for a vCloud.
• A provider vDC can map to one and only one cluster. Once a cluster is attached to a provider vDC, it is no
longer available for attachment to another provider vDC.
• While it is possible to back a provider vDC with a resource pool instead of a cluster, the best practice is to use
a cluster. This will allow preservation of resource allocations should additional hosts be added to the cluster.
• It is not possible to attach a second cluster to a provider vDC at this time. If additional compute capacity is
required, add more hosts in the vCenter cluster on the vSphere end.
• One or more datastores can be attached to a provider vDC. A datastore can be assigned to multiple provider
vDCs. As a best practice in segmenting storage, datastores should not be shared by multiple provider vDCs.
• Create multiple provider vDCs to differentiate different levels or characteristics of a service offering. Segment
by capacity or performance type. For example, provider vDC01 = fast storage, provider vDC02 = medium
storage. Or Provider vDC_A = high-end hosts, provider vDC_B = mid-tier hosts.
• As the level of expected consumption increases for a given provider vDC, add additional hosts to the cluster
from vCenter and attach more datastores.
• If the cluster backing a provider vDC has reached the maximum number of hosts per vSphere design
guidelines, create a new provider vDC backed by a new resource pool associated with a new cluster. A
provider vDC cannot span multiple host clusters.

TECH N I C AL WH ITE PAPE R / 13

 Architecting a vCloud

Refer to the service definition for guidance on the size of vSphere clusters and datastores to attach when
creating a provider vDC.
Consider:
• Expected number of VMs
• Size of VMs (CPU, RAM, disk)
Service Provider Considerations
Considerations for a service provider (public) vCloud include creating multiple provider virtual datacenters
(Prov vDCs) based on tiers of service that will be provided.
Because Prov vDCs contain only CPU, memory, and storage resources and those are common across all of the
requirements in the public cloud service definition, you should create one large Prov vDC attached to a vSphere
cluster that has sufficient capacity to run 1,500 VMs. You should also leave overhead to grow the cluster with
more resources up to the maximum of 32 hosts, should organizations need to grow in the future.
If you determine that your hosts do not have sufficient capacity to run the maximum number of VMs called out
by the public cloud service definition, you should separate the Pay-As-You-Go service tier from the Resource
Pool service tier by creating two separate Prov vDCs.
Private Cloud Considerations
Given that a provider virtual datacenter (Prov vDC) represents a vSphere cluster and resource pool, it’s
commonly accepted that a single Prov vDC be established. Refer to the service definition for private cloud for
details on the Service Tier(s) called for.
Because Prov vDCs contain only CPU, memory, and storage resources, and those are common across all of the
requirements in the private cloud service definition, you should create one large Prov vDC attached to a cluster
that has sufficient capacity to run 400 VMs.
Should it be determined that existing host capacity can’t meet the requirement, or there’s a desire to segment
capacity along the lines of equipment type (for example, CPU types in different Prov vDCs), then establish a
Prov vDC for Pay-As-You-Go use cases and a separate Prov vDC for the resource-reserved use cases.

3.3 Establish Organizations

A vCloud contains one or more organizations. Each organization represents a collection of end consumers,
groups, and computing resources. Users authenticate at the organization level, using credentials established by
an organization administrator within vCloud Director or LDAP.
Users in an organization consume resources by selecting vApps from a predefined catalog.
When creating organizations the name of the organization will be used in the URL to access the GUI for that
organization. As an example, ACME would be accessed at https://<hostname>/cloud/org/ACME. You should
take care to avoid special characters or spaces in the organization name since that will affect the URL in
undesirable ways.
The service definition does not specifically call out the use of LDAP for organizations, so each organization will
be set up to not use LDAP, and instead use local users. See Security Considerations in this document for more
information on LDAP authentication.
You can use the system defaults for most of the other organization settings. The one exception is leases, quotas,
and limits. There are no specific requirements called out by the service definition for leases, quotas, and limits.
The provider should set these values to whatever works best in their cloud.
Administrative Organization
A vCloud requires at least one organization. As a best practice, the first organization to be created will be an
administrative organization. This organization will own a master catalog of vApp templates that are published
and shared with all other (standard) organizations.

TECH N I C AL WH ITE PAPE R / 14

 Architecting a vCloud

Administrators assigned to the administrative organization will also be responsible for creating official template
VMs for placement in the master catalog for other organizations to use. VMs in development should be stored in
a separate development catalog that is not shared with other organizations.
As a note of reference, there is already a default System organization in the vCloud Director environment. The
administrative organization being created here is different from the built-in System organization since it can
actually create vApps and catalogs and share them.
Make sure that when you create the administrative organization you set it up to allow publishing of catalogs.
Standard Organizations
Create an organization for each tenant of the vCloud as necessary. Each of the standard organizations will be
created with the following considerations:
• Do not use LDAP
• Cannot publish catalogs
• Use system defaults for SMTP
• Use system defaults for notification settings
• Use Leases, Quotas, and Limits meeting the provider’s requirements

3.4 Establish Networking Options – Public vCloud

External Networks
Referencing the service definition for a public cloud, all service tiers use a shared public Internet connection. To
fulfill this, create a single external provider network. Make sure to give the network a descriptive name such as
Provider-Internet for the case here. You will connect this External network to a vSphere port group which is
actually connected to the Internet. Make sure you have the IP information for the physical network you have
attached to, including the network mask, default gateway, and DNS information. Lastly, you will create a pool of
static IP addresses that will be consumed by vShield Edge appliances (which facilitate a routed connection) each
time you connect an organization network to this external network.
For sizing purposes, you should create a large enough IP address pool so that each of your organizations can
have access to an external network. Per the service definition, the estimated number of organizations for 1,500
VMs is 25 organizations, so make sure you have at least 25 IP addresses in your static IP pool.
Network Pools
In addition to access to external networks, each organization in a public vCloud will have organization-specific
private networks. vCloud Director instantiates Isolated L2 networks through the use of network pools.
Create a single large network pool for all organizations to share, and limit the use of this network pool when you
create each individual organization. The network pool created will use vCloud Network Isolation for separating
the traffic. This will use an existing vNetwork Distributed Switch previously created for connecting hosts. You
can optionally use a VLAN to further segregate all of the vCD-NI traffic.
Because the network pools will be used by both the external organization network and private vApp networks,
you will need at least 11 networks in the network pool per organization. Ten of the networks in the pool will be for
the private vApp networks according to the public cloud service definition. One of the networks will be used for
the protected external organization network. Given the estimate of 25 organizations, you need at least 275
networks in the pool. There is a limitation of a maximum of 4096 networks in a network pool due to the port
limitation on the vNetwork Distributed Switch. When connecting the network pool to a vNetwork Distributed
Switch, make sure you have enough free ports left on the switch (at least 275).

TECH N I C AL WH ITE PAPE R / 15

 Architecting a vCloud

vCloud Datacenter

Organization “ACME Corp.”

Org Net: Network Pool

“ACME-Private”
Private Internal

Org Net: “ACME-Internet”

Private Routed

“Provider Internet”

Figure 5 – Example Diagram of Provider Networking for a Public vCloud

Organization Networks
Create 2 different organization networks for each organization, one external organization network and one
private internal organization network. You can do this as one step in the vCloud Director UI wizard by selecting
the default (recommended) option when creating a new organization network. When naming a organization
network, it is a best practice to start with the organization name and a hyphen, for example, ACME-Internet.
Per the Service Definition for Public Cloud, the external network will be connected as a routed connection that
will leverage vShield Edge for firewalling and NAT to keep traffic separated from other organizations on the
same external provider network. Both the external organization network and the internal organization networks
will leverage the same vCD-NI network pool previously established. For both the internal network and the
external network, you will need to provide a range of IP addresses and associated network information. Since
both of the networks will be private networks behind a vShield Edge, you can use RFC 1918 addresses for both
static IP address pools.
The Service Definition for Public Cloud defines a limit of external connections with a maximum of 8 IP addresses,
so you should provide a range of 8 IP addresses only when creating the static IP address pool for the external
network. For the private network, you can make the static IP address pool as large as desired. Typically, a full
RFC 1918 class C is used for the private network IP pool.
The last step is to add external public IP addresses to the vShield Edge configuration on the external
organization network. By selecting Configure Services on the external organization network, you can add
8 public IP addresses that can be used by that particular organization. These IP addresses should come from
the same subnet as the network that you assigned to the system’s external network static IP pool.

TECH N I C AL WH ITE PAPE R / 1 6

 Architecting a vCloud

Figure 6 – Configure External IPs

3.5 Establish Networking Options – Private vCloud

External Networks
In general, for a private vCloud, the networking needs are simplified and direct compared to a Public vCloud.
As such, direct connections from inside the organization to the networking backbone provided by the enterprise
are all that’s necessary. This is analogous to “extending a wire” from the network switch that contains the
network or VLAN to be used all the way through the cloud layers to the organization and into the vApp.
One of these direct networks must be established for each network or VLAN to be used in the private vCloud.

Enterprise vCloud

Organization “Software Design”

Network Pool

Org Net:
“Internal Network”
Private Internal (optional)

Org Net: “External Access”

Private Direct

“Corporate Backbone”

Figure 7 – Example Diagram of Provider Networking for a Private vCloud

TECH N I C AL WH ITE PAPE R / 17

 Architecting a vCloud

An important differentiation to keep an eye on is an “External Network”, a function of the vCloud foundational
layer under all the private vClouds that may get established, and “Organization External Networks”, a
component of each organization that gets established at its creation time. This section is focused on the first
external network mentioned, the foundational object.
At least one external network is required to enable organization networking to connect to. The external provider
network in a private vCloud is a network outside of the scope of the cloud, i.e., it is not managed by either the
vCloud layer or the vSphere layer. It is a network that already exists within the address space used by the
enterprise.
To establish this network, follow the wizard, filling in the network mask, default gateway and other specifications
of the LAN segment as required. When building this, specify enough address space for use as static
assignments, as this is where vCloud Director draws “Public IP Pool” addresses from. A good starting range is
30 addresses that do not conflict with existing addresses in use, or ranges already committed for DHCP.
Note: Static IP Pool address space is not used for DHCP, but the function is similar to that. This pool will be used
to provision NAT-type connectivity between the Organizations and the cloud services below it.
Network Pools
A network pool is a collection of virtual machine networks that are available to be consumed by organizations to
create organization networks and vApp networks. Network traffic on each network in a pool is isolated at layer 2
from all other networks.
You will need a network in the network pool for every private organization network and external organization
network in the vCloud environment. The private cloud service definition calls for one external organization
network and the ability for the organization to create private vApp networks. Because there is no minimum
called out in the service definition for the number of vApp networks, a good number of networks to start out
with is 10 per organization. Make your network pool as large as the number of organizations times 10.
Organization Networks
At least one organization external network is required to connect vApps created within the Organization to other
vApps and/or the networking layers beyond the Private vCloud.
To accomplish this, create an external network in the Cloud Resources section (under Manage & Monitor of the
System Administration section of the vCloud Director UI). In the wizard, be sure to select a direct connection.
This external network maps to an existing vSphere network for VM use as defined in the External Networks
section (above).
Other networking options are available, like a routed organization external network, and could be used, but add
complexity to the design that is normally not needed. For the purpose of this design there are no additional
network requirements. For more information on adding additional network options please refer to the vCloud
Director Administrator’s Guide.

3.6 Establish Organization Virtual Datacenters (Org vDCs)

An organization virtual datacenter (Org vDC) allocates resources from a Prov vDC and makes it available for use
for a given organization. Multiple Org vDCs can take from the same Prov vDC. An organization can have multiple
Org vDCs.
Resources are taken from a Provider vDC and allocated to an Organization vDC using one of three resource
allocation models:
• Pay as you go. Resources are only reserved and committed for vApps as vApps are created. There is no
upfront reservation of resources.
• Allocation. A baseline amount (“guarantee”) of resources from the provider vDC is reserved for the
organization vDC’s exclusive use. An additional percentage of resources is available to oversubscribe CPU
and memory, but this taps into compute resources that are shared by other organization vDCs drawing from
the provider vDC.

TECH N I C AL WH ITE PAPE R / 1 8

 Architecting a vCloud

• Reservation. All resources assigned to the organization vDC are reserved exclusively for the organization
vDC’s use.
With all of the above models the organization can be limited to deploy a certain number of VMs. Or, this can also
be set to unlimited.
The first organization vDC to be created should be an administration organization vDC for use by the
administration organization. The allocation model is set to “Pay as you go” so as not to take resources from other
organization vDCs until they are needed.
Subsequent organization vDCs should be created to serve the organizations previously established. In selecting
the appropriate allocation model, the service definition and organization’s use cases of workloads should be
taken into consideration.
Service Provider Considerations
The organization virtual datacenter allocation model maps directly to a corresponding vCenter Chargeback
billing model:
• Pay as you go. Pricing can be set per VM, and a corresponding speed of a vCPU equivalent can be specified.
Billing is unpredictable as it is tied directly to actual usage.
• Allocation. Consumers are allocated a baseline set of resources but have the ability to burst by tapping into
additional resources as needed, but are typically charged at higher rates for exceeding baseline usage. This
model will result in more variable billing but allows for the possibility of more closely aligning variable
workloads to their cost.
• Reservation. Consumers are allocated and billed for a fixed container of resources, regardless of usage. This
model allows for predictable billing and level of service, but consumers may pay for a premium if they do not
consume all their allocated resources.
These allocation models also map directly to the service tiers found in the public cloud service definition. The
Basic VDC model will use the Pay-as-you-go allocation model since instances are only charged for the resources
they consume and there is no commitment required from the consumer. The Committed VDC model will use the
Allocation Pool model since the consumer is required to commit to a certain level of usage but is also allowed to
exceed that usage. The Dedicated VDC model will use the Reservation Pool model since this service tier requires
dedicated and guaranteed resources for the consumer.
The Service Definition for Public Cloud provides detailed and descriptive guidance on how much a provider
should charge for each service tier. Chargeback functionality is provided by VMware vCenter Chargeback, which
is integrated with VMware vCloud Director. You should follow the steps in the vCloud Chargeback Models to set
up the appropriate charging profiles for each of your service tiers. You can further reference the VMware vCenter
Chargeback User’s Guide for information on how to customize the individual reports generated.
For further information, refer to the vCloud Chargeback Models Implementation Guide, which details how to set
up vCloud Director and vCenter Chargeback to accommodate instance-based pricing (pay as you go),
reservation-based pricing, and allocation-based pricing.
Private Cloud Considerations
The organization vDC allocation model used depends on the type of workloads to be expected.
• Pay as you go. A transient environment where workloads are repeatedly deployed and undeployed, such as a
demonstration or training environment, would be suited for this model.
• Allocation. Elastic workloads that have a steady state but during certain periods of time surge due to special
processing needs would be suited for this model.
• Reservation. Since a fixed set of resources are guaranteed, infrastructure-type workloads that demand a
predictable level of service would run well using this model.
When an organization vDC is created in vCloud Director, vCenter Server automatically creates child resource pools
with the appropriate resource reservations and limits, under the resource pool representing the provider vDC.

TECH N I C AL WH ITE PAPE R / 1 9

 Architecting a vCloud

As part of creating an organization vDC, a storage limit can be set on the amount of storage to draw from the
provider vDC backing the organization vDC. By default, this setting is left to unlimited. For the purpose of this
architecture there will be no limit on storage consumed by the vApps since we are providing static values for the
individual VM storage and we are also limiting the number of VMs in an organization.
An option to “enable thin provision” allows provisioning VMs using thin disks to conserve disk usage. vSphere
best practices apply in the use of thin-provisioned virtual disks. This feature can save substantial amounts of
storage and have very little performance impact on workloads in the vCloud infrastructure. It is recommended
to enable this feature when creating each organization. For more information about this feature please refer to
the vCloud Director Administrator’s Guide or the VMware knowledge base.

3.7 Create vApp Templates and Media Catalogs

The way to consume services in a cloud environment is from a catalog. Catalogs are stored in an organization vDC.
The administrative organization vDC will have two catalogs:
• Internal. Used for developing and staging new vApps and media.
• Master. Published and shared to all other organization vDCs.
Organizations will use the master catalog that has been published from the administrative organization vDC with
the default cloud templates. In addition, organizations will have a private catalog created by the organization
administrator and used for uploading new vApps or media to the individual organization.
There are no other configuration requirements for the catalogs or templates in this cloud architecture. Please
refer to the service definition for a full listing of recommended templates.

3.8 Establish Policies

During the creation of an organization, you can set policies around the number of deployed and stored VMs:
• Deployed VMs refers to the number of running VMs.
• Stored VMs refers to the total number of VMs including VMs that are not powered on.
You can also specify runtime policies to control vApps and vApp templates in an organization vDC. Specify the
maximum length of time vApps and vApp templates can run and be stored in the organization vDCs:
• The runtime lease can be set to allow vApps or vApp templates to run for a defined period of time after which
time vApps will be powered off, or set to “never expire”.
• The storage lease can be specified, allowing vApps or vApp templates to be stored for a defined period of
time, after which time vApps or vApp templates will be automatically cleaned up, or set to “never expire”.
When any option for storage lease (with the exception of “never expire”) is selected, the storage will be
automatically cleaned up. Additional options include:
• Permanently deleted. After the specified period of time, the vApps or vApp templates will automatically be
deleted.
• Moved to expired items. This flags the vApps or vApp templates for deletion, which hides them from users so
that they can no longer be used, allowing an Administrator to remove them.
The public cloud service definition has specific requirements for the maximum number of VMs each organization
can have based on size. Refer to the public cloud service definition for the maximum VM count for each of the
three tiers of reservation pools.

TECH N I C AL WH ITE PAPE R / 20

 Architecting a vCloud

3.9 Accessing your vCloud

The vCloud is now ready for self-service use. Each organization should have a public URL configured to access
the organization’s cloud portal using vCloud Director. These URLs will have the format of https://<vCD-cell-
hostname>/cloud/org/<org-Name>. Each time a user of an organization logs in they should point their browser
to the organization-specific URL.

4. Managing the vCloud

4.1 Monitoring
To ensure the vCloud operates with minimal downtime, monitor all vCloud components. At the vSphere level,
typical procedures for monitoring physical and vSphere components apply. This document will not detail
specifics on setting up a monitoring solution since every provider has very different monitoring solutions in
place to be integrated.
A centralized monitoring tool such as Hyperic can be used to monitor some of the servers (Oracle Server, SQL
Server, Active Directory Server, DNS Server, Red Hat Enterprise Linux Server, Windows Server) that are needed
to run a vCloud Director environment. SNMP and SMASH are not supported for monitoring vCloud Director cells.
Alternatively, cells can be monitored through integration with a third party monitoring platform via JMX Beans.
Each vCloud Director cell is dependent on the following to be operational:
• vCloud Director Database
• vCenter Server (which depends on vCenter Database)
• vShield Manager (to deploy vShield Edge virtual appliances)
• VMware ESXi hosts (via vCenter Server)
vCenter Chargeback Server is needed to generate reports and is dependent on the vCenter Chargeback
Database. vCenter Chargeback is also dependent on data collectors to collect usage information. Downtime of
data collectors can impact reporting but does not affect the ability to generate reports.
To ensure that vCloud Director and vCloud Director-related components are running, here are the vCloud
dependent processes to monitor for each vCloud component.
vCloud Director
Within Red Hat Enterprise Linux where vCloud Director is installed, executing the following commands will
provide the status of the cell and the watchdog process that monitors the cell.
# service vmware-vcd status
vmware-vcd-watchdog is running
vmware-vcd-cell is running

vCloud Director is basically a java process. One can search for java processes with the process status (ps)
command to make sure that the cells are running. If you see java process listed then the cell should be running,
otherwise you will get no output from the command below.
# ps -ef | grep java

vcloud 27721 1 0 Aug20 ? 00:16:01 /opt/vmware/cloud-director/jre/

bin/java -Xms512M -Xmx1024M -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/opt/vmware/cloud-director/logs -Dservicemix.home=/opt/vmware/cloud-
director -Dservicemix.base=/opt/vmware/cloud-director -Djava.util.logging.config.file=/
opt/vmware/cloud-director/etc/java.util.logging.properties -Dorg.apache.servicemix.
filemonitor.configDir=/opt/vmware/cloud-director/etc -Dorg.apache.servicemix.filemonitor.

TECH N I C AL WH ITE PAPE R / 21

 Architecting a vCloud

monitorDir=/opt/vmware/cloud-director/deploy -Dorg.apache.servicemix.filemonitor.
generatedJarDir=/opt/vmware/cloud-director/data/generated-bundles -Dorg.apache.
servicemix.filemonitor.scanInterval=86400000 -Dservicemix.startLocalConsole=false
-Dservicemix.startRemoteShell=false -Dorg.ops4j.pax.logging.DefaultServiceLog.level=ERROR
-Dservicemix.name=root -Djava.awt.headless=true -DVCLOUD_HOME=/opt/vmware/cloud-director
-Djava.io.tmpdir=/opt/vmware/cloud-director/tmp -Djava.library.path=/opt/vmware/cloud-
director -Djava.net.preferIPv4Stack=true -Doracle.jdbc.defaultNChar=true -Dlog4j.
configuration=file:/opt/vmware/cloud-director/etc/log4j.properties -jar /opt/vmware/
cloud-director/system/org.eclipse.osgi-3.4.3.R34x_v20081215-1030.jar -configuration /opt/
vmware/cloud-director/etc

Running a tail command on the vCloud Director’s log files (cell.log, vcloud-container-debug.log, and vcloud-
container-info.log) located in /opt/vmware/cloud-director/logs, contains a lot of information related to
understanding the execution and health of each individual cell.
For example, the following error message could appear in the vcloud-container-debug.log file:
2010-08-23 15:33:34,407 | ERROR | pool-jetty-6 | LdapProviderImpl | LDAP search error.

com.vmware.ssdc.backend.ldap.LdapSearchException: “Problem encountered search searching

LDAP or retrieving object from LDAP.”

at com.vmware.ssdc.backend.ldap.LdapProviderImpl.getUsersByName(LdapProviderImpl.java:818)
at com.vmware.ssdc.backend.ldap.LdapProviderImpl.getUserByUsername(LdapProviderImpl.java:844)
at com.vmware.ssdc.backend.ldap.LdapProviderImpl.testLdapSettings(LdapProviderImpl.java:212)

This entry reveals that there is a problem with LDAP. This would give some information in narrowing down the
problem to a specific component in place (LDAP in this case). Searching for a string “ERROR” in the log files
such as vcloud-container-debug.log and vcloud-container-info.log will show all the errors that happened to an
individual cell at execution time.
In a multi-cell environment, this could be more challenging because one has to log into different servers to
monitor the health of all of the cells. For multi-cell environments you should enable syslog collection to a
centralized logging server. Please refer to the vCloud Director Administrator’s Guide for instructions on how to
setup syslog redirection.
Analyzing errors from the log files is also possible from the vCloud Director’s Administrator portal. For detailed
instructions on how to access the log files in the Administrator portal please refer to the vCloud Director
Administrator’s Guide.

TECH N I C AL WH ITE PAPE R / 2 2

 Architecting a vCloud

Figure 8 – vCloud Director Administrator Portal

vSphere: ESXi hosts

Follow vSphere best practices to ensure hosts are running. In addition, ensure that vCloud dependencies are
monitored. “Vslad” is the vCloud agent and “vpxa” and “hostd” are the vSphere agents that run on ESX/ESXi
hosts. All of the agents run as services.
To do a sanity check, one can run a process status (ps) command and make sure that these processes are up and
running.
# ps aux | grep vslad
45832 5659 worker /opt/vmware/vslad/vslad
5659 5659 worker /opt/vmware/vslad/vslad
5670 5659 poll /opt/vmware/vslad/vslad
5671 5659 worker /opt/vmware/vslad/vslad

For more information on monitoring the vSphere components refer to the vSphere Resource Management Guide.

TECH N I C AL WH ITE PAPE R / 2 3

 Architecting a vCloud

vShield Manager and vShield Edge

Once the vShield Manager is installed and configured successfully to work with vCloud Director, there are two
ways to manage and leverage the monitoring aspect that vShield Manager provides. You can log in directly to
vShield Manager’s administrator portal (UI) or the vShield Manager itself with a vSphere Client plug-in (vShield
Manager will show up in the vSphere Client under “Solutions and Applications”).
By navigating through the administrator UI, and checking the System Events and Audit Logs (under Setting &
Reports), you can see the necessary details to monitor the functionalities of vShield Edge devices.
You can also directly log in to the vShield Manager virtual appliance from its console. A console shell will be
provided after successful login with which limited monitoring is possible with the restricted set of command
line options.
vShield Edge devices are under the control of vShield Manager. There is no console access for a vShield Edge
device. The recommended way to monitor them is though the vShield Manager’s Administrator UI.

Figure 9 – vShield Manager’s Administrator UI

Apart from the Administrator UI or vShield Manager vSphere Client plugin, there is currently no external
mechanism to do health monitoring of vShield Manager or vShield Edge devices.
For more detailed information on the monitoring aspects of vShield Manager and vShield Edge refer to the
vShield Manager Administrator Guide.

vCloud Resource Consumption Monitoring

Within vCloud Director, the following items should be proactively monitored to ensure sufficient resources will
be available for consumption.

SCOPE ITEM

vCloud Director System Organizations Leases

Quotas
Limits

vSphere Resources CPU

Memory
Network static IP address pool
Storage free space

TECH N I C AL WH ITE PAPE R / 24

 Architecting a vCloud

Once logged in as Administrator to vCloud Director, the UI shows the availability and current status of both
virtual and pure virtual resources (where virtual resources are vCenters, resource pools, hosts, datastores,
switches, and ports; and pure virtual resources are vCloud cells, provider virtual datacenters [Prov vDCs],
organization virtual datacenters [Org vDCs], external networks, organization networks, and network pools).

Figure 10 – vCloud Director Manage and Monitor UI

4.2 Logging
Logs of vCloud components can be analyzed for troubleshooting, auditing, and additional monitoring purposes.
As with vSphere, the use of a centralized logging server is recommended. The primary methods for remote
event notification include syslog, SNMP, and MOM (Windows). Refer to the Administrator’s Guide for each
respective VMware product.
vCloud Director cells can be configured to send logs to a centralized server. The following settings will need to
be modified:
• /opt/vmware/cloud-director/etc/global.properties
• /opt/vmware/cloud-director/etc/responses.properties

and these lines should be changed:

• audit.syslog.host = ip.or.hostname.of.your.syslog.server
• audit.syslog.port = 514

Replace “ip.or.hostname.of.your.syslog.server” with the appropriate IP address or hostname, and, if needed,

change port 514 to the port for your syslog server.
vShield Manager does not support remote transmission of logs. Connect to the vShield Manager and use “show
log” commands to view vShield Manager logs.
It is possible to configure the vShield Edge devices to redirect their syslog messages to a centralized syslog
server (example vMA – vManagement Appliance). This is done through the vShield Manager’s Administrator UI.

TECH N I C AL WH ITE PAPE R / 2 5

 Architecting a vCloud

The following table shows the primary log files for each vCloud component, and whether remote logging is
supported.

COMPONENT LO G LO C AT I O N R E M OT E LO G G I N G ?

vCloud Director %VCLOUD%/logs/* Yes

/var/log/messages
/var/log/secure

vSphere ESXi /var/log/vmware/vslad/installer.log

/var/log/vmware/vslad/vslad.log
/var/log/vmware/esxupdate.log
/var/log/vmware/esxcfg-boot.log
/var/log/vmkernel
/var/log/vmware/esxcfg-firewall.log
/var/log/vmware/vpx/vpxa.log

vCenter Server Windows Logs No

vCenter Chargeback Server Windows Logs No

%ProgramFiles%VMware\VMware
vCenter Chargeback\apache-
tomcat-6.0.18\logs
%ProgramFiles%\VMware\VMware
vCenter Chargeback\Apache2.2\logs
%ProgramFiles%\VMware\VMware
vCenter Chargeback\DataCollector-
Embedded\logs

vShield Manager View from UI or console: No

 “show log” or “show manager log” on
console

vShield Edge View from vShield Manager Yes

4.3 Security Considerations

Security in a vCloud can be considered at three levels—the overall vCloud environment, user access, and
workloads.
Securing the vCloud Environment
While vCloud Director is designed for secure multi-tenancy so that multiple organizations do not impact each
other, there are additional steps that can be taken to harden the environment. This is especially important for a
service provider environment where multiple organizations coexist and most are connected to the Internet.
For detailed information on hardening your VMware vCloud Director environment, refer to the VMware vCloud
Director Security Hardening Guide.
Securing User Access
Security for the consumers of vCloud resources is done through authentication and authorization mechanisms
built into VMware vCloud Director. Integration with LDAP or Active Directory can be configured for user
authentication. For more information on how to set up LDAP or Active Directory integration, refer to the VMware
vCloud Director Administration Guide.

TECH N I C AL WH ITE PAPE R / 26

 Architecting a vCloud

The current public cloud service definition does not call out a requirement for setting up LDAP or Active
Directory integration, so it is up to the individual provider. This is also the case for an enterprise running a private
vCloud.
User access and privileges within vCloud Director is controlled through role-based access control (RBAC). For
additional information on permissions, roles, and default settings, refer to the VMware vCloud Director
Administration Guide.
Securing Workloads
Workloads in the vCloud environment are protected from a networking perspective through network visibility
(external or internal to an organization or vApp) and connection types (direct or NAT routed).
vShield Edge devices are deployed automatically by vCloud Director to facilitate routed network connections.
vShield Edge uses MAC encapsulation for NAT routing. This prevents any Layer 2 network information from
being seen by other organizations in the environment. vShield also provides firewall services which can be
configured to not allow any inbound traffic to any virtual machines connected to a public access organization
network.
For service providers, the Service Definition for Public Cloud specifies how the networking options should be set
up, which in turn takes into consideration network security requirements. Each of the organization networks are
connected to the shared public network through a routed connection.
In order to meet the requirements of the service definition, allow up to 8 public IP addresses inbound access to
virtual machines in the organization. The organization administrator is the actual user that will be responsible for
making this configuration change. Once a vApp is created and VMs are added to it and connected to the public
access organization network, the vApp will obtain a private IP address from the static IP pool previously
established. The organization administrator can then configure the firewall and the NAT external IP mapping for
the newly created VM and private IP address using the network configure services wizard as shown below.

Figure 11 – Configure Firewall Services

TECH N I C AL WH ITE PAPE R / 27

 Architecting a vCloud

For a private vCloud, network routing and firewall requirements will depend on the security policies of the
enterprise as they apply to the specific workloads, organizations, and the enterprise itself.

4.4 Workload Availability Considerations

vCloud Director provisions VMs by transparently working with vCenter Server to deploy VMs on hosts.
Provisioned VMs can be protected by VMware HA. VMs can also be protected using backup tools within the
Guest OS.
At this time, VMs provisioned by vCloud Director cannot be protected by VMware FT, vCenter Site Recovery
Manager, or VMware Data Recovery. While these VMs are accessible from vCenter Server and can be set up for
protection irrespective of vCloud Director, this approach can lead to problems in the recovery of VMs because
vCloud Director adds additional logical constructs and management information not visible to vCenter. VMs
protected and recovered using processes that are not integrated with vCloud Director can lead to VMs that will
not work properly with vCloud Director.

5. Sizing the vCloud

5.1 Sizing Considerations
When sizing your vCloud environment there are 4 main resources you should consider:
• CPU
• Memory
• Storage
• Networking
These core resources are divided into 2 types of resource clusters:
• The management cluster
• The workload resource group clusters
Sizing for each of these environments is slightly different. The management cluster has a fairly predictable
workload with very prescriptive guidance from the service definitions, and this architecture document, on what
should run there. The workload resource group has very unpredictable usage, although some guidance can be
given based on the assumptions from the service definitions. The rest of this section will guide you through
sizing your vCloud environment appropriately.

5.2 Sizing the Management Cluster

The following table lists out the requirements for each of the components that will run in the vCloud Director
management cluster. For the number of VMs and organizations listed in the service definitions you will not need
to worry about scaling too far beyond the provided numbers.

ITEM VC P U MEMORY S TO R AG E N E T WOR KING

vCenter Server 2 8 GB 20 GB 100 MB

Oracle Database 4 16 GB 100 GB 1 GigE

vCloud Director Cells 2 4 GB 10 GB 1 GigE

(2 – stats for each)

vCenter Chargeback 2 8 GB 30 GB 1 GigE

TECH N I C AL WH ITE PAPE R / 2 8

 Architecting a vCloud

ITEM VC P U MEMORY S TO R AG E N E T WOR KING

vShield Manager 1 4 GB 512 MB 100 MB

TOTAL 11 40 GB 161 GB* 3 GigE*

* Numbers rounded up or down will not impact overall sizing

For the table above, the Oracle Database will be shared between the vCenter Server, the vCloud Director cells,
and the vCenter Chargeback Server. Different users and instances should be used for each database instance
in-line with VMware best practices.
In addition to the storage requirements above, a NFS volume is required to be mounted and shared by each
vCloud Director cell to facilitate uploading of vApps from cloud consumers. The size for this volume will vary
depending on how many concurrent uploads are in progress. Once an upload completes the vApp is moved to
permanent storage on the datastores backing the catalogs for each organization and the data no longer resides
on the NFS volume. The recommended starting size for the NFS transfer volume is 250 GB. You should monitor
this volume and increase the size should you experience more concurrent or larger uploads in your environment.

5.3 Sizing the Workload Resource Group Clusters

Sizing for the workload resource group clusters can be difficult to predict since the provider is not in charge of
what the consumer may run. The provider is also not aware of existing usage statistics for VMs that are run in
the cloud. The information below should assist in initial sizing of the vCloud environment and is based on
information from the service definition. This information is being provided as examples. It is highly
recommended that you engage you local VMware representative for detailed sizing of your environment.
The service definition states that 50% of the total number of VMs will be run in the reservation pool model and
50% will be run in the Pay-As-You-Go model. Furthermore, the reservation pool is split into small, medium, and
large pools with a respective split of 75%, 20%, and 5%. Using the 50% above this means that small represents
37.5% of the total, medium represents 10% of the total, and large represents 2.5% of the total number of VMs in
the environment.
The definition for these resource pools and the split with the VMs is listed below. The total number of VMs of
1,500 from the public cloud service definition is used in the example below. You can change this total to reflect
your own target VM count.

TYPE OF RESOURCE POOL TOTA L P E R C E N TAG E TOTA L V M S

Pay-As-You-Go 50% 750

Small Reservation Pool 37.5% 563*

Medium Reservation Pool 10% 150

Large Reservation Pool 2.5% 37*

TOTAL 100% 1,500

* Note that some total VMs are rounded up or down due to percentages

TECH N I C AL WH ITE PAPE R / 2 9

 Architecting a vCloud

The service definition also calls out the distribution for VMs in the environment with 45% small, 35% medium, 15%
large, and 5% extra large. Below is a chart that shows the total amount of memory, CPU, storage, and networking
based on the service definition assumptions and the total VM count from the public cloud service definition.

ITEM PERCENT VC P U S MEMORY S TO R AG E N E T WOR KING

Small 45% 675 675 GB 40.5 TB 400 GB

Medium 35% 1,050 1,050 GB 31.5 TB 300 GB

Large 15% 900 900 GB 54 TB 400 GB

Extra Large 5% 600 600 GB 4.5 TB 200 GB

TOTAL (1,500) 100% 3,225 3,225 GB 130.5 1,300 GB

The above numbers may shock you. Before you determine your final sizings you should refer to VMware best
practices for common consolidation ratios on the above resources. An example table has been provided below
to show you what final numbers could look like using typical consolidation ratios seen in field deployments.

RESOURCE BEFORE R AT I O AFTER

CPU 3,225 8:1 403 vCPUs

Memory 3,225 GB 1.6:1 2,016 GB

Storage 130.5 TB 2.5:1 52 TB

Network 1,300 GB 6:1 217 GB

The above calculations could be served by 16 of the following hosts.

Socket count: 4
Core count: 6
Hyper threading: Yes
Memory: 128 GB
Networking: Dual 10 GigE
The above calculations do not take into account the storage consumed by consumer’s or provider’s templates.
The above calculations also do not take into account the resources consumed by the vShield Edge appliances
that are deployed for each organization. There will be a vShield Edge for each private organization network and
external organization network. Given the current service definition target of 25 organization a maximum of 275
vShield Edge appliances will be created.
The specifications for each vShield Edge appliance are listed below.
CPU: 1 vCPU
Memory: 64 MB
Storage: 16 MB
Network: 1 GigE (this is already calculated in the throughput of the workloads and should not be added again)

VMware, Inc. 3401 Hillview Avenue Palo Alto CA 94304 USA Tel 877-486-9273 Fax 650-427-5001 www.vmware.com
Copyright © 2010 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed
at http://www.vmware.com/go/patents. VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be
trademarks of their respective companies. Item No: VMW_11Q4_WP_Architecting_p30_A_R2