Escolar Documentos
Profissional Documentos
Cultura Documentos
实现双机的一些讨论
• VIOS Basic
• 实现分区双机需要几个VIOS
• 心跳/网卡/HBA的数量
• 访问存储vSCSI or NPIV?哪种性能更好?
What is PowerVM?
Hardware and software that delivers industry-leading virtualization on
IBM POWER processor-based processors for UNIX, Linux and i5/OS
customers
PowerVM Editions:
PowerVM Enterprise Edition
PowerVM Standard Edition
PowerVM Express Edition
IBM虚
虚拟化技术发展历程
A 40 year tradition culminates with PowerVM
client quote source: Brakes India case study published at http://w w w .ibm.com/softw are/success/cssdb.nsf/CS/STRD-76DHWE?OpenDocument&Site=eserverpseries&cty=en_us
PowerVM Edition
Servers
p520 / p550 p5, p6 and JS2X p6 and JS22
Supported
VIOS Y es Y es Yes
Y es Yes
Multiple Shared Processor Pools No
(HMC) (HMC)
Operating Systems AIX & Linux AIX & Linux AIX & Linux
• Virtual I/O
- Storage
- vSCSI (Storage Virtualizer)
- NPIV (Pass-through)
- Virtual Networking
- Shared Ethernet Adapter (SEA)
- Integrated Virtual Ethernet (IVE)
Power™ Systems Isolates I/O Virtualization with the Virtual
I/O Server
Workload Management and Provisioning
Virtual
I/O
Servers Unassigned
AIX Linux i5/OS
Virtual On Demand
Partitions Partitions Partitions
Adapters Resources
Virtual
Hardware Disks
Management
Console POWER
Virtual Networks
Hypervisor
Firmware Virtual Processors Virtual Memory
POWER Processors
Server
Hardware Memory
I/O Expansion Slots
Service
Subsyst. Local Devices and Storage
Virtualization of POWER5 and POWER6 servers is accomplished using two layers of firmware:
• A thin core hypervisor that virtualizes processors, memory, and local networks
• One or more Virtual I/O Server partitions that virtualize I/O adapters and devices
VIOS Virtual Storage
Micro-partition sees disks as vSCSI Multiple LPARs can use same or different
(Virtual SCSI) devices physical disk
Virtual SCSI devices added to partition Configure as logical volume on VIOS
via HMC Appear a hdisk on the micro-partition
LUNs on VIOS accessed as vSCSI disk Can assign entire hdisk to a single client
VIOS must be active for client to boot
Available via optional Advance POWER Virtualization or POWER Hypervisor and VIOS features.
Virtual SCSI Basic Architecture
AIX Partition Virtual I/O Server
PV LV Optical
VSCSI VSCSI VSCSI
LVM
Multi-Path
Optical
DVD Hdisk or
Driver
Disk Drivers
vSCSI vSCSI
Client Server Adapter /
Adapter Adapter Drivers
POWER5 Hypervisor
FC or SCSI
Device
Virtual SCSI optical devices
• Complexity
– Simpler to setup and manage than dual VIOS
– No specialized setup on the client
• Resilience
– VIOS, SCSI adapter, SCSI disk are potential
single points of failure
– The loss of a physical disk may impact more
than one client
• Throughput / Scalability
– Performance limited by single SCSI adapter and
internal SCSI disks.
• Notes
– Low cost disk alternative
Virtual SCSI Options (1) – Details
Single VIOS, LV VSCSI Disks
hdisk0 hdisk0
vscsi0 vscsi0
Hypervisor
vhost0 vhost1
vtscsi0 vtscsi1
LV LV
VIOS 1 scsi0
A disk
B disk
Virtual SCSI Options (2)
Single VIOS, PV VSCSI Disks
• Complexity
– Simpler to setup and manage than dual VIOS
– No specialized setup on the client
• Resilience
– VIOS, SCSI adapter, SCSI disk are potential
single points of failure
– The loss of a single physical client disk will
affect only that client
• Throughput / Scalability
– Performance limited by single SCSI adapter and
internal SCSI disks.
Virtual SCSI Options (2) – Details
Single VIOS, PV VSCSI Disks
hdisk0 hdisk0
vscsi0 vscsi0
Hypervisor
vhost0 vhost1
vtscsi0 vtscsi1
VIOS 1 scsi0
A disk
B disk
Virtual SCSI Options (3)
Single VIOS with Multi-Path I/O
• Complexity
– Simpler to setup and manage than dual VIO
servers
– Requires Multi-Path I/O setup on the VIOS
– No specialized setup on the client
• Resilience
– VIOS is a single point of failure
• Throughput / Scalability
– Potential for increased bandwidth due to multi-path
I/O.
– Could divide clients across independent VIOS
allowing more VIOS adapter bandwidth.
Virtual SCSI Options (3) – Details
Single VIOS with Multi-Path I/O
hdisk0 hdisk0
vscsi0 vscsi0
Hypervisor
vhost0 vhost1
vtscsi0 vtscsi1
Multi-Path Driver
B disk
PV LUNs
Virtual SCSI Options (4)
AIX Client Mirroring with direct attach SCSI and VIOS PV VSCSI disks
• Complexity
– Requires LVM mirroring to be setup on the VIOC
– Multi-Path I/O setup on the VIOS
– If a VIOS is rebooted, the mirrored disks will need
to be resynchronized via a varyonvg on the VIOC.
– Additional complexity due to multiple disk types,
Multi-Path I/O setup, and client mirroring.
• Resilience
– Protection against failure of single adapter failure
(or path) or disk
– Potential protection against FC adapter failures
within VIOS (if Multi-Path I/O is configured)
* Note: See the slide labeled VIOS Multi-Path Options for a high level overview of MPATH options.
Virtual SCSI Options (4) – Details
AIX Client Mirroring with direct attach SCSI and VIOS PV VSCSI disks
Hypervisor
A disk B disk
vhost0 vhost1
vtscsi0 vtscsi1
Multi-Path Driver
A’ disk
B’ disk
PV LUNs
Virtual SCSI Options (5)
AIX Client Mirroring, Single Path in VIOS, PV VSCSI Disks
• Complexity
– More complicated than single VIO server
but does not require SAN ports or setup
– Requires LVM mirroring to be setup on the
client
– If a VIOS is rebooted, the mirrored disks
will need to be resynchronized via a
varyonvg on the VIOC
• Resilience
– Protection against single VIOS / SCSI
disk / SCSI controller
– The loss of a single physical disk would
affect only one client
• Throughput / Scalability
– VIOS performance limited by single SCSI
adapter and internal SCSI disks.
Virtual SCSI Options (5) – Details
AIX Client Mirroring, Single Path in VIOS, PV VSCSI Disks
A disk A’ disk
B disk B’ disk’
Virtual SCSI Options (6)
AIX Client Mirroring, Multi-Path I/O in VIOS, LV or PV VSCSI FC Disks
• Complexity
– Requires LVM mirroring to be setup on the VIOC
– Requires Multi-Path I/O setup on the VIOS
– If a VIOS is rebooted, the mirrored disks will
need to be resynchronized via a varyonvg on the
VIOC
• Resilience
– Protection against failure of single VIOS / FC
adapter failure (or path)
– Protection against FC adapter failures within
VIOS
• Throughput / Scalability
– Potential for increased bandwidth due to multi-
path I/O
• Notes
– LUNs used for this purpose can only be assigned
to a single VIOS
– LV VSCSI LUNs could also be PV VSCSI LUNs.
* Note: See the slide labeled VIOS Multi-Path Options for a high level overview of MPATH options.
Virtual SCSI Options (6) – Details
AIX Client Mirroring, Multi-Path I/O in VIOS, LV VSCSI FC Disks
A disk A’ disk
B disk B’ disk
PV LUNs PV LUNs
NPIV specifics
– Fibre Channel industry standard for adapter sharing
• Pass-through model
– unique WWPN generation (allocated in pairs)
- Each virtual FC HBA has a unique and persistent identity
– Each physical NPIV capable FC HBA will support 64 virtual ports
– HMC-managed and IVM-managed servers
vSCSI NPIV
vio client vio client
generic generic
scsi disk scsi disk EMC
EMC IBM 2105
SAN SAN
The vSCSI model for sharing storage resources is With NPIV, the VIOS's role is fundamentally
storage virtualizer. Heterogeneous storage is different. The VIOS facilitates adapter sharing
pooled by the VIOS into a homogeneous pool of only, there is no device level abstraction or
block storage and then allocated to client LPARs emulation. Rather than a storage virtualizer, the
in the form of generic SCSI LUNs. The VIOS VIOS serving NPIV is a passthru, extending an
performs SCSI emulation and acts as the SCSI FCP connection from the client LPAR to the SAN.
Target.
Data flow using LRDMA for vSCSI and NPIV
data
buffer vSCSI vSCSI physical
& & adapter
VFC VFC
driver
Da
ta
(L control
RD
M phyp
A)
*
pci adapter
VIOS#1 AIX
Passthru module A
NPIV
NPIV
Fibre
Fibre
HBA
HBA
POWER Hypervisor
Storage Controller
SAN Sw itch SAN Sw itch
A B C D
A’ B’ C’ D’
Virtual Networking
- Shared Ethernet Adapter (SEA)
-Integrated Virtual Ethernet (IVE)
POWER5 Server
External Servers
Micro-partitions
VIOS
AIX 5L AIX 5L
VLAN 1 vSCSI Linux Linux
V5.3 V5.3
VLAN 2
VLAN 3 vLAN
Shared
Ethernet
Ethernet Switch Adapter
Virtual Ethernet Switch
•Virtual Ethernet
POWER Hypervisor
–Partition to partition communication
–Requires AIX 5L V5.3 and POWER5
•Shared Ethernet Adapter
–Provides access to outside world •IEEE VLANS
–Uses Physical Adapter in the Virtual I/O Server –Up to 4096 VLANS
•VLAN – Virtual LAN –Up to 65533 vENET adapters
–Provide ability for one adapter to be on multiple subnets –21 VLANS per vENET adapter
–Provide isolation of communication to VLAN members
–Allows a single adapter to support multiple subnets
Available via optional Advance POWER Virtualization or POWER Hypervisor and VIOS features.
Shared Ethernet Adapter
– A Shared Ethernet Adapter (SEA) can be used to connect a
physical Ethernet network to a virtual Ethernet network. It
also provides the ability for several client partitions to share
one physical adapter.
– The SEA hosted in the Virtual I/O Server (VIOS) acts as a
layer-2 bridge between the internal and externalnetwork.
– The Shared Ethernet Adapter service runs in the Virtual I/O
Server. It cannot be run in a general purpose AIX 5L partition
ent0
POWER5 Server
ent0
(virt)
Logic ent2
(SEA)
ent0
Port
(phy)
• Complexity
– Easy to setup and manage
– No specialized configuration on switch
– No specialized configuration on client
• Resilience
– VIOS Ethernet adapter, switch port
and switch are single points of failure
– Throughput / Scalability
– Performance limited to a single
Ethernet adapter
Virtual Ethernet Options (1) – Details
Single VIOS – Single LAN Segment
ent0 ent0
(virt) (virt)
PVID=2 PVID=2
Hypervisor
VLAN 2
PVID=2
VIOS 1 ent1
(virt)
ent2
(SEA)
ent0
(phy)
Untagged Port
Ethernet Switch
Virtual Ethernet Options (2)
Single VIOS – Multiple LAN Segments
• Complexity
– Easy to setup and manage
– No specialized configuration on switch
– No specialized configuration on client
• Resilience
– VIOS Ethernet adapter, switch port, and
switch are single points of failure
• Throughput / Scalability
– Performance limited to a single Ethernet
adapter for each LAN segment
Virtual Ethernet Options (2) – Details
Single VIOS – Multiple LAN Segments
VLAN 3
VLAN 2
PVID=3 PVID=2
• Complexity
– Requires specialized setup on client (NIB)
– Needs to ping outside host from the client to
initiate NIB failover
• Resilience
– Protects against single VIOS / switch port /
switch / Ethernet adapter failures
• Throughput / Scalability
– Allows load-sharing between VIOS’s
• Notes
– NIB does not support tagged VLANs on
physical LAN
– Must use external switches not hubs
Virtual Ethernet Options (3) – Details
AIX Network Interface Backup (NIB), Dual VIOS
ent2 ent2
(la) (la)
NIB NIB
ent1 ent0 ent0 ent1
(virt) (virt) (virt) (virt)
PVID=2 PVID=3 PVID=2 PVID=3
Hypervisor
VLAN 2
ent2 ent2
(SEA) (SEA)
ent0 ent0
(phy) (phy)
Untagged VLAN
Ethernet Switch Ethernet Switch
Virtual Ethernet Options (4)
AIX Network Interface Backup (NIB) , Dual VIOS with Link Aggregation
(LA)
• Complexity
– Requires specialized setup on client (NIB)
– Requires link aggregation setup on external
switches
– Needs to ping outside host from the client to
initiate NIB failover.
• Resilience
– Protection against single VIOS/ switch port /
switch / Ethernet adapter failure
– Protection against adapter failures within VIOS
• Throughput / Scalability
– Allows each client to use a different primary
VIOS sharing network load across multiple
VIOS’s.
– Potential for increased bandwidth with LA
• Notes
– NIB does not support tagged VLANs on
physical LAN
– Must use external switches not hubs
Virtual Ethernet Options (4) – Details
AIX Network Interface Backup (NIB), Dual VIOS with Link Aggregation
ent2 ent2
(LA) (LA)
NIB NIB
ent1 ent0 ent0 ent1
(virt) (virt) (virt) (virt)
PVID=2 PVID=3 PVID=2 PVID=3
Hypervisor
VLAN 2
Untagged VLAN
Ethernet Switch Ethernet Switch
Integrated Virtual Ethernet Overview(Power
6)
•Naming
–Integrated Virtual Ethernet – External name in marketing
–Host Ethernet Adapter (HEA) – Name used on user interfaces & documentation
Or
VIOS本身就是一个定制的AIX系统,只运行几个物理和虚拟device driver管理程序,
本身非常稳定。所以,对于普通应用要求,一个VIOS即可, 但对于需要如下需求,建
议配置2个VIOS。
• 网络心跳
• dedicated 串口心跳
• 共享磁盘心跳
-- vSCSI disk
-- NPIV
网卡/HBA卡数量
•网卡 -- 至少两个
•HBA卡 -- 至少两个
Shared Ethernet Adapter
VIOS 1 Client 1 Client 2
en8 en7
(if) (if)
ent2 ent1 ent4 ent5 ent0 ent3 ent0 ent1 ent0 ent1
(Phy) (Phy) (Vir) (Vir) (Phy) (Vir) (Vir) (Vir) (Vir) (Vir)
VID PVID VID 300 PVID 100 PVID PVID PVID
200 2 ( PVID 3 ) 300 2 200
Multi-path Multi-path
M M
Mirror M
M
NPIV redundancy options
VIO Client VIO Client
Mirror M
VIO Client
MPIO
VIOS VIOS
vSCSI vs NPIV
访问存 储vSCSI or NPIV?哪
哪种性能更好
vSCSI NPIV
vio client vio client
generic generic
scsi disk scsi disk EMC
EMC IBM 2105
SAN SAN
vSCSI