Power and SVCZoning V2

Advanced Technical Skills (ATS) North America
Overview
Dan Braden and Bill Wiegand

IBM Advanced Technical Skills
8/8/16
2012 IBM Corporation
Power VIO and SVC Disk Path Design
Dan Braden IBM Power ATS

Bill Wiegand IBM Storage ATS
December 14, 2011
Agenda
Determine and examine all possible 4 path designs
Dual VIOS, dual SAN fabric and SVC
Examine each for availability characteristics
Handling multiple VIOCs
Extend to more SVC IO groups
Increasing bandwidth
Objective
The objective of this presentation is to show alternative disk path designs to
setup a dual VIOS environment with SVC storage, and their availability
characteristics.
This will examine zoning with 4 vFCs, each connected to a single SVC port
to generate 4 paths/LUN.
Definitions
SAN link a cable connecting a storage port to the SAN or a server port to the SAN
Disk path a logical path from a specific host port to a specific storage port
WWPN World Wide Port Name unique 16 digit ID for a host or storage port
Active/passive disk subsystem a disk subsystem with dual controller/processors
where a controller handles all IOs for a LUN except under failure scenarios
Half the LUNs are typically owned by each controller to use all resources
VIOS Virtual IO Server
VIOC Virtual IO Client
vFC Virtual Fibre Channel adapter created with VIO for a VIOC
Tied to a specific real FC adapter in a VIOS
Has an online and offline WWPN to facilitate LPM
The active WWPN changes with each LPM
LPM Live Partition Mobility the ability to move a running VIOC LPAR from one
Power system to another
Requires all IO for the VIOC to be virtualized thru VIOSs
WWPN zone: Typically a single initiator (host port) zone indicating which storage
ports with which the host port may communicate:
host port WWPN, storage port WWPN, storage port WWPN, .
Zoning nomenclature
Single initiator zone a zone with one host port, and one or more storage ports
A best practice
Alias Used to give a descriptive name to WWPNs
Samples:
vFC0_myhost the first virtual adapter port on the myhost LPAR
LPARa_fcs0 - fcs0 on a LPAR representing a host port
N1_P3 SVC Node 1 Port 3
This presentation assumes WWPN zoning
Availability Requirements
Wed like to setup the zoning such that we can survive a double failure
Failure of two of : VIOS, SAN fabric or SVC node
But not failure of both VIOSs, both SAN fabrics or both SVC nodes
which will always cause an outage
With 2 ports per VIOS or SVC node, we can ignore failure of individual ports
Two ports failing in a VIOS or SVC is equivalent to the VIOS/SVC failing
Why 4 paths?
Minimizing the number of paths minimizes the time needed by AIX to
determine paths have failed and to recover
AIX must distinguish between a slow IO and a path failure
We dont want to mark paths as failed if the IO is just slow
SAN cabling
Power Server
VIOC
server port 1
VIOS1
VIOS2
Assumes 2 host ports per VIOS

Additional ports can be used for
additional bandwidth
One may create up to 4 vFCs here
per VIOC
Availble host HBAs have 1, 2 or 4
ports
Each SVC node has 4 ports
server port 4
SAN1
SAN2
N2P3
Node 2 Port 3
N1P1
Node 1 Port 1
SVC
node1
SVC
node2
SVC IO Group
How do we get 4 paths?

Given 4 host ports and 4 storage ports we have 16 potential paths
Paths reduced by half due to dual SAN fabric
Can be further reduced by SAN zoning at the switch
Can be further reduced by LUN masking at the SVC
A LUN served from the SVC is served from one IO group only
Connections to other IO groups do not increase the number of paths
Options for 4 paths:
4 vFCs each connected to a different storage port
2 vFCs each connected to two storage ports
Doesnt meet availability requirements as each vFC only uses one SAN fabric, so
failure of a SAN fabric and the real adapter the other vFC uses causes an outage
1 vFC connected to 4 storage ports
Doesnt meet availability requirements as this can only use one SAN fabric
Thus well use sets of ports that include 4 unique host ports and 4 unique storage ports
Well examine all the possible ways to use 4 vFCs connected to 4 storage ports
Skip to slide 28 to see designs meeting the requirement
Or go thru them to understand what designs to avoid and why
How many ways are there to set this up?

Given 4 host ports and 4 storage ports, with one host port connected to one storge
port, there are 24 ways to pair up the ports 4!=24
Not all can use all 4 storage ports
Many are equivalent
Table entries are host port#, storage port#
Design#
1
11
22
33
44
2
11
22
34
43
3
11
23
32
44
4
11
23
34
42
5
11
24
32
43
6
11
24
33
32
7
12
21
33
44
8
12
21
34
43
9
12
23
31
44
10
12
23
34
41
11
12
24
31
43
12
12
24
33
41
13
13
21
32
44
14
13
21
34
42
15
13
22
31
44
16
13
22
34
41
17
13
24
31
42
18
13
24
32
41
19
14
21
32
43
20
14
21
33
42
21
14
22
31
43
22
14
22
33
41
23
14
23
31
42
24
14
23
32
41
Since theres no difference between ports on a VIOS, or ports on a SVC node, we

can redo the table showing VIOS# and SVC node# for the ports
Table entries are VIOS#, SVC node#
Design#
1
11
11
22
22
2
11
11
22
22
3
11
12
21
22
4
11
12
22
21
5
11
12
21
22
6
11
12
22
21
7
11
11
22
22
8
11
11
22
22
9
11
12
21
22
10
11
12
22
21
11
11
12
21
22
12
11
12
22
21
13
12
11
21
22
14
12
11
22
21
15
12
11
21
22
16
12
11
22
21
17
12
12
21
21
18
12
12
21
21
19
12
11
21
22
20
12
11
22
21
21
12
11
21
22
22
12
11
22
21
23
12
12
21
21
24
12
12
21
21
How many ways are there to set this up?

Examining each design for unique ones, and reducing the design name to (SVCN#,
SVCN#, SVCN#, SVCN#) for connection to VIOS1, VIOS1, VIOS2, VIOS2 respectively
yields these 6 designs:
VIOS1 port
1
VIOS1 port
2
VIOS2
port1
VIOS2
port2
Connected to
Connected to
Connected to
Connected to
SVCN#
SVCN#
SVCN#
SVCN#
Design
Or simply:
1. 1122
2. 1212
3. 1221
4. 2112
5. 2121
6. 2211
Design 1 with 4 paths/LUN - 1122

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Zone1: vFC1 WWPN1, N1P1
Italic zones are for inactive WWPNs
Disk Paths
vFC1 - N1P1
vFC2 N1P3
vFC3 N2P1
vFC4 N2P3
SVC IO Group
Design 1 with 4 paths/LUN VIOS failure

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 - N1P1
vFC2 N1P3
vFC3 N2P1
vFC4 N2P3
Failure of SVC node2 results in an outage
Lost zones and paths shown in red
SVC IO Group

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Zone1: vFC1 WWPN1, N1Px
Disk Paths
vFC1 - N1Px
vFC2 N2Px
vFC3 N1Px
vFC4 N2Px
Not possible to zone this to use all

storage ports
SVC IO Group

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 - N1P1
vFC2 N2P3
vFC3 N2P1
vFC4 N1P3
SVC IO Group

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 - N1P1
vFC2 N2P3
vFC3 N2P1
vFC4 N1P3
SVC IO Group
Design 3 with 4 paths/LUN SAN fabric failure

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 - N1P1
vFC2 N2P3
vFC3 N2P1
vFC4 N1P3
SVC IO Group
Design 3 with 4 paths/LUN SVC node failure

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 - N1P1
vFC2 N2P3
vFC3 N2P1
vFC4 N1P3
SVC IO Group

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 N2P1
vFC2 N1P3
vFC3 N1P1
vFC4 N2P3
SVC IO Group

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 N2P1
vFC2 N1P3
vFC3 N1P1
vFC4 N2P3
SVC IO Group
Design 4 with 4 paths/LUN SAN fabric failure

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 N2P1
vFC2 N1P3
vFC3 N1P1
vFC4 N2P3
SVC IO Group
Design 4 with 4 paths/LUN SVC node failure

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 N2P1
vFC2 N1P3
vFC3 N1P1
vFC4 N2P3
SVC IO Group

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SVC IO Group
SAN zoning
Disk Paths
vFC1 N2Px
vFC2 N1Px
vFC3 N2Px
vFC4 N1Px
Not possible to zone this to use all

storage ports
Design 6 with 4 paths/LUN

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 N2P1
vFC2 N2P3
vFC3 N1P1
vFC4 N1P3
SVC IO Group

Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node1
SVC
node2
SAN zoning
Disk Paths
vFC1 N2P1
vFC2 N2P3
vFC3 N1P1
vFC4 N1P3
SVC IO Group
Designs meeting availability requirements

Only designs 3 and 4 provide availability in case of a double failure
There are inferior ways to zone
Design 3
Design 4
Power Server
Power Server
LPAR1
LPAR1
vFC4
vFC3
vFC2
vFC1
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
VIOS1
VIOS2
SAN1
SAN2
SAN1
SAN2
SVC
node
1
SVC
node
2
SVC
node
1
SVC
node
2
SVC IO Group
SVC IO Group
Designs meeting availability requirements

VIOS1 port
1
VIOS1 port
2
VIOS2
port1
VIOS2
port2
Connected to
Connected to
Connected to
Connected to
SVCN#
SVCN#
SVCN#
SVCN#
Design
Only designs 3 and 4 meet availability requirements avoid the others
Adding more VIOC LPARs

Power Server
LPAR2
LPAR1
vFC4
vFC3
vFC2
vFC1
vFC4
vFC3
vFC2
vFC1
vFC4
vFC3
vFC2
vFC1
vFC4
vFC3
vFC2
vFC1
LPAR1
Power Server
VIOS1
VIOS2
VIOS1
VIOS2
LPAR2
4 host ports
8 host ports
SAN1
SAN2
SAN1
SAN2
SVC
node1
SVC
node2
SVC
node1
SVC
node2
SVC IO Group
Using the same FCs on the VIOS

Rotate LPARs across SVC ports
SVC IO Group
Using other FCs on the VIOS

Rotate LPARs across port sets
Using a balanced resource approach is cost effective

Where bandwidth of host ports = bandwidth of storage ports
Bandwidth of ports varies depending on the model
Latest HW: SVC port bandwidth of about 40,000 IOPs, Host port bandwidth around 50,000 IOPS
Option on right is more balanced
Adding SVC IO Groups

Allows additional SVC resources to be used
Node processors and cache
LUNs are served up from one IO group only
Still 4 paths/LUN
The zoning changes:
Power Server
LPAR1
vFC4
vFC3
vFC2
vFC1
SVC
node
1
VIOS1
VIOS2
SAN1
SAN2
SVC
node
2
SVC IO Group
SVC
node
3
SVC
node
4
Zone1: vFC1 WWPN1, N1P1, N3P1

SVC IO Group
Adding IO bandwidth to a VIOC LPAR

To add bandwdith to a VIOC, one must add more physical adapter ports
Add one port with 1 additional path or
Add a set of ports
Power Server
LPAR1
vFC8
vFC7
vFC6
vFC5
vFC4
vFC3
vFC2
vFC1
VIOS1
VIOS2
SAN1
SAN2
SVC
node
1
SVC
node
2
Zone the new ports like the old ports

Storage administrator assigns LUNs to new host ports, or
Move half or some of the LUNs to the new port set
SVC IO Group
Error message from SVC

The current SVC code (as of 12/2011) displays a Degraded path warning message when a host port
is zoned to only one node in an I/O group, even if the second host port is zoned to the other node in
the same I/O group providing path redundancy to a volume:
The GUI panel shows the 4 paths to the inactive host WWPNs used by LPM as Offline
This is normal and expected behavior with LPM
The GUI panel also shows the 4 active host paths used for I/O as Degraded
This is expected behavior but development is looking at a change in the future
The # Nodes Logged In column shows how many nodes in cluster see the host ports
References
Disk Path Design tech doc:
http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101914
SDDPCM manual:
http://www-01.ibm.com/support/docview.wss?rs=540&context=ST52G7&uid=ssg1S7000303
Dynamically adding a FC adapter to a partition tech doc:
http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD105218
vFC adapter properties using NPIV tech doc:
http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/FQ128819
Systems Hardware information center: Virtual Fibre Channel
http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=/iphat/iphblconfigvfc.htm

Power and SVCZoning V2

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Power and SVCZoning V2

Enviado por

Direitos autorais:

Formatos disponíveis

Advanced Technical Skills (ATS) North America

Dan Braden and Bill Wiegand

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

Power VIO and SVC Disk Path Design

Dan Braden IBM Power ATS

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

Assumes 2 host ports per VIOS

Advanced Technical Skills (ATS) North America

How do we get 4 paths?

Advanced Technical Skills (ATS) North America

How many ways are there to set this up?

Since theres no difference between ports on a VIOS, or ports on a SVC node, we

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

How many ways are there to set this up?

2012 IBM Corporation

Advanced Technical Skills (ATS) North America

Design 1 with 4 paths/LUN - 1122

Advanced Technical Skills (ATS) North America

Design 1 with 4 paths/LUN VIOS failure

Advanced Technical Skills (ATS) North America

Design 2 with 4 paths/LUN - 1212

Not possible to zone this to use all

Advanced Technical Skills (ATS) North America

Design 3 with 4 paths/LUN - 1221

Advanced Technical Skills (ATS) North America

Design 3 with 4 paths/LUN VIOS failure

Advanced Technical Skills (ATS) North America

Design 3 with 4 paths/LUN SAN fabric failure

Advanced Technical Skills (ATS) North America

Design 3 with 4 paths/LUN SVC node failure

Advanced Technical Skills (ATS) North America

Design 4 with 4 paths/LUN - 2112

Advanced Technical Skills (ATS) North America

Design 4 with 4 paths/LUN VIOS failure

Advanced Technical Skills (ATS) North America

Design 4 with 4 paths/LUN SAN fabric failure

Advanced Technical Skills (ATS) North America

Design 4 with 4 paths/LUN SVC node failure

Advanced Technical Skills (ATS) North America

Design 5 with 4 paths/LUN - 2121

Not possible to zone this to use all

Advanced Technical Skills (ATS) North America

Design 6 with 4 paths/LUN

Advanced Technical Skills (ATS) North America

Design 6 with 4 paths/LUN VIOS failure

Advanced Technical Skills (ATS) North America