Introduction to Fibre Channel

This course presents the student with the necessary information to understand the basics of Fibre Channel protocol, Fibre Channel topologies, WWN usage, and Fibre Channel logins in a SAN or NAS environment. Upon completion of this course, you will be able to: Discuss basics of Fibre Channel protocol Identify differences between the Fibre Channel topologies Identify three types of Fibre Channel logins and their functions Define WWN and discuss its role in Fibre Channel environments

Fibre Channel Overview

Fibre Channel is a connectivity technology designed to transmit data between devices, commonly in a storage area network. The objectives covered in this module are: Explain the role of Fibre Channel as a transmission interface List the Fibre Channel levels and discuss each level Identify characteristics of Fibre Channel transmission media

Page 1 of 46

Fundamentals
Fibre Channel extends networking concepts to storage, including distance, speed, connectivity, scalability, and reconfiguration. The Fibre Channel technology provides for improved data transfer rates and greater distances than traditional Small Computer System Interface (SCSI) cable. Upon completion of this lesson, you will be able to: Define Fibre Channel and describe its use in a storage area network Compare and contrast Fibre Channel and SCSI Identify EMC products that use Fibre Channel connectivity

Definition and Use

Fibre Channel is a serial data transfer interface that operates over copper wire and optical fiber. Fibre Channel solves many problems associated with input/output (I/O) interfaces and interconnections among systems in an environment.

Page 2 of 46

Performance Impact

Simultaneous communication among hosts, servers, storage systems and other devices is possible through Fibre Channel technology, using SCSI, IP, and other protocols. This interconnection is available for different topologies, providing reliability and high bandwidth.

Associated Products

Products associated with an Enterprise Storage Network facilitate fast, reliable interconnects, including switches, hubs, adapters and storage arrays.

Page 3 of 46

Standards

The Fibre Channel Physical and Signaling standard, ANSI, and ISO specify standards for Fibre Channel. A Standard is a set of documents that is widely recognized and employed, defining the set of rules and configurations of a product or practice. All Standards are subject to revision, and all parties are subject to agreements based on a Standard.

Protocol Standards

The Fibre Channel alliance determines the standards for the Fibre Channel protocol. Protocol standards can be obtained by going to www.t11.org.

Page 4 of 46

Comparison to SCSI

The first generation of storage was connected using SCSI. By adding an external SCSI connection from a host to a storage system, the amount of storage was drastically increased. Certain conditions and limitations were imposed on this configuration, based on the SCSI standards.

Increased Distance and Throughput

Replacing the hardwired standard SCSI cable with fiber cable in a Fibre Channel environment resulted in an increase in both the distance between a host system and a storage system, and the throughput.

Page 5 of 46

Protocols Mapped

Networking and I/O protocols such as SCSI commands are mapped to Fibre Channel constructs, then encapsulated and transported within Fibre Channel frames. This allows the SCSI protocol to be used over a Fibre Channel network.

Benefit of Single Technology

Fibre Channel is designed to provide throughput that is not protocol dependent. It allows a consolidation of storage systems and hardware such as cables and connectors.

Page 6 of 46

Internet Protocol

Traditionally, SCSI has been used as an I/O interface, utilizing bus-oriented interconnection. Today we see the emergence of IP as another possible protocol that could be enhanced by being used with Fibre Channel. It is possible to have several Upper Level Protocols (ULP) transmitted within a Fibre Channel environment, simplifying the physical layout of a storage area network.

Fibre Channel and EMC Products, Services, and Offerings

EMC offers a series of Fibre Channel devices and services. Among these offerings is the Connectrix Family of Switches. Connectrix switches are generally divided into two categories: M-Series and B-Series.

Page 7 of 46

Management Software

Management tools are available for each type of switch. EMC ESN Manager is a consolidated tool for managing zoning and volume access on switches and directors. Each switch series also has a native management tool that provides its own flexible scheme for zoning management and switch management applications, including Connectrix Manager for the M-Series switches, and web and telnet-based management tools for B-Series switches.

Storage Systems

EMC offers Fibre Channel connectivity to its storage systems: Symmetrix and CLARiiON.

Page 8 of 46

Lesson Summary

Fibre Channel provides protocol independent distance, throughput, and speed that is superior to the capabilities of SCSI. EMC provides fibre channel connectivity to storage arrays through the Connectrix family of switches and storage systems.

Fibre Channel Topologies

Page 9 of 46

Fibre Channel Levels

The Fibre Channel Architecture has been designed in levels to allow for manageability and future growth. Fibre channel levels can be compared to the ISO/OSI 7-layer model in that it attempts to define the different aspects of Fibre Channel in manageable pieces. However all five levels of Fibre Channel would fit within the first three layers of the 7-layer model, and the routing capabilities of Fibre Channel are limited. Upon completion of this lesson, you will be able to: Describe attributes of FC-0, FC-1, and FC-2 Identify the roles of FC-3 and FC-4 in Fibre Channel protocol

Designed for Flexibility

The Fibre Channel Architecture has been designed in levels to allow for manageability and future growth. Similar to the ISO/OSI 7Layer model, any of the levels could be updated or modified without impact to the other levels.

Page 10 of 46

FC-0

FC-0 is the lowest level and defines the physical link in the system, including the fibre, connectors, optical and electrical components. This level covers a variety of media and the associated drivers and receivers capable of operating at a wide range of speeds.

Fibre and Port Attachment

Each fibre cable is attached to a transmitter of a port at one end and a receiver of another port at the other end. When a fabric is present in the configuration, a fibre cable may attach between an N_Port and an F_Port. In an Arbitrated Loop, the cable attaches between NL_Ports. Other devices may be present and function as repeaters, concentrators or fibre converters.

Page 11 of 46

Connectors

Optical and Electrical connectors are specified in Fibre Channel. The SC connector is the standard connector for fiber optic cables. It is a push-pull connector and is favored over the ST connector. If the cable is pulled, the tip of the cable in the connector does not retract, which would result in loss of signal quality.

Multimode Cable

Multimode cable is dominant for short distances of 500 meters or less. Multimode has an inner diameter of 62.5 or 50 microns, allowing light to enter the cable in multiple modes, including straight and at different angles. The many light beams tend to lose shape as they move down the cable. This loss of shape is called modal dispersion and limits the distance.

Page 12 of 46

Single Mode Cable

Single mode cable is used for long distance cable runs, only limited by the power of the laser at the transmitter and by the sensitivity of the receiver. Single mode cable has an inner diameter of 9 microns and is always used with a long wave laser, which limits the effects of modal dispersion. Therefore, with single mode cables the distance is greatly increased.

FC-1

FC-1 defines the transmission protocol including serial encoding and decoding, special characters and error control. Information is encoded 8 bits at a time into a 10 bit transmission character and transmitted over the cable.

Page 13 of 46

Encoding Process

Fibre Channel transmits information using an adaptive 8B/10B code to bound the maximum run length of the code, maintain DCbalance, and provide word alignment. The encoding process results in the generation of Transmission Characters.

Ordered Sets

Two types of Transmission Characters (Data and Special) are defined. Certain combinations of Transmission Characters, referred to as Ordered Sets, are designated to have special meaning. Ordered Sets are used to identify frame boundaries, transmit primitive function requests, and maintain proper link transmission characteristics during periods of inactivity.

Page 14 of 46

FC-2

The FC-2 level serves as the transport mechanism of Fibre Channel. The transported data is transparent to FC-2 and visible to FC3 and above.

Frames

Frames are the basic building blocks of a Fibre Channel connection. The frames contain the information to be transmitted, the address of the source and destination ports and link control information.

Page 15 of 46

Role of Frames

All information in Fibre Channel is passed in frames. The maximum amount of data carried in a frame is 2112 bytes, total frame size is 2148 bytes. The general structure of a Frame is specific.

Header Contents

The header contains the Source and Destination Addresses which allows the frame to be routed to the correct port. The Type field interpretation is dependent on whether the frame is a link control frame or a Fibre Channel data frame. For example, if the frame is a data frame, a 08 in the type field would indicate a SCSI FCP information in the Data field.

Page 16 of 46

Exchange

An Exchange is a unidirectional or bi-directional set of non-concurrent Sequences. An Exchange is the largest construct understood by FC-2. FC-2 manages Exchanges that map directly to operations.

Sequences

FC-2 manages Sequences as unidirectional transfers of one or more frames. A Sequence is contained within an Exchange and is comprised of one or more Frames. FC-2 names each Sequence and tracks each Sequence to completion. The purpose of the Sequence is to reorder data when it is received at the other end.

Page 17 of 46

Buffer to Buffer Credit

FC-2 provides flow control for buffer management. When nodes initialize on the fabric they agree on operational parameters such as the number of buffers available (Buffer Credits). Transmitting nodes can continue to transmit as long as there are buffer credits. R_RDY are sent to a transmitting node as buffers are cleared.An FC switch also performs buffering and flow control internally and externally.

Classes of Service

Classes of Service are different types of topology independent services provided by the Fabric and used by the communicating N_Ports destination. The allocation and retention method between the N_Ports and the level of delivery integrity required for an application distinguish classes of service. If the Fabric is not present, the service is provided as a special case of point-to-point. Fabrics and N_Ports are not required to support all Classes of service.

Page 18 of 46

FC-4 and FC-3

The FC-4 level of Fibre Channel is designed to hand off to another protocol such as SCSI. Fundamentally, the commands at FC-4 for SCSI allow SCSI initiator and target to communicate over Fibre Channel. FC-3 level functions are not implemented at this time.

Purpose of FC-3

FC-3 was put into Fibre Channel as a placeholder. In concept FC-4 would pass requests to FC-3 that would perform the desired service and then pass onto FC-2.

Page 19 of 46

Purpose of FC-4

The FC-4 level consists of several standards documents describing how different upper-level protocols (ULPs) use the transport services provided by levels FC-2, FC-1 and FC-0. The purpose of an FC-4 protocol mapping is to make a logical connection between the ULPs and Fibre Channel's transport facilities.

Lesson Summary

Fibre Channel levels, like the OSI model, are divided into manageable layers that have different functions. FC-0 and FC-1 are connectivity and transmission based, FC-2 is the transport mechanism and FC-4 connects these levels to ULPs.

Page 20 of 46

Module Summary

Fibre Channel-based enterprise storage networks dramatically increase the number of systems that connect to enterprise storage. The extended distance characteristics of Fibre Channel break through the walls of the data center to let departmental servers and databases participate in enterprise storage. The throughput that Fibre Channel provides make connecting hundreds of systems to centralized enterprise storage possible from a performance standpoint.

Using Fibre Channel

Fibre Channel connectivity transmits data in three main topologies. Fibre Channel frames provide routing information, and a series of logins allows information to be transmitted. Upon completion of this module, you will be able to: Differentiate between Arbitrated Loop and Switched Fabric topologies Discuss the role of Worldwide Names in Fabric Addressing Identify the use of fabric, port, and process logins in a fabric environment

Page 21 of 46

Fibre Channel Topologies

The three basic topologies in Fibre Channel are point-to-point, Arbitrated Loop (FC_AL) , and Switched Fabric (FC-SW). EMC supports FC-AL and FC-SW, and refers to a two node FC-AL as Direct Connect. Upon completion of this lesson, you will be able to: Describe characteristics of Direct Connect topology Describe characteristics of Arbitrated Loop topology Describe characteristics of Switched Fabric topology

Nodes

Nodes are central to any communication between two devices. In general, a node is a point of contact where data enters or leaves a device. An HBA in a host or a fibre port in a storage system can be referred to as a node.

Page 22 of 46

Node Ports

Each node has at least one port that connects to other ports on other nodes.

Port Types

There are several different port types, including: NL_Port, N_Port, F_Port, and E_Port.

Page 23 of 46

Direct Connect and Arbitrated Loop

In a direct connect system, the nodes are connected directly to one another. Arbitrated loop can also be used as a method of direct connection in which a hub is used and each node is cabled to the hub.

FC_AL Hub

Fibre Channel Arbitrated Loop allows only one pair of nodes to transfer data at a time, and nodes arbitrate for use. EMC requires this to be implemented using a FC-AL hub. The hub provides a full bandwidth loop that is shared between the nodes on the loop, however only one virtual circuit can be established at a time.

Page 24 of 46

Private Loop

The FC-AL architecture will support from 2 to 126 nodes on a private loop. During the loop initialization process (LIP), each node on the loop is assigned an Arbitrated Loop Physical Address (AL-PA).The nodes in the loop then follow a standard arbitration protocol that defines the rules for gaining access to the loop.

Loop Access

Loop access priority is based upon the AL-PA of the nodes. At anytime there can be only a single full bandwidth circuit open between 2 nodes on the loop.

Page 25 of 46

EMC Specification

EMC specifies a 2 node FC-AL connection called Direct Connect. This Topology is the same as the standard Arbitrated Loop except that there are two nodes in the Loop and the FC-AL Hub is not used.

Limitations

Direct connect and Arbitrated Loop pose limitations in certain environments.

Page 26 of 46

Port Density

A direct Fibre Channel direct connect between host and storage array provides greater distance and throughput than SCSI, but allows only a 1:1 port density ratio.

Performance Impact

Arbitrated loop solves the issue of port density by allowing multiple hosts to access multiple storage systems. However, only one virtual circuit, thus impacting overall SAN performance at a time is possible between nodes, thus limiting bandwidth in a storage area network.

Page 27 of 46

Switched Fabric

A Fabric is a switch or switches, managed centrally, and is capable of routing frames by using only the D_ID information in a FC-2 frame header. A fabric can be comprised of either a single switch or a multi-switch configuration.

Dedicated Bandwidth

In a switched fabric environment, the total network bandwidth is determined by the number of ports on the switch. For instance, an 8 port 100MB switch can support up to 800 MB/s aggregate bandwidth.

Page 28 of 46

Virtual Paths

A switched fabric allows multiple virtual circuits to transport data at a time, eliminating the bandwidth issues presented by Arbitrated Loop.

Greater Expandability

EMC has defined the maximum number of switches in a Fabric to be 16. Currently this allows over 1024 ports to be available in a single fabric. This number will increase over time.

Page 29 of 46

Price and Performance

Although the price of FC-SW devices is higher than that of FC-AL, the performance and throughput far exceed the capabilities of FC-AL Hub devices. Additionally, the management and redundancy features of FC-SW devices allow for a very stable, manageable and high-throughput environment.

Lesson Summary

The three basic topologies of Fibre Channel are direct connect, arbitrated loop, and switched fabric. Switched fabric provides the most throughput, flexibility, and scalability, allowing the highest number of ports in a single fabric.

Page 30 of 46

SCSI to Fibre Channel

Addressing and Worldwide Names

Worldwide Names are assigned to entities including ports. Worldwide Names are assigned by the vendor and do not change. When nodes generate frames, they are routed by their addresses, not their WWNs. A World Wide Name is a 64 bit value (16 characters).Upon completion of this lesson, you will be able to: Identify the contents of a Fibre Channel frame State the role of the WWN in a Fibre Channel environment Identify characteristics of Fibre Channel name formats

Page 31 of 46

Fibre Channel Addressing

Fibre Channel addresses are used to designate the source and destination of frames in the Fibre Channel network. The Fibre Channel address field is 24 bits /3 bytes in length.

Assigned Addresses

Unlike Ethernet, these addresses are not burned in, but are assigned when the node either enters the loop or is connected to the switch. There are reserved addresses, which are used for services rather than interface addresses.

Page 32 of 46

S_ID and D_ID

When an N_Port connects to the fabric, the switch assigns and transmits a physical fabric address to the N_Port. This address becomes the Source ID (S_ID) on the outbound frames and the Destination ID (D_ID) on the inbound frames.

Address Formats

Address identifiers are three bytes in length. The Frame Header contains two three-byte fields for address identifiers, the (D_ID) field and the (S_ID) field. Each N_Port has a fabric-unique identifier, the N_Port Identifier, by which it is known. The source and destination N_Port Identifiers and alias address identifiers are used to route frames within the fabric.

Page 33 of 46

Address Changes

The Physical address of an N_Port can change, such as when a link is moved from one switch port to another switch port. The WWN of the N_Port, however, is static.

Worldwide Name

Worldwide names are assigned by the vendor to all Fibre Channel entities.

Page 34 of 46

WWN Assignment

Each N_Port attached to a fabric has a World Wide Name. Most HBAs have the WWN "burned in" to the hardware. WWNs can also be assigned through software.

Routing

When nodes generate frames, they are routed by their addresses, not their WWNs. However, tables can be built which can associate WWNs to the destination addresses. A World Wide Name is a 64 bit value (16 characters).

Page 35 of 46

Fabric Environment

The most significant 8 bits of the Fibre Channel address contain the Domain ID, which basically identifies the switch. In the Fabric environment this allows frames to be routed between switches. The middle 8 bits contain the area address, which has been implemented as the port address within the switch.

Fibre Channel Name Formats

Fibre Channel specifications allow for multiple formats of the World Wide Name. The example shown is that of the IEEE Registered Name Format.

Page 36 of 46

Length of Fibre Channel Name

The Fibre Channel name is 64 bits in length The first 4 bits identify the format being used. The 5 in this example identifies the format as IEEE Registered Name Format.

IEEE company_id

Values for worldwide name formats are based on the IEEE company_id. More information on these formats can be found at www.standards.ieee.org.

Page 37 of 46

Lesson Summary

Fibre Channel protocol uses Worldwide Names to identify entities such as ports. Fibre Channel Addresses are used for routing of frames from their source to their destination.

Fibre Channel Logins

When using Fibre Channel to establish communication, a series of Fibre Channel logins are initiated. Upon completion of this lesson, you will be able to: Identify logins used in a Switched Fabric environment Identify logins used in Arbitrated Loop environment

Page 38 of 46

Switched Fabric

The fabric login is specific to Switched Fabric and does not occur in Arbitrated Loop or Direct Connect.

Fabric Login Process

An N_Port uses fabric login (FLOGI) to determine the presence of a fabric. The fabric login communicates identities and exchanges service parameters to establish a session with the fabric. It is also the mechanism by which the fabric assigns an address to an N_Port. A fabric login process begins when the N_Port performs a login (to address FFFFFE) using a source address of 000000. The Fabric Login service returns a frame that assigns a 24-bit address to the port.

Page 39 of 46

Distributed Name Service

Prior to performing an N_Port Login, the incoming Node should perform a registration with the distributed name service (dNS). The service which handles this is called the Directory Server and appears as a normal N_Port with the address of x' FF FF FC'.

Switched Fabric and Arbitrated Loop

Port and Process Logins occur in all Fibre Channel topologies. All relevant logins must occur in order for device discovery to take place.

Page 40 of 46

Arbitrated Loop

Loop Initialization is performed to assign an AL_PA to a Node entering a Loop. During the Loop Initialization Process, Primitives are sent to ensure that all Nodes have unique AL_PAs. Nodes previously within the Loop will retain their previous addresses, while new Nodes entering may acquire either a Hard or Soft assigned address.

Loop Initialization Process

Once a port has logged in, it can query the Name Service database for information about other logged in ports.

Page 41 of 46

Port Login

Fibre Channel logins are performed between ports prior to initiating higher-level protocol operations. The port login (PLOGI) follows the fabric login process, allowing two N_Ports to establish a session and exchange identities and service parameters.

Port Login Function

Port logins exchange information such as host source identification frame, frame size, and Port name; the output refers to these values as host SID, receive buffer size, and WWPN respectively.

Page 42 of 46

Port Data Table

When an N_Port logs in to another N_Port, tables are built which will keep track of the WWN of the logged-in port along with its Fibre Channel address.

Process Login

The process login establishes a session between two FC-4 level processes at different N_Ports, basically performing a binding between two processes.

Page 43 of 46

Process Information Exchange

Once the process login has been performed, the two ports may exchange information that can be used to identify a specific process at the initiator and target ports.

Lesson Summary

Fabric, port, and process logins allow SCSI device discovery to take place in a switched environment: The fabric login establishes a session with the fabric and assigns an address to an N_Port. The port login follows, allowing two N_Ports to establish a session. The process login then binds two processes, and the two ports may exchange information.

Page 44 of 46

Module Summary

Fibre Channel addressing is determined by the topology used. Addresses are dynamically assigned. The addresses include the Domain, Area, and Port fields. The Domain ID portion of an address identifies the switch. Fibre Channel N_ports are usually referred to by their WWN. Everything, including the fabric itself, in a Fibre Channel environment has a WWN.

Course Summary

Fibre Channel provides a number of benefits to enterprise storage capabilities, including the flexibility of encapsulating and transporting SCSI commands in fibre channel frames. Each device in a fabric has a node that transports information, such as the HBA on the host, the FA on Symmetrix, and the SP on CLARiiON. Addresses containing source and destination information are assigned when the node logs into the fabric. Node ports are each identified with a unique Worldwide Name. Nodes use a series of logins based on Fibre Channel protocol. The fabric login is first used to establish a session. The port login allows information exchange between ports. The process login then binds the two ports.

Page 45 of 46

Course Conclusion

Page 46 of 46