White Paper

Ethernet now offers the most comprehensive OAM for packet-based solutions
Introduction
OAM (Operations, Administration and Maintenance) describes the instrumentation of telecommunications networks that provide operators with the tools to remotely manage their network infrastructure. At the device level, OAM protocols generate messages that are used by operations staff to help identify problems in the network. For example, OAM allows automatic network monitoring, so that the operator receives notification of equipment outages. In the event of a fault, the information generated by OAM helps the operator troubleshoot the network to locate the fault, identify which services have been impacted and take the appropriate action. Also, just as it is important to keep the customers service running, operators must be able to prove that is the case, this is usually measured against an SLA (service level agreement) and the operator must have the performance measurements to manage customer SLAs. Finally, administration features include collecting accounting data for the purpose of billing and network usage data for capacity planning exercises. Networks with poor OAM are harder to debug; for example, a network operator reported a case where they sent maintenance teams to a customer site four times to check their MPLS equipment before discovering the problem was a customer misconfigured router. Carrier-grade OAM remote loopback testing in the data path would have proved that there was nothing wrong with the service. In short, OAM creates the environment that operators require to perform mature network and service management operations that allow operators to control their networks cost-effectively and manage their customers expectations. OAM functionality in traditional TDM networks is well-defined and is an important building block in ensuring operators can deliver carriergrade performance. However, OAM represents a significant challenge for next-generation technologies, such as Carrier Ethernet. Carrier Ethernet is a

term that has emerged to describe the evolution of Ethernet from a Local Area Network (LAN) technology, to one suitable for deployment in metropolitan and wide area networks for both Ethernet services and transport infrastructure. From an operators perspective, the attraction is that they can take advantage of Ethernets low cost points to build more cost-effective networks. Traditional Ethernet does not have the kind of OAM functionality required by the majority of network operators, since LANs do not need remote monitoring when the equipment is located in the building. If Carrier Ethernet is to fulfill its promise as the next-generation packet-based infrastructure for metropolitan and wide area networks, OAM capabilities must be added to Ethernet. This paper discusses the new standards that provide Ethernet with the most comprehensive OAM for packet-based solutions.

Alarm if CCM timeout A

M CC

E D

Figure 1. Fault Detection through Continuity Check Messages (CCM)

M CC

LB

E D

Figure 2. Fault Verification through Loopback Messages (LBM)

LB R

Ethernet OAM standards

The main Ethernet OAM standards are: IEEE 802.1ag (Connectivity Fault Management), which is expected to complete at the end of 2006 ITU-T Y.1731 (Fault and Performance Monitoring) completed in May 2006 IEEE 802.3ah (Ethernet in the first mile) completed in September 2004 IEEE 802.1aj (Two-port MAC Relay) introduced as a draft in May 2005 IEEE 802.1ag and ITU-T Y.1731 address OAM for end-to-end Ethernet services and infrastructure. While the ITU-T standard covers both fault management and performance monitoring, IEEE 802.1ag covers a subset of the fault management which is aligned with the functionality defined by the ITU-T.
2

IEEE 802.3ah and IEEE 802.1aj address link layer OAM. Unlike the previous standards which address endto-end OAM, these standards support OAM on a single Ethernet link. As the names suggest, their intended use is in first mile access networks to provide a basic level of monitoring to remote devices. Fault management There are two main areas of OAM: fault management and performance monitoring. Fault management ensures that when a defect occurs in the network that this is reported to the operator, who can then take the appropriate action. This is divided into the following functions: Fault Detection IEEE 802.1ag and ITU-T Y.1731 support fault detection through Continuity Check Messages (CCM). These allow end-

points to detect an interruption in service. CCMs are sent from the source to destination node at periodic intervals; if either end does not receive a CCM within a specified duration, then a fault is detected against the service. Fault verification IEEE 802.1ag and ITU-T Y.1731 support fault verification through Loopback Messages (LBM) and Loopback Reply (LBR). These can be used during initial set-up or after a fault has been detected to verify that the fault has occurred between two end points. In the example above, node A sends a LBM to node E. A Loopback Reply (LBR) is then sent back to node A, verifying connectivity between these two service end points. If the process was repeated for node F, no LBR would be received, allowing the operator to verify that a fault had occurred between these two nodes.

 Fault isolation IEEE 802.1ag and ITU-T Y.1731 support fault isolation through Linktrace Messages (LTM) and Linktrace Reply (LTR). In the example (see Figure 3), node A initiates an LTM, each intermediate node along the path (B and E) sends an LTR back and forwards the LTM towards node F. This serves two purposes. Under normal conditions, it allows the operator to determine the path used by the service through the network. While under fault conditions, it allows the operator to isolate the fault location without making a site visit. Fault notification ITU-T Y.1731 supports fault notification through Alarm Indication Signal (AIS). In the example (see Figure 4.), a failure between nodes B and E triggers AIS packets in both directions towards the service end points. This functionality alerts the operator to a fault in the network, before it is reported by customers. At nodes A and F, the service end points, the alarm can be replicated across all services supported at that UNI (User Network Interface) that are impacted by the fault. The AIS packets are issued periodically by nodes B and E, to ensure that while the fault still exists, a failure state is maintained. Additionally, the AIS packets can be used to trigger a redundancy sub-system.

B A
LTM
M LT

LTR

Figure 3. Fault Isolation through Linktrace Messages (LTM)

AI

VI S(

) Dy

Figure 4. Fault Notification through Alarm Indication Signal Messages (AIS)

Performance monitoring
In many respects the fault management concepts above have been adopted from existing practices in traditional TDM networks. However, while connection-orientated TDM services offer customers a predictable and guaranteed service, packet or frame-

based services are connectionless and can have varying performance levels. This is because each individual frame in a service can suffer varying delays due to possible queuing, while network congestion can result in actual loss of frames. The emergence of Carrier Ethernet networks is due in part to the bandwidth explosion resulting from residential triple play services (VoIP, IPTV and Internet access). The realtime nature of these video and voice services is particularly susceptible to the effects of latency and jitter. As a result, Carrier Ethernet networks require advanced performance monitoring to enforce customer SLAs and this functionality is introduced by

R LT

LT M

AI

(V

ID

D
y)

ITU-T Y.1731. The following features are supported. Frame Loss Ratio ITU-T Y.1731 calculates frame loss by sending transmit and receive counters within the CCM for dual-ended measurements. The far end counters can then be compared with those produced locally to derive frame loss as a percentage. Frame Delay Similarly, ITU-T Y.1731 calculates frame delay (or latency) by using a timestamp in the DM (Delay Measurement). The receiving end can derive the time delay experienced across the network. This requires each service end point to have synchronized clocks.

Alternatively, DMM/DMR (Delay Measurement Message and Reply) can be used to calculate the two-way frame delay; this method does not require clock synchronization. Frame Delay Variation Finally, ITU-T Y.1731 calculates frame delay variation (or jitter) by tracking frame delay measurements.

Service statistics Total Tx packets Total Rx packets A

E D

MPLS OAM
An alternative architecture for Carrier Ethernet is VPLS (Virtual Private LAN Services). VPLS uses an MPLS control plane to create circuits that support Ethernet connectivity across an IP/ MPLS network, to provide E-LINE and E-LAN services. This allows the network operator to provide OAM functionality through MPLS. However, in comparison with Ethernet, MPLS OAM functionality is immature as although it can support some fault management, it has no performance monitoring capabilities. In comparison, Ethernet OAM as detailed by IEEE 802.1ag and ITU-T Y.1731 provides more detailed network element instrumentation that leads to more powerful network and service management tools. Table 1 provides a comparison of MPLS and Ethernet OAM functions. Furthermore, the MPLS OAM that does exist is added in the control plane and this introduces a potential disconnect between whats reported by the OAM and what is actually happening to the Ethernet frames as they travel across the network. For example, if a fault occurs in the control plane, the network management system could misinterpret this as a problem in the data plane. In contrast, Ethernet OAM is embedded in the actual data frame. This means that the network management system can be sure that when a
Figure 5. Frame Loss through CCMs

B A E

Service layer

Step 1:
A sends ETH-LB with timestamp through 802.1ag packet

Step 2:
F replies with original timestamp maintained

Step 3:
A determines the round-trip latency by comparing timestamp with current time

Figure 6. Latency through CCMs

fault is reported, it relates to a problem that the Ethernet frame is experiencing and not the control plane. The use of MPLS tunnels also creates OAM problems for the network operator. As discussed, VPLS uses MPLS tunnels to establish Ethernet services across the network and each tunnel is identified by an MPLS label. However, the label offers no indication of the source or destination of the data.

To establish this information, the network operator has to delve deeper into the packet, to the actual Ethernet frame, making trouble-shooting more complicated. Also, as a result of the additional processing required, the fault management functions in MPLS cannot be performed in real-time. Ethernet OAM removes this additional layer of complexity and provides direct and accurate fault and performance monitoring for each Ethernet service.

Table 2 summarizes the comparison between Ethernet and MPLS OAM. As a result of these issues, several network operators have reported difficulties getting sufficiently useful information from their MPLS devices to effectively operate their networks. This in turn makes it difficult to deliver carrier-grade services to the end customer. Although Ethernet OAM is superior to MPLS OAM, there is acknowledgement that Ethernet and MPLS networks will co-exist. As a result, it is possible to have interworking between Ethernet and MPLS OAM. Such functionality is expected to be addressed within the Layer 2 VPN and Pseudowire Working Groups in the IETF. The available functionality in such deployments, however, falls back to the lowest common denominator in this case, the MPLS OAM capabilities. As a result, there are also proposals to allow extensions of VCCV (Virtual Circuit Connection Verification) to carry Ethernet OAM as the payload.

Table 1. Ethernet and MPLS OAM functionality

OAM functionality Ethernet OAM (IEEE 802.1ag, ITU-T Y.1731)
Yes (CCM) Yes (LBM/LBR) Yes (LTM/LTR) Yes (AIS and RDI) Yes (multicast LBM, LTM/LTR) Yes (CCM, LMM/LMR) Yes (1DM, DMM/DMR) Yes (APS) Yes (MCC) Yes (TST, LBM/LBR)

MPLS (LSP Ping)

Yes (LSP Ping Echo Request) Yes (LSP Ping Echo Reply) Yes (LSP Ping) No No No No No No No

Fault Detection Fault Verification Fault Isolation Fault Notification Discovery Frame Loss Frame Delay Protection Switching Remote Management Diagnostics (Data Plane)

Table 2. MPLS and Ethernet OAM summary MPLS

Some tools for fault management (e.g. LSP ping, Status TLV, VCCV, BFD) No standardized tools for performance monitoring Packets are not self-identifying OAM relies on control resources

Ethernet
Fault management in IEEE 802.1ag and ITU-T Y.1731 Performance monitoring supported by ITU-T Y.1731 Each packet is self-identifying OAM in the forwarding plane

Summary
The emergence of carrier-grade Ethernet has driven the need for improved Ethernet OAM functionality. Ethernet OAM allows the exchange of management information from the network elements to the management layer. Without this capability, it is impossible to provide the comprehensive network management tools that operators have today in their TDM networks. The combination of IEEE 802.1ag and ITU-T Y.1731 provides powerful fault management and performance monitoring capabilities to Ethernet. It is now clear that full implementations of these standards will leapfrog the capabilities in MPLS to provide the most comprehensive OAM solution for any packet-based network. This improved network instrumentation allows network operators to manage Carrier Ethernet networks effectively and brings the operational values previously only available in traditional TDM networks to the packet-based networks which are replacing them.

In the United States: Nortel 35 Davis Drive Research Triangle Park, NC 27709 USA In Canada: Nortel 195 The West Mall Toronto, Ontario M9C 5K1 Canada In Caribbean and Latin America: Nortel 1500 Concorde Terrace Sunrise, FL 33323 USA

In Europe: Nortel Maidenhead Office Park, Westacott Way Maidenhead Berkshire SL6 3QH UK In Asia: Nortel United Square 101 Thomson Road Singapore 307591 Phone: (65) 6287 2877

Nortel is a recognized leader in delivering communications capabilities that enhance the human experience, ignite and power global commerce, and secure and protect the worlds most critical information. Serving both service provider and enterprise customers, Nortel delivers innovative technology solutions encompassing end-to-end broadband, Voice over IP, multimedia services and applications, and wireless broadband designed to help people solve the worlds greatest challenges. Nortel does business in more than 150 countries. For more information, visit Nortel on the Web at www.nortel.com. For more information, contact your Nortel representative, or call 1-800-4 NORTEL or 1-800-466-7835 from anywhere in North America. Nortel, the Nortel logo, Nortel Business Made Simple and the Globemark are trademarks of Nortel Networks. All other trademarks are the property of their owners. Copyright 2006 Nortel Networks. All rights reserved. Information in this document is subject to change without notice. Nortel assumes no responsibility for any errors that may appear in this document.
N N 1 1 9 6 6 0 - 1 1 2 7 0 6

> BUSINESS MADE SIMPLE