Escolar Documentos
Profissional Documentos
Cultura Documentos
v 0.1
The purpose of this document is to provide A CUSTOMER with a low level design document proposing the
design for the technology refresh taking place at their Headquarters based in Gloucestershire. This
document will cover the core network and the DC environment only specifically the Nexus environment,
all other components of the network infrastructure are out of scope. This low level design will be used for
a definitive reference and a base as to how the fixed network will be developed and implemented.
The information in this document is conveyed with the assumption that engineers possess CCNP or
equivalent knowledge of network protocols and design fundamentals. It is not intended to explain every
fine detail and every configuration line for both Catalyst and Nexus devices that will be used in the design,
as its expected that engineers understand these devices and their configurations to a decent level.
Record of Modifications
Distribution List
Contributors
Referenced Documents
Sign Off
TABLE OF FIGURES
TABLE OF TABLES
Overview
A Customer has engaged Resolution to assist them with their core network infrastructure, which requires
an urgent technology refresh. The existing Chassis are at full capacity, out of support and are creating
bottlenecks on the corporate network. Resolution have been given the task to design and migrate to a
new core network which will meet the growing demands of their user base and business requirements
over the next 5 years.
The Core and DC environment will undergo a technology refresh as per this document. The Access Layer,
WAN and additional network services refresh such as load balancers and firewalls are out of scope in this
design. The design will cover methods of connectivity for these services only.
Approach
Due to the risk involved, the new infrastructure will be implemented in two phases:
Phase 1:
The initial phase will look to replace the existing core Catalyst 6509 switch as this is deemed the most
critical device to the enterprise infrastructure, the new equipment being installed will be integrated with
the existing server estate environment consisting of a Cisco UCS chassis. The new installed core switches
will be deployed as a VSS pair
Phase 2:
The second phase will see the additions of two Nexus 5548-UP switches. These will be configured as a vPC
pair and will make links to the newly installed core switches and the Fabric Interconnect (FI) switches for
the server estate.
Migration Plan
Test Plan
Installation and migration of the existing Catalyst core to the new Nexus core network
Described methods of connectivity from the Core switches to the Nexus Switches in the FlexPod environment
UPS specification
New Technologies
Standard Practice
Core
Server Aggregation
External Access
OSPF Design
Quality Of Service
Multicast
Security
A FlexPod will be deployed for the server estate. FlexPod components include Cisco Unified
Computing System (Cisco UCS) servers, Cisco Nexus switches, and NetApp unified storage
systems. The FlexPod architecture can scale up or out and it can be optimized for a variety of
mixed workloads, in both virtualized and non-virtualized environments.
The A CUSTOMER core network will be contained to one server room only. This will be hosting the entire
core and DC infrastructure. The network infrastructure will utilize a tiered model providing core, server
aggregation and server access.
Note: The FlexPod implementation is considered as part of the overall design; however the UCS
and Netapp unified storage components are considered out of scope.
The next sections will detail each sites physical and logical design for that particular network layer. This
includes hostnames, IP addressing, device types etc.
Abbreviation Description
Mgmt
Legend FCoE Only
Converged
10 GbE Only
OIR OIR
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 STATUS 1 2 3 4 5 6 7 8 OIR 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 STATUS 1 2 3 4 5 6 7 8 OIR
UID STATUS PS1 PS2 FAN USB UID STATUS PS1 PS2 FAN USB
vPC vPC
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 1 2 3 4 9 10 11 12 17 18 19 20
PO
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 1 2 3 4 9 10 11 12 17 18 19 20
N5K-C56-72UP
N5K-C56-72UP
ID
ID
5 6 7 8 13 14 15 16 21 22 23 24
STAT
5 6 7 8 13 14 15 16 21 22 23 24
STAT
PO PO
vPC vPC
L1 MGMTO FAN FAN
PS1
FAN1 L1 MGMTO FAN FAN
PS1
STAT STAT FAN1
STAT STAT
FAN2 FAN2
FAIL FAIL
FAIL FAIL
OK OK
ID ID OK OK
PS2
PS2
L2 CONSOLE
STAT STAT L2 CONSOLE
UCS 5108
!
SLOT SLOT
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
CONSOLE
!
SLOT SLOT
5 6
UCS
C240 M3
SLOT SLOT
7 8
UCS 5108
2x 10 GbE with
!
SLOT SLOT
2x 10 GbE with
1 2
! Console Reset ! Console Reset
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
PWR
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
SYS
CONSOLE
!
SLOT SLOT
5 6
UCS
C240 M3
SLOT SLOT
7 8
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
DS2246
2x 10 GbE 2x 10 GbE
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
FCoE 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
DS2246
FCoE
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
900GB
FAS8020
A B
DS4243
DS4243
Using a VSS deployment enables cross-chassis high-availability capabilities such as In-Service Software
Upgrade (ISSU), Nonstop Forwarding (NSF), and Stateful Switchover (SSO). Within the new design the
two Catalyst 4500-X will provide the core infrastructure utilizing a VSS design.
Note: For VSS capability, the minimum software requirement is Cisco IOS XE Software Release 3.4.0SG
The airflow of the Catalyst 4500-X switches is front to back for the power supplies, as such special
considerations need to be taken into account regarding the clearance on the front and back for the rack of
the chassis, ensuring that all slots and openings on a chassis remain unobstructed, especially the fan
assembly vent at the back of the chassis. To maintain proper air circulation through the switch chassis, it is
recommended that you maintain a minimum 6-inch (15 cm) separation between a wall and the chassis hot
air exhaust
Power Recommendations
Each power supply should be connected to separate input sources; otherwise the chassis could be
susceptible to a power failure
3.3.2 Licensing
Each Catalyst 4500-X will chassis have the Enterprise Services which will allow for every feature
available in the 4500-X platform
1 N5K-C5672UP 14.51Kg
Total 14.51Kg
Displayed below are the physical dimensions of each Nexus 5672UP chassis:
4.2.2 Licensing
No special licensing is required for the Nexus 5000 series switches as these switches are acting as pure
Layer 2 switches. The switches will be running the standard LAN Base feature set.
Each technology shown in the following sections includes configuration examples that are based on Cisco
best practices which should be adhered to throughout the network design unless otherwise stated. This
information will not be mentioned again so it should be known that where remaining parts of the
document mentions a technology (which is mentioned in this section), the configuration will be more or
less the same.
A VSS combines a pair of Catalyst 4500 or 4500-X series switches into a single network element. The VSS
manages the redundant links, which externally act as a single port channel.
The VSS simplifies network configuration and operation by reducing the number of Layer 3 routing
neighbours and by providing a loop-free Layer 2 topology.
The VSS domain must match between both switches. The virtual switch domain is a number between 1
and 255, and must be unique for each VSS in your network (the domain number is incorporated into
various identifiers to ensure that these identifiers are unique across the network).
Within the VSS, you must configure one switch to be switch number 1 and the other switch to be switch
number 2. To configure the virtual switch domain and switch number on both switches, perform these
tasks:
Config Example
Config Example
Config Example
Note After you confirm the command (by entering yes at the prompt), the running configuration is
automatically saved as the startup configuration and the switch reboots. After the reboot, the switch is in
virtual switch mode, so you must specify interfaces with three identifiers (switch/module/port). When
switches are being converted to VSS, you should not set them to ignore startup-config. If done, the switch
can be enabled to parse the startup-config at the rommon prompt. Ignoring startup-config in VSS mode,
causes a switch to boot in a semi-VSS mode, which can only be corrected by a reboot and by enabling the
parsing of startup-config.
The VSL link failure is a rare scenario, but should be protected against and all devices should be dual
homed across each of the VSS member switches. The desired design is to have no orphan ports connected
to the VSS domain.
5.8.1 Overview
Virtual PortChannels (vPCs) allow links that are physically connected to two different Cisco switches to
appear to a third downstream device as coming from a single device and as part of a single port channel.
The downstream device can be a switch, a server, or any other networking device that supports IEEE
802.3ad Port Channels.
vPC allows the creation of Layer 2 PortChannels that span two switches and at present vPC is
implemented on the Nexus 7000 and 5000 series platforms running NX-OS version 4.1(4) or higher.
Benefits of vPC are device level redundancy with faster convergence than using spanning tree, elimination
of blocked ports which promotes a loop-free topology, and much better bandwidth utilisation.
Figure 3 vPC
The vPC peer switches are connected through a link called peer link, also known as multi-chassis
PortChannel trunk. This is a standard 802.1Q trunk that carries vPC and non vPC VLANs, Cisco Fabric
Services messages (which are tagged as CoS 4), flooded traffic from the other vPC peer, and other
protocols like STP BPDUs, IGMP joins/updates, HSRP hellos etc.
The ports that form the PortChannel are split between the vPC peers and are referred to as vPC member
ports.
An out-of-band link is used to resolve dual-active scenarios where the peer-link connectivity is lost. This
link is referred to as vPC peer-keepalive or fault-tolerant link.
Some devices can be single-attached to the topology, like server 1 and server 3. The ports connecting
devices in a non-vPC mode to a vPC topology are called orphaned ports. Design considerations often
surround these types of ports as its extremely important for the design.
While mismatched Spanning-Tree and vPC priorities do not any impact on traffic forwarding under normal
conditions, it is desirable to keep the priorities matched so as to have Spanning-Tree root and vPC primary
on the same device and Spanning-Tree secondary root and vPC secondary on the same device.
For the two switches (vPC peers) to form a vPC system, the domain-id of these switches need to match.
The domain-id is used to generate the system-id which is used for the LAGID in the LACP negotiation and
BPDU generation.
Shown below is the configuration to create and set the vPC domain, and set the matching STP priorities:
It should also be noted that the vPC role is not preemptive, so a device may be operationally primary but
secondary from a configuration perspective. This is not an issue.
The recommendation is to use the Nexus out-of-band management interface for this task. However, you
should not use the mgmt0 interface for a direct back-to-back connection between Nexus 7000 systems
because the mgmt0 IP address is only ever active on one supervisor, and you cannot discern which
supervisor is active at any given time. For the Nexus 5600 this is acceptable as only one mgmt interface
exists, but not recommended as it means device management would have to be in-band.
The keepalive itself is a UDP message on port 3200, is 96 bytes long (32 byte payload), and includes
version, time stamp, local and remote IPs, and the domain ID.
The following configuration shows how to use the management interface for peer-keepalive. This
assumes that out-of-band management hasnt currently been set up:
On the N7K Series, this PortChannel should be configured on 10 Gigabit Ethernet interfaces configured in
dedicated-mode across two different 10 Gigabit Ethernet line cards. However there are exceptions to this
which will be explained in later site specific sections.
The peer-link is by far the most important component in the vPC system, in that its failure may impair the
establishment of new flows and isolate orphan ports.
Configuring the peer link in a redundant fashion ensures uninterrupted connectivity between the vPC
peers in almost every likely failure scenario.
The following configuration illustrates how to configure the peer-link across two 10Gbps interfaces which
in this case is PortChannel1 using LACP active negotiation at both ends.
N5K-1(config)# feature lacp
Note: You should always create a Port-Channel interface first and specify if its Layer 2 or
Layer 3 using the switchport or no switchport command before bundling any links.
It is always good practice to use the Link Aggregation Control Protocol (LACP) for bundling of the ports in
the vPC. This is because LACP determines that the ports being bundled are actually part of the same
physical or virtual switch, preventing erroneous configurations.
The configuration shown below places a single 10Gbps interface on each vPC peer into a PortChannel and
then applies vPC to the PortChannel on each peer. Following this is an example configuration of a
PortChannel on a downstream device such as a Catalyst 2950:
Note: The vpc ID doesnt need to be the same as the PortChannel ID, however its recommended; also the
recommended configuration of downstream devices is to use passive negotiation.
To view these you can use the show vpc consistency-parameters command set.
A new feature in NX-OS 5.0(2) allows you to configure the Nexus 7000 to restore vPC services when its
remote peer fails to come online.
Its recommended to use the command reload restore on both vPC peers to avoid this rare scenario.
Particularly with some network-attached storage (NAS) devices or load-balancers, they may have features
which are aimed at optimising application performance. Behaviour of these devices will reply to traffic
using the MAC address of the sender Cisco Nexus 7000 device rather than the common HSRP gateway.
This behaviour is non-complaint with some basic Ethernet RFC standards. Packets reaching a vPC device
for the non-local router MAC address are sent across the peer-link and could be dropped by the built in
vPC loop avoidance mechanism if the final destination is behind another vPC.
The vPC peer-gateway capability allows a vPC switch to act as the active gateway for packets that are
addressed to the router MAC address of the vPC peer. This feature enables local forwarding of such
packets without the need to cross the vPC peer-link. In this scenario, the feature optimizes use of the
peer-link and avoids potential traffic loss.
Note: When enabling this feature it is also required to disable IP redirects on all interface VLANs mapped
over a vPC VLAN to avoid generation of IP redirect messages.
vPC Peer-switch
The vPC peer switch feature addresses performance concerns around STP convergence. This feature
allows a pair of Cisco Nexus devices to appear as a single STP root in the Layer 2 topology and it eliminates
the need to pin the STP root to the vPC primary switch as well as improving vPC convergence if the vPC
primary switch fails.
To avoid loops the vPC peer link is excluded from the STP computation. In vPC peer switch mode, STP
BPDUs are sent from both vPC peer devices to avoid issues related to STP BPDU timeout on the
downstream switches, which can cause traffic disruption.
NX-OS
N5K(config)#vpc domain 1
N5K(config-vpc-domain)#peer-switch
As mentioned earlier domain-id defined for the vPC domain is used to generate the system-id of the
system comprised of the vPC peers. Because of this it is important to ensure that each vPC system in a
topology such as the one described utilizes a different domain-id number. This ensures uniqueness in the
Bridge ID used for BPDUs and the LAGID used by LACP. Alternatively its possible to define the same
domain-id as long as a different system-id (system-priority) has been configured under the vPC domain.
Its recommended to configure the system-priority manually the same as the domain ID.
Overview
In a vPC environment the most important link in the technology is the Peer-Link. If the Peer-Link fails for
whatever reason you have the potential for a split brain scenario where both vPC peers think everything is
normal, and continue to advertise subnets to non-vpc peers and continue forwarding on downlinks (vPC
member ports). This behavior is avoided by the use of the Peer Keepalive which determines if both peers
are still responsive on the out-of-band link, and if so, the currently acting secondary role will shut all of its
vPC member ports, and bring down all of its SVI interfaces for any VLANs that are configured across the
Peer-Link. This behavior is fine for situations when all devices are dual homed to the vPC domain and Port-
Channeled to both peers.
If you have single homed devices attached to the domain, you need to think carefully about this failure
scenario and determine if the failure of the peer-link would cause extended loss of service because its
broken the path of traffic, and cannot be resolved without manual intervention.
Within this design vPC is only configured on the server aggregation layer of the network, where devices
are dual homed.
When we consider the possibility of losing the vPC Peer-Link either by misconfiguration, or disconnected
fibers etc. The secondary vPC peer will shut its member ports because the keep alive has identified that
both are still reachable. For A CUSTOMER network devices are dual homed into both the core and server
aggregation Nexus 7000 switches.
5.8.6 HSRP
The use of HSRP in the context of vPC does not require any special configuration. With vPC, only the active
HSRP interface answers ARP requests, but both HSRP interfaces (active and standby) can forward traffic.
If an ARP request coming from a server arrives on the secondary HSRP device, it is forwarded to the active
HSRP device through the peer-link.
If configurations are pertinent to certain network layers it will be specified otherwise the section is global
to each network layer and/or site. Reason for this is that different areas of the network require different
policy and security or trust levels.
6.2 Layer 2
6.2.3 VTP
VTP is a Layer 2 messaging protocol that maintains VLAN configuration consistency by managing the
addition, deletion, and renaming of VLANs within a VTP domain. Nexus switches will support the standard
VTP modes of client, server, and transparent, with an additional mode of vtp off. The main difference
with VTP being turned off is that the device will not relay any VTP messages between devices. A
CUSTOMER makes use of VTP to automatically configure VLANs on switches within the same VTP domain,
this function ability will be enabled on the nexus switches which by default have it turned off. Within the
design it is recommended that the core/access layer and the server aggregation/server access layer reside
in separate VTP domains to prevent accident deletion of VLANs should Layer 2 links be added between the
two domains. The Nexus 7000 switches will act as the VTP servers and all VLANs will be created in the
appropriate VDCs, all other switches in the network will act as VTP clients.
The recommended spanning-tree protocol across the board for A CUSTOMER is Rapid-PVST protocol.
Root Bridge Placement & STP Load Balancing (Core / Server Aggregation)
For every instance of spanning-tree (per-VLAN) there is one device elected Root Bridge. The root bridge is
typically at the centre of the network topology and all of its Layer 2 links in forwarding state.
In a typical network design, spanning-tree is also used to provide load balancing across Layer 2 paths
where multiple VLANs exist. This normally achieved by configuring multiple devices to act as the root
bridge, and then mapping the root bridge with other protocols such as HSRP.
Root bridge placement in vPC deployments is a much more simplified configuration because one of the
key characteristics of vPC is active/active forwarding on Layer 2 paths. In this scenario the root bridge
placement is less of a concern since load balancing is an inherent part of vPC and the use of the peer-
switch command.
As such the root bridge placement in every Layer 2 domain of the A CUSTOMER design is to use the same
priority for all VLANs to Nexus 7000 VDCs, and make the root priorities the same as the vPC domain
priorities where appropriate.
Config Example:
vPC Primary
N5K-1(config)# spanning-tree mode rapid-pvst
N5K-1(config)# spanning-tree vlan 1-4096 priority 4096
vPC Secondary
N5K-2(config)# spanning-tree mode rapid-pvst
N5K-2(config)# spanning-tree vlan 1-4096 priority 4096
For vPC ports only, the operational primary switch generates and processes BPDUs. The operational secondary switch forwards
BPDUs to the primary switch.
2Gbps links = 3
3Gbps links = 3
4Gbps links = 3
Considering that spanning tree was designed before the availability of 10 GigabitEthernet or even
GigabitEthernet, the default spanning-tree reference cost for a link is inadequate, as most links with 10
GigabitEthernet bandwidth or bandwidth with multiples of 10 Gigabit Ethernet will appear to have the
same cost.
With spanning-tree pathcost method long enabled, spanning tree calculates the best forwarding path
according to the link bandwidth to the root and not based on hop count. With this command, the cost of
links is as follows (these numbers are weights so they dont have a specific measurement unit):
Path cost method long should be enabled on every device participating in the spanning tree. This will be
enabled for all devices being installed as part of the core LAN refresh, however all remaining switches such
as the access layer switches remain the responsibility of the A CUSTOMER IT team to update.
Spanning-Tree Portfast
Portfast immediately brings an interface configured as an access or trunk port to the forwarding state
from a blocking state, bypassing the listening and learning states. Portfast should be used on interfaces
connected to a single workstation or server, to allow those devices to immediately connect to the
network, rather than waiting for the spanning tree to converge (up to 50 seconds delay).
OR
Normal ports - By default a switchport is considered a normal port for the purpose of spanning-tree. Normal ports remain
unmodified and operate as standard spanning-tree ports
Network ports - Network ports are used to define connections between two bridges. By default Bridge Assurance is enabled on
these ports, which is described in the next section.
Edge ports - Previously known as Portfast, a port configured as a spanning-tree edge should transition immediately into a
forwarding state, bypassing the listening and learning states. Only non-bridging L2 devices should be configured as edge ports.
This port type should be reserved for data centre hosts which are not capable of creating a L2 loop; this includes single
attached hosts, L3 routers and firewalls, or multi-homed devices that leverage some form of NIC teaming. Trunk ports to these
types of devices can also be configure as edge ports (this is called TrunkFast and generally used for virtualisation hosts).
It is recommended that all ports are left in the default configuration of normal so that spanning-tree will
protect the topology when needed, and explicit configuration is given for any ports that should be
network or edge.
OR
Warning: Edge port type (portfast) should only be enabled on ports connected to
a single host. Connecting hubs, concentrators, switches, bridges, etc...
to this interface when edge port type (portfast) is enabled, can cause temporary
bridging loops.
Use with CAUTION
OR
BPDU Guard
BPDU guard prevents a host port (port type edge or portfast) from participating in spanning tree. Under
normal circumstances, Layer 2 access ports connected to a single workstation or server should not
participate in spanning tree. When enabled on a port, BPDU guard shuts down the port as soon as a BPDU
is received on that port. In this way, BPDU guard helps prevent unauthorized access and the illegal
injection of forged BPDUs.
BPDU guard can be configured per port, or globally. When configured globally, BPDU guard is effective
only on ports in the operational port type edge mode. Its recommended that global BPDU Guard be
enabled across the A CUSTOMER infrastructure therefore affecting all ports that are placed in to edge
mode.
BPDU guard should also be enabled on individual interfaces that are in portfast or edge mode:
STP Root guard should be enabled on all downlinks away from the root bridge in each network layer,
server access ports, and user access ports.
Root guard is mutually exclusive with Loop guard, as Root guard is to be used on designated ports and
does not allow the port to become non-designated. Loop guard works on non-designated ports and does
not allow the port to become designated via max_age expiry.
Its recommend that Loop guard be enabled on root and alternate ports for all possible combinations of
active topologies where Bridge Assurance is not supported. An example of this would be the 2960
switches.
LACP has the same characteristics as PAgP, which is Cisco proprietary. As PAgP and LACP are not
interoperable, LACP will be the preferred negotiation protocol within A CUSTOMER.
LACP will be set to active/passive mode and the medium type specified as point-to-point. The active device
should always be closer towards the Core of the network with the other end as passive.
Config Example:
When bundling links its important to ensure that they are all the same speed, their duplexing mode is full,
and any configurations are mirrored across all ports. Additionally, PortChannel configurations should
always be consist ports to the power of 2 to provide the best load distribution.
Copyright 2015 Re-Solution. All rights reserved. Page 26 of 32
6.3 Layer 3
When configuring priorities the best practice is to align the active HSRP device to the primary root bridge
or vPC primary switch for every HSRP instance. When specifying HSRP timers, In the A CUSTOMER design
single supervisors will be used, however in the case that vPC dual supervisors are used in the future, the
recommendation is to configure and extended hold timer on all devices to support NSF during ISSU and
supervisor switchovers sub second hellos should not be used. Values of 1s hello and 3s hold timers are
ideal. Lastly IP redirects should be disabled and MD5 authentication is recommended.
Primary
N5K-1(config)# feature hsrp
N5K-1(config)# feature interface-vlan
Secondary
N5K-2(config)# feature hsrp
N5K-2(config)# feature interface-vlan
Overview
Bidirectional Forwarding Detection (BFD) is a network protocol used to detect faults between
two forwarding engines connected by a link. It provides detection of faults for various physical media
type, encapsulations, topologies, and routing protocols.
BFD can enable sub-second failure detection between two adjacent devices without the need of
configuring aggressive CPU intensive protocol hello messages, this is because some of the BFD load can
be distributed onto the data plane with some I/O modules.
The key to providing sub-second failover using BFD is that it talks directly to other network protocols and
sends them a failure detection notice when network changes occur, this circumvents the delay usually
experienced by the time it takes for line protocol to go down, or the dead intervals to expire.
EIGRP
OSPF
IS-IS
HSRP
PIM
Static Routes
Within the design, BFD should be used for all OSPF and PIM neighbours where Nexus and/or Catalyst
6500s are directly connected together. HSRP is not included as there are number of instances that could
cause the SVI to flap. In any case, the use of vPC reduces the need to enabled BFD on SVI interfaces as
HSRP is active/active.
You can configure BFD echo mode on one or both ends of a BFD monitored link. Echo mode slows down
the required minimum receive interval, based on the configured slow timer. The RequiredMinEchoRx BFD
session parameter is set to zero if echo mode is disabled. The slow timer becomes the required minimum
receive interval if echo mode is enabled. Echo mode is enabled by default and is the recommended
configuration.
Note: Echo mode requires that the interface participating in BFD has the no ip redirects command
applied.
For Layer 3 PortChannels and Layer 2 PortChannels used by SVI interfaces, LACP needs to be enabled.
Default VDC
N7K(config)# no hardware ip verify address identical
Non-Default VDC
N7K(config)# feature bfd
N7K(config)# bfd interval 100 min_rx 100 multiplier 4
N7K(config)# bfd slow-timer 3000
OR
For other PIM configurations please refer to the Multicast Design section.
Cisco Catalyst 6500 series switches support fault resistance by allowing a redundant supervisor engine to
take over if the primary supervisor engine fails. Cisco NSF works with SSO to minimize the amount of time
a network is unavailable to its users following a switchover while continuing to forward IP packets.
A manual switchover
SSO
SSO establishes one of the supervisor engines as active while the other supervisor engine is designated as
standby, and then SSO synchronizes information between them. A switchover from the active to the
redundant supervisor engine occurs when the active supervisor engine fails, or is removed from the
switch, or is manually shut down for maintenance. This type of switchover ensures that Layer 2 traffic is
not interrupted.
In networking devices running SSO, both supervisor engines must be running the same configuration so
that the redundant supervisor engine is always ready to assume control following a fault on the active
supervisor engine. SSO switchover also preserves FIB and adjacency entries and can forward Layer 3
traffic after a switchover. Configuration information and data structures are synchronized from the active
to the redundant supervisor engine at startup and whenever changes to the active supervisor engine
configuration occur. Following an initial synchronization between the two supervisor engines, SSO
maintains state information between them, including forwarding information.
During switchover, system control and routing protocol execution is transferred from the active
supervisor engine to the redundant supervisor engine. The switch requires between 0 and 3 seconds to
switchover from the active to the redundant supervisor engine. The configuration is shown in the
example below.
Switch(config)# power redundancy-mode redundant
Cisco NSF is supported by the OSPF protocol for routing and is supported by Cisco Express Forwarding
(CEF) for forwarding. OSPF has been enhanced with NSF-capability and awareness, which means that
routers running these protocols can detect a switchover and take the necessary actions to continue
forwarding network traffic and to recover route information from the peer devices. Its also known as
graceful restart.
Each protocol depends on CEF to continue forwarding packets during switchover while the routing
protocols rebuild the Routing Information Base (RIB) tables. After the routing protocols have converged,
CEF updates the FIB table and removes stale route entries. CEF then updates the line cards with the new
FIB information.
Config Example: