E-VPN & PBB-EVPN: the Next Generation

of MPLS-based L2VPN

sbng@cisco.com
ttheera@cisco.com
limfung@cisco.com

Agenda
Technical Overview
Flows and Use Cases
Ciscos PBB-EVPN Implementation
Summary

2014 Cisco and/or its affiliates. All rights reserved.

Technical Overview
Highlights and Solution Requirements
4

DCI Brings New Requirements

Data Center Interconnect requirements not fully addressed by current
L2VPN technologies
All-active Redundancy and Load Balancing
Simplified Provisioning and Operation
Optimal Forwarding
Fast Convergence
MAC Address Scalability

Ethernet Virtual Private Network (E-VPN) and Provider Backbone Bridging

EVPN (PBB-EVPN) designed to address these requirements
2014 Cisco and/or its affiliates. All rights reserved.

Towards EVPN
Solve Challenges of VPLS for All-Active Redundancy
Existing VPLS solutions do not offer an
All-Active per-flow redundancy
Looping of Traffic Flooded from PE
Duplicate Frames from Floods from the
Core
MAC Flip-Flopping over Pseudowire
E.g. Port-Channel Load-Balancing does not
produce a consistent hash-value for a frame
with the same source MAC (e.g. non MAC
based
Hash-Schemes)

M1

CE1

PE1

CE2

M2

Echo !

PE4

PE2
M1

CE1

PE1

PE3

M1

CE1

CE2 Duplicate !

PE4

PE2

PE1

PE3

CE2

M2

MAC
Flip-Flop

PE2

2014 Cisco and/or its affiliates. All rights reserved.

PE3

PE4

Solution Requirements
All-Active Redundancy and Load Balancing
Flow-based Multi-pathing
WAN

Flow-based Load
balancing

Geo-Redundancy
Backdoor

Site 1

Site N
Site 2

All-Active Redundancy to maximize bisectional bandwidth

Load-balance traffic among PEs and exploit core ECMP based on flow entropy (flow can
be L2/L3/L4 or combinations)
Support geo-redundant PE nodes with optimal forwarding
Flexible Redundancy Grouping of PEs
2014 Cisco and/or its affiliates. All rights reserved.

Solution Requirements
All-Active Redundancy and Load Balancing (cont.)
Active / Active Multi-Homing with
flow-based load balancing in CE
to PE direction
Maximize bisectional bandwidth
Flows can be L2/L3/L4 or combinations

Flow-based load balancing in PE

to PE direction
Multiple RIB entries associated for a
given MAC
Exercises multiple links towards CE

2014 Cisco and/or its affiliates. All rights reserved.

Flow Based Load-balancing CE to PE direction

Vlan X F1

Vlan X
F2

P
E

P
E

P
E

P
E

Flow Based Load-balancing PE to PE direction

Vlan X F1X
Vlan
F2

P
E

P
E

P
E

P
E

Solution Requirements
All-Active Redundancy and Load Balancing (cont.)
Flow-based Multi-Pathing
Load balancing across
equal cost multiple paths
in the MPLS core
Load balancing at PE and
P routers based on
Entropy MPLS labels

2014 Cisco and/or its affiliates. All rights reserved.

Vlan X - F1
Vlan X
F2X
Vlan
F3X
Vlan
F4

Flow Based Multi-Pathing in the Core

PE

PE

PE

P
PE

Solution Requirements
MAC Address Scalability
N * 1M
WAN

1Ms
10Ks
1Ks

DC Site 1
DC Site 2

DC Site N

Server Virtualization fueling growth in MAC Address scalability:

1 VM = 1 MAC address.
1 server = 10s or 100s of VMs

MAC address scalability most pronounced on Data Center WAN Edge for Layer 2
extensions over WAN.
Example from a live network: 1M MAC addresses in a single SP data center
2014 Cisco and/or its affiliates. All rights reserved.

11

Ethernet VPN
Highlights
Next generation solution for Ethernet
multipoint connectivity services

Data-plane address
learning from Access

Control-plane address
advertisement / learning
over Core

Leverage similarities with L3VPN

PEs run Multi-Protocol BGP to

advertise & learn MAC addresses over
Core

PE1

PE3

VID 100
SMAC: M1
DMAC: F.F.F

CE1

CE3

Learning on PE Access Circuits via

data-plane transparent learning

MPLS

No pseudowire full-mesh required

Unicast: use MP2P tunnels
Multicast: use ingress replication over MP2P
tunnels or use LSM

Under standardization at IETF draftietf-l2vpn-evpn

2014 Cisco and/or its affiliates. All rights reserved.

PE2

PE4
BGP MAC adv. Route
E-VPN NLRI
MAC M1 via PE1

12

PBB Ethernet VPN

Highlights
Combines Ethernet Provider Backbone Bridging
(PBB - IEEE 802.1ah) with Ethernet VPN
PEs perform as PBB Backbone Edge Bridge (BEB)

Reduces number of BGP MAC advertisements

routes by aggregating Customer MACs (CMAC) via Provider Backbone MAC (B-MAC)
Addresses virtualized data centers with C-MAC count
into the millions
PEs advertise local Backbone MAC (B-MAC)
addresses in BGP
C-MAC and C-MAC to B-MAC mapping learned in
data-plane

Under standardization at IETF draft-ietf-l2vpnpbb-evpn

Control-plane address
advertisement / learning
over Core (B-MAC)

Data-plane address
learning from Core
Remote C-MAC to remote
B-MAC binding
PE1

Data-plane address
learning from Access
Local C-MAC to local BMAC binding

PE3

B-MAC:
B-M1

B-M2

CE1

CE3

MPLS
B-MAC:
B-M1

B-M2

PE2

PE4
BGP MAC adv. Route
E-VPN NLRI
MAC B-M1 via PE2

2014 Cisco and/or its affiliates. All rights reserved.

13

Technical Overview
Concepts
14

E-VPN / PBB-EVPN Concepts

E-VPN Instance (EVI)

Ethernet Segment
SHD

EV
I

BD

EV
I

BD

P
E
EVI identifies a VPN in the

network
Encompass one or more

bridge-domains,
depending on service
interface type
Port-based
VLAN-based (shown above)

CE1

ESI1
MHD

PE1

C
E2

ESI2

PE2

Represents a site

connected to one or more

PEs
Uniquely identified by a

10-byte global Ethernet

Segment Identifier (ESI)
Could be a single device

or an entire network

BGP Routes

BGP Route
Attributes

Route Types

Extended Communities

[1] Ethernet Auto-Discovery (AD) Route

ESI MPLS Label

[2] MAC Advertisement Route

ES-Import

[3] Inclusive Multicast Route

MAC Mobility

[4] Ethernet Segment Route

Default Gateway

E-VPN and PBB-EVPN

New BGP extended

define a single new BGP

NLRI used to carry all EVPN routes
NLRI has a new SAFI (70)
Routes serve control

plane purposes,
including:

communities defined
Expand information

carried in BGP routes,

including:
MAC address moves
C-MAC flush notification
Redundancy mode

VLAN-bundling

Single-Homed Device (SHD)

MAC address reachability

MAC / IP bindings of a GW

VLAN aware bundling (NEW)

Multi-Homed Device (MHD)

MAC mass withdrawal

Split-horizon label encoding

Single-Homed Network (SHN)

Split-Horizon label adv.

Multi-Homed Network (MHN)

Aliasing
Multicast endpoint discovery
Redundancy group discovery

2014 Cisco and/or its affiliates. All rights reserved.

Designated forwarder election

15

E-VPN Instance (EVI) & Service Interfaces

E-VPN Instance (EVI) identifies a VPN in the MPLS/IP network
EVI may encompass one or more bridge-domains, depending on PEs service
interface type:

New!
Port Based Service
Interface
C
All CEVLANs

VLAN Based Service

C
Interface

UE
NI

VLA
NX
VLAN
Y

VPN A

U
NI

U
NI

C
E

UE
NI
VPN
VPN
A
B
U
NI

U
NI

U
NI

U
NI

C
E

EV
I

BD

EV
I

2014 Cisco and/or its affiliates. All rights reserved.

P
E

U
NI

U
NI

EV
I

EV
I

PE

C
E

VP
N
A

BD

BD
BD

UE
NI

CEVLAN
subset

UE
NI
VPN
A

CEVLAN
subset

C
E

VLAN Aware Bundling Service

C
Interface

EV
I

BD

BD

VLAN Bundling Service

Interface
C

PE

P
E
16

Ethernet Segment
Definition
SHD
ESI1

PE5

MHN

PE1

MHD

CE5

CE2
PE2

ESI2
MHD
ESI3

ESI4

CE4

CE1

CE3

ESI5

CE6
PE4

SHN

PE3

Ethernet Segment is a site connected to one or more PEs

Ethernet Segment could be a single device (i.e. CE) or an entire network

Single-Homed Device (SHD)

Multi-Homed Device* (MHD) using Ethernet Multi-chassis Link Aggregation Group
Single-Homed Network (SHN)
Multi-Homed Network* (MHN)

Uniquely identified by a 10-byte global Ethernet Segment Identifier (ESI)

2014 Cisco and/or its affiliates. All rights reserved.

(*) Includes Dual-Homed Device

17

Ethernet Segment
ESI Auto-Sensing
CE1

LACPDU

BPDU
PE1

PE1

CE
PE2

MST

MPLS

PE2

MPLS

CE2

LACPDU

BPDU

MHD with Multi-chassis LAG

MHN with MST

ESI is auto-discovered via LACP

ESI is encoded using the CEs LACP

parameters:

ESI is auto-discovered via MST BPDU

snooping

ESI is encoded using the ISTs root

parameters:

System
Priority

2 bytes

2014 Cisco and/or its affiliates. All rights reserved.

System MAC
Address

Port Key

6 bytes

2 bytes

Bridge Priority

2 bytes

Root Bridge MAC

6 bytes

0x0000

2 bytes
18

Split Horizon
For Ethernet Segments E-VPN
Challenge:
How to prevent flooded traffic from echoing
back to a multi-homed Ethernet Segment?

ESI-1

CE1

PE1

ESI-2
PE3

Echo !

CE4

CE3
CE5

PE2

PE4

PE advertises in BGP a split-horizon label (ESI MPLS Label) associated with

each multi-homed Ethernet Segment
Split-horizon label is only used for multi-destination frames (Unknown
Unicast, Multicast & Broadcast)
When an ingress PE floods multi-destination traffic, it encodes the SplitHorizon label identifying the source Ethernet Segment in the packet
Egress PEs use this label to perform selective split-horizon filtering over the
attachment circuit
2014 Cisco and/or its affiliates. All rights reserved.

19

Split Horizon
For Ethernet Segments PBB-EVPN
Challenge:
How to prevent flooded traffic from echoing
back to a multi-homed Ethernet Segment?

ESI-1
CE1

B-MAC1 PE1

ESI-2
PE3

Echo !

CE5

B-MAC1

CE4

PE2

CE3

PE4

PEs connected to the same MHD use the same B-MAC address for the
Ethernet Segment
1:1 mapping between B-MAC and ESI (for All-Active Redundancy with flow-based LB)

Disposition PEs check the B-MAC source address for Split-Horizon filtering
Frame not allowed to egress on an Ethernet Segment whose B-MAC matches the BMAC source address in the PBB header

2014 Cisco and/or its affiliates. All rights reserved.

20

Split Horizon
For Core Tunnels
Challenge:
How to prevent flooded traffic from looping back
over the core?

ESI-1
CE1

ESI-2

PE1

PE3

CE3

Loop !

CE5

CE4
PE2

PE4

Traffic received from an MPLS tunnel over the core is never forwarded
back to the MPLS core
This is similar to the VPLS split-horizon filtering rule

2014 Cisco and/or its affiliates. All rights reserved.

21

Designated Forwarder (DF)

DF Election
Challenge:
How to prevent duplicate copies of flooded
traffic from being delivered to a multi-homed
Ethernet Segment?

ESI-2

ESI-1
CE1

PE1

PE3

CE2
Duplicate !

PE2

PE4

PEs connected to a multi-homed Ethernet Segment discover each other via

BGP
These PEs then elect among them a Designated Forwarder responsible for
forwarding flooded multi-destination frames to the multi-homed Segment
DF Election granularity can be:
Multiple DFs for load-sharing
Per Ethernet Tag on Ethernet Segment (E-VPN)
Per I-SID on Ethernet Segment (PBB-EVPN)
2014 Cisco and/or its affiliates. All rights reserved.

22

Designated Forwarder (DF)

DF Filtering
MHD All-Active with Per-Flow Load Balancing

MHD / MHN All-Active with Per-Service

Load Balancing
PE1
CE1

Legend

PE1
CE

MPLS

PE2

MHN

PE2

CE2

Multi-destination
Traffic

MPLS

DF Filtering

Unicast Traffic

DF Filtering

PE1
CE

MPLS

PE2

!
DF Filtering

Filtering
Direction:

Core to Segment

Filtering
Direction:

Core to Segment
Segment to Core

Filtered Traffic:

Flooded multi-destination

Filtered Traffic:

Flooded multidestination
Unicast

2014 Cisco and/or its affiliates. All rights reserved.

23

Aliasing

I can reach
MAC1 via ESI1
I can
reach
ESI1

E-VPN
Challenge:
How to load-balance traffic towards a multihomed device across multiple PEs when MAC
addresses are learnt by only a single PE?

MAC1

PE1

(All-Active)

PE3

MAC1 ESI1
PE1

PE2

MAC1 CE1

ESI-1

CE3

PE2

I can
reach
ESI1

PE4

CE4

(All-Active)

PEs advertise in BGP the ESIs of local multi-homed Ethernet Segments

All-Active Redundancy Mode indicated

When PE learns MAC address on its AC, it advertises the MAC in BGP
along with the ESI of the Ethernet Segment from which the MAC was
learnt
Remote PEs can load-balance traffic to a given MAC address across all
PEs advertising the same ESI
2014 Cisco and/or its affiliates. All rights reserved.

24

Aliasing

I can reach
MAC1 via BMAC1
I can
reach BMAC1

PBB-EVPN
Challenge:
How to load-balance traffic towards a multihomed device across multiple PEs when MAC
addresses are learnt by only a single PE?

MAC1

MAC1 CE1

PE1

MAC1 B-MAC1PE1

PE2

PE3
CE3

B-M1

B-M1

ESI-1

PE2

I can
reach BMAC1

PE4

CE4

PEs connected to the same MHD use the same B-MAC address for the Ethernet
Segment
1:1 mapping between B-MAC and ESI (for All-Active Redundancy with flow-based LB)

PEs advertise their B-MAC addresses independent of the C-MAC learning state

Remote PEs can load-balance traffic to a given C-MAC across all PEs advertising
the same associated B-MAC

2014 Cisco and/or its affiliates. All rights reserved.

25

MAC Mass-Withdraw

I can reach
MAC1 via ESI1

I can reach
MAC2 via ESI1

Challenge:
How to inform remote PEs of a failure affecting
many MAC addresses quickly while the controlplane re-converges?

MAC1

MAC1, (All-Active)
MAC2,
PE1
MACn

I lost ESI1

MAC1,.. MACn ESI1 PE1

PE2

I can
reach
ESI1

E-VPN

I can reach
MACn via ESI1

PE3

CE1

ESI-1

CE3

PE2

I can
reach
ESI1

PE4

CE4

(All-Active)

PEs advertise two sets of information:

MAC addresses along with the ESI from the address was learnt
Connectivity to ESI(s)

If a PE detects a failure impacting an Ethernet Segment, it withdraws the route

for the associated ESI
Remote PEs remove failed PE from the path-list for all MAC addresses associated
with an ESI
This effectively is a MAC mass-withdraw function
2014 Cisco and/or its affiliates. All rights reserved.

27

Flows and Use Cases

PBB-EVPN Startup Sequences
31

PBB-EVPN Startup Sequence

Segment Auto-Discovery

ESI and B-MAC Auto-Sensing

VPN Auto-Discovery

Multicast Tunnel ID / Endpoint

Discovery

Redundancy Group Membership

Auto-Discovery

Backbone MAC (B-MAC)

Reachability Advertisement

2014 Cisco and/or its affiliates. All rights reserved.

32

PBB-EVPN Startup Sequence (cont.)

ESI (10B) can be auto-generated*
from CEs LACP information ->
concatenation of CEs LACP
System Priority + Sys ID + Port Key

ESI and B-MAC Auto-Sensing

Segment Auto-Discovery

Example:
0000. 0011.0022.0033.0018

System
Priority

2 bytes

System MAC
Address

6 bytes

PE1

ESI and B-MAC Auto-Sensing

CE LACP info:
LACP System ID (MAC) (6B)
e.g. 0011.0022.0033
LACP System Priority (2B)
e.g. 0000
LACP Port Key (2B)
e.g. 0018

LACP PDU
exchange

Example: 0211.0022.0033

2 bytes
PE3

B-MAC
B-MAC

CE1

Source B-MAC used at PBB-EVPN PE on a

given ESI can be auto-generated* from CEs
LACP information -> CEs LACP System ID
MAC with U/L** (Universal / Locally
Administered) bit flipped

2014 Cisco and/or its affiliates. All rights reserved.

Port Key

CE3

MPLS
B-MAC
B-MAC

PE2

PE4

(*) ESI and B-MAC can also be manually configured

(**) U/L is second-least-significant bit of most significant byte
33

PBB-EVPN Startup Sequence (cont.)

BGP Ethernet Segment Route

PE 1 Eth Segment Route

RD = RD10

Segment Auto-Discovery

RD RD unique per
advertising PE

ESI = ESI1
ES-Import ext. comm.

MAC address portion

of ESI (6B)

e.g. 0011.0022.0033
PE1

PE3

ESI and B-MAC Auto-Sensing

CE1

CE3

Redundancy Group Membership

Auto-Discovery

MPLS

PE2

PE4
PE 2 Eth Segment Route
RD = RD20

ESI = ESI1
ES-Import ext. comm.
2014 Cisco and/or its affiliates. All rights reserved.

e.g. 0011.0022.0033

34

Ordered List of discovered PEs

starting from zero (lowest IP add)

PBB-EVPN Startup Sequence

Designated Forwarder (DF) Election*
Segment Auto-Discovery

Modulo Operation

Result of modulo
operation is used to
determine DF and
BDF status

I-SID

I-SID mod N
(N = # of PEs)
(e.g. I-SID mod 2)

100

PE Ordered List
Position

PE

PE1

101

PE2

102

103

Example:
PE1 DF for I-SIDs 100, 102
PE1 BDF for I-SIDs 101, 103

PE1

ESI and B-MAC Auto-Sensing

PE3
Exchange of Ethernet
Segment Routes

CE1

CE3

Redundancy Group Membership

Auto-Discovery
Modulo Operation
I-SID

(I-SID mod 2)

100

101

102

103

(*) DF election with Service Carving shown (i.e. one DF per I-SID in the segment)
2014 Cisco and/or its affiliates. All rights reserved.

MPLS

PE Ordered List
Position

PE

PE1

PE2

Example:
PE2 DF for I-SIDs 101, 103
PE2 BDF for I-SIDs 100, 102

PE2

PE4

DF Designated Forwarder
BDF Backup Designated Forwarder
I-SID PBB 24-bit Service Instance ID

35

PBB-EVPN Startup Sequence (cont.)

BGP MAC Advertisement Route (B-MAC)
Segment Auto-Discovery

RD RD unique per
advertising PE per EVI

PE1 MAC Route

RD = RD-1a

MP2P VPN Label

downstream allocated label
used by other PEs to send
traffic to advertised (MAC,EVI)

ESI reserved ESI

indicates advertised
MAC is a B-MAC

ESI = all 1s
MAC = B-M1

B-MAC advertised
by route

Label = L1
PE1

RT ext. community

RT-a

PE3

ESI and B-MAC Auto-Sensing

B-M1

B-M2

CE1

CE3

Redundancy Group Membership

Auto-Discovery

Backbone MAC (B-MAC)

Reachability Advertisement

MPLS

PE2 MAC Route

RD = RD-2a
ESI = all 1s
MAC = B-M1

PE2

PE4
Path List

PE3 / PE4 RIB

Label = L2

VPN

MAC

ESI

NH

RT ext. community

RT-a

B-M1

n/a

PE1

RT-a
2014 Cisco and/or its affiliates. All rights reserved.

B-M2

B-M1

PE2

36

PBB-EVPN Startup Sequence

BGP Inclusive Multicast Route
VPN Auto-Discovery

Multicast Tunnel ID / Endpoint

Discovery1

PE 1 Inclusive Multicast Route

RD = RD-1a

Tunnel Type Ingress

Replication or P2MP LSP

PMSI Tunnel Attribute

Tunnel Type (e.g. Ing. Repl.)

Mcast MPLS Label used to

transmit BUM traffic downstream assigned (ing.
repl.) or upstream assigned
(Aggregate Inclusive P2MP
LSP2)
RT RT associated with a
given EVI

RD RD unique per
adv. PE per EVI

Label (e.g. L1)

RT ext. community
RT-a
PE1

PE3

CE1

CE3

MPLS
PE 2 Inclusive Multicast Route
RD = RD-2a
PMSI Tunnel Attribute
Tunnel Type (e.g. Ing. Repl.)

PE2

PE4

Label (e.g. L2)

(1) Inclusive Multicast Route advertized per I-SID

(2) Multicast MPLS label is not set for Inclusive Trees (P2MP LSP)
2014 Cisco and/or its affiliates. All rights reserved.

RT ext. community
RT-a

PMSI - P-Multicast Service Interface

BUM Broadcast / Unknown Unicast / Multicast
37

Flows and Use Cases

PBB-EVPN Life of a Packet
38

PBB-EVPN Life of a Packet

Ingress Replication Multi-destination Traffic Forwarding

PE1 receives broadcast
traffic from CE1. PE1
adds PBB encapsulation
and forwards it using
ingress replication 3
copies created

During start-up sequence,

PE1, PE2, PE3, PE4 sent
Inclusive Multicast route
which include Mcast label
PE1

PE3

B-M1

VID 100
SMAC: M1
DMAC: F.F.F
B-M2

CE1

CE3

PSN MPLS label

to reach PE3

PE1
B-M1

B-M2

L3 PBB

CE1

CE3
L2 PBB

MPLS

Mcast MPLS
Label assigned by
PE3 for incoming
BUM traffic on a
given EVI
PE3 as DF, it
forwards BUM
traffic towards
PE3
segment

MPLS
L4 PBB

PE 2 Inclusive Multicast
Route

B-M2

B-M1

B-M2

B-M1

RD = RD-2a
PMSI Tunnel Attribute

PE2

Tunnel Type = Ing. Repl.

Label = L2
RT ext. community
RT-a
2014 Cisco and/or its affiliates. All rights reserved.

PE2

PE4
Mcast MPLS Label used to
transmit BUM traffic downstream assigned (for
ingress replication)

PE2 drops BUM

traffic originated
on same source
B-MAC (B-M1)

PE4
PE3 MAC Table
I-SID xyz
C-MAC

B-MAC

M1

B-M1

PE4 non-DF for

given I-SID drops
BUM traffic
Data-plane based
MAC learning for
C-MAC / B-MAC
association

39

PBB-EVPN Life of a Packet (cont.)

Unicast Traffic Forwarding and Aliasing
PE1 MAC Route

MP2P VPN Label

downstream allocated label
used by other PEs to send
traffic to advertised MAC

RD = RD-1a

PSN MPLS label

to reach PE1

MAC advertised
by route

ESI = all 1s
MAC = B-M1
Label = L1

VID 100
SMAC: M1
DMAC: F.F.F

PE1

RT ext. community

RT-a

PE1

PE3

B-M1

CE1

PE3 forwards traffic

on a flow (flow 2) to
M1 using B-MAC BM1 towards PE2

VID 100
SMAC: M3
DMAC: M1
B-M2

L1 PBB
CE3

CE1

VID 100
SMAC: M4
DMAC: M1

CE3

MPLS

MPLS

L2 PBB
B-M2

B-M1

PE2 MAC Route

RD = RD-2a
ESI = all 1s
MAC = B-M1

PE2

PE4

PE3 RIB

Path List

Data-plane based
MAC learning for CMAC / B-MAC
association

PE3 MAC Table

I-SID xyz

Label = L2

VPN

MAC

ESI

NH

RT ext. community

RT-a

B-M1

n/a

PE1

C-MAC

B-MAC

PE2

M1

B-M1

RT-a

PE3

B-M1

B-M2

During start-up sequence,

PE1 & PE2 advertised
MAC route for B-MAC (BM1)

PE3 forwards traffic

on a flow (flow 1) to
MP2P VPN
M1 using B-MAC BLabel
M1 towards PE1
assigned by
PE1 for
incoming traffic
for target EVI

2014 Cisco and/or its affiliates. All rights reserved.

B-M2

B-M1

PE2
PSN MPLS label
to reach PE2

PE4
MP2P VPN Label
assigned by PE2
for incoming traffic
for target EVI

40

Flows and Use Cases

PBB-EVPN Operational / Failure scenarios
42

PBB-EVPN Operational Scenarios

3

MAC Mobility

Host M1 moves
from CE1 to CE3s
location

1
PE1 learns C-MAC M1 on local
port and forwards across core
according to C-MAC DA to
Remote B-MAC mapping

PE3

B-M1

MPLS

PE3 MAC Table

I-SID xyz

PE4

M1

B-M2

B-M1

C-MAC

B-MAC

M1

B-M1

PE2
PE1 MAC Table
I-SID xyz

5
2014 Cisco and/or its affiliates. All rights reserved.

CE3

M1

B-M2

PE2

VID 100
SMAC: M1
B-M2 DMAC: F.F.F

L3 L4 PBB

B-M1

M1

PE3

CE1

CE3

MPLS

B-MAC

PE1

B-M1

B-M2

M1

C-MAC

Via data-plane
learning, PE1
updates C-MAC M1
location (via B-MAC
B-M2)

4
After host sends traffic at
new location, PE3 updates
C-MAC M1 location (local
port.) PE3 also forwards
across core according to CMAC DA to Remote B-MAC
mapping

L1 L2 PBB

PE1 MAC Table

I-SID xyz

CE1

2
Via data-plane
learning, PE3 learns
C-MAC M1 via BMAC B-M1

PE1
VID 100
SMAC: M1
DMAC: M2

MAC Mobility event handled entirely by dataplane learning

PE4
PE3 MAC Table
I-SID xyz

C-MAC

B-MAC

C-MAC

B-MAC

M1

B-M2

M1

4
43

PBB-EVPN Failure Scenarios / Convergence

Link / Segment Failure Active/Active per Flow
2
1
PE1 detects failure
of one of its
attached segments

PE1 withdraws B-MAC

advertised for failed
segment (B-M1)
PE1

PE3

B-M1

B-M2

CE1

CE3

MPLS

2
PE1 withdraws Ethernet
Segment Route

B-M2

B-M1

4
PE2 reruns DF election.
Becomes DF for all ISIDs on segment

PE2

PE3 / PE4
remove PE1 from
path list for BMAC (B-M1)

PE4
Path List

PE3, PE4 RIB

VPN

MAC

ESI

NH

RT-a

B-M1

n/a

PE1
PE2

2014 Cisco and/or its affiliates. All rights reserved.

44

PBB-EVPN Failure Scenarios / Convergence

Core Isolation

PE Failure
2
PE1 experiences a
node failure (e.g.
power failure)

PE1 sends LACP

OUT_OF_SYNC for
CE1 to take port out
of the bundle

BGP RR / PE3 detects

BGP session time-out
with PE1

PE1

PE3

B-M1

B-M2

LACP PDU

CE1

CE3

PE3

B-M1

B-M2

CE1

CE3

MPLS

2
B-M1

PE2

BGP RR / PE4
detects BGP
session timeout with PE1

4
PE2 reruns DF election.
Becomes DF for all ISIDs on segment
2014 Cisco and/or its affiliates. All rights reserved.

BGP RR / PE3 detects

BGP session time-out
with PE1

PE1

MPLS
BGP RR / PE2 detects
BGP session time-out
with PE1

PE1 looses
connectivity to the
core

B-M2

PE3 / PE4
invalidate routes
from PE1

PE4
Path List

PE3, PE4 RIB

VPN

MAC

ESI

NH

RT-a

B-M1

n/a

PE1
PE2

2
BGP RR / PE2
detects BGP
session timeout with PE1

B-M1

PE2

5
PE2 reruns DF election.
Becomes DF for all ISIDs on segment

4
B-M2

BGP RR / PE4
detects BGP
session timeout with PE1

PE3 / PE4
invalidate routes
from PE1

PE4
Path List

PE3, PE4 RIB

VPN

MAC

ESI

NH

RT-a

M1

ES1

PE1
PE2

45

Use Cases
46

PBB-EVPN Model
Customer Bridge
Domain
Ethernet
Segment
Identifier
ESI 1

BD

Core Bridge
Domain

B-MAC1

BD
BD
BD

ESI 2
BD

E-VPN
Forwarder

B-MAC2

BD

EFP

2014 Cisco and/or its affiliates. All rights reserved.

I-Component

B-Component

47

PBB-EVPN
Sample Use Access
Single Home Device (SHD)
Single Home Network (SHN)

Dual Home Device (DHD)

Active / Active Per-Flow LB

Dual Home Device (DHD)

Active / Active Per-Service LB

PE1

PE1

CE1
ESI Null PE1

VID
X

BMAC
1
ESI W

VID
X

CE1

VID X

BMAC
1
ESI W

CE1

MPLS
Core

MPLS
Core

MPLS
Core

CE2
ESI Null

VID
X

VID X

VID
Y

BMAC
1
ESI W

PE2

Null Ethernet Segment

Identifier (ESI)

2014 Cisco and/or its affiliates. All rights reserved.

BMAC
2
ESI W

PE2

Identical B-MAC on PBBEVPN PEs (PE1 / PE2)

Different B-MAC on PBBEVPN PEs (PE1 / PE2)

Identical ESI on PBB-EVPN

PEs

Identical ESI on PBB-EVPN

PEs

Per service (I-SID) carving

(manual or automatic)

48

PBB-EVPN
Sample Use Access (cont.)
Multi Home Device (MHD)
Active / Active Per-Flow LB

Multi Home Device (MHD)

Active / Active Per-Service LB

PE1

PE1

BMAC 1
ESI W

VID X

VID X

CE1

CE1

BMAC 1
ESI W

BMAC 2
ESI W

MPLS
Core

VID X

BMAC 1
ESI W

VID Y

PE2

VID X

MPLS
Core
PE2

VID Z

BMAC 1
ESI W

PE3

BMAC 3
ESI W

PE3

More than two (2) PEs in

redundancy group

More than two (2) PEs in

redundancy group

Same as DHD Act/Act

per-flow LB

Same as DHD Act/Act

per-service LB

2014 Cisco and/or its affiliates. All rights reserved.

49

PBB-EVPN
Sample Use Access (cont.)
Dual Home Network (DHN)
ITU-T G.8032
CE1

ESI Null

Dual Home Network (DHN)

REP

PE1

PE1
CE1

VID X

MPLS
Core

G.8032
Open Sub-ring

REP

REP
Edge No
Neighbou
r

VID Y

ESI Null

PE2

Treated as SHN by PBBEVPN PEs (PE1 / PE2)

ALT
port

Null ESI; No DF election / No

service carving

Ring operation controlled by

R-APS protocol

2014 Cisco and/or its affiliates. All rights reserved.

REP-AG
REP-AG

PE1

MPLS
Core

MPLS
Core
VID Y

VID Y

CE2

BMAC 1
ESI W

VID X

VID X

VID X

CE1

VID Y

R-APS

CE2

ESI Null
VID X

VID Y

RPL
Link

Dual Home Network (DHN)

Active / Active Per-Service LB

ESI Null

PE2

CE2

BMAC 2
ESI W
PE2

Treated as SHN by PBBEVPN PEs (PE1 / PE2)

Different B-MAC on PBBEVPN PEs (PE1 / PE2)

Null ESI; No DF election / No

service carving

Identical ESI on PBB-EVPN

PEs

Per service (I-SID) carving

(manual or automatic)

Segment operation
controlled by REP protocol

50

PBB-EVPN IOS-XR Implementation

Configuration and Examples
51

MINIMAL
Configuration

PBB-EVPN Single Home Device (SHD)

PE1

Global B-MAC SA
Auto RT for EVI
Auto RD for EVI
Auto RD for Segment Route

interface Bundle-Ether1.777 l2transport

encapsulation dot1q 777
l2vpn
bridge group gr1
bridge-domain bd1
interface Bundle-Ether1.777
pbb edge i-sid 100 core-bridge-domain core_bd1
bridge group gr2
bridge-domain core_bd1
pbb-core
evpn evi 1000
router bgp 64
address-family l2vpn evpn
!
neighbor <x.x.x.x>
remote-as 64
address-family l2vpn evpn

2014 Cisco and/or its affiliates. All rights reserved.

PBB I-component
Includes I-SID assignment
PBB B-component
No need to define B-VLAN

PE1

CE1

BundleEth1.777

MPLS
Core

Mandatory - Globally
unique identifier for all PEs
in a given EVI

BGP configuration with

new E-VPN AF

Note: MPLS / LDP configuration

required on core-facing interfaces (not
shown)
52

MINIMAL
Configuration

PBB-EVPN Dual Home Device (DHD)

Active / Active per-FLOW Load Balancing
PE1
redundancy
iccp
group 66
mlacp node 1
mlacp system mac 0aaa.0bbb.0ccc
mlacp system priority 1
mode singleton
interface Bundle-Ether25
mlacp iccp-group 66
interface Bundle-Ether25.1 l2transport
encapsulation dot1q 777
l2vpn
bridge group gr1
bridge-domain bd1
interface Bundle-Ether25.1
pbb edge i-sid 100 core-bridge-domain core_bd1
bridge group gr2
bridge-domain core_bd1
pbb-core
evpn evi 1000
router bgp 64
address-family l2vpn evpn
neighbor <x.x.x.x> remote-as 64
address-family l2vpn evpn
2014 Cisco and/or its affiliates. All rights reserved.

Auto ESI
Auto B-MAC SA
A/A Per-flow LB (default)
Auto RT for EVI
Auto RD for EVI
Auto RD for Segment Route
PE2 should use same RG
number
PE 2 should use different mlacp
node id
PE2 should use same mlacp
system mac and system priority
ICCP in singleton mode (i.e.No
peer neighbor configuration)

PBB I-component and Bcomponent configuration. ISIDs

must match on both PEs
No need to define B-VLAN
Mandatory EVI ID configuration
BGP configuration with
new EVPN AF

PE1
BundleEth25.1

CE1
MPLS
Core

BundleEth25.1

PE2

Note: MPLS / LDP configuration

required on core-facing interfaces (not
shown)
53

MINIMAL
Configuration

PBB-EVPN Dual Home Device (DHD)

Active / Active per-Service Load Balancing and Dynamic Service Carving

PE1

interface Bundle-Ether25.1 l2transport

encapsulation dot1q 777
evpn
interface Bundle-Ether25
ethernet-segment
identifier system-priority 1 system-id 0000.0b25.00ce
load-balancing-mode per-service
l2vpn
bridge group gr1
bridge-domain bd1
interface Bundle-Ether25.1
pbb edge i-sid 100 core-bridge-domain core_bd1
bridge group gr2
bridge-domain core_bd1
pbb-core
evpn evi 1000
router bgp 64
address-family l2vpn evpn
neighbor <x.x.x.x> remote-as 64
address-family l2vpn evpn

2014 Cisco and/or its affiliates. All rights reserved.

Global B-MAC SA
Default Service Carving
Auto RT for EVI
Auto RD for EVI
Auto RD for Segment Route
A/A per-service (per-ISID)
load balancing with
dynamic Service Carving
ESI must match on both
PEs
PBB I-component and Bcomponent configuration.
ISIDs must match on both
PEs
No need to define B-VLAN
Mandatory EVI ID
configuration
BGP configuration with
new EVPN AF

PE1
BundleEth25

CE1
MPLS
Core

BundleEth25

PE2

Note: MPLS / LDP configuration

required on core-facing interfaces (not
shown). ICCP (singleton) config (not
shown)
54

Summary
55

Summary
E-VPN / PBB-EVPN are next-generation L2VPN solutions based on a BGP
control-plane for MAC distribution/learning over the core
E-VPN / PBB-EVPN were designed to address following requirements:

All-active Redundancy and Load Balancing

Simplified Provisioning and Operation
Optimal Forwarding
Fast Convergence

In addition, PBB-EVPN and its inherent MAC-in-MAC hierarchy provides:

Scale to Millions of C-MAC (Virtual Machine) Addresses
MAC summarization co-existence with C-MAC (VM) mobility

E-VPN / PBB-EVPN applicability goes beyond DCI into Carrier Ethernet use
cases

2014 Cisco and/or its affiliates. All rights reserved.

References
draft-ietf-l2vpn-evpn
draft-ietf-l2vpn-pbb-evpn
draft-ietf-l2vpn-trill-evpn

2014 Cisco and/or its affiliates. All rights reserved.

Acronyms IP and MPLS

Acronym

Description

Acronym

Description

AC

Attachment Circuit

PW

Pseudo-Wire

AS

Autonomous System

PWE3

Pseudo-Wire End-to-End Emulation

BFD

Bidirectional Failure Detection

QoS

Quality of Service

CoS

Class of Service

RD

Route Distinguisher

ECMP

Equal Cost Multipath

RIB

Routing Information Base

EoMPLS

Ethernet over MPLS

RR

Route Reflector

E-VPN

Ethernet Virtual Private Network

RSVP

Resource Reservation Protocol

EVI

E-VPN Instance

RSVP-TE

RSVP based Traffic Engineering

FRR

Fast Re-Route

RT

Route Target

IGP

Interior Gateway Protocol

TE

Traffic Engineering

LDP

Label Distribution Protocol

tLDP

Targeted LDP

LER

Label Edge Router

VC

Virtual Circuit

LFIB

Labeled Forwarding Information Base

VCID

VC Identifier

LSM

Label Switched Multicast

VFI

Virtual Forwarding Instance

LSP

Label Switched Path

VPLS

Virtual Private LAN Service

LSR

Label Switching Router

VPN

Virtual Private Network

MPLS

Multi-Protocol Label Switching

VPWS

Virtual Private Wire Service

NLRI

Network Layer Reachability Information

VRF

Virtual Route Forwarding Instance

PSN

Packet Switch Network

VSI

Virtual Switching Instance

2014 Cisco and/or its affiliates. All rights reserved.

58

Acronyms Ethernet/Bridging
Acronym

Description

Acronym

Description

ACL

Access Control List

MVRP

Multiple VLAN Registration Protocol

BD

Bridge Domain

PE

Provider Edge device

BPDU

Bridge Protocol Data Unit

PoA

Point of Attachment

CE

Customer Equipment (Edge)

REP

Resilient Ethernet Protocol

REP-AG

REP Access Gateway

RG

Redundancy Group

STP

Spanning Tree Protocol

C-VLAN / CEVLAN

Customer / CE VLAN

CoS

Class of Service

DHD

Dual Homed Device

LACP

Link Aggregation Control Protocol

LAN

Local Area Network

MEF

Metro Ethernet Forum

MEN

Metro Ethernet Network

MIRP

Multiple I-Tag Registration Protocol

mLACP

Multi-Chassis LACP

MST / MSTP

Multiple Instance STP

MSTG-AG

MST Access Gateway

2014 Cisco and/or its affiliates. All rights reserved.

59

Acronyms Provider Backbone Bridging

Acronym

Description

Acronym

Description

B-BEB

B-Component BEB

I-BEB

I-Component BEB

BCB

Backbone Core Bridge

IEEE

Institute of Electrical and Electronics Engineers

B-DA

Backbone Destination Address

BEB

Backbone Edge Bridge

I-SID

Instance Service Identifier

B-MAC

Backbone MAC Address

I-Tag

I-SID Tag

B-SA

Backbone Source Address

MAC

Media Access Control

B-Tag

B-VLAN Tag

N-PE

Network-facing Provider Edge device

B-VLAN

Backbone VLAN

PB

Provider Bridge

C-DA

Customer Destination Address

PBB

Provider Backbone Bridge / Bridging

CE

Customer Equipment (Edge)

PBBN

Provider Backbone Bridging Network

C-MAC

Customer MAC Address

PBN

Provider Bridging Network

C-SA

Customer Source Address

PE

Provider Edge device

80

C-VLAN Tag

Q-in-Q

VLAN tunneling using two 802.1Q tags

SA

Source MAC Address

C-VLAN / CEVLAN

Customer / CE VLAN

S-Tag

S-VLAN Tag

DA

Destination MAC Address

S-VLAN

Service VLAN (Provider VLAN)

FCS

Frame Check Sequence

UNI

User to Network Interface

IB-BEB

Combined I-Component & B-Component BEB

U-PE

User-facing Provider Edge device

VLAN

Virtual LAN

2014 Cisco and/or its affiliates. All rights reserved.

60

BGP Routes and Attributes

62

BGP Routes
Overview
(PBB) E-VPN defines a single new BGP NLRI used to carry all E-VPN routes.

The NLRI has a new SAFI (70).

1. Ethernet Auto (PBB) E-VPN speakers must first exchange BGP

capability for
E-VPN
AFI /
Discovery
(AD)
Route
SAFI per RFC4760.
2. MAC Advertisement
Route
1 byte
Route Type
3. Inclusive Multicast
1 byte
Length
Route
Variable Route Type Specific
4. Ethernet Segment
Route
2014 Cisco and/or its affiliates. All rights reserved.

63

BGP Routes
Route Types and Usage
Route

Usage

Applicability

Ethernet A-D Route

E-VPN only

MAC Advertisement
Route

MAC Mass-Withdraw
Aliasing
Advertising Split-Horizon Labels
Advertise MAC Address Reachability
Advertise IP/MAC Bindings

E-VPN &
PBB-EVPN

Inclusive Multicast
Route

Multicast Tunnel Endpoint Discovery

E-VPN &
PBB-EVPN

Ethernet Segment
Route

E-VPN &
PBB-EVPN

2014 Cisco and/or its affiliates. All rights reserved.

Redundancy Group Discovery

DF Election

64

BGP Routes
Route Attributes and Usage
Attribute

Usage

Route
Applicability

ESI MPLS Label

Extended Community

Encode Split-Horizon Label for Ethernet

Segment.
Indicate Redundancy Mode (Active/Standby vs.
All-Active)
Limit the import scope of the Ethernet Segment
routes.

Ethernet A-D
Route

E-VPN: Indicate that a MAC address has moved

from one segment to another across PEs.
PBB-EVPN: Signal C-MAC address flush
notification
Indicate the MAC/IP bindings of a gateway

MAC
Advertisement
Route

ES-Import Extended
Community

MAC Mobility Extended

Community

Default Gateway
Extended Community
2014 Cisco and/or its affiliates. All rights reserved.

Ethernet
Segment Route

MAC
Advertisement
Route

65