Escolar Documentos
Profissional Documentos
Cultura Documentos
DO NOT DISTRIBUTE
Rev 2 (02-May-2014)
Introduction ........................................................................................................................................................................... 2
Statistics/Errors .................................................................................................................................................................... 82
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 1
Introduction
DDOS protection infrastructure is introduced with the TRIO ASIC which is mainly used to monitor, inspect, classify and
police the host bound traffic flows to avoid any misbehaved flows from causing any unexpected host queue congestion in
different part of the system (ASIC, uKern and RE). This is enabled by default with user-configurable pre-defined threshold
for various packet types.
In this documents, we will go through the implementation of DDOS on MX platform with TRIO MPC and explain the how
the policers are applied on different part of the system. The followings are based on JUNOS 13.3 version.
Once the host bound traffic is received via the PUNT nexthop with different PUNT reasons, it will be tagged to a DDOS
protocol ID according to their packet type. If the packet is control packet, for example, an IPv4/IPv6 packet, the Host
bound classification filter (HBC) (ie. HOSTBOUND_IPv4_FILTER / HOSTBOUND_IPv6_FILTER) filter is used to further
look into the packet content like ip-protocol , source / destination port numbers to determine the packet type and further
classify a DDOS protocol ID for it.
Once the packet is tagged with the DDOS protocol ID, the corresponding policer will be applied to rate limit that specific
packet type. Here is the HOSTBOUND_IPv4_FILTER.
NPC2(Dokinchan-re0 vty)# show filter index 46137345 program
Filter index = 46137345
Optimization flag: 0x0
Filter notify host id = 0
Filter properties: None
Filter state = CONSISTENT
term HOSTBOUND_IGMP_TERM
term priority 0
payload-protocol
2
then
accept
ddos proto 69
term HOSTBOUND_OSPF_TERM
term priority 0
payload-protocol
89
then
accept
ddos proto 70
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 2
term HOSTBOUND_RSVP_TERM
term priority 0
payload-protocol
46
then
accept
ddos proto 71
term HOSTBOUND_PIM_TERM
term priority 0
payload-protocol
103
then
accept
ddos proto 72
term HOSTBOUND_DHCP_TERM
term priority 0
payload-protocol
17
destination-port
67-68
then
accept
ddos proto 24
term HOSTBOUND_RIP_TERM
term priority 0
payload-protocol
17
destination-port
520-521
then
accept
ddos proto 73
term HOSTBOUND_PTP_TERM
term priority 0
payload-protocol
17
destination-port
319-320
then
action next-hop, type (set ptp nh)
ddos proto 74
term HOSTBOUND_BFD_TERM1
term priority 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 3
payload-protocol
17
destination-port
3784-3785
then
action next-hop, type (inline keepalive BFD nh)
ddos proto 75
term HOSTBOUND_BFD_TERM2
term priority 0
payload-protocol
17
destination-port
4784
then
accept
ddos proto 75
term HOSTBOUND_LMP_TERM
term priority 0
payload-protocol
17
destination-port
701
then
accept
ddos proto 76
term HOSTBOUND_ANCP_TERM
term priority 0
payload-protocol
6
destination-port
6068
then
accept
ddos proto 85
term HOSTBOUND_LDP_TERM1
term priority 0
payload-protocol
6
destination-port
646
then
accept
ddos proto 77
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 4
term HOSTBOUND_LDP_TERM2
term priority 0
payload-protocol
6
source-port
646
then
accept
ddos proto 77
term HOSTBOUND_LDP_TERM3
term priority 0
payload-protocol
17
destination-port
646
then
accept
ddos proto 77
term HOSTBOUND_LDP_TERM4
term priority 0
payload-protocol
17
source-port
646
then
accept
ddos proto 77
term HOSTBOUND_MSDP_TERM1
term priority 0
payload-protocol
6
destination-port
639
then
accept
ddos proto 78
term HOSTBOUND_MSDP_TERM2
term priority 0
payload-protocol
6
source-port
639
then
accept
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 5
ddos proto 78
term HOSTBOUND_BGP_TERM1
term priority 0
payload-protocol
6
destination-port
179
then
accept
ddos proto 79
term HOSTBOUND_BGP_TERM2
term priority 0
payload-protocol
6
source-port
179
then
accept
ddos proto 79
term HOSTBOUND_VRRP_TERM
term priority 0
payload-protocol
112
destination-address
224.0.0.18/32
then
action next-hop, type (inline keepalive VRRP nh)
ddos proto 80
term HOSTBOUND_TELNET_TERM1
term priority 0
payload-protocol
6
destination-port
23
then
accept
ddos proto 81
term HOSTBOUND_TELNET_TERM2
term priority 0
payload-protocol
6
source-port
23
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 6
then
accept
ddos proto 81
term HOSTBOUND_FTP_TERM1
term priority 0
payload-protocol
6
destination-port
20-21
then
accept
ddos proto 82
term HOSTBOUND_FTP_TERM2
term priority 0
payload-protocol
6
source-port
20-21
then
accept
ddos proto 82
term HOSTBOUND_SSH_TERM1
term priority 0
payload-protocol
6
destination-port
22
then
accept
ddos proto 83
term HOSTBOUND_SSH_TERM2
term priority 0
payload-protocol
6
source-port
22
then
accept
ddos proto 83
term HOSTBOUND_SNMP_TERM1
term priority 0
payload-protocol
17
destination-port
161
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 7
then
accept
ddos proto 84
term HOSTBOUND_SNMP_TERM2
term priority 0
payload-protocol
17
source-port
161
then
accept
ddos proto 84
term HOSTBOUND_DTCP_TERM
term priority 0
payload-protocol
17
destination-port
652
destination-address
224.0.0.36/32
then
accept
ddos proto 148
term HOSTBOUND_RADIUS_TERM_SERVER
term priority 0
payload-protocol
17
destination-port
1812
then
accept
ddos proto 151
term HOSTBOUND_RADIUS_TERM_ACCOUNT
term priority 0
payload-protocol
17
destination-port
1813
then
accept
ddos proto 152
term HOSTBOUND_RADIUS_TERM_AUTH
term priority 0
payload-protocol
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 8
17
destination-port
3799
then
accept
ddos proto 153
term HOSTBOUND_NTP_TERM
term priority 0
payload-protocol
17
destination-port
123
destination-address
224.0.1.1/32
then
accept
ddos proto 154
term HOSTBOUND_TACACS_TERM
term priority 0
payload-protocol
17
destination-port
49
then
accept
ddos proto 155
term HOSTBOUND_DNS_TERM1
term priority 0
payload-protocol
6
destination-port
53
then
accept
ddos proto 156
term HOSTBOUND_DNS_TERM2
term priority 0
payload-protocol
17
destination-port
53
then
accept
ddos proto 156
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 9
term HOSTBOUND_DIAMETER_TERM1
term priority 0
payload-protocol
6
destination-port
3868
then
accept
ddos proto 157
term HOSTBOUND_DIAMETER_TERM2
term priority 0
payload-protocol
132
destination-port
3868
then
accept
ddos proto 157
term HOSTBOUND_L2TP_TERM
term priority 0
payload-protocol
17
destination-port
1701
then
accept
ddos proto 162
term HOSTBOUND_GRE_TERM
term priority 0
payload-protocol
47
then
accept
ddos proto 163
term HOSTBOUND_ICMP_TERM
term priority 0
payload-protocol
1
then
accept
ddos proto 68
term HOSTBOUND_TCP_FLAGS_TERM_INITIAL
term priority 0
payload-protocol
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 10
6
tcp-flags
value & 0x12 = 0x02
then
accept
ddos proto 146
term HOSTBOUND_TCP_FLAGS_TERM_ESTAB
term priority 0
payload-protocol
6
tcp-flags
value & 0x14 != 0x00
then
accept
ddos proto 147
term HOSTBOUND_TCP_FLAGS_TERM_UNCLS
term priority 0
payload-protocol
6
tcp-flags
value & 0x3f != 0x00
then
accept
ddos proto 145
term HOSTBOUND_IP_FRAG_TERM_FIRST
term priority 0
is-fragment
value & 0x3fff = 0x2000
then
accept
ddos proto 160
term HOSTBOUND_IP_FRAG_TERM_TRAIL
term priority 0
is-fragment
value & 0x1fff != 0x0000
then
accept
ddos proto 161
term HOSTBOUND_AMT_TERM1
term priority 0
payload-protocol
17
destination-port
2268
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 11
then
accept
ddos proto 198
term HOSTBOUND_AMT_TERM2
term priority 0
payload-protocol
17
source-port
2268
then
accept
ddos proto 198
term HOSTBOUND_IPV4_DEFAULT_TERM
term priority 0
then
accept
NPC2(Dokinchan-re0 vty)#
Policer Hierarchy
The DDOS configuration is mainly a combination of three different levels ASIC, uKern and Routing Engine. Each of
them will apply a rate limit on the corresponding packet type. The DDOS is enabled by default. Although it can be disabled
via a configuration knob, thats not recommended.
# set system ddos-protection global ?
Possible completions:
disable-fpc Disable FPC policing for all protocols
disable-logging Disable event logging for all protocols
disable-routing-engine Disable Routing Engine policing for all protocols
However, if we disable the DDOS for a specific protocol, it doesnt mean that it will fail thru the other term within the
DDOS filter it just means that we will accept all those packets without policing.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 12
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 13
Lets take IPv4 unclassified packets (ie. host bound packet which doesnt fall into any of the pre-defined IPv4 protocol type
above) as an example. Under unclassified protocol type, we have separated policer configuration on per-packet host
bound notification type. (Note: The unclassified protocol type should cover IPv6 as well but I take out the IPv6 part to
simplify it a bit. Also, the flow- related configuration will be covered under the SCFD section.)
# set system ddos-protection protocols unclassified ?
Possible completions:
> aggregate Configure aggregate for all unclassified host-bound traffic
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
> control-layer2 Configure unclassified layer2 control packets
> control-v4 Configure unclassified v4 control packets
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 14
Under each notif type, we can define the policer rate and the burst size for the whole system (ie. Routing Engine level) or
under each FPC (uKern level). Under each FPC, each PFE (ie. ASIC level) will take the FPC policer configuration and
apply that on the ASIC level under LUchip as well.
# set system ddos-protection protocols unclassified host-route-v4 ?
Possible completions:
bandwidth Policer bandwidth (1..100000 packets per second)
burst Policer burst size (1..100000 packets)
bypass-aggregate Bypass aggregate policer
disable-fpc Turn off policing on all fpc's
disable-logging Disable event logging for protocol violation
disable-routing-engine Turn off policing on routing engine
> fpc Flexible PIC Concentrator parameters
recover-time Time for protocol to return to normal (1..3600 seconds)
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 15
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 16
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 17
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 18
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 19
We can find exactly the same thing for other protocols. For example, PPP.
# set system ddos-protection protocols ppp ?
Possible completions:
> aggregate Configure aggregate for all PPP control traffic
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
> authentication Configure Authentication Protocol
> echo-rep Configure LCP Echo Reply
> echo-req Configure LCP Echo Request
> ipcp Configure IP Control Protocol
> ipv6cp Configure IPv6 Control Protocol
> isis Configure ISIS Protocol
> lcp Configure Link Control Protocol
> mlppp-lcp Configure MLPPP LCP
> mplscp Configure MPLS Control Protocol
> unclassified Configure unclassified PPP control traffic
[edit]
lab@Dokinchan-re0# set system ddos-protection protocols ppp echo-req ?
Possible completions:
+ apply-groups Groups from which to inherit configuration data
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 20
UKERN-Config PFE-Config
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 21
We will cover the relationship of the policers in each level under the following sections.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 22
ASIC Level
The policer on ASIC is done by the LUchip. The following is a map of protocol type and policer being applied. Under
DDOS, each protocol / frame type will have an index and protocol ID defined (which is NOT the IPv4-protocol ID). The
DDOS policer will map the corresponding protocol / frame type to the corresponding protocol ID for classification.
Here is a list of each protocol type and the corresponding protocol ID and index. For each of them, there are uKern level
and PFE (ie. LUchip) level configurations. There is a priority for each protocol type but its only between the protocols
(For example, lcp, auth, ipcp..etc) under the same group (i.e. PPP).
# show ddos policer configuration all
DDOS Policer Configuration:
UKERN-Config PFE-Config
idx prot group proto on Pri rate burst rate burst
--- --- ------------ ------------ -- -- ------ ----- ------ -----
0 0 host-path aggregate Y -- --- --- 25000 25000
1 100 ipv4-uncls aggregate Y Md 2000 10000 2000 10000
2 200 ipv6-uncls aggregate Y Md 2000 10000 2000 10000
3 300 dynvlan aggregate Y Lo 1000 500 1000 500
4 400 ppp aggregate Y Md 16000 16000 --- ---
5 401 ppp unclass Y Lo 1000 500 1000 500
6 402 ppp lcp Y Lo 12000 12000 12000 12000
7 403 ppp auth Y Md 2000 2000 2000 2000
8 404 ppp ipcp Y Hi 2000 2000 2000 2000
9 405 ppp ipv6cp Y Hi 2000 2000 2000 2000
10 406 ppp mplscp Y Hi 2000 2000 2000 2000
11 407 ppp isis Y Hi 2000 2000 2000 2000
12 408 ppp echo-req Y Lo 12000 12000 12000 12000
13 409 ppp echo-rep Y Lo 12000 12000 12000 12000
14 40a ppp mlppp-lcp Y Lo 12000 12000 12000 12000
15 500 pppoe aggregate Y Md 2000 2000 --- ---
16 501 pppoe unclass.. Y -- --- --- 0 0
17 502 pppoe padi Y Lo 500 500 500 500
18 503 pppoe pado Y Lo 0 0 0 0
19 504 pppoe padr Y Md 500 500 500 500
20 505 pppoe pads Y Lo 0 0 0 0
21 506 pppoe padt Y Hi 1000 1000 1000 1000
22 507 pppoe padm Y Lo 0 0 0 0
23 508 pppoe padn Y Lo 0 0 0 0
24 600 dhcpv4 aggregate Y Md 5000 5000 5000 5000
25 601 dhcpv4 unclass.. Y Lo 300 150 --- ---
26 602 dhcpv4 discover Y Lo 500 500 --- ---
27 603 dhcpv4 offer Y Lo 1000 1000 --- ---
28 604 dhcpv4 request Y Md 1000 1000 --- ---
29 605 dhcpv4 decline Y Lo 500 500 --- ---
30 606 dhcpv4 ack Y Md 500 500 --- ---
31 607 dhcpv4 nak Y Lo 500 500 --- ---
32 608 dhcpv4 release Y Hi 2000 2000 --- ---
33 609 dhcpv4 inform Y Lo 500 500 --- ---
34 60a dhcpv4 renew Y Hi 2000 2000 --- ---
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 23
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 24
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 25
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 26
Each protocol will be associated to different policers under different levels. Here is a nexthop and host bound queue under
MQ mapping for each PUNT traffic type.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 27
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 28
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 29
Here, the violation report message is one of the notification to the PPC. Hence, its also rate limited too 100pps by default.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 30
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 31
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 32
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 33
Lets trace some of the nexthop here to explain how the policers are associated to each other.
If we check the policer nexthop for this type, here is the policer configuration.
# show jnh 0 decode 0xc03c106607a21001
PolicerISSU_NH: Absolute Caddr = 0xc0f442, nextNH = 0x7820c, , type:0, color=0, op=0 use_layer3_len = 0x0,
num_nh = 0x0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 34
% bits 13 20 2 3 4 22
0xa3d0000047c00000
Wid 13 20 2 3 4 22
Bin 1010001111010 00000000000000000000 10 001 1111 0000000000000000000000
Hex 147a 0 2 1 f 0
Dec 5242 0 2 1 15
This is a policer with rate = 5242 * 1562.5 = 8,190,625 bps. On LUchip, the packet policer is using a fixed packet size (512
bytes), hence, that becomes 2000 pps which matches the policer configuration.
#define PKT_BASED_POLICER_PKT_SIZE (512)
Furthermore, if we check the ddos-nh, its actually pointing to another policer configuration.
NPC2(Dokinchan-re0 vty)# show jnh 0 decode 0xe01454000010000
NPC2(Dokinchan-re0 vty)#
0x8000000057c00000
Wid 13 20 2 3 4 22
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 35
The above policer is programmed with 4096 * 25000 = 25000 pps. Thats the host-path policer, which is trying to police
an aggregated traffic from some protocols to the host.
NPC2(Dokinchan-re0 vty)# show ddos policer configuration all
DDOS Policer Configuration:
UKERN-Config PFE-Config
NPC2(Dokinchan-re0 vty)#
This aggregated policer also applies to multiple protocols. For example, PUNT_REDIRECT, PUNT_REJECT,
PUNT_REJECT_FW, PUNT_RESOLVE etc
PUNT codes directly mapped to DDOS proto:
code PUNT name group proto idx q# bwidth burst
---- -------------------- --------- ------ ---- -- ------ ------
3 PUNT_REDIRECT redirect aggregate 3e00 0 2000 10000
40 PUNT_REJECT reject aggregate 4600 6 2000 10000
33 PUNT_RESOLVE
PUNT_REDIRECT
NPC2(Dokinchan-re0 vty)# show jnh 0 decode c03c108e07b7f001
PolicerISSU_NH: Absolute Caddr = 0xc0f6fe, nextNH = 0x78211, , type:0, color=0, op=0 use_layer3_len = 0x0,
num_nh = 0x0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 36
0xa3d0000047c00000
Wid 13 20 2 3 4 22
Bin 1010001111010 00000000000000000000 10 001 1111 0000000000000000000000
Hex 147a 0 2 1 f 0
Dec 5242 0 2 1 15 0
NPC2(Dokinchan-re0 vty)#
PUNT_REJECT
NPC2(Dokinchan-re0 vty)# show jnh 0 decode c03c111607b79001
PolicerISSU_NH: Absolute Caddr = 0xc0f6f2, nextNH = 0x78222, , type:0, color=0, op=0 use_layer3_len = 0x0,
num_nh = 0x0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 37
0xa3d257b447f4db1e
Wid 13 20 2 3 4 22
Bin 1010001111010 01001010111101101000 10 001 1111 1101001101101100011110
Hex 147a 4af68 2 1 f 34db1e
Dec 5242 307048 2 1 15 3463966
NPC2(Dokinchan-re0 vty)#
PUNT_RESOLVE
NPC2(Dokinchan-re0 vty)# show jnh 0 exceptions terse
Reason Type Packets Bytes
==================================================================
Routing
----------------------
resolve route PUNT(33) 7199596 460774144
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 38
0xc04819 2 : 0xda00602d26800b04
0xc0481a 3 : 0xda00602d20800b04
0xc0481b 4 : 0xdaf060208180c810
NPC2(Dokinchan-re0 vty)#
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 39
NPC2(Dokinchan-re0 vty)#
svl-jtac-tool02% bits 13 20 2 3 4 22
0xa3d0000047c00000
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 40
Wid 13 20 2 3 4 22
Bin 1010001111010 00000000000000000000 10 001 1111 0000000000000000000000
Hex 147a 0 2 1 f 0
Dec 5242 0 2 1 15 0
Policer Rate = 5242 * 1562.5 / 512 bytes = 2000 pps. Next it hits the DDOS-nh, which points to the host-path PFE policer.
NPC2(Dokinchan-re0 vty)# show jnh 0 decode e02eb9000010000
0xc05d70 0 : 0x42f07fffff800eb0
0xc05d71 1 : 0xdaf060208180c810
0xc0423d 0 : 0x42f07fffff800010
0xc0423e 1 : 0xc0040096078c1001
0xc0423f 2 : 0x127fffffe00003f8
PolicerISSU_NH: Absolute Caddr = 0xc0f182, nextNH = 0x8012, , type:0, color=0, op=0 use_layer3_len = 0x0, num_nh
= 0x0
NPC2(Dokinchan-re0 vty)#
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 41
11 PUNT_MLP |---------------+
32 PUNT_PROTOCOL | |
33 PUNT_RESOLVE | |
34 PUNT_RECEIVE | |
36 PUNT_REJECT_FW | |
54 PUNT_SEND_TO_HOST_FW | |
69 PUNT_RESOLVE_V6 | |
|
------------------------------------------------------------------
type subtype group proto idx q# bwidth burst
------ ---------- ---------- ---------- ---- -- ------ ------
contrl LACP lacp aggregate 2c00 3 20000 20000
contrl STP stp aggregate 2d00 3 20000 20000
contrl ESMC esmc aggregate 2e00 3 20000 20000
contrl OAM_LFM oam-lfm aggregate 2f00 3 20000 20000
contrl EOAM eoam aggregate 3000 3 20000 20000
contrl LLDP lldp aggregate 3100 3 20000 20000
contrl MVRP mvrp aggregate 3200 3 20000 20000
contrl PMVRP pmvrp aggregate 3300 3 20000 20000
contrl ARP arp aggregate 3400 2 20000 20000
contrl PVSTP pvstp aggregate 3500 3 20000 20000
contrl ISIS isis aggregate 3600 1 20000 20000
contrl POS pos aggregate 3700 3 20000 20000
contrl MLP mlp packets 3802 2 2000 10000
contrl JFM jfm aggregate 3900 3 20000 20000
contrl ATM atm aggregate 3a00 3 20000 20000
contrl PFE_ALIVE pfe-alive aggregate 3b00 3 20000 20000
NPC2(Dokinchan-re0 vty)#
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 42
LACP
NPC2(Dokinchan-re0 vty)# show jnh 0 decode c03c0f2e07a2a001
PolicerISSU_NH: Absolute Caddr = 0xc0f454, nextNH = 0x781e5, , type:0, color=0, op=0 use_layer3_len = 0x0,
num_nh = 0x0
NPC2(Dokinchan-re0 vty)#
svl-jtac-tool02% bits 13 20 2 3 4 22
0xccc8000053c00000
Wid 13 20 2 3 4 22
Bin 1100110011001 00000000000000000000 10 100 1111 0000000000000000000000
Hex 1999 0 2 4 f 0
Dec 6553 0 2 4 15 0
STP
NPC2(Dokinchan-re0 vty)# show jnh 0 decode c03c0ebe07a29001
PolicerISSU_NH: Absolute Caddr = 0xc0f452, nextNH = 0x781d7, , type:0, color=0, op=0 use_layer3_len = 0x0,
num_nh = 0x0
NPC2(Dokinchan-re0 vty)#
svl-jtac-tool02% bits 13 20 2 3 4 22
0xccc8000053c00000
Wid 13 20 2 3 4 22
Bin 1100110011001 00000000000000000000 10 100 1111 0000000000000000000000
Hex 1999 0 2 4 f 0
Dec 6553 0 2 4 15 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 43
---- -------------
2 PUNT_OPTIONS |
4 PUNT_CONTROL |
6 PUNT_HOST_COPY |
11 PUNT_MLP |---------------+
32 PUNT_PROTOCOL | |
33 PUNT_RESOLVE | |
34 PUNT_RECEIVE | |
36 PUNT_REJECT_FW | |
54 PUNT_SEND_TO_HOST_FW | |
69 PUNT_RESOLVE_V6 | |
|
------------------------------------------------------------------
type subtype group proto idx q# bwidth burst
------ ---------- ---------- ---------- ---- -- ------ ------
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 44
Take OSPF as an example. As the L2 control traffic, the only policer applied to this is the OSPF one. Once the packet
passes this policer, the packet will be sent to the host queue.
PolicerISSU_NH: Absolute Caddr = 0xc0f474, nextNH = 0x781a5, , type:0, color=0, op=0 use_layer3_len = 0x0,
num_nh = 0x0
NPC2(Dokinchan-re0 vty)#
0xccc8000053c00000
Wid 13 20 2 3 4 22
Bin 1100110011001 00000000000000000000 10 100 1111 0000000000000000000000
Hex 1999 0 2 4 f 0
Dec 6553 0 2 4 15 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 45
PolicerISSU_NH: Absolute Caddr = 0xc0f43e, nextNH = 0x78232, , type:0, color=0, op=0 use_layer3_len = 0x0,
num_nh = 0x0
0xccc800004fc00000
Wid 13 20 2 3 4 22
Bin 1100110011001 00000000000000000000 10 011 1111 0000000000000000000000
Hex 1999 0 2 3 f 0
Dec 6553 0 2 3 15 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 46
PolicerISSU_NH: Absolute Caddr = 0xc0f43a, nextNH = 0x78231, , type:0, color=0, op=0 use_layer3_len = 0x0,
num_nh = 0x0
0xccc8000053c00000
Wid 13 20 2 3 4 22
Bin 1100110011001 00000000000000000000 10 100 1111 0000000000000000000000
Hex 1999 0 2 4 f 0
Dec 6553 0 2 4 15 0
From the DDOS policer configuration, we see a protocol under the same group (ip-opt) as non-v4v6 and unclassified. The
unclassified is for the packet holding options which is not rt-alert and the non-v4v6 is for the non-v4v6 packet being sent
up with the PUNT_OPTION, which will be policed by the option punt nh policer.
Packet Exceptions
----------------------
IP options PUNT( 2) 121976 22902560
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 47
For example, this is for OSPF as we cant (doesnt need to ?) parse it with different types like Hello, LSA requestetc.
NPC2(Dokinchan-re0 vty)# show ddos policer configuration ospf
DDOS Policer Configuration:
UKERN-Config PFE-Config
idx prot group proto on Pri rate burst rate burst
--- --- ------------ ------------ -- -- ------ ----- ------ -----
70 b00 ospf aggregate Y Hi 20000 20000 20000 20000
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 48
NPC2(Dokinchan-re0 vty)#
For these types of packet, that would go thru the hbc policer.
NPC2(Dokinchan-re0 vty)# show jnh 0 exceptions nh 21 discard
Nexthop Chain:
CallNH:desc_ptr:0xc05c48, mode=0, rst_stk=0x0, count=0x3
0xc05c44 0 : 0x2ffffffe07caba00
0xc05c45 1 : 0xc03c152607cb9001
0xc05c46 2 : 0x127fffffe00003fe
0xc05c47 3 : 0x260081d80000000c
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 49
NPC2(Dokinchan-re0 vty)#
0x29f0000043c00000
Wid 13 20 2 3 4 22
Bin 0010100111110 00000000000000000000 10 000 1111 0000000000000000000000
Hex 53e 0 2 0 f 0
Dec 1342 0 2 0 15 0
Rate = 1342 * 781.25 ~= 1Mbps. This will be implemented as packet base policer as well which is = 256 pps.
Here is a table to list down the ASIC policer(s) applied to each host bound packet type.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 50
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 51
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 52
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 53
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 54
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 55
Here is a table to list down the host queue being used for packet hitting the exception ucode.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 56
Q2 (Q_L2_LO) PACKET_PUNT_L2PT_ERROR(14),
PACKET_PUNT_HOST_COPY(6),
PACKET_PUNT_AUTOSENSE(35),
PACKET_PUNT_MAC_FWD_TYPE_HOST(7),
PACKET_PUNT_PPPOE_PADI(45),
PACKET_PUNT_PPPOE_PADR(46),
PACKET_PUNT_PPPOE_PADT(47),
PACKET_PUNT_PPP_LCP(48),
PACKET_PUNT_LCP_ECHO_REQ(60),
PACKET_PUNT_LCP_ECHO_REP(63),
PACKET_PUNT_PPP_AUTH(49),
PACKET_PUNT_PPP_IPV4CP(50),
PACKET_PUNT_PPP_IPV6CP(51),
PACKET_PUNT_PPP_MPLSCP(52),
PACKET_PUNT_PPP_ISIS(57), PACKET_PUNT_MLPPP_LCP(64),
PACKET_PUNT_PPP_UNCLASSIFIED_CP(53),
PACKET_PUNT_SEND_TO_HOST_FW(54),
PACKET_PUNT_SEND_TO_HOST_FW_INLINE_SVCS(59),
PACKET_PUNT_MLP(11), PACKET_PUNT_MLFR_CONTROL(65),
PACKET_PUNT_MFR_CONTROL(66)
Q3 (Q_L2_HI) PACKET_PUNT_CONTROL(4), PACKET_PUNT_VC_HI(55),
PACKET_PUNT_KEEPALIVE(58), PACKET_PUNT_INLINE_KA(61),
PACKET_PUNT_DDOS_POLICER_VIOL(15)
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 57
PACKET_PUNT_SAMPLE_TAP(44),
PACKET_PUNT_SAMPLE_SFLOW(71),
PACKET_PUNT_FAB_OUT_PROBE_PKT(5)
For the exception traffic hitting the HBC policer, its the discard exception type with TRKL tagged.
- PACKET_ERR_ENUM_CHK_MISMATCH (mcast rpf mismatch)
- PACKET_ERR_MTU_EXCEEDED (mtu exceeded)
- PACKET_ERR_FRAG_NEED_DF_SET (frag needed but DF set)
Furthermore, DDOS will classify the packets and apply the corresponding policer before sending to the host via the MQ
host bound queue. There are 8 host bound queues (ie. MQchip Qsys 0 queue 1016-1023) and each of them will carry
different types of traffic.
// Host bound queue offsets
#define Q_HOST_L3_LO_OFF 0
#define Q_HOST_L3_HI_OFF 1
#define Q_HOST_L2_LO_OFF 2
#define Q_HOST_L2_HI_OFF 3
#define Q_HOST_OPTN_OFF 4
#define Q_HOST_IIF_MMTCH_TTL_EXPR_OFF 5
#define Q_HOST_OTHER_ERRS_OFF 6
#define Q_HOST_SAMPLE_OFF 7
The following provides a mapping between protocol packets and the host bound queue being used.
src/pfe/common/pfe-arch/trinity/tooklits/jnh_app/jnh_ddos.c - jnh_ddos_setup_asic_proto_id_maps()
Here is a table to list down the mapping between protocols and host bound queue being used after the classification and
policing. For example, once an IP option packet hits the PACKET_PUNT_OPTIONS exception, this PUNT will go thru the
HBC and be classified to either router-alert option (IP_OPT_RT_ALERT Q1) or others (IP_OPT_UNCLS Q4) protocol,
then, be assigned to the correct host bound queue.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 58
Q1 (Q_L3_HI) IGMP, OSPF, RSVP, PIM, RIP, PTP, BFD, LMP, LDP, MSDP, VRRP,
ANCP, IGMPV6, EGPV6, RSVPV6, PIMV6, IGMPV4V6, RIPV6,
BFDV6, LMPV6, LDPV6, MSDPV6, VRRPV6, ANCPV6, OSPFV3V6,
SEND_TO_HOST_SVCS, ISIS, IP_OPT_RT_ALERT
Q2 (Q_L2_LO) AUTOSENSE, PPPOE_PADI, PPP_LCP, PPP_LCP_ECHO_REQ,
PPP_LCP_ECHO_REP, PPP_UNCLASSIFIED_CP, MLPPP_LCP,
VC_LO, VC_TTL_ERROR, MAC_FWD_TYPE_HOST, MLP,
L2PT_ERROR, SEND_TO_HOST_FW_INLINE_SVCS,
MLFR_CONTROL, MFR_CONTROL, ARP, MLP
Q3 (Q_L2_HI) PPPOE_PADR, PPPOE_PADT, PPP_AUTH, PPP_IPV4CP,
PPP_IPV6CP, PPP_MPLSCP, PPP_ISIS, VC_HI, KEEPALIVE,
INLINE_KA, LACP, STP, ESMC, OAM_LFM, EOAM, LLDP, MVRP,
PMVRP, PVSTP, POS, JFM, ATM, PFE_ALIVE
Q4 (Q_OPTN) IGMP_SNOOP, PIM_SNOOP, IP_OPT_UNCLS,
IP_OPT_NON_V4V6
Q5 TTL
(Q_IIF_MMTCH_TTL_EXPR)
Q6 (Q_OTHER_ERRS) REJECT, REJECT_V6,
Q7 (Q_SAMPLE) SAMPLE_SYSLOG, SAMPLE_HOST, SAMPLE_PFE,
SAMPLE_TAP, SAMPLE_SFLOW
uKern Level
After each PFE policed the host bound traffic, they will hit the uKern on the FPC and the aggregated traffic might be
policed again according to the DDOS policer configuration. The policer implementation on the uKern is a simple token
bucket algorithm policer rate is per-packet policer and the burst is the maximum number of accumulated credits.
Take IP option packets as an example. After each PFE applies a policer to police the corresponding optioned packet,
when the traffic from all PFEs hit the uKern, the corresponding protocol policer will police all the traffic again. As a result,
the packet will have to go through another round of policing.
- ip-option unclassified packets from all PFEs within the MPC will hit a policer (10000 pps : uKern-config)
- ip-option rt-alert packets from all PFEs within the MPC will hit a policer (20000 pps : uKern-config)
- The sum of both ip-option packet types will go thru an aggregate policer on uKern to make sure the sum of them
wont exceed 20000 pps (Ukern-config)
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 59
NPC2(Dokinchan-re0 vty)#
Here, the priority plays an important role. The priority here becomes a strict priority (until that traffic exceeds its own
policer for sure). Here, we have both rt-alert packets and unclassified ip-option packet. Both hit the same PFE and FPC.
When the rt-alert is hitting the maximum rate, which is the aggregate policer rate on the uKern, none of the rt-alert packet
will be dropped.
NPC2(Dokinchan-re0 vty)# show ddos policer configuration ip-options
DDOS Policer Configuration:
UKERN-Config PFE-Config
idx prot group proto on Pri rate burst rate burst
--- --- ------------ ------------ -- -- ------ ----- ------ -----
123 3d00 ip-opt aggregate Y Hi 20000 20000 --- ---
124 3d01 ip-opt unclass.. Y Lo 10000 10000 10000 10000
125 3d02 ip-opt rt-alert Y Hi 20000 20000 20000 20000
126 3d03 ip-opt non-v4v6 Y Lo 10000 10000 10000 10000
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 60
NPC2(Dokinchan-re0 vty)#
If we reduce the rt-alert rate a bit, then, can we see the higher rate for the ip-option unclassified packet.
NPC2(Dokinchan-re0 vty)# show ddos policer stats ip-options
DDOS Policer Statistics:
arrival pass # of
idx prot group proto on loc pass drop rate rate flows
--- --- ----------- ----------- -- ------ -------- -------- ------ ------ -----
123 3d00 ip-opt aggregate Y UKERN 4561353 0 18222 18222 0
124 3d01 ip-opt unclass.. Y UKERN 269188 0 1065 1065 0
PFE-0 2774347 21530507 138973 10003 0
125 3d02 ip-opt rt-alert Y UKERN 4292165 0 17156 17156 0
PFE-0 4640605 6936119 17166 17166 0
126 3d03 ip-opt non-v4v6 Y UKERN 0 0 0 0 0
PFE-0 0 0 0 0 0
NPC2(Dokinchan-re0 vty)#
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 61
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 62
lab@Dokinchan-re0>
The alarm will go off if the violation is cleared and last for recover time configured.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 63
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 64
Taking the ip-option as an example again, the Routing Engine will just police the sum of all ip-option packets using the
aggregate policer rate (20000 pps). Also, the priority of each individual protocol packet still play a role here.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 65
The detail implementation of the policer in Routing Engine could be found under src/junos/bsd/sys/netpfe/ddos_policers.c.
By default, the SCFD is disabled. This can be enabled by the folllwing configuration.
# set system ddos-protection global ?
flow-detection Enable flow detection for all protocols
Once its enabled, the DDOS system will monitor the host bound traffic from 3 levels of flow granularity in LUchip once an
violation happens.
- Subscriber level (SUB)
- IFL level (DDOS protocol ID, IIF, Aggregation-level as key)
- IFD level (DDOS protocol ID, IFD, Aggregation-level as key)
When the DDOS violation happens, the SCFD check all the packets within that protocol. The idea is to use a hash
function to filter out the suspicious flow. Then, the flow is inserted into a LU hardware hash table.
If the flow has rate consistently above its allowed bandwidth for a detect-time period (flow-detect-time 3 secs by
default), we declare the suspicious flow to be a culprit flow. The traffic form it will be dropped consequently unless we
disable the drop. If a flow does not exceed its allowed bandwidth for the detect-time period, we assume that its a false
positive and remove that from the hardware hash table.
Once a suspicious flow rate is below its bandwidth for the recover time period (recover-time 60 secs by default), the
SCFD declares the flow to be normal, removes it from hardware flow table and let traffic resume.
[edit]
lab@Dokinchan-re0# set system ddos-protection protocols ospf aggregate ?
Possible completions:
flow-detect-time Time to determine a flow is bad (1..60 seconds)
flow-detection-mode Flow detection mode for the packet type
> flow-level-bandwidth Bandwidth for flows at various levels
> flow-level-control Specify how discovered flows are controlled
> flow-level-detection Specify detection mode at various levels
flow-recover-time Time to return to normal after last violation (1..3600 seconds)
flow-timeout-time Time to timeout the flow since found (1..7200 seconds)
no-flow-logging Disable logging of violating flows
recover-time Time for protocol to return to normal (1..3600 seconds)
timeout-active-flows Allow timeout active violating flows
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 66
This is to configure the action once a suspicious flow is detected on different level.
[edit]
lab@Dokinchan-re0# set system ddos-protection protocols ospf aggregate flow-level-control ?
Possible completions:
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
logical-interface Specify how logical-interface flows are controlled
physical-interface Specify how physical-interface flows are controlled
subscriber Specify how subscriber flows are controlled
[edit]
lab@Dokinchan-re0# set system ddos-protection protocols ospf aggregate flow-level-control logical-interface ?
Possible completions:
drop Drop all traffic of flows of this level
keep Keep all traffic of flows of this level
police Police flows to within the bandwidth of this level
[edit]
lab@Dokinchan-re0# set system ddos-protection protocols ospf aggregate flow-level-control physical-interface ?
Possible completions:
drop Drop all traffic of flows of this level
keep Keep all traffic of flows of this level
police Police flows to within the bandwidth of this level
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 67
[edit]
lab@Dokinchan-re0# set system ddos-protection protocols ospf aggregate flow-level-detection logical-interface ?
Possible completions:
automatic Detect flows at logical-interface level if needed
off Do not detect flows at logical-interface level
on Always detect flows at logical-interface level
[edit]
lab@Dokinchan-re0# set system ddos-protection protocols ospf aggregate flow-level-detection physical-interface ?
Possible completions:
automatic Detect flows at physical-interface level if needed
off Do not detect flows at physical-interface level
on Always detect flows at physical-interface level
[edit]
lab@Dokinchan-re0#
Here is the default SCFD configuration or each protocol. When the SCFD is enabled, by default, the flow detection mode
is auto (op-mode:a) and once the suspicious flow is detected, the action is to drop the packets (fc-mode:d). The detection
rate on all 3 levels is protocol dependent. For example, in OSPF, the sub level is 10pps (which is not being used I
believe), the ifl level is 10pps and the IFD level is 20000pps. When the mode is set to on, the new flow will be added to
the table automatically.
By default, the active-flow-timeout is disabled. If active-flow-timeout is enabled, the flow will be removed from the list when
its there for active-flow-timeout time (300 secs by default). Once its removed, the flow will generate a violation event
again and it will be added back to the list.
NPC2(Dokinchan-re0 vty)# show ddos scfd proto-states all
(sub|ifl|ifd)-cfg: op-mode:fc-mode:bwidth(pps)
op-mode: a=automatic, o=always-on, x=disabled
fc-mode: d=drop-all, k=keep-all, p=police
d-t: detect time, r-t: recover time, t-t: timeout time
aggr-t: last aggregated/deaggreagated time
idx prot group proto mode detect agg flags state sub-cfg ifl-cfg ifd-cfg d-t r-t t-t aggr-t
--- ---- -------- -------- ---- ------ --- ----- ----- --------- --------- --------- --- --- --- ------
0 0 host-path aggregate auto no 1 2 0 a:d: 10 a:d: 10 a:d:25000 3 60 300 0
1 100 ipv4-uncls aggregate auto no 1 2 0 a:d: 10 a:d: 10 a:d: 2000 3 60 300 0
2 200 ipv6-uncls aggregate auto no 1 2 0 a:d: 10 a:d: 10 a:d: 2000 3 60 300 0
3 300 dynvlan aggregate auto no 1 2 0 a:d: 10 a:d: 10 a:d: 1000 3 60 300 0
4 400 ppp aggregate auto no 1 2 0 a:d: 10 a:d: 10 a:d:16000 3 60 300 0
5 401 ppp unclass auto no 1 2 0 a:d: 10 a:d: 10 a:d: 1000 3 60 300 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 68
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 69
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 70
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 71
NPC2(Dokinchan-re0 vty)#
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 72
State
#define DDOS_SCFD_STATE_CLEARING 0x00000001 /* is clearing */
#define DDOS_SCFD_STATE_RATE_MOD 0x00000002 /* on rate mod list */
#define DDOS_SCFD_STATE_AGGRED 0x00000010 /* prev op is aggr */
#define DDOS_SCFD_STATE_DEAGGRED 0x00000020 /* prev op is de-aggr */
#define DDOS_SCFD_STATE_AGGR_MASK 0x00000030 /* prev aggr op mask */
Agg
#define DDOS_SCFD_AGGR_ON_MAP(p) \
((((p)->aggr_levels[DDOS_SCFD_AGGR_LEVEL_SUB].flags & \
SCFD_AGGR_FLAG_ACTIVE) ? (1 << DDOS_SCFD_AGGR_LEVEL_SUB) : 0) | \
(((p)->aggr_levels[DDOS_SCFD_AGGR_LEVEL_IFL].flags & \
SCFD_AGGR_FLAG_ACTIVE) ? (1 << DDOS_SCFD_AGGR_LEVEL_IFL) : 0) | \
(((p)->aggr_levels[DDOS_SCFD_AGGR_LEVEL_IFD].flags & \
SCFD_AGGR_FLAG_ACTIVE) ? (1 << DDOS_SCFD_AGGR_LEVEL_IFD) : 0))
Flags
#define SCFD_PROTO_FLAG_LOCAL_MASK 0x0000FFFF
#define SCFD_PROTO_FLAG_RUN_UKERN 0x00000001
#define SCFD_PROTO_FLAG_RUN_ASIC 0x00000002
#define SCFD_PROTO_FLAG_NO_LOG 0x00010000
#define SCFD_PROTO_FLAG_TO_ACTV 0x00020000 /* Allow timeout of flow */
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 73
Once a suspicious flow is detected, it will be deaggreagted from the subscriber/IFL levels, depending on the rate. With the
flow installed, none of these packets would hit the host as the action is drop by default.
Nov 20 13:57:52.659 Dokinchan-re0 jddosd[1723]: %DAEMON-4-DDOS_SCFD_FLOW_FOUND: A new flow of protocol IP-
Options:unclassified on ge-2/0/0.0 with source addr -- -- -- is found at 2013-11-20 13:57:52 JST
Nov 20 13:57:52.659 Dokinchan-re0 jddosd[1723]: %DAEMON-4-DDOS_SCFD_FLOW_FOUND: A new flow of protocol IP-
Options:router-alert on ge-2/0/0.0 with source addr -- -- -- is found at 2013-11-20 13:57:52 JST
Nov 20 13:57:52.659 Dokinchan-re0 jddosd[1723]: %DAEMON-4-DDOS_SCFD_FLOW_FOUND: A new flow of protocol IP-
Options:router-alert on ge-2/0/0.0 with source addr 192.1.1.2 is found at 2013-11-20 13:57:52 JST
Nov 20 13:57:52.659 Dokinchan-re0 jddosd[1723]: %DAEMON-4-DDOS_SCFD_FLOW_FOUND: A new flow of protocol IP-
Options:unclassified on ge-2/0/0.0 with source addr 192.1.1.2 is found at 2013-11-20 13:57:52 JST
Nov 20 13:57:52.659 Dokinchan-re0 jddosd[1723]: %DAEMON-4-DDOS_SCFD_FLOW_FOUND: A new flow of protocol IP-
Options:unclassified on ge-2/0/0 with source addr -- -- -- is found at 2013-11-20 13:57:52 JST
Nov 20 13:57:54.597 Dokinchan-re0 jddosd[1723]: %DAEMON-4-DDOS_SCFD_FLOW_DEAGGREGATED: Flows of protocol IP-
Options:router-alert on slot fpc 2 are deaggregated to subscriber, logical-interface level(s)
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 74
lab@Dokinchan-re0>
arrival pass # of
idx prot group proto on loc pass drop rate rate flows
--- --- ----------- ----------- -- ------ -------- -------- ------ ------ -----
123 3d00 ip-opt aggregate Y UKERN 76227465 0 0 0 0
124 3d01 ip-opt unclass.. Y UKERN 1880974 0 0 0 1
PFE-0 42059480 591983015 138910 0 1
125 3d02 ip-opt rt-alert Y UKERN 74346491 0 0 0 1
PFE-0 79020937 29282748 24513 0 1
126 3d03 ip-opt non-v4v6 Y UKERN 0 0 0 0 0
PFE-0 0 0 0 0 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 75
PFE: 0
Flow Record Index: 5
Flow Key:
Proto-ID: 3d02
Key type: 1
IIF: 339
Src IP addr: c0010102 (192.1.1.2)
Dst IP addr: c0010101 (192.1.1.1)
Src port: 0
Dst port: 0
Flow Context Data:
Rcvd ack_add: 1
Rcvd ack_del: 0
Rcvd last flow op: 2
Flow state: 2
Aggr level: 0
Proto idx: 125
Policer idx: 3
Time inserted: 1944001488
Time last violated: 1944507734
Last received: 12408018
Flow Statitics:
Packet Count: 12410556
Byte Count: 968023290
PFE: 0
Flow Record Index: 3
Flow Key:
Proto-ID: 3d01
Key type: 1
IIF: 339
Src IP addr: c0010102 (192.1.1.2)
Dst IP addr: c0010101 (192.1.1.1)
Src port: 0
Dst port: 0
Flow Context Data:
Rcvd ack_add: 1
Rcvd ack_del: 0
Rcvd last flow op: 2
Flow state: 2
Aggr level: 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 76
NPC2(Dokinchan-re0 vty)#
If active-flow-timeout is configured, the active monitoring flow will be removed form the list. If the rate of that flow still
exceeds the protocol DDOS rate, it will genereate another violation event and it will be re-added to the list.
[edit]
lab@Dokinchan-re0# show system ddos-protection
global {
flow-detection;
}
protocols {
ip-options {
aggregate {
timeout-active-flows;
}
unclassified {
timeout-active-flows;
}
router-alert {
timeout-active-flows;
}
}
}
NPC2(Dokinchan-re0 vty)#
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 77
PFE: 0
Flow Record Index: 12
Flow Key:
Proto-ID: 3d02
Key type: 1
IIF: 339
Src IP addr: c0010102 (192.1.1.2)
Dst IP addr: c0010101 (192.1.1.1)
Src port: 0
Dst port: 0
Flow Context Data:
Rcvd ack_add: 1
Rcvd ack_del: 0
Rcvd last flow op: 2
Flow state: 2
Aggr level: 0
Proto idx: 125
Policer idx: 3
Time inserted: 1946184735
Time last violated: 1946475734
Last received: 7132354
Flow Statitics:
Packet Count: 7152878
Byte Count: 557924406
PFE: 0
Flow Record Index: 11
Flow Key:
Proto-ID: 3d01
Key type: 1
IIF: 339
Src IP addr: c0010102 (192.1.1.2)
Dst IP addr: c0010101 (192.1.1.1)
Src port: 0
Dst port: 0
Flow Context Data:
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 78
Rcvd ack_add: 1
Rcvd ack_del: 0
Rcvd last flow op: 2
Flow state: 2
Aggr level: 0
Proto idx: 124
Policer idx: 4
Time inserted: 1946184734
Time last violated: 1946476734
Last received: 40555616
Flow Statitics:
Packet Count: 40662069
Byte Count: 2683696488
NPC2(Dokinchan-re0 vty)#
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 79
[edit]
lab@Dokinchan-re0# set system ddos-protection protocols ospf aggregate fpc 0 ?
Possible completions:
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
bandwidth-scale Bandwidth scale from 1% to 100% (1..100 percent)
burst-scale Burst scale from 1% to 100% (1..100 percent)
disable-fpc Turn off policing on this slot
[edit]
lab@Dokinchan-re0#
The
bandwidth-scale/burst-scale
configuration
under
the
FPC
is
used
to
configure
how
much
bandwidth
(bandwidth
*
bandwith-scale%
/
burst
*
burst-scale%)
should
be
applied
on
that
FPC.
For
example,
with
50%
for
both
bandwidth
and
burst
scale,
the
OSPF
protocol
policer
becomes:
[edit]
lab@Dokinchan-re0# run show ddos-protection protocols ospf parameters
Packet types: 1, Modified: 0
* = User configured value
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 80
[edit]
lab@Dokinchan-re0# set system ddos-protection protocols ospf aggregate fpc 2 bandwidth-scale 50
[edit]
lab@Dokinchan-re0# set system ddos-protection protocols ospf aggregate fpc 2 burst-scale 50
[edit]
lab@Dokinchan-re0# commit
commit complete
[edit]
lab@Dokinchan-re0# run show ddos-protection protocols ospf parameters
Packet types: 1, Modified: 1
* = User configured value
[edit]
lab@Dokinchan-re0#
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 81
Statistics/Errors
We can capture the per-protocol statistics before and after the policers being applied on the packets.
lab@Dokinchan-re0> show ddos-protection protocols ip-options unclassified
Currently tracked flows: 1, Total detected flows: 1
* = User configured value
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 82
arrival pass # of
idx prot group proto on loc pass drop rate rate flows
--- --- ----------- ----------- -- ------ -------- -------- ------ ------ -----
123 3d00 ip-opt aggregate Y UKERN 17574 18504 0 0 0
^^^^^ Agg Policer + Protocol Policer in uKern
124 3d01 ip-opt unclass.. Y UKERN 36078 18504 0 0 1
^^^^^ Protocol Policer in uKern
PFE-0 36078 98645 7841 0 1
^^^^^ Protocol Policer + SCFD drops
125 3d02 ip-opt rt-alert Y UKERN 0 0 0 0 0
PFE-0 0 0 0 0 0
126 3d03 ip-opt non-v4v6 Y UKERN 0 0 0 0 0
PFE-0 0 0 0 0 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 83
IP Option Statistics:
0 loose source routes
0 strict source routes
0 record routes
0 router alerts
16963 other options
IP Option Errors:
0 runts
0 bad versions
0 runt header lengths
0 giant header lengths
0 null frames
0 bad option lengths
0 duplicate options
0 bad option pointers
0 source route frames dropped
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 84
NPC2(Dokinchan-re0 vty)#
If we check the aggregate policer drop, the system wide statistics will count the uKern aggregate policer drop. Here, we
inject 30K pkts for each ip-frag type. The following might be confusing as the pass count is including the drop pkts.
PR942813 has filed to enhance this command output.
arrival pass # of
idx prot group proto on loc pass drop rate rate flows
--- --- ----------- ----------- -- ------ -------- -------- ------ ------ -----
158 4f00 ip-frag aggregate Y UKERN 12751 47249 0 0 0
^^^^^ Sum of the drop below
159 4f01 ip-frag unclass.. Y N/A --- --- --- --- ---
160 4f02 ip-frag first-frag Y UKERN 30000 23484 0 0 0
^^^^^ Drop by uKern Agg policer. This is PR942813
PFE-0 30000 0 0 0 0
161 4f03 ip-frag trail-frag Y UKERN 30000 23765 0 0 0
^^^^^ Drop by uKern Agg policer. This is PR942813
PFE-0 30000 0 0 0 0
NPC2(Dokinchan-re0 vty)#
Total drop on MPC is 23484 + 23765 = 47249. With 7 pkts drop on RE, the total drop becomes 47256.
lab@Dokinchan-re0> show ddos-protection protocols ip-fragments aggregate
Currently tracked flows: 0, Total detected flows: 0
* = User configured value
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 85
Global Policer:
policer_nexthop: 0xC03C152607CB9001
policer_result: 0x4C3F2360
dropped packets: 0
Hostbound policer packet drops: 0 Sum of HBC policer drop for exception nhs.
Hostbound policer byte drops: 0
Aggregate policer packet drops: 40160393 Sum of all DDOS IPv4 policer drops.
Aggregate policer byte drops: 4871701502
Aggregate IPv6 policer packet drops: 76521499 Sum of all DDOS IPv6 policer drops.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 86
NPC2(Dokinchan-re0 vty)#
Here are some DDOS error counters to record errors when it parses the received protocol frames.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 87
35 PUNT_AUTOSENSE mismatch-id 0
36 PUNT_REJECT_FW non-exist-id 0
37 --- unsupported 0 PUNT_UNUSED
38 PUNT_SERVICES mismatch-id 0
39 PUNT_DEMUXAUTOSENSE mismatch-id 0
40 PUNT_REJECT mismatch-id 0
41 PUNT_SAMPLE_SYSLOG mismatch-id 0
42 PUNT_SAMPLE_HOST mismatch-id 0
43 PUNT_SAMPLE_PFE mismatch-id 0
44 PUNT_SAMPLE_TAP mismatch-id 0
45 PUNT_PPPOE_PADI mismatch-id 0
46 PUNT_PPPOE_PADR mismatch-id 0
47 PUNT_PPPOE_PADT mismatch-id 0
48 PUNT_PPP_LCP mismatch-id 0
49 PUNT_PPP_AUTH mismatch-id 0
50 PUNT_PPP_IPV4CP mismatch-id 0
51 PUNT_PPP_IPV6CP mismatch-id 0
52 PUNT_PPP_MPLSCP mismatch-id 0
53 PUNT_PPP_UNCLASSIFIED_CP mismatch-id 0
54 PUNT_SEND_TO_HOST_FW non-exist-id 0
55 PUNT_VC_HI mismatch-id 0
56 PUNT_VC_LO mismatch-id 0
57 PUNT_PPP_ISIS mismatch-id 0
58 PUNT_KEEPALIVE mismatch-id 0
59 PUNT_SEND_TO_HOST_FW_INLINE_SVCS mismatch-id 0
60 PUNT_PPP_LCP_ECHO_REQ mismatch-id 0
61 PUNT_INLINE_KA mismatch-id 0
62 --- unsupported 0 PUNT_UNUSED
63 PUNT_PPP_LCP_ECHO_REP mismatch-id 0
64 PUNT_MLPPP_LCP mismatch-id 0
65 PUNT_MLFR_CONTROL mismatch-id 0
66 PUNT_MFR_CONTROL mismatch-id 0
67 --- unsupported 0 PUNT_UNUSED
68 PUNT_REJECT_V6 mismatch-id 0
69 PUNT_RESOLVE_V6 non-exist-id 0
70 PUNT_SEND_TO_HOST_SVCS mismatch-id 0
71 PUNT_SAMPLE_SFLOW mismatch-id 0
Here are the IPC msg stats between the DDOS module on the MPC and the Routing Engine (jddosd).
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 88
violation set 0 0 0 7
violation clr 0 0 0 5
protocol_stats_get 24 0 0 0
protocol_stats_clr 0 0 0 0
protocol_stats_rts 0 0 0 0
policer 5 0 0 0
policer_rts 0 0 0 0
pstates 0 0 0 0
pstates_rts 0 0 0 0
pfe_peer_info 0 0 0 0
flow_get 0 0 0 0
flow_clr 0 0 0 0
scfd_proto_get 0 0 0 0
Here is the global configuration and statistics summary for the SCFD module.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 89
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 90
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 91
NPC2(Dokinchan-re0 vty)#
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 92
When we check the DDOS statistics, there is a gap in-between the ASIC and uKern. For example, in the followings, we
could see uKern having arrival rate far less than the once measured on PFE(ASIC). However, between the ASIC and
uKern, the drop could happen in TOE/MQ if the host bound traffic rate is too high. In this case, the drop happens on the
MQ hostbound queue and thats why uKern sees far less traffic volume than the PFE.
NPC2(Dokinchan-re0 vty)# show ddos policer stats ip-fragments
DDOS Policer Statistics:
arrival pass # of
idx prot group proto on loc pass drop rate rate flows
--- --- ----------- ----------- -- ------ -------- -------- ------ ------ -----
158 4f00 ip-frag aggregate Y UKERN 3828251 0 16623 16623 0
159 4f01 ip-frag unclass.. Y N/A --- --- --- --- ---
160 4f02 ip-frag first-frag Y UKERN 1913970 0 8310 8310 0
PFE-0 4594919 0 19997 19997 0
161 4f03 ip-frag trail-frag Y UKERN 1914281 0 8313 8313 0
PFE-0 4594921 0 19998 19998 0
NPC2(Dokinchan-re0 vty)#
The same happens on the path between PPC and RE (ie. TTP drop..etc).
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 93
Indeed, there are cases that the DDOS might not help.
https://gnats.juniper.net/web/default/878789
This PR is related to SCFD flow detection against arp storm. When an arp packet comes in, it will be handled by the
default arp policer (__default_arp_policer__) before it hits the HBC. Since the default arp policer is stateless, it will just
drop arp packets based on the policer rate without considering that the passed arp packets are actually the same. As a
result, the non-attack arp packets might be dropped by the default arp policer and the attack arp storm will be dropped by
the SCFD once it detects the flow.
In order to workaround this, we need to disable the default arp policer by configuring a high arp policer rate, which is the
same as passing all the arp packets to the SCFD. In SCFD, it will identify the attack flow(s) and drop it from there.
https://gnats.juniper.net/web/default/934869
As mentioned above, DDOS requires a steady traffic volume to detect the suspicious flow. In this PR, its related to a
bursty traffic source typical case for start up multicast flow.
In this PR, when we receive multicast packets hitting the resolve nh, the resolve request will come up to the RPD in the
Routing Engine and create a multicast route on the PFE. Starting from that point, the flow hitting the resolve nh will no
longer be there and thats why the DDOS couldnt detect that.
Even if we turn on SCFD, since it requires some time (in the order of secs) to detect the flow, it wont be quick enough to
stop the resolve requests from the same multicast group from entering the resolve queue on the host (resolve_nh -> host
queue -> PPC -> resolve queue -> RPD[RE]) and let other multicast group to enter the resolve queue. Hence, enabling
DDOS wont help much to speed up the multicast route setup time in this case.
https://gnats.juniper.net/web/default/871500
The problem is that the MLP packets are processed differently. The packets do not go through the regular exception
processing path. The MLP packets sent in general at 200 pps directly to host by the learning process. It actually bypasses
most of the DDOS processing. This is why you cannot control it. The MLP is self paced. This means that MLP pose NO
DDOS threats.
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 94
https://gnats.juniper.net/web/default/832740
This is mainly a code enhancement for DDOS and add the supportability on XM chip. Would suggest a customer to pick
up this fix for DDOS usage.
https://gnats.juniper.net/web/default/924807
This is a major design flaw in DDOS where the packet hitting the resolve/fw reject nexthop will be classified as protocol
control packet as long as the protocol field matches the specific DDOS term. With this fix, we separate the notification
hitting the resolve and reject nexthops to a different DDOS term.
NPC1(currypanman-re0 vty)# show ddos asic punt-proto-maps
PUNT exceptions directly mapped to DDOS proto:
code PUNT name group proto idx q# bwidth burst
---- -------------------- --------- ------ ---- -- ------ ------
1 PUNT_TTL ttl aggregate 3c00 5 2000 10000
3 PUNT_REDIRECT redirect aggregate 3e00 0 2000 10000
5 PUNT_FAB_OUT_PROBE_PKT fab-probe aggregate 5700 0 20000 20000
7 PUNT_MAC_FWD_TYPE_HOST mac-host aggregate 4100 2 20000 20000
8 PUNT_TUNNEL_FRAGMENT tun-frag aggregate 4200 0 2000 10000
11 PUNT_MLP mlp packets 3802 2 2000 10000
12 PUNT_IGMP_SNOOP igmp-snoop aggregate 4300 4 20000 20000
13 PUNT_VC_TTL_ERROR vchassis vc-ttl-err 805 2 4000 10000
14 PUNT_L2PT_ERROR l2pt aggregate 5a00 2 20000 20000
35 PUNT_AUTOSENSE dynvlan aggregate 300 2 1000 500
38 PUNT_SERVICES services aggregate 4400 0 2000 10000
39 PUNT_DEMUXAUTOSENSE demuxauto aggregate 4500 0 2000 10000
40 PUNT_REJECT reject aggregate 4600 6 2000 10000
41 PUNT_SAMPLE_SYSLOG sample syslog 5602 7 1000 1000
42 PUNT_SAMPLE_HOST sample host 5603 7 1000 1000
43 PUNT_SAMPLE_PFE sample pfe 5604 7 1000 1000
44 PUNT_SAMPLE_TAP sample tap 5605 7 1000 1000
45 PUNT_PPPOE_PADI pppoe padi 502 2 500 500
46 PUNT_PPPOE_PADR pppoe padr 504 3 500 500
47 PUNT_PPPOE_PADT pppoe padt 506 3 1000 1000
48 PUNT_PPP_LCP ppp lcp 402 2 12000 12000
49 PUNT_PPP_AUTH ppp auth 403 3 2000 2000
50 PUNT_PPP_IPV4CP ppp ipcp 404 3 2000 2000
51 PUNT_PPP_IPV6CP ppp ipv6cp 405 3 2000 2000
52 PUNT_PPP_MPLSCP ppp mplscp 406 3 2000 2000
53 PUNT_PPP_UNCLASSIFIED_CP ppp unclass 401 2 1000 500
55 PUNT_VC_HI vchassis control-hi 802 3 10000 5000
56 PUNT_VC_LO vchassis control-lo 803 2 8000 3000
57 PUNT_PPP_ISIS ppp isis 407 3 2000 2000
58 PUNT_KEEPALIVE keepalive aggregate 5b00 3 20000 20000
59 PUNT_SEND_TO_HOST_FW_INLINE_SVCS inline-svcs aggregate 5d00 2 20000 20000
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 95
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 96
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 97
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 98
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 99
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 100
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 101
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 102
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 103
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 104
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 105
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 106
NPC1(currypanman-re0 vty)#
https://gnats.juniper.net/web/default/942816
This is the DDOS statistics output after PR942816 fix.
<-- No SCFD
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 107
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 108
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 109
arrival pass # of
idx prot group proto on loc pass drop rate rate flows
--- --- ----------- ----------- -- ------ -------- -------- ------ ------ -----
159 4f00 ip-frag aggregate Y UKERN 953127 0 0 0 0
160 4f01 ip-frag unclass.. Y N/A --- --- --- --- ---
161 4f02 ip-frag first-frag Y UKERN 0 0 0 0 0
PFE-0 0 0 0 0 0
PFE-1 0 0 0 0 0
<-- 964156 is missing.
162 4f03 ip-frag trail-frag Y UKERN 953127 0 0 0 0
PFE-0 1917283 9759087 0 0 0
PFE-1 0 0 0 0 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 110
PFE Info:
configured: rate=20000 (pps) burst=20000 (pkts)
SCFD Info:
op-mode=automatic, state=normal, flags=0x1(never timeout, log)
detect-time=3000(ms), recover-time=60000(ms), timeout-time=300000(ms)
aggr-level allowed active force ctrl rate(pps) flow-count
sub yes yes no drop 10 0
ifl yes yes no drop 10 0
ifd yes yes no drop 20000 0
total 3 3 0 --- --- 0
flow drop rate=0, flow drop trend=ff, pol viol trend=0
Packet Statistics:
stats PFE-0 PFE-1 UKERN TOTAL
----------------- --------- --------- --------- ---------
received 11676370 0 953127 11676370
arrived at policer 11676370 0 953127 ---
dropped: indv pol 9759087 0 0 9759087
dropped: aggr pol --- --- 0 0
dropped: indv flow 0 0 --- 0
dropped: aggr flow --- --- --- ---
total dropped 9759087 0 0 9759087
final passed 1917283 0 953127 953127
arrival rate(pps) 0 0 0 0
max arvl rate(pps) 122585 0 9998 122585
pass rate(pps) 0 0 0 0
Global Policer:
policer_nexthop: 0xC03C15360754B001
policer_result: 0x4CC84D48
dropped packets: 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 111
NPC1(currypanman-re0 vty)#
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 112
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 113
lab@currypanman-re0>
arrival pass # of
idx prot group proto on loc pass drop rate rate flows
--- --- ----------- ----------- -- ------ -------- -------- ------ ------ -----
159 4f00 ip-frag aggregate Y UKERN 17786 0 0 0 0
160 4f01 ip-frag unclass.. Y N/A --- --- --- --- ---
161 4f02 ip-frag first-frag Y UKERN 0 0 0 0 0
PFE-0 0 0 0 0 0
PFE-1 0 0 0 0 0
162 4f03 ip-frag trail-frag Y UKERN 17786 0 0 0 1
<-- 5066511 = SCFD + DDOS policer drop
PFE-0 46606 5066511 0 0 1
PFE-1 0 0 0 0 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 114
SCFD Info:
op-mode=automatic, state=detect, flags=0x1(never timeout, log)
detect-time=3000(ms), recover-time=60000(ms), timeout-time=300000(ms)
aggr-level allowed active force ctrl rate(pps) flow-count
sub yes yes no drop 10 1
ifl yes no no drop 10 0
ifd yes no no drop 20000 0
total 3 1 0 --- --- 1
flow drop rate=0, flow drop trend=ff, pol viol trend=0
Packet Statistics:
stats PFE-0 PFE-1 UKERN TOTAL
----------------- --------- --------- --------- ---------
<-- pkt can reach ukern = 46606 - 28820 = 17786
received 5113117 0 17786 5113117
<-- After SCFD
arrived at policer 211479 0 17786 ---
<-- 211479 - 164873 = 46606 = final pass up to ukern
dropped: indv pol 164873 0 0 164873
dropped: aggr pol --- --- 0 0
<-- 5113117 - 4901638 = 211479 = pkt sent to DDOS policer term
dropped: indv flow 4901638 0 --- 4901638
dropped: aggr flow --- --- --- ---
total dropped 5066511 0 0 5066511
final passed 46606 0 17786 17786
arrival rate(pps) 0 0 0 0
max arvl rate(pps) 122657 0 2159 122657
pass rate(pps) 0 0 0 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 115
Global Policer:
policer_nexthop: 0xC03C15360754B001
policer_result: 0x4CC84D48
dropped packets: 0
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 116
NPC1(currypanman-re0 vty)#
The aggregate policer packet drop counter is always 1 less than the actual drop from the above test. Thats because it is
counted as a violation. When a policer is in normal mode (not starting detecting flow yet), and when violation is detected
(we are going to drop), the drop will be converted to a violation report and sent to ukern. This drop is counted at the
violating policer but not at the global counter. These violations are never dropped and not processed as the original
exception, and are only used as an indication of a policer violation. This was introduced in 12.3 with SCFD. Also we could
keep sending these violation reports until host acks the receiving or switching to flow detection. Apparently, in your test
case, the first violation got acked right away and you only lost one packet. The ack feature is just introduced in this PR
fix. We used to keep sending violation reports if we are not doing SCFD.
==================================================================
Packet Exceptions
----------------------
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 117
Reference
1. http://www-in.juniper.net/eng/cvs_pdf/sw-projects/platform/trinity/pfe/host/host.doc
2. http://cvs.juniper.net/cgi-bin/viewcvs.cgi/*checkout*/sw-projects/platform/commonedge/arch/RLI15473-
DDOS-SCFD-FS.pdf?rev=1.4
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 118
Changes
18-Nov-2013
(Rev
0)
Initial
Draft
JUNIPER NETWORKS CONFIDENTIAL DO NOT DISTRIBUTE Juniper Networks, Inc. 119