Escolar Documentos
Profissional Documentos
Cultura Documentos
Troubleshooting
LAB GUIDE
Table of contents
Lab Exercises:
1. Alcatel Support
2. Debug Logs and Log filters
3. Sending debug output to a Syslog Server
4. Using OAM tools and SAA
5. Troubleshooting Hardware Faults
6. Troubleshooting OSPF
7. Troubleshooting ISIS
8. Troubleshooting LDP
9. Troubleshooting RSVP
2/126
Contents
Lab 1. Alcatel Support ................................................................................................................................... 7
Exercise 1.1. Log into the external Alcatel-Lucent page and demonstrate how to get to customer
documents and release notes on OLCS. Log into the SRKB and demonstrate how to search for tips best
practices and submit a question as well as search the KB database. ....................................................... 7
Lab 2. Debug Logs and Log filters................................................................................................................ 10
Exercise 2.1. Create a file-id, log filter, and log to store the security debug output. ............................. 10
Lab 3. Sending debug output to a Syslog Server......................................................................................... 18
Exercise 3.1. Configuring a loopback interface ....................................................................................... 18
Exercise 3.2. Configure Syslog on all core routers .................................................................................. 19
Exercise 3.3. Configure a Log to direct change events to syslog ............................................................ 19
Lab 4. Using OAM tools and SAA ................................................................................................................ 22
Exercise 4.1. Create an SAA test, set the loss-event threshold to 1, and enable SNMP traps. .............. 22
Lab 5. Troubleshooting Hardware faults .................................................................................................... 33
Lab 6. Troubleshooting OSPF ...................................................................................................................... 49
Exercise 6.1 Router adjacency study....................................................................................................... 49
Exercise 6.1.1 Create a log to capture your debug output. .................................................................... 49
Exercise 6.1.2 On your Core router enable debug-trace for OSPF packets; only debug the interface to
your Edge router. .................................................................................................................................... 49
Exercise 6.1.3 Shut down OSPF on the edge router; verify neighbor state is down on your core router.
................................................................................................................................................................ 49
Exercise 6.1.4 Enable ospf on the edge router verify neighbor state is full. .......................................... 50
Exercise 6.1.5 Stop the debug and view the output of the debug log.................................................... 50
Exercise 6.2 Router Adjacency fault 1..................................................................................................... 56
Exercise 6.2.1 using the commands from the previous lab isolate the fault causing the adjacency to
fail. .......................................................................................................................................................... 56
Exercise 6.3 Router Adjacency fault 2..................................................................................................... 59
Exercise 6.3.1 using the commands from the previous lab isolate the fault causing the adjacency to
fail. .......................................................................................................................................................... 59
Exercise 6.4 Router Adjacency fault 3..................................................................................................... 62
Exercise 6.4.1 using the commands from the previous lab isolate the fault causing the adjacency to
fail. .......................................................................................................................................................... 62
Lab 7. Troubleshooting ISIS ......................................................................................................................... 67
3/126
4/126
System Address
10.10.10.6/32
System Address
10.10.10.5/32
1/1/2
Edge 1
1/1/2
10.2.12.0/28
.5
Edge 2
.6
1/1/4
.5
1/1/4
.1
10.1.15.0/28
10.10.10.1/32
Core 1
1/1/5
1/1/2
1/1/1
1/1/3
.2
.1
10.1.12.0/28
.2
.1
10.10.10.2/32
Core 2
1/1/2
1/1/3
.6
.2
10.1.26.0/28
1/1/5
1/1/1
.1
.2
10
.1
.1
0/
3.
10
.1
.2
4.
0/
28
10.1.23.0/28
28
10.1.14.0/28
.4
.4
1/1/1
10.1.48.0/28
.4
1/1/5
1/1/2
10.10.10.4/32
Core 4
.8
Edge 4
.3
.3
1/1/3
1/1/3
.4
1/1/1
.3
.3
10.1.34.0/28
1/1/2
1/1/5
10.10.10.3/32
Core 3
10.1.37.0/28
1/1/4
.7
1/1/4
1/1/2
.8
1/1/2
10.2.34.0/28
Edge 3
.7
System Address
10.10.10.7/32
System Address
10.10.10.8/32
5/126
Lab 1
Alcatel Support
6/126
Exercise 1.1. Log into the external Alcatel-Lucent page and demonstrate how
to get to customer documents and release notes on OLCS. Log into the SRKB
and demonstrate how to search for tips best practices and submit a question
as well as search the KB database.
http://www.alcatel-lucent.com/support
http://srkb.ca.alcatel-lucent.com/kb
Objective:
In this exercise, each student will demonstrate how to create a tech-support file to provide to ALU
support.
Table 1. Commands to capture tech-support file
admin tech-support <file-url>
Please Note: If this exercise is to be completed on simulators they will need floppy or hard disks installed
to simulate the CF card.
Exercise 2.1. Create a tech-support file and store it on CF1.
Px# admin tech-support cf1:\coreXDayMonthYear
7/126
01:23p
<DIR>
05:25p
05:25p
04:15p
09:28a
<DIR>
3 File(s)
2 Dir(s)
.ssh/
7564 Core1_Interfaces
2068 Core1_Interfaces.ndx
110132 core1July302013
log/
119764 bytes.
1335296 bytes free.
To allow Alcatel-Lucent support personel access to the tech-support file, you will need to move it to an
external ftp locaton.
To copy the file to a FTP server use the following command.
#copy <local_filename> ftp://admin:admin@10.10.10.1/cf1:/<remote_filename>
To allow FTP access to the router use the following command.
#configure system security ftp-server
8/126
Lab 2
Debug Logs and Log filters
9/126
Exercise 2.1. Create a file-id, log filter, and log to store the security debug
output.
In this exercise, each student will create his/her own Log File instance on a router. Use the following
commands create a Log File on your assigned router.
Table 2. Commands to configure and verify Debug Logs and log filters
configure log file-id <instance-number>
configure log file-id <instance-number> location < cf1:, cf2:, or cf3>
configure log filter <instance-number>
configure log file-id <instance-number> rollover <minutes> retention <hours>
show log file-id <log-file-id>
configure log filter <filter-id >
configure log log-id <log-id >
show log log-id
file type <file-url >
debug router ip packet
Please Note: If this exercise is to be completed on simulators they will need floppy or hard disks installed
to simulate the CF card.
Exercise 2.1. Create File ID with a compact flash location to store output of log filter.
Px# configure log file-id 1
Px# >config>log>file-id$ location cf1:
Px# >config>log>file-id$ rollover 5 retention 8
10/126
Exercise 2.2. Create a log filter with the required match criteria.
Px#
Px#
Px#
Px#
Px#
Exercise 2.3. Create a log id to capture security events, apply the filter created in exercise
2.2 and send the output to the file created in exercise 2.1.
Px# configure log log-id 3
Px# >config>log>log-id$ from security
11/126
Exercise 2.4. Create security event by launching another session and logging into to your
node.
02:30p
<DIR>
11:33a
<DIR>
02:20p
02:29p
02:30p
02:31p
4 File(s)
2 Dir(s)
408
515
526
407
./
../
log0301-20130624-121058
log0301-20130624-122020
log0301-20130624-122915
log0301-20130624-123033
1856 bytes.
1453568 bytes free.
12/126
Please note the more specific the debug command often the more useful the results. See below for an
example.
Px# debug router ip packet
13/126
.Y.&....
........
........
........
.Y.;....
........
........
........
.Y.&....
........
........
........
14/126
Stop the debug for OSPF packets and start a specific debug for ingress ospf hello packets on an interface.
Px# no debug
15/126
:
:
:
:
:
:
:
:
255.255.255.248
10
02 -------E-1
40
0.0.0.0
0.0.0.0
10.10.10.1
16/126
Lab 3
Sending debug output to a Syslog
Server
17/126
interface syslog
loopback
address 10.1.11.25/32
ospf area 0 interface "syslog"
After creating the loopback interface, verify it is reachable from your core router.
18/126
ttl=64
ttl=64
ttl=64
ttl=64
ttl=64
time=12.4ms.
time=1.59ms.
time=1.61ms.
time=1.60ms.
time=1.62ms.
---- 10.1.11.25 PING Statistics ---5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 1.59ms, avg = 3.77ms, max = 12.4ms, stddev = 4.33ms
19/126
Please Note: syslog only works correctly on real equipment not simulators
20/126
Lab 4
Using OAM tools and SAA
21/126
Objective:
The students will configure service assurance agent to run an OAM test and evaluate the output.
Exercise 4.1. Create an SAA test, set the loss-event threshold to 1, and
enable SNM P traps.
22/126
Note: the instructor will execute a script to cause intermittent reachability prior to
the execution of this test.
Run the SAA test using the OAM SAA command.
*A:CORE1# oam saa "SAA_ping_Test" owner "admin" start
23/126
Loss-rt
Falling
Rising
Falling
None
1
1
None
10
None
Never
None
06/25/2013 16:46:02 1
Never
None
===============================================================================
Test Run: 1
Total number of attempts: 100
Number of requests that failed to be sent out: 10
Number of responses that were received: 90
Number of requests that did not receive any response: 0
Total number of failures: 10, Percentage: 10
(in ms)
Min
Max
Average
Jitter
Outbound :
0.000
0.000
0.000
0.000
Inbound
:
0.000
0.000
0.000
0.000
Roundtrip :
1.52
14.5
2.50
0.959
Per test packet:
Sequence
Outbound
Inbound
RoundTrip Result
1
0.000
0.000
11.2 Response Received
2
0.000
0.000
1.54 Response Received
... Output omitted ...
21
0.000
0.000
2.83 Response Received
22
0.000
0.000
0.000 No route to destination
23
0.000
0.000
0.000 No route to destination
24
0.000
0.000
0.000 No route to destination
25
0.000
0.000
0.000 No route to destination
26
0.000
0.000
0.000 No route to destination
27
0.000
0.000
0.000 No route to destination
28
0.000
0.000
1.59 Response Received
... Output omitted ...
76
0.000
0.000
1.55 Response Received
77
0.000
0.000
0.000 No route to destination
78
0.000
0.000
0.000 No route to destination
79
0.000
0.000
0.000 No route to destination
80
0.000
0.000
0.000 No route to destination
81
0.000
0.000
1.60 Response Received
... Output omitted ...
100
0.000
0.000
1.61 Response Received
===============================================================================
24/126
1.
CRON The CRON feature allows the user to configure a CRON job, schedule it to run at a
specific time (CHRONOS) and day and define how many times the job should run. In this exercise the
student will configure a CRON to run a pre-configured SAA test at a specific time. The first step is to
create a script file using vi on the 7X50. The script file will contain the command oam saa "<test
name>" owner "admin" start be sure to press the enter key at the completion of the script.
CORE1# file vi cf1:\SAAcron
Defaulting to terminal type vt100
oam saa SAA_ping_Test owner "admin" start
~
~
~
~
~
To save your changes while in the editor hit the <esc> key once followed by : you will see a colon
appear at the bottom of the page. Type w (write) then q (quit) followed by enter.
oam saa SAA_ping_Test owner admin start
~
~
~
~
~
~
:wq
You should see the number of lines and characters written if it was successful.
oam saa SAA_ping_Test owner admin start
~
~
~
~
~
~
~
~
~
"cf1:\SAAcron" 2 lines, 44 characters
Note: vi commands are beyond the scope of this training guide if the student in not familiar with the vi
editor please refer to instructors directions.
Once this script file has been created and saved configure a CRON to run this script at a specific time and
date.
25/126
There are three main areas that must be configured in a CRON, script, action, and schedule.
First, configure the script information. This will tell the CRON job where to go to find the script file.
Next, configure the action for the CRON. The action area identifies the name of the script field that the
CRON will use (PingCronScript) and where the CRON is to send the report. This report will show the
command that is in the script file and log it under a date time stamp.
CORE1# configure cron action
*A:CORE1>config>cron>action#
*A:CORE1>config>cron>action#
*A:CORE1>config>cron>action#
*A:CORE1>config>cron>action#
*A:CORE1>config>cron#
"PingCronAction"
script "PingCronScript"
results cf1:\
no shutdown
back
The next step is to set the schedule of when you want this SAA test to run, how often you want it to run
and what action field you want the scheduled CRON to go to. In this case the configuration is for a
oneshot or one time run; however, you could schedule it to run on a regular basis.
26/126
*A:CORE1>config>cron#schedule PingCronSchedule
*A:CORE1>config>cron>sched# action "PingCronAction"
*A:CORE1>config>cron>sched# type oneshot
*A:CORE1>config>cron>sched# day-of-month 22
*A:CORE1>config>cron>sched# month july
*A:CORE1>config>cron>sched# hour 21
*A:CORE1>config>cron>sched# minute 55
*A:CORE1>config>cron>sched# weekday wednesday
*A:CORE1>config>cron>sched# no shutdown
*A:CORE1>config>cron>sched# back
27/126
30/126
Falling
Rising
Falling
Latency-in Rising
Falling
Latency-out Rising
Falling
Latency-rt Rising
Falling
Loss-in
Rising
Falling
Loss-out
Rising
Falling
Loss-rt
Rising
Falling
Jitter-rt
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
Never
Never
Never
Never
Never
Never
Never
Never
Never
Never
Never
Never
Never
Never
Never
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
===============================================================================
The result portion of the action field will be written with a time/date stamp on the compact flash
specified. In the screen capture below you can locate the result file by the time/date stamp. This shows
that the CRON was executed.
02:44p
03:10p
04:05p
3 File(s)
0 Dir(s)
7141 OSPF
45 SAAcron
121 _20130723-140501.out
7307 bytes.
1449472 bytes free.
31/126
Lab 5
Hardware Faults
32/126
show version
Note this is NOT an exhaustive list of show commands please use the ?, tab, and space bar commands to
investigate other show commands and options.
33/126
2013
by
builder
in
34/126
___
___
___
___
/\ \
/\__\
/\ \
/\ \
\:\ \
___
/::| |
/::\ \
/::\ \
\:\ \
/\__\
/:|:| |
/:/\:\ \
/:/\:\ \
/::\ \
_\/__/ /:/|:|__|__
/:/ \:\ \
_\:\~\:\ \
/:/\:\__\ /\__\
/:/ |::::\__\ /:/__/ \:\__\ /\ \:\ \:\__\
/:/ \/__/ /:/ /
\/__/~~/:/ / \:\ \ /:/ / \:\ \:\ \/__/
/:/ /
/:/ /
/:/ /
\:\ /:/ /
\:\ \:\__\
\/__/
\/__/
/:/ /
\:\/:/ /
\:\/:/ /
/:/ /
\::/ /
\::/ /
\/__/
\/__/
\/__/
Initializing VMM
Time from clock is MON JUN 24 03:19:36 2013 UTC
Initial DNS resolving preference is ipv4-only
Attempting to exec primary configuration file:
'ftp://*:*@192.168.192.113/./config/7/derek.cfg' ...
System Configuration
System Security Configuration
System Login Control Configuration
Log Configuration
System Security Cpm Hw Filters and PKI Configuration
QoS Policy Configuration
Card Configuration
Port Configuration
System Sync-If-Timing Configuration
Management Router Configuration
Router (Network Side) Configuration
OSPFv2 Configuration
MPLS Configuration
RSVP Configuration
MPLS LSP Configuration
LDP Configuration
Web Portal Protocol Configuration
Service Configuration
Router (Service Side) Configuration
OSPFv2 Configuration
Mirror Configuration
Executed
752
lines
in
1.2
ftp://*:*@192.168.192.113/./config/7/derek.cfg
seconds
from
file
Login: ===============================================================================
cf3:/bootlog_prev.txt
===============================================================================
Boot log started on CPU#0
Build: X-10.0.R8 on Thu Jan 10 13:18:52 PST 2013 by builder
CPUCTL FPGA version: 17
Boot rom version is v38
35/126
2013
by
builder
in
___
___
___
___
/\ \
/\__\
/\ \
/\ \
\:\ \
___
/::| |
/::\ \
/::\ \
\:\ \
/\__\
/:|:| |
/:/\:\ \
/:/\:\ \
/::\ \
_\/__/ /:/|:|__|__
/:/ \:\ \
_\:\~\:\ \
/:/\:\__\ /\__\
/:/ |::::\__\ /:/__/ \:\__\ /\ \:\ \:\__\
/:/ \/__/ /:/ /
\/__/~~/:/ / \:\ \ /:/ / \:\ \:\ \/__/
/:/ /
/:/ /
/:/ /
\:\ /:/ /
\:\ \:\__\
36/126
\/__/
\/__/
/:/ /
/:/ /
\/__/
\:\/:/ /
\::/ /
\/__/
\:\/:/ /
\::/ /
\/__/
Initializing VMM
Time from clock is MON MAY 13 15:07:39 2013 UTC
Initial DNS resolving preference is ipv4-only
Attempting to exec primary configuration file:
'ftp://*:*@192.168.192.113/./config/7/derek.cfg' ...
MINOR: CLI Host '192.168.192.113' is unknown or unreachable.
Cannot find or access primary configuration file:
'ftp://*:*@192.168.192.113/./config/7/derek.cfg' ...
CRITICAL: CLI #1001 Cannot locate the configuration file Using default configuration values.
MAJOR: CLI #1008 The SNMP daemon is disabled.
To enable SNMP, execute the command
'config>system>snmp no shutdown'.
TiMOS-C-10.0.R8 cpm/hops ALCATEL ESS 7450 Copyright (c) 2000-2013 Alcatel-Lucent.
All rights reserved. All use subject to applicable license agreements.
Built on Thu Jan 10 13:48:35 PST 2013 by builder in /rel10.0/b1/R8/panos/main
37/126
The show log command below will detail any alarms that have been logged specifically against the
chassis.
#show log log-id 99 application chassis | match "base card a" ignore-case
17 2013/05/13 05:57:32.63 UTC MAJOR: CHASSIS #2026 Base Card A
6 2013/05/13 05:57:27.03 UTC MINOR: CHASSIS #2002 Base Card A
38/126
[size=500
next event=113
(not wrapped)]
The show chassis environment displays the current status of the router fans indicating any error
conditions. The command show chassis power-supply displays the current status of the router power
supplies indicating any error conditions.
show chassis environment
===============================================================================
Chassis Information
===============================================================================
Environment Information
Number of fan trays
Number of fans
: 2
: 4
: 1
: half speed
: up
:
:
:
:
1
dc
pem
up
39/126
sfm-400g
up
down/standby
(not equipped)
===============================================================================
PE3# show card A detail
===============================================================================
Card A
===============================================================================
Slot
Provisioned Type
Admin Operational
Comments
Equipped Type (if different)
State State
------------------------------------------------------------------------------A
sfm-400g
up
up/active
:
:
:
:
:
N/A
MON MAY 30 13:26:31 2011 UTC
N/A
2013/07/01 10:27:49
primary
Flash - cf1:
Administrative State
Operational state
Serial number
Firmware revision
Model number
Size
Free space
:
:
:
:
:
:
:
up
up
serial-1
v1.0
PC HD 1
1,424 KB
1,304 KB
Flash - cf2:
Administrative State
Operational state
Serial number
Firmware revision
Model number
Size
Free space
:
:
:
:
:
:
:
up
up
serial-2
v1.0
PC HD 2
0 Bytes
0 Bytes
Flash - cf3:
Administrative State
Operational state
Serial number
Firmware revision
Model number
Size
Free space
:
:
:
:
:
:
:
up
up
serial-3
v1.0
PC HD 3
18,432 Bytes
0 Bytes
Hardware Data
Platform type
Part number
CLEI code
Serial number
Manufacture date
Manufacturing deviations
Manufacturing assembly number
Administrative state
Operational state
Temperature
Temperature threshold
Software boot (rom) version
Software version
Time of last boot
:
:
:
:
:
:
:
:
:
:
:
:
:
:
7750
Sim Part#
Sim CLEI
card-11
01012003
Sim MfgDeviation card-11
01-2345-67
up
up
-1C
75C
simulated
TiMOS-B-11.0.R1 both/i386 ALCATEL SR 7750 *
2013/07/01 09:54:25
40/126
PE3# show
mda
===============================================================================
MDA Summary
===============================================================================
Slot Mda
Provisioned Type
Admin
Operational
Equipped Type (if different)
State
State
------------------------------------------------------------------------------No Matching Entries
===============================================================================
By default only the SFM is provisioned; no MDAs will be present: No Matching Entries until you
provision the IOM.
PE3# configure card 1 card-type "iom3-xp"
PE3# show card
===============================================================================
Card Summary
===============================================================================
Slot
Provisioned Type
Admin Operational
Comments
Equipped Type (if different)
State State
------------------------------------------------------------------------------1
iom3-xp
up
up
A
sfm-400g
up
up/active
B
sfm-400g
up
down/standby
(not equipped)
===============================================================================
PE3# show mda
===============================================================================
MDA Summary
===============================================================================
Slot Mda
Provisioned Type
Admin
Operational
Equipped Type (if different)
State
State
------------------------------------------------------------------------------1
1
(not provisioned)
up
unprovisioned
m20-1gb-xp-tx
2
(not provisioned)
up
unprovisioned
m20-1gb-xp-tx
===============================================================================
41/126
:
:
:
:
20
20
default
Ethernet
Hardware Data
Platform type
Part number
CLEI code
Serial number
Manufacture date
Manufacturing deviations
Manufacturing assembly number
Administrative state
Operational state
Temperature
Temperature threshold
Software version
Time of last boot
Current alarm state
Base MAC address
Firmware version
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
7750
Sim Part#
Sim CLEI
mda-1
01012003
Sim MfgDeviation mda-1
01-2345-67
up
up
-128C
75C
N/A
2013/07/11 16:58:44
alarm cleared
9a:c9:01:01:00:01
B-11.0.R1
------------------------------------------------------------------------------QOS Settings
------------------------------------------------------------------------------Ing. Named Pool Policy
: None
Egr. Named Pool Policy
: None
===============================================================================
MDA 1/2 detail
===============================================================================
Slot Mda
Provisioned Type
Admin
Operational
Equipped Type (if different)
State
State
------------------------------------------------------------------------------2
(not provisioned)
up
unprovisioned
42/126
m20-1gb-xp-tx
: 0
: 0
: default
:
Hardware Data
Platform type
Part number
CLEI code
Serial number
Manufacture date
Manufacturing deviations
Manufacturing assembly number
Administrative state
Operational state
Software version
Time of last boot
Current alarm state
Base MAC address
Firmware version
:
:
:
:
:
:
:
:
:
:
:
:
:
:
7750
Sim Part#
Sim CLEI
mda-2
01012003
Sim MfgDeviation mda-2
01-2345-67
up
unprovisioned
N/A
N/A
alarm cleared
9a:c9:01:02:00:01
N/A
------------------------------------------------------------------------------QOS Settings
------------------------------------------------------------------------------Ing. Named Pool Policy
: None
Egr. Named Pool Policy
: None
===============================================================================
(not provisioned)
up
unprovisioned
m20-1gb-xp-tx
: 0
: 0
: default
:
Hardware Data
Platform type
Part number
CLEI code
Serial number
Manufacture date
Manufacturing deviations
Manufacturing assembly number
Administrative state
Operational state
Software version
Time of last boot
Current alarm state
Base MAC address
Firmware version
:
:
:
:
:
:
:
:
:
:
:
:
:
:
7750
Sim Part#
Sim CLEI
mda-2
01012003
Sim MfgDeviation mda-2
01-2345-67
up
unprovisioned
N/A
N/A
alarm cleared
9a:c9:01:02:00:01
N/A
------------------------------------------------------------------------------QOS Settings
------------------------------------------------------------------------------Ing. Named Pool Policy
: None
Egr. Named Pool Policy
: None
43/126
By default all ports are administratively down. If a port is correctly configured but not up, possibly the
port is administratively down and requires a no shutdown to bring it up. The show port command allows
us to verify some key port parameters including Admin state, MTU Encapsulation/LAG.
The show port 1/1 or 1/2 card/MDA will display all ports for a specific MDA.
The show port 1/1/1 allows us to see more specific L2 parameters like link type, speed, duplex mode, L2
address, traffic statistics for that port as well as L2 error statistics.
The show version command will display the current software release running on the router
44/126
===============================================================================
Ports on Slot B
===============================================================================
Port
Admin Link Port
Cfg Oper LAG/ Port Port Port
C/QS/S/XFP/
Id
State
State
MTU MTU Bndl Mode Encp Type
MDIMDX
------------------------------------------------------------------------------B/1
Up
No
Ghost
1514 1514
- netw null faste
===============================================================================
show port 1/1
===============================================================================
Ports on Slot 1
===============================================================================
Port
Admin Link Port
Cfg Oper LAG/ Port Port Port
C/QS/S/XFP/
Id
State
State
MTU MTU Bndl Mode Encp Type
MDIMDX
------------------------------------------------------------------------------1/1/1
Down No
Down
1578 1578
- netw null xcme
1/1/2
Down No
Down
1578 1578
- netw null xcme
1/1/3
Down No
Down
1578 1578
- netw null xcme
1/1/4
Down No
Down
1578 1578
- netw null xcme
1/1/5
Down No
Down
1578 1578
- netw null xcme
1/1/6
Down No
Down
1578 1578
- netw null xcme
1/1/7
Down No
Down
1578 1578
- netw null xcme
1/1/8
Down No
Down
1578 1578
- netw null xcme
1/1/9
Down No
Down
1578 1578
- netw null xcme
1/1/10
Down No
Down
1578 1578
- netw null xcme
1/1/11
Down No
Down
1578 1578
- netw null xcme
1/1/12
Down No
Down
1578 1578
- netw null xcme
1/1/13
Down No
Down
1578 1578
- netw null xcme
1/1/14
Down No
Down
1578 1578
- netw null xcme
1/1/15
Down No
Down
1578 1578
- netw null xcme
1/1/16
Down No
Down
1578 1578
- netw null xcme
1/1/17
Down No
Down
1578 1578
- netw null xcme
1/1/18
Down No
Down
1578 1578
- netw null xcme
1/1/19
Down No
Down
1578 1578
- netw null xcme
1/1/20
Down No
Down
1578 1578
- netw null xcme
===============================================================================
show port 1/1/1
===============================================================================
Ethernet Interface
===============================================================================
Description
: 10/100/Gig Ethernet TX
Interface
: 1/1/1
Oper Speed
: N/A
Link-level
: Ethernet
Config Speed
: 1 Gbps
Admin State
: down
Oper Duplex
: N/A
Oper State
: down
Config Duplex
: full
Physical Link
: No
MTU
: 1578
Single Fiber Mode : No
Min Frame Length : 64 Bytes
IfIndex
: 35684352
Hold time up
: 0 seconds
Last State Change : 07/11/2013 16:58:43
Hold time down
: 0 seconds
Last Cleared Time : N/A
Phys State Chng Cnt: 0
Configured Mode
:
Dot1Q Ethertype
:
PBB Ethertype
:
Ing. Pool % Rate
:
Ing. Pool Policy
:
Egr. Pool Policy
:
Net. Egr. Queue Pol:
Egr. Sched. Pol
:
network
0x8100
0x88e7
100
n/a
n/a
default
n/a
Encap Type
QinQ Ethertype
: null
: 0x8100
45/126
Auto-negotiate
Oper Phy-tx-clock
Accounting Policy
Acct Plcy Eth Phys
Egress Rate
Load-balance-algo
:
:
:
:
:
:
true
not-applicable
None
None
Default
Default
MDI/MDX
: unknown
Collect-stats
Collect Eth Phys
Ingress Rate
LACP Tunnel
:
:
:
:
Down-when-looped
: Disabled
Loop Detected
: False
Use Broadcast Addr : False
Keep-alive
Retry
: 10
: 120
: Disabled
: Disabled
: sdh
Disabled
Disabled
Default
Disabled
: Disabled
: Disabled
: 10 seconds
:
:
:
:
:
:
:
0
0
0
0
0
0
0
Sngl Collisions
Mult Collisions
Late Collisions
Excess Collisns
Int MAC Tx Errs
Int MAC Rx Errs
Out Pause Frames
:
:
:
:
:
:
:
0
0
0
0
0
0
0
46/126
===============================================================================
Slot
Provisioned Type
Admin Operational
Comments
Equipped Type (if different)
State State
------------------------------------------------------------------------------1
iom3-xp
up
up
:
:
:
:
:
:
SR
none
Disabled
Disabled
2
2
FP 1 Specific Data
WRED Admin State
WRED buffer-allocation max
WRED buffer-allocation min
WRED reserved-cbs max
WRED reserved-cbs min
WRED Slope Policy
hi-bw-mc-srcEgress Alarm
hi-bw-mc-srcEgress Group
mc-path-mgmt Admin State
Ingress Bandwidth Policy
:
:
:
:
:
:
:
:
:
:
Out Of Service
2500
2500
2500
2500
default
2
0
Out Of Service
default
Hardware Data
Platform type
: 7750
Part number
: Sim Part#
CLEI code
: Sim CLEI
Serial number
: card-1
Manufacture date
: 01012003
Manufacturing string
: Sim MfgString card-1
Manufacturing deviations
: Sim MfgDeviation card-1
Manufacturing assembly number : 01-2345-67
Administrative state
: up
Operational state
: up
Temperature
: 46C
Temperature threshold
: 75C
Software boot (rom) version
: simulated
Software version
: TiMOS-B-10.0.R1 both/i386 ALCATEL SR 7750 *
Time of last boot
: 2013/07/23 14:56:10
Current alarm state
: alarm cleared
Base MAC address
: 9a:dd:01:00:00:00
Last bootup reason
: hard boot
Memory capacity
: 2,031 MB
* indicates that the corresponding row element may have been truncated.
===============================================================================
show version
TiMOS-B-10.0.R1 both/i386 ALCATEL SR 7750 Copyright (c) 2000-2012 Alcatel-Lucent.
All rights reserved. All use subject to applicable license agreements.
Built on Sun Mar 4 17:21:43 PST 2012 by builder in /rel10.0/b1/R1/panos/main
47/126
Lab 6
Troubleshooting OSPF
48/126
Exercise 6.1.2 On your Core router enable debug-trace for OSPF packets;
only debug the interface to your Edge router.
*A:CORE1# debug router ospf packet toEdge1
Exercise 6.1.3 Shut down OSPF on the edge router; verify neighbor state is
down on your core router.
49/126
Exercise 6.1.4 Enable ospf on the edge router verify neighbor state is full.
*A:Edge1# configure router ospf no shutdown
Verification
Verify the packets exchanged to form the OSPF adjacency
Note for real time debug you could set the debug log output to the current session. This can make it
difficult to enter commands as the debut output scrolls on your session. Optionally you can open a
second session to enter commands as well as stop start the debug or log.
Exercise 6.1.5 Stop the debug and view the output of the debug log.
50/126
*A:CORE1# no debug
*A:CORE1# show log log-id 22
===============================================================================
Event Log 22
===============================================================================
Description : (Not Specified)
Memory Log contents [size=300
next event=22 (not wrapped)]
21 2013/07/24 04:14:31.15 CEST MINOR: DEBUG #2001 Base OSPFv2
"OSPFv2: PKT
>> Incoming OSPF packet on I/F toEdge1
OSPF Version
: 2
Router Id
: 10.10.10.5
Area Id
: 0.0.0.0
Checksum
: db46
Auth Type
: Null
Auth Key
: 00 00 00 00 00 00 00 00
Packet Type
: LS_ACK
Packet Length
: 44 "
20 2013/07/24 04:14:31.09 CEST MINOR: DEBUG #2001 Base OSPFv2
"OSPFv2: PKT
>> Outgoing OSPF packet on I/F toEdge1
OSPF Version
: 2
Router Id
: 10.10.10.1
Area Id
: 0.0.0.0
Checksum
: 9d88
Auth Type
: Null
Auth Key
: 00 00 00 00 00 00 00 00
Packet Type
: LS_UPD
Packet Length
: 172 "
19 2013/07/24 04:14:30.80 CEST MINOR: DEBUG #2001 Base OSPFv2
"OSPFv2: PKT
>> Outgoing OSPF packet on I/F toEdge1
OSPF Version
: 2
Router Id
: 10.10.10.1
Area Id
: 0.0.0.0
Checksum
: 3bb0
Auth Type
: Null
Auth Key
: 00 00 00 00 00 00 00 00
Packet Type
: LS_ACK
Packet Length
: 44 "
18 2013/07/24 04:14:30.75 CEST MINOR: DEBUG #2001 Base OSPFv2
"OSPFv2: PKT
>> Incoming OSPF packet on I/F toEdge1
OSPF Version
: 2
Router Id
: 10.10.10.5
Area Id
: 0.0.0.0
Checksum
: d9a2
Auth Type
: Null
Auth Key
: 00 00 00 00 00 00 00 00
Packet Type
: LS_UPD
Packet Length
: 88 "
51/126
52/126
Packet Length
: 32 "
53/126
Auth Key
Packet Type
Packet Length
: 00 00 00 00 00 00 00 00
: DB_DESC
: 192 "
54/126
Checksum
Auth Type
Auth Key
Packet Type
Packet Length
:
:
:
:
:
e79f
Null
00 00 00 00 00 00 00 00
HELLO
44 "
55/126
Instructor change
The instructor will introduce a fault in the network causing some of the adjacencies to fail. In this
exercise you will work within your own pod and only use show commands. This is a discovery
exercise only please do not make any changes to the routers configuration.
Determine why your Core and Edge routers are not forming an adjacency.
Exercise 6.2.1 using the commands from the previous lab isolate the fault
causing the adjacency to fail.
Verify the neighbor state of your Core and Edge routers.
*A:CORE1# show router ospf neighbor
===============================================================================
OSPF Neighbors
===============================================================================
Interface-Name
Rtr Id
State
Pri RetxQ
TTL
Area-Id
------------------------------------------------------------------------------toCORE2
10.10.10.2
Full
1
0
31
0.0.0.0
toCORE3
10.10.10.3
Full
1
0
39
0.0.0.0
toCORE4
10.10.10.4
Full
1
0
34
0.0.0.0
toEdge1
10.10.10.5
ExchStart 1
0
38
0.0.0.0
------------------------------------------------------------------------------No. of Neighbors: 4
===============================================================================
56/126
Open an additional session to your Core router and configure an ospf debug log to that session.
Examine the packets exchanged between your core and edge router then disable the debug.
57/126
:
:
:
:
1300
000042
7
48079
:
:
:
:
1564
000042
7
48079
58/126
toEdge1
3170 2013/07/24 04:45:40.28 CEST WARNING: OSPF #2043 Base VR: 1 OSPFv2 (0)
"LCL_RTR_ID 10.10.10.1: Conflicting configuration mtuMismatch on interface
from 10.1.15.5 in dbDescript"
toEdge1
Instructor change
The instructor will introduce a fault in the network causing some of the adjacencies to fail. In this
exercise you will work within your own pod and only use show commands. This is a discovery
exercise only please do not make any changes to the routers configuration.
Determine the fault preventing your Core and Edge routers forming an adjacency.
Exercise 6.3.1 using the commands from the previous lab isolate the fault
causing the adjacency to fail.
Verify the neighbor state of your Core and Edge routers.
59/126
Open an additional session to your Core router and configure an ospf debug log to that session.
*A:CORE1# configure log log-id 33
*A:CORE1>config>log>log-id$ from debug-trace
*A:CORE1>config>log>log-id$ to session
*A:CORE1>config>log>log-id$exit all
Examine the packets exchanged between your core and edge router then disable the debug.
60/126
:
:
:
:
:
:
:
255.255.255.240
10
02
1
40
0.0.0.0
0.0.0.0
:
:
:
:
:
:
:
255.255.255.240
10
02
1
40
0.0.0.0
0.0.0.0
61/126
interface
Instructor change
The instructor will introduce a new fault in the network causing some of the adjacencies to fail.
In this exercise you will work within your own pod and only use show commands. This is a
discovery exercise only please do not make any changes to the routers configuration.
Exercise 6.4.1 using the commands from the previous lab isolate the fault
causing the adjacency to fail.
Verify the neighbor state of your Core and Edge routers.
62/126
Open an additional session to your Core router and configure an ospf debug log to that session.
*A:CORE1# configure log log-id 33
*A:CORE1>config>log>log-id$ from debug-trace
*A:CORE1>config>log>log-id$ to session
*A:CORE1>config>log>log-id$exit all
Examine the packets exchanged between your core and edge router then disable the debug.
63/126
*A:Edge1#
25 2013/07/24 15:31:27.28 CEST MINOR: DEBUG #2001 Base OSPFv2
"OSPFv2: PKT
>> Outgoing OSPF packet on I/F toCORE1
OSPF Version
: 2
Router Id
: 10.10.10.5
Area Id
: 0.0.0.0
Checksum
: e79f
Auth Type
: Null
Auth Key
: 00 00 00 00 00 00 00 00
Packet Type
: HELLO
Packet Length
: 44
Network Mask
Hello Interval
Options
Rtr Priority
Dead Interval
Designated Router
Backup Router
"
:
:
:
:
:
:
:
255.255.255.240
10
02
1
40
0.0.0.0
0.0.0.0
:
:
:
:
:
:
:
:
255.255.255.240
10
02
1
40
10.1.15.1
0.0.0.0
10.10.10.5
64/126
Overall Questions
1.
2.
3.
4.
5.
Which state are the routers in if the authentication password does not match?
65/126
Lab 7
Troubleshooting IS-IS
66/126
Exercise 7.1.2 On your Core router enable debug-trace for ISIS packets; only
debug the interface to your Edge router.
*A:CORE1# debug router isis packet "toEdge1" detail
Exercise 7.1.3 Shut down ISIS on the edge router; verify neighbor state is
down on your core router.
*A: Edge1# configure router isis shutdown
67/126
Exercise 7.1.4 Enable ISIS on the edge router; verify neighbor state is full to
the Core route.
*A:Edge1# configure router isis no shutdown
Verification
Verify the packets exchanged to form the ISIS adjacency.
Note for real time debug you could set the debug log output to the current session. This can make it
difficult to enter commands as the debut output scrolls on your session. Optionally you can open a
second session to enter commands as well as stop start the debug or log.
Exercise 7.1.5 Stop the debug and view the output of the debug log.
68/126
*A:CORE1# no debug
*A:CORE1# show log log-id 22
===============================================================================
Event Log 22
===============================================================================
Description : (Not Specified)
Memory Log contents [size=300
next event=57 (not wrapped)]
56 2013/07/24 16:26:21.50 CEST MINOR: DEBUG #2001 Base ISIS PKT
"ISIS PKT:
TX ISIS PDU ifId 5 len 69:
DMAC
: 09:00:2b:00:00:05
Proto Disc
: 131
Header Len
: 20
Version PID
: 1
ID Length
: 0
Version
: 1
Reserved
: 0
Max Area Addr : 3
PDU Type
: (11) Point-2-Point IS-IS Hello Pdu
Circuit Type : L1L2
Source Id
: 01 00 10 01 00 01
Hold Time
: 27
Packet length : 52
Circuit Id
: 0
Area Addresses:
Area Address : (3) 49.0051
Supp Protocols:
Protocols
: IPv4
I/F Addresses :
I/F Address
: 10.1.15.1
3Way Adjacency
:
State
: UP
Ext ckt ID
: 5
NbrSysID
: 01 00 10 01 00 05
Nbr ext ckt ID : 2
"
55 2013/07/24 16:26:20.83 CEST MINOR: DEBUG #2001 Base ISIS PKT
"ISIS PKT:
RX ISIS PDU ifId 5 len 163:
DMAC
: 09:00:2b:00:00:05
Proto Disc
: 131
Header Len
: 33
Version PID
: 1
ID Length
: 0
Version
: 1
Reserved
: 0
Max Area Addr : 3
PDU Type
: (18) Level 1 CSNP PDU
Packet length : 163
Source Id
: 01 00 10 01 00 05 00
Start LSP Id : 00 00 00 00 00 00 00 00
End LSP Id
: FF FF FF FF FF FF FF FF
LSP Entries
:
Remaining Life: 1190
LSP ID
: 01 00 10 01 00 01 00 00
Sequence Num : 0000000c
69/126
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
b198
1113
01 00 10
00000006
772f
1073
01 00 10
00000006
3ced
1049
01 00 10
00000005
6873
1192
01 00 10
00000007
3abb
1164
01 00 10
00000004
8333
1081
01 00 10
00000004
c6ad
1056
01 00 10
00000004
0a28
01 00 02 00 00
01 00 03 00 00
01 00 04 00 00
01 00 05 00 00
01 00 06 00 00
01 00 07 00 00
01 00 08 00 00
"
54 2013/07/24 16:26:20.84 CEST MINOR: DEBUG #2001 Base ISIS PKT
"ISIS PKT:
RX ISIS PDU ifId 5 len 163:
DMAC
: 09:00:2b:00:00:05
Proto Disc
: 131
Header Len
: 33
Version PID
: 1
ID Length
: 0
Version
: 1
Reserved
: 0
Max Area Addr : 3
PDU Type
: (19) Level 2 CSNP PDU
Packet length : 163
Source Id
: 01 00 10 01 00 05 00
Start LSP Id : 00 00 00 00 00 00 00 00
End LSP Id
: FF FF FF FF FF FF FF FF
LSP Entries
:
Remaining Life: 1192
LSP ID
: 01 00 10 01 00 01 00 00
Sequence Num : 00000017
Checksum
: 8c32
Remaining Life: 1191
LSP ID
: 01 00 10 01 00 02 00 00
Sequence Num : 0000000f
Checksum
: 9dbf
Remaining Life: 1191
LSP ID
: 01 00 10 01 00 03 00 00
Sequence Num : 0000000e
Checksum
: 4ec1
Remaining Life: 1191
70/126
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
01 00 10
0000000e
ce0e
1194
01 00 10
0000000e
3859
1191
01 00 10
0000000c
74f4
1190
01 00 10
0000000c
c17d
1190
01 00 10
0000000c
759f
01 00 04 00 00
01 00 05 00 00
01 00 06 00 00
01 00 07 00 00
01 00 08 00 00
"
53 2013/07/24 16:26:20.51 CEST MINOR: DEBUG #2001 Base ISIS PKT
"ISIS PKT:
TX ISIS PDU ifId 5 len 180:
Proto Disc
: 131
Header Len
: 33
Version PID
: 1
ID Length
: 0
Version
: 1
Reserved
: 0
Max Area Addr : 3
PDU Type
: (19) Level 2 CSNP PDU
Packet length : 163
Source Id
: 01 00 10 01 00 01 00
Start LSP Id : 00 00 00 00 00 00 00 00
End LSP Id
: FF FF FF FF FF FF FF FF
LSP Entries
:
Remaining Life: 1192
LSP ID
: 01 00 10 01 00 01 00 00
Sequence Num : 00000017
Checksum
: 8c32
Remaining Life: 1191
LSP ID
: 01 00 10 01 00 02 00 00
Sequence Num : 0000000f
Checksum
: 9dbf
Remaining Life: 1191
LSP ID
: 01 00 10 01 00 03 00 00
Sequence Num : 0000000e
Checksum
: 4ec1
Remaining Life: 1191
LSP ID
: 01 00 10 01 00 04 00 00
Sequence Num : 0000000e
Checksum
: ce0e
Remaining Life: 1194
LSP ID
: 01 00 10 01 00 05 00 00
Sequence Num : 0000000e
Checksum
: 3859
Remaining Life: 1191
LSP ID
: 01 00 10 01 00 06 00 00
Sequence Num : 0000000c
Checksum
: 74f4
71/126
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
Remaining Life:
LSP ID
:
Sequence Num :
Checksum
:
1191
01 00 10 01 00 07 00 00
0000000c
c17d
1191
01 00 10 01 00 08 00 00
0000000c
759f
"
52 2013/07/24 16:26:20.51 CEST MINOR: DEBUG #2001 Base ISIS PKT
"ISIS PKT:
TX ISIS PDU ifId 5 len 180:
Proto Disc
: 131
Header Len
: 33
Version PID
: 1
ID Length
: 0
Version
: 1
Reserved
: 0
Max Area Addr : 3
PDU Type
: (18) Level 1 CSNP PDU
Packet length : 163
Source Id
: 01 00 10 01 00 01 00
Start LSP Id : 00 00 00 00 00 00 00 00
End LSP Id
: FF FF FF FF FF FF FF FF
LSP Entries
:
Remaining Life: 1191
LSP ID
: 01 00 10 01 00 01 00 00
Sequence Num : 0000000c
Checksum
: b198
Remaining Life: 1113
LSP ID
: 01 00 10 01 00 02 00 00
Sequence Num : 00000006
Checksum
: 772f
Remaining Life: 1074
LSP ID
: 01 00 10 01 00 03 00 00
Sequence Num : 00000006
Checksum
: 3ced
Remaining Life: 1049
LSP ID
: 01 00 10 01 00 04 00 00
Sequence Num : 00000005
Checksum
: 6873
Remaining Life: 1192
LSP ID
: 01 00 10 01 00 05 00 00
Sequence Num : 00000007
Checksum
: 3abb
Remaining Life: 1164
LSP ID
: 01 00 10 01 00 06 00 00
Sequence Num : 00000004
Checksum
: 8333
Remaining Life: 1081
LSP ID
: 01 00 10 01 00 07 00 00
Sequence Num : 00000004
Checksum
: c6ad
Remaining Life: 1056
LSP ID
: 01 00 10 01 00 08 00 00
Sequence Num : 00000004
Checksum
: 0a28
"
72/126
"
50 2013/07/24 16:26:16.83 CEST MINOR: DEBUG #2001 Base ISIS PKT
"ISIS PKT:
RX ISIS PDU ifId 5 len 43:
DMAC
: 09:00:2b:00:00:05
Proto Disc
: 131
Header Len
: 17
Version PID
: 1
ID Length
: 0
Version
: 1
Reserved
: 0
Max Area Addr : 3
PDU Type
: (1b) Level 2 PSNP Pdu
Packet length : 35
Source Id
: 01 00 10 01 00 05 00
LSP Entries
:
Remaining Life: 1196
LSP ID
: 01 00 10 01 00 01 00 00
Sequence Num : 00000017
Checksum
: 8c32
"
49 2013/07/24 16:26:16.50 CEST MINOR: DEBUG #2001 Base ISIS PKT
"ISIS PKT:
TX ISIS PDU ifId 5 len 52:
DMAC
: 09:00:2b:00:00:05
Proto Disc
: 131
Header Len
: 17
Version PID
: 1
73/126
ID Length
: 0
Version
: 1
Reserved
: 0
Max Area Addr : 3
PDU Type
: (1a) Level 1 PSNP Pdu
Packet length : 35
Source Id
: 01 00 10 01 00 01 00
LSP Entries
:
Remaining Life: 1196
LSP ID
: 01 00 10 01 00 05 00 00
Sequence Num : 00000007
Checksum
: 3abb
"
48 2013/07/24 16:26:16.51 CEST MINOR: DEBUG #2001 Base ISIS PKT
"ISIS PKT:
TX ISIS PDU ifId 5 len 52:
DMAC
: 09:00:2b:00:00:05
Proto Disc
: 131
Header Len
: 17
Version PID
: 1
ID Length
: 0
Version
: 1
Reserved
: 0
Max Area Addr : 3
PDU Type
: (1b) Level 2 PSNP Pdu
Packet length : 35
Source Id
: 01 00 10 01 00 01 00
LSP Entries
:
Remaining Life: 1198
LSP ID
: 01 00 10 01 00 05 00 00
Sequence Num : 0000000e
Checksum
: 3859
"
47 2013/07/24 16:26:15.91 CEST MINOR: DEBUG #2001 Base ISIS PKT
"ISIS PKT:
Continued...
Default Metric: (I) 20
Delay Metric : (I) 0
Expense Metric: (I) 0
Error Metric : (I) 0
IP Address
: 10.10.10.7
IP Mask
: 255.255.255.255
Default Metric: (I) 20
Delay Metric : (I) 0
Expense Metric: (I) 0
Error Metric : (I) 0
IP Address
: 10.10.10.6
IP Mask
: 255.255.255.255
Default Metric: (I) 20
Delay Metric : (I) 0
Expense Metric: (I) 0
Error Metric : (I) 0
IP Address
: 10.10.10.8
IP Mask
: 255.255.255.255
Default Metric: (I) 10
Delay Metric : (I) 0
Expense Metric: (I) 0
74/126
Instructor change
The instructor will introduce a fault in the network causing some of the adjacencies to fail. In this
exercise you will work within your own pod and only use show commands. This is a discovery
exercise only please do not make any changes to the routers configuration.
75/126
Determine why your Core and Edge routers are not forming an adjacency.
Exercise 7.2.1 using the commands from the previous lab isolate the fault
causing the adjacency to fail.
Verify the neighbor state of your Core and Edge routers.
*A:CORE1# show router isis adjacency
===============================================================================
ISIS Adjacency
===============================================================================
System ID
Usage State Hold Interface
MT Enab
------------------------------------------------------------------------------CORE2
L1L2 Up
21
toCORE2
No
CORE3
L1L2 Up
23
toCORE3
No
CORE4
L1L2 Up
24
toCORE4
No
------------------------------------------------------------------------------Adjacencies : 3
===============================================================================
Open an additional session to your Core router and configure an ospf debug log to that session.
*A:CORE1# configure log log-id 33
*A:CORE1>config>log>log-id$ from debug-trace
*A:CORE1>config>log>log-id$ to session
*A:CORE1>config>log>log-id$exit all
Examine the packets exchanged between your core and edge router then disable the debug.
76/126
"
2 2013/07/24 17:10:18.67 CEST MINOR: DEBUG #2001 Base ISIS PKT
"ISIS PKT:
RX ISIS PDU ifId 5 len 43:
DMAC
: 09:00:2b:00:00:05
Proto Disc
: 131
Header Len
: 20
Version PID
: 1
ID Length
: 0
Version
: 1
Reserved
: 0
Max Area Addr : 3
PDU Type
: (11) Point-2-Point IS-IS Hello Pdu
Circuit Type : L1
Source Id
: 01 00 10 01 00 05
Hold Time
: 27
Packet length : 42
Circuit Id
: 0
Area Addresses:
Area Address : (3) 49.0051
Supp Protocols:
Protocols
: IPv4
I/F Addresses :
I/F Address
: 10.1.15.5
3Way Adjacency
:
State
: DOWN
Ext ckt ID
: 2
*A:CORE1# no debug
77/126
Instructor change
The instructor will introduce a fault in the network causing some of the adjacencies to fail. In this
exercise you will work within your own pod and only use show commands. This is a discovery
exercise only please do not make any changes to the routers configuration.
Objective
Determine why your Core and Edge routers are not forming an adjacency.
Exercise 7.2.1 using the commands from the previous lab isolate the fault
causing the adjacency to fail.
Verify the neighbor state of your Core and Edge routers.
78/126
10.10.10.5,
10.10.10.5,
10.10.10.5,
10.10.10.5,
10.10.10.5,
Router:
Router:
Router:
Router:
Router:
Base
Base
Base
Base
Base
---- 10.10.10.5 PING Statistics ---5 packets transmitted, 0 packets received, 100% packet loss
Open an additional session to your core router and configure a debug log to that session.
79/126
80/126
81/126
Lab 8
Troubleshooting LDP
82/126
83/126
84/126
85/126
10.10.10.4
10.10.10.6/32
131064U
131064
--10.10.10.5
10.10.10.7/32
131067U
131065
--10.10.10.2
10.10.10.7/32
131067N
131069
1/1/3
10.1.13.3
10.10.10.3
10.10.10.7/32
131067U
131069
--10.10.10.4
10.10.10.7/32
131067U
131065
--10.10.10.5
10.10.10.8/32
131066U
131068
--10.10.10.2
10.10.10.8/32
131066U
131068
--10.10.10.3
10.10.10.8/32
131066N
131068
1/1/1
10.1.14.4
10.10.10.4
10.10.10.8/32
131066U
131069
--10.10.10.5
------------------------------------------------------------------------------No. of Prefix Bindings: 32
===============================================================================
===============================================================================
LDP Service FEC 128 Bindings
===============================================================================
Type
VCId
SvcId
SDPId
Peer
IngLbl EgrLbl LMTU RMTU
------------------------------------------------------------------------------------------------------------------------------------------------------------No Matching Entries Found
===============================================================================
===============================================================================
LDP Service FEC 129 Bindings
===============================================================================
AGI
SAII
TAII
Type
SvcId
SDPId
Peer
IngLbl EgrLbl LMTU RMTU
------------------------------------------------------------------------------------------------------------------------------------------------------------No Matching Entries Found
===============================================================================
86/126
10.10.10.2/32
131067N
131070
1/1/4
10.1.15.1
10.10.10.1
10.10.10.3/32
131066N
131068
1/1/4
10.1.15.1
10.10.10.1
10.10.10.4/32
131070N
131069
1/1/4
10.1.15.1
10.10.10.1
10.10.10.5/32
131071U
---10.10.10.1
10.10.10.6/32
131064N
131064
1/1/4
10.1.15.1
10.10.10.1
10.10.10.7/32
131065N
131067
1/1/4
10.1.15.1
10.10.10.1
10.10.10.8/32
131069N
131066
1/1/4
10.1.15.1
10.10.10.1
------------------------------------------------------------------------------No. of Prefix Bindings: 8
===============================================================================
===============================================================================
LDP Service FEC 128 Bindings
===============================================================================
Type
VCId
SvcId
SDPId
Peer
IngLbl EgrLbl LMTU RMTU
------------------------------------------------------------------------------------------------------------------------------------------------------------No Matching Entries Found
===============================================================================
===============================================================================
LDP Service FEC 129 Bindings
===============================================================================
AGI
SAII
TAII
Type
SvcId
SDPId
Peer
IngLbl EgrLbl LMTU RMTU
------------------------------------------------------------------------------------------------------------------------------------------------------------No Matching Entries Found
===============================================================================
87/126
Instructor change
The instructor will introduce a fault in the network causing some of the LDP sessions to fail. In
this exercise you will work within your own pod and only use show commands. This is a
discovery exercise only please do not make any changes to the routers configuration.
88/126
Remember for LDP to function correctly we require LDP enabled on the interfaces and route
reachability including the system address. We must also receive a label offer from the active next
hop router in the route table.
LDP Fault 1
Objective:
For this exercise some of your LDP sessions are no longer available. Using show commands from the
previous exercise try to identify what sessions are missing.
89/126
10.10.10.6,
10.10.10.6,
10.10.10.6,
10.10.10.6,
10.10.10.6,
Router:
Router:
Router:
Router:
Router:
Base
Base
Base
Base
Base
90/126
===============================================================================
*A:CORE2# show router route-table 10.10.10.6
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags]
Type
Proto
Age
Pref
Next Hop[Interface Name]
Metric
------------------------------------------------------------------------------------------------------------------------------------------------------------No. of Routes: 0
Flags: L = LFA nexthop available
B = BGP backup route available
n = Number of times nexthop is repeated
===============================================================================
91/126
No session
Verify the LDP interfaces
*A:Edge3# show router ldp interface
===============================================================================
LDP Interfaces
===============================================================================
Interface
Adm Opr Hello Hold KA
KA
Transport
Factor Time Factor Timeout Address
------------------------------------------------------------------------------toCORE3
Up Up
3
15
3
30
System
------------------------------------------------------------------------------No. of Interfaces: 1
===============================================================================
92/126
ttl=64
ttl=64
ttl=64
ttl=64
ttl=64
time=1.68ms.
time=1.63ms.
time=1.49ms.
time=1.44ms.
time=1.73ms.
Ping OK problem with the TCP connection port blocked by filter or firewall
Verify the connection to Core3
*A:Edge3# show system connections address 10.10.10.3
===============================================================================
Connections
===============================================================================
Prot RecvQ
TxmtQ
Local Address
State
MSS Remote Address
vRtrID
------------------------------------------------------------------------------No Entries Found.
===============================================================================
No connection!
Check for firewall or filter
93/126
ldp
session
==============================================================================
LDP Sessions
==============================================================================
Peer LDP Id
Adj Type
State
Msg Sent Msg Recv Up Time
----------------------------------------------------------------------------------------------------------------------------------------------------------No Matching Entries Found
======================================================================
No session
Verify the LDP interfaces
==============================================================================
*A:Edge4# show router ldp interface
===============================================================================
LDP Interfaces
===============================================================================
Interface
Adm Opr Hello Hold KA
KA
Transport
Factor Time Factor Timeout Address
------------------------------------------------------------------------------toCORE4
Up Up
3
15
3
30
System
------------------------------------------------------------------------------No. of Interfaces: 1
===============================================================================
94/126
95/126
Instructor change
The instructor will introduce a fault in the network causing some of the LDP active bindings to
fail. In this exercise you will work within your own pod and only use show commands. This is a
discovery exercise only please do not make any changes to the routers configuration.
Remember for LDP to function correctly we require LDP enabled on the interfaces and route
reachability including the system address. We must also receive a label offer from the active next
hop router in the route table.
LDP Fault 2
Objective:
For this exercise some of your LDP active bindings are no longer available. Using show commands from
the previous exercise try to identify what bindings are missing.
96/126
The route does not follow the LDP interface! We must receive a label offer from the active next
hop router in the route table
97/126
98/126
Lab 9
Troubleshooting RSVP
99/126
100/126
LSP Name
To
Fastfail
Adm
Opr
Config
------------------------------------------------------------------------------Edge1-Edge3
10.10.10.7
Yes
Up
Up
------------------------------------------------------------------------------LSPs : 1
===============================================================================
The RSVP protocol maintains path state information for every LSP, in every router an LSP transits
through. In a core node with a large number of LSPs this may be a vast amount of information to sort
through. The following commands will allow you to trace a single LSP and its FRR paths through your
network.
Exercise 9.1.3 Display all the RSVP sessions on your router.
Remember the name of the LSP from step 8.1.1
101/126
Note the LSP name and LSP id of the LSP originating from your router this will also be in the state
information on the transiting and terminating routers.
Exercise 9.1.5 Examine the details of the session originating from your router
102/126
Exercise 9.1.6 Log into each router along the path of your LSP; verify the LSP details.
103/126
104/126
105/126
Exercise 9.1.7 Return to the originating node and display the LSP path details using the
show router mpls lsp <LSPname> path <PathName> detail command. You will find this command
one of the most useful commands for troubleshooting RSVP signaled MPLS LSPs.
106/126
Disabled
Enabled
Disabled
Disabled
Enabled
Enabled
16
Disabled
Oper
Oper
Oper
Oper
Oper
Oper
Oper
Oper
Adspec :
CSPF
:
CSPF-FL:
LeastF*:
FRR
:
FRR NP :
FRHopL*:
PropAG :
Disabled
Enabled
Disabled
Disabled
Enabled
Enabled
16
Disabled
Neg MTU
:
Bandwidth
:
Hop Limit
:
Record Route:
Record Label:
SetupPriori*:
Hold Priori*:
Class Type :
Backup CT
:
MainCT Retry:
Rem
:
MainCT Retry:
Limit
:
Include Grps:
None
Exclude Grps:
None
1560
No Reservation
255
Record
Record
7
0
0
None
n/a
Oper
Oper
Oper
Oper
Oper
Oper
Oper
Oper
MTU
:
Bw
:
HopLim*:
RecRou*:
RecLab*:
SetupP*:
HoldPr*:
CT
:
1560
0 Mbps
255
Record
Record
7
0
0
Adaptive
: Enabled
Preference : n/a
Path Trans : 1
Failure Code: noError
ExplicitHops:
No Hops Specified
Actual Hops :
10.1.15.5(10.10.10.5) @ n
-> 10.1.15.1(10.10.10.1) @ n
-> 10.1.13.3(10.10.10.3) @
-> 10.1.37.7(10.10.10.7)
Oper InclGr*:
None
Oper ExclGr*:
None
Oper Metric : 300
CSPF Queries: 1
Failure Node: n/a
Record
Record
Record
Record
Label
Label
Label
Label
:
:
:
:
N/A
131068
131068
131066
107/126
ComputedHops:
10.1.15.5(S)
-> 10.1.15.1(S)
-> 10.1.13.3(S)
-> 10.1.37.7(S)
ResigEligib*: False
LastResignal: n/a
CSPF Metric : 300
===============================================================================
Code for the fault (see user guide for current list of fault codes)
Failure node
Note: if CSPF is not enabled on the LSP the Failure node will always indicate the originating node of
the LSP. This is not necessarily the node that is causing the problem.
ExplicitHops
This represents the explicitly configured (S) strict and (L) loose hops that were defined in the LSP path
Actual Hops
Identifies the hops the LSP is currently using, where and what type of FRR is available and if FRR is
currently active on the LSP.
Note: if FRR facility is active actual hops does not show all hops it only shows up to the node where
FRR facility bypass tunnel is active not the actual bypass hops.
(@ n) identifies where node protect is available
(@) identifies where link protect is available
(@ # n) identifies where FRR is currently active
Node where fault occurred
Please note if CSPF is not enabled on the LSP the Failure node will always indicate the originating node
of the LSP. This is not necessarily the node that is causing a problem.
ComputedHops
These are the hops CSPF returned based on the Traffic Engineering constraints provided.
108/126
Exercise 9.1.8 Next we will log into all the nodes that provide protection, verify the bypass
tunnel labels and ports in use.
Note: The functionality of FRR facility bypass is to temporarily detour to avoid the next downstream
node (first) or link (second) and get back onto the original LSP as close as possible to the failure. This is
done with the additional label push for the bypass tunnel.
Output from Exercise 8.1.7 above:
Actual Hops :
10.1.15.5(10.10.10.5) @ n
Record Label
: N/A
-> 10.1.15.1(10.10.10.1) @ n
Record Label
: 131070
-> 10.1.13.3(10.10.10.3) @
Record Label
: 131069
-> 10.1.37.7(10.10.10.7)
Record Label
: 131070
The first detour is originating on Edge1 10.10.10.5 providing node protect for Node 10.10.10.1
109/126
110/126
111/126
112/126
113/126
114/126
detail
===============================================================================
RSVP Sessions (Detailed)
===============================================================================
------------------------------------------------------------------------------LSP : bypass-node10.10.10.3
------------------------------------------------------------------------------From
: 10.10.10.1
To
: 10.2.34.7
Tunnel ID
: 61440
LSP ID
: 2
Style
: FF
State
: Up
Session Type
: Bypass Tunnel
In Interface
: n/a
Out Interface : 1/1/1
In Label
: n/a
Out Label
: 131070
Previous Hop
: n/a
Next Hop
: 10.1.14.4
SetupPriority : 7
Hold Priority : 0
Class Type
: 0
SubGrpOrig ID : 0
SubGrpOrig Addr:
P2MP ID
: 0
FrrAvailType
: N/A
FrrSrlgStrict : N/A
SrlgDisjoint
: N/A
Path Recd
: 0
Path Sent
: 7955
Resv Recd
: 7925
Resv Sent
: 0
Summary msgs: :
SPath Recd
: 0
SPath Sent
: 0
SResv Recd
: 0
SResv Sent
: 0
------------------------------------------------------------------------------LSP : bypass-node10.10.10.3
------------------------------------------------------------------------------From
: 10.10.10.7
To
: 10.1.14.1
Tunnel ID
: 61440
LSP ID
: 2
Style
: FF
State
: Up
Session Type
: Terminate
In Interface
: 1/1/1
Out Interface : n/a
In Label
: 131065
Out Label
: n/a
Previous Hop
: 10.1.14.4
Next Hop
: n/a
SetupPriority : 7
Hold Priority : 0
Class Type
: 0
SubGrpOrig ID : 0
SubGrpOrig Addr:
P2MP ID
: 0
FrrAvailType
: N/A
FrrSrlgStrict : N/A
SrlgDisjoint
: N/A
Path Recd
: 7877
Path Sent
: 0
Resv Recd
: 0
Resv Sent
: 7879
Summary msgs: :
SPath Recd
: 0
SPath Sent
: 0
SResv Recd
: 0
SResv Sent
: 0
===============================================================================
115/126
detail
===============================================================================
RSVP Sessions (Detailed)
===============================================================================
------------------------------------------------------------------------------LSP : bypass-node10.10.10.3
------------------------------------------------------------------------------From
: 10.10.10.1
To
: 10.2.34.7
Tunnel ID
: 61440
LSP ID
: 2
Style
: FF
State
: Up
Session Type
: Transit
In Interface
: 1/1/1
Out Interface : 1/1/5
In Label
: 131070
Out Label
: 131070
Previous Hop
: 10.1.14.1
Next Hop
: 10.1.48.8
SetupPriority : 7
Hold Priority : 0
Class Type
: 0
SubGrpOrig ID : 0
SubGrpOrig Addr:
P2MP ID
: 0
FrrAvailType
: N/A
FrrSrlgStrict : N/A
SrlgDisjoint
: N/A
Path Recd
: 7958
Path Sent
: 7913
Resv Recd
: 7960
Resv Sent
: 7927
Summary msgs: :
SPath Recd
: 0
SPath Sent
: 0
SResv Recd
: 0
SResv Sent
: 0
------------------------------------------------------------------------------LSP : bypass-node10.10.10.3
------------------------------------------------------------------------------From
: 10.10.10.7
To
: 10.1.14.1
Tunnel ID
: 61440
LSP ID
: 2
Style
: FF
State
: Up
Session Type
: Transit
In Interface
: 1/1/5
Out Interface : 1/1/1
In Label
: 131066
Out Label
: 131065
Previous Hop
: 10.1.48.8
Next Hop
: 10.1.14.1
SetupPriority : 7
Hold Priority : 0
Class Type
: 0
SubGrpOrig ID : 0
SubGrpOrig Addr:
P2MP ID
: 0
FrrAvailType
: N/A
FrrSrlgStrict : N/A
SrlgDisjoint
: N/A
Path Recd
: 7977
Path Sent
: 7879
Resv Recd
: 7880
Resv Sent
: 7911
Summary msgs: :
SPath Recd
: 0
SPath Sent
: 0
SResv Recd
: 0
SResv Sent
: 0
===============================================================================
116/126
detail
===============================================================================
RSVP Sessions (Detailed)
===============================================================================
------------------------------------------------------------------------------LSP : bypass-node10.10.10.3
------------------------------------------------------------------------------From
: 10.10.10.1
To
: 10.2.34.7
Tunnel ID
: 61440
LSP ID
: 2
Style
: FF
State
: Up
Session Type
: Transit
In Interface
: 1/1/4
Out Interface : 1/1/2
In Label
: 131070
Out Label
: 131069
Previous Hop
: 10.1.48.4
Next Hop
: 10.2.34.7
SetupPriority : 7
Hold Priority : 0
Class Type
: 0
SubGrpOrig ID : 0
SubGrpOrig Addr:
P2MP ID
: 0
FrrAvailType
: N/A
FrrSrlgStrict : N/A
SrlgDisjoint
: N/A
Path Recd
: 7916
Path Sent
: 7896
Resv Recd
: 7943
Resv Sent
: 7963
Summary msgs: :
SPath Recd
: 0
SPath Sent
: 0
SResv Recd
: 0
SResv Sent
: 0
------------------------------------------------------------------------------LSP : bypass-node10.10.10.3
------------------------------------------------------------------------------From
: 10.10.10.7
To
: 10.1.14.1
Tunnel ID
: 61440
LSP ID
: 2
Style
: FF
State
: Up
Session Type
: Transit
In Interface
: 1/1/2
Out Interface : 1/1/4
In Label
: 131065
Out Label
: 131066
Previous Hop
: 10.2.34.7
Next Hop
: 10.1.48.4
SetupPriority : 7
Hold Priority : 0
Class Type
: 0
SubGrpOrig ID : 0
SubGrpOrig Addr:
P2MP ID
: 0
FrrAvailType
: N/A
FrrSrlgStrict : N/A
SrlgDisjoint
: N/A
Path Recd
: 7966
Path Sent
: 7979
Resv Recd
: 7913
Resv Sent
: 7916
Summary msgs: :
SPath Recd
: 0
SPath Sent
: 0
SResv Recd
: 0
SResv Sent
: 0
===============================================================================
117/126
detail
===============================================================================
RSVP Sessions (Detailed)
===============================================================================
------------------------------------------------------------------------------LSP : bypass-node10.10.10.3
------------------------------------------------------------------------------From
: 10.10.10.1
To
: 10.2.34.7
Tunnel ID
: 61440
LSP ID
: 2
Style
: FF
State
: Up
Session Type
: Terminate
In Interface
: 1/1/2
Out Interface : n/a
In Label
: 131069
Out Label
: n/a
Previous Hop
: 10.2.34.8
Next Hop
: n/a
SetupPriority : 7
Hold Priority : 0
Class Type
: 0
SubGrpOrig ID : 0
SubGrpOrig Addr:
P2MP ID
: 0
FrrAvailType
: N/A
FrrSrlgStrict : N/A
SrlgDisjoint
: N/A
Path Recd
: 7923
Path Sent
: 0
Resv Recd
: 0
Resv Sent
: 7973
Summary msgs: :
SPath Recd
: 0
SPath Sent
: 0
SResv Recd
: 0
SResv Sent
: 0
------------------------------------------------------------------------------LSP : bypass-node10.10.10.3
------------------------------------------------------------------------------From
: 10.10.10.7
To
: 10.1.14.1
Tunnel ID
: 61440
LSP ID
: 2
Style
: FF
State
: Up
Session Type
: Bypass Tunnel
In Interface
: n/a
Out Interface : 1/1/2
In Label
: n/a
Out Label
: 131065
Previous Hop
: n/a
Next Hop
: 10.2.34.8
SetupPriority : 7
Hold Priority : 0
Class Type
: 0
SubGrpOrig ID : 0
SubGrpOrig Addr:
P2MP ID
: 0
FrrAvailType
: N/A
FrrSrlgStrict : N/A
SrlgDisjoint
: N/A
Path Recd
: 0
Path Sent
: 7992
Resv Recd
: 7946
Resv Sent
: 0
Summary msgs: :
SPath Recd
: 0
SPath Sent
: 0
SResv Recd
: 0
SResv Sent
: 0
===============================================================================
Check the final bypass originating at Core3 10.10.10.3 providing link protection for link 10.1.37.7 this
bypass LSP transits Core4 and Edge4 re-joining the original LSP on terminating node Edge3
118/126
119/126
120/126
Exercise 9.2.1 The instructor will make a change in the network causing a fault. Using the
commands from Exercise 8.1.8, verify the LSP path and any bypass tunnel labels and ports in
use.
Look at the path details on your router using the show router mpls lsp <lsp-name> path detail
Command.
121/126
Disabled
Enabled
Disabled
Disabled
Enabled
Enabled
16
Disabled
Oper
Oper
Oper
Oper
Oper
Oper
Oper
Oper
Adspec :
CSPF
:
CSPF-FL:
LeastF*:
FRR
:
FRR NP :
FRHopL*:
PropAG :
Disabled
Enabled
Disabled
Disabled
Enabled
Enabled
16
Disabled
Neg MTU
:
Bandwidth
:
Hop Limit
:
Record Route:
Record Label:
SetupPriori*:
Hold Priori*:
Class Type :
Backup CT
:
MainCT Retry:
Rem
:
MainCT Retry:
Limit
:
Include Grps:
None
Exclude Grps:
None
1560
No Reservation
255
Record
Record
7
0
0
None
n/a
Oper
Oper
Oper
Oper
Oper
Oper
Oper
Oper
MTU
:
Bw
:
HopLim*:
RecRou*:
RecLab*:
SetupP*:
HoldPr*:
CT
:
1560
0 Mbps
255
Record
Record
7
0
0
0
Oper InclGr*:
None
Oper ExclGr*:
None
Adaptive
: Enabled
Preference : n/a
Path Trans : 1
Failure Code: tunnelLocallyRepaired
ExplicitHops:
10.1.15.1(S)
-> 10.1.13.3(S)
Actual Hops :
10.1.15.5(10.10.10.5) @ n
-> 10.1.15.1(10.10.10.1) @ # n
-> 10.2.34.7(10.10.10.7)
ComputedHops:
: N/A
: 131068
: 131066
122/126
10.1.15.5(S)
-> 10.1.15.1(S)
-> 10.1.13.3(S)
-> 10.1.37.7(S)
ResigEligib*: False
LastResignal: n/a
CSPF Metric : 300
In Prog MBB :
MBB Type
: GlobalRevert
NextRetryIn : 6 sec
Started At : 07/22/2013 17:02:08
RetryAttempt: 22
FailureCode: noCspfRouteToDestination
Failure Node: 10.10.10.5
Signaled BW: 0 Mbps
===============================================================================
* indicates that the corresponding row element may have been truncated.
From this output we can see it is on a FRR path at # Core1 (10.10.10.1). Use the OAM LSP trace
command to see the current path for this lsp.
123/126
124/126
125/126
126/126