Basic and Advanced Troubleshooting

of Nexus 1000v
LABDCT-2333

Clifford Aldan
Aaron Lam
Agenda

• Cisco Nexus 1000V Architecture

• VSM to VEM
• MAC Tracing
• Additional Resources
• Conclusion

LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Cisco Nexus 1000V Architecture
Cisco Nexus 1000V Architecture
Network team manages virtual & physical networks
Virtual Appliance
Network VSM-1 (active) NX-OS
Admin Control Plane
VSM-2 (standby)

Supervisor-1 (Active)
Supervisor-2 (StandBy)
Back Plane

Linecard-1
Linecard-2
… NX-OS
Linecard-N Data Plane
Modular Switch VEM-1 VEM-2 VEM-N

Hypervisor Hypervisor Hypervisor

VSM: Virtual Supervisor Module Server
VEM: Virtual Ethernet Module Admin
LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
 Cisco Nexus 1000V Architecture
VM VM VM VM VM VM VM VM VM VM VM VM

Cisco Cisco Cisco

Nexus Nexus Nexus
1000V 1000V 1000V
VEM VEM VEM
Hypervisor Hypervisor Hypervisor

Server Server Server

Virtual Supervisor Module (VSM) Virtual Ethernet Module (VEM)

• Virtual or Physical appliance running Cisco • Enables advanced networking capability on
NXOS (supports high-availability) the hypervisor
• Performs management, monitoring, and • Provides each virtual machine with a
configuration dedicated “switch port”
• Tight integration with management platforms • Collection of VEMs : 1 Virtual Distributed
Switch

Nexus 1000V VSM

vCenter / SCVMM
LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 6
VSM to VEM
Opaque Data

• Opaque Data is an essential configuration that is needed for the VEMs to

initiate communication with the VSM
• Bootstrapping communication to bring up the VEM-VSM path
• It carries important data such as, Control VLAN, Packet VLAN, Domain ID,
System Port-profiles, VSM IP address, VSM MAC address, etc.
• Opaque Data takes this route VSM  vCenter  VEM(ESXi host)

LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
VSM to VEM
L2 vs L3 mode

• L2 Mode
– VSM is Layer 2 accessible and can ONLY control hosts that reside in the SAME Layer
2 network
• L3 Mode
– VSM can be Layer 3 accessible and can control hosts that reside in a separate Layer
2 network
– Requires a dedicated vmkernel interface that is assigned to a vethernet port-profile
with “capability l3control”

LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
VSM to VEM
System VLANs

• Used for critical communication

• MUST be used for:
1. Control/Packet VLAN (when using L2 mode)
2. L3 control vmk (when using L3 mode)
3. Management VLAN (VSM mgmt, ESXi mgmt)
4. Storage VLAN (iSCSI / NFS)
• Ports get straight access to the kernel even if the DVS is not up
• Good for bringing the ports online faster
• Should NOT be used for
1. VM traffic
2. vMotion
LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
 VSM to VEM
vEthernet port-profile examples
VSM# sh run port-profile VM2 VSM# sh run port-profile
port-profile type vethernet VM2
vmware port-group l3control
switchport mode access port-profile type vethernet
switchport access vlan 20 l3control
no shutdown
state enabled
capability l3control
vmware port-group
VSM# sh run port-profile iscsi-a
port-profile type vethernet iscsi-a
switchport mode access
capability iscsi-multipath switchport access vlan 170
vmware port-group no shutdown
switchport mode access
system vlan 170
switchport access vlan 150
no shutdown state enabled
system vlan 150
state enabled

LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
 VSM to VEM
Ethernet port-profile examples
VSM# sh run port-profile Uplink1 VSM# sh run port-profile Uplink2
port-profile type ethernet Uplink1 port-profile type ethernet Uplink2
vmware port-group vmware port-group
switchport mode trunk switchport mode trunk
switchport trunk allowed vlan switchport trunk allowed vlan
20,150,170 20,150,170
channel-group auto on mode channel-group auto mode active
mac-pinning no shutdown
no shutdown system vlan 150,170
system vlan 150,170 state enabled
state enabled

LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Port-channels
MAC-Pinning
• No “port-channel” configuration needed on upstream channel-group auto mode
switch on mac-pinning
• Each Eth port gets its own SGID
• You can also manually pin the port-profiles to a
specific SGID
• Recommended 
• channel-group auto mode on mac-pinning
LACP
Single Switch Port-Channel
• Servers with 2 or more links connected to the same
upstream access switch or multiple uplinks that
connects using a port-channel Channel-group auto mode
• Supports up to 16 physical links [active|passive]
• Uses the native VLAN for LACPDU exchange
• channel-group auto mode
[active|passive]
Sub-group CDP/Manual Multi-Chassis EtherChannel
• 
• channel-group auto mode on cdp
Channel-group auto mode
• NOT RECOMMENDED
[active|passive]

LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
 MAC Tracing

13
 MAC Tracing
Mappings

Nexus 5548 Nexus 5548

FI-6248-A FI-6248-B

B250-M2 IOM-2208-B
IOM-2208-A
LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
MAC Tracing

LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 15
MAC Tracing

MAC:A MAC:B
LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Additional Resources
• Nexus 1000v deployment guide:
http://www.cisco.com/c/en/us/products/collateral/switches/nexus-1000v-switch-
vmware-vsphere/guide_c07-556626.html
• Nexus 1000v install and upgrade guide:
http://www.cisco.com/c/en/us/support/switches/nexus-1000v-switch-vmware-
vsphere/products-installation-guides-list.html
• Nexus 1000v configuration guide:
http://www.cisco.com/c/en/us/support/switches/nexus-1000v-switch-vmware-
vsphere/products-installation-and-configuration-guides-list.html
• Nexus 1000v troubleshooting guide:
http://www.cisco.com/c/en/us/support/switches/nexus-1000v-switch-vmware-
vsphere/products-troubleshooting-guides-list.html

LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
IPv6-only Experimental SSID (with NAT64)
SSID: IPV6ONLYEXP
PASS: iknowbesteffort
Addressing: SLAAC + stateless DHCPv6
Offsite NAT64 (Thanks to Go6 Institute)

Questions/support: @ayourtch
Hashtag: #IPV6ONLYEXP
SLA: it’s in the password 
LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Complete Your Online Session Evaluation
• Please complete your online session
evaluations after each session.
Complete 4 session evaluations
& the Overall Conference Evaluation
(available from Thursday)
to receive your Cisco Live T-shirt.

• All surveys can be completed via

the Cisco Live Mobile App or the
Communication Stations

LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Call to Action
• Visit the World of Solutions for
– Cisco Campus
– Walk in Labs
– Technical Solution Clinics
• Meet the Engineer
• Lunch time Table Topics
• DevNet zone related labs and sessions
• Recommended Reading: for reading material and further resources for this
session, please visit www.pearson-books.com/CLMilan 2015

LABDCT-2333 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 20