DMVPN Troubleshooting Commands

Some basic IOS IPsec/DMVPN commands to aid with troubleshooting:

show dmvpn detail
show ip nhrp
show ip nhrp multicast

show crypto isakmp sa
show crypto isakmp policy
show crypto ipsec sa

show run | s isakmp
show run | s ipsec
show run | s interface Tunnel

debug crypto isakmp
debug crypto ipsec
(remember to enable terminal monitor if you are remote)


IP SLA, Track Lists and Route Tracking

IP SLA is a powerful tool, which I will only scratch the surface of in this post in regards to route tracking. I had a scenario where the primary default route should be removed if the internet was no longer accessible through the primary interface. This can be achieved with weighted static routing, but the primary default route will only be removed if the primary interface is physically down. The more likely scenario exists where the primary interface is physically connected and up, but the ISP is having routing/peering issues and takes down your connection to the outside world. IP SLA can be used to simply ping an IP address, and if the ping is unsuccessful, the route is removed.

Here is a simple IP SLA configuration to ping every 5 seconds:

ip sla 1
icmp-echo source-ip
frequency 5
ip sla schedule 1 life forever start-time now

To track this IP SLA operation, we use the following command:

track 1 rtr 1 reachability
delay down 15 up 15

On newer versions of IOS, “rtr” will be replaced with “ip sla”.  I added the “delay down 15 up 15” statement so that the track doesn’t report as “down” until ip sla has failed 3 times in a row.  This is because I don’t want one ping to fail and have the route switched immediately.  Conversely, on fail-back, I don’t just one ping to be successful before failing back to the primary route.  This prevents route-flapping when interfaces/service is flapping.

I would consider it best practice to statically route to whatever IP address you are pinging for your IP SLA operation.  Since I am pinging, and my gateway IP is (theoretically), I add:

ip route

And finally, to add the track to my default route, I would use:

ip route track 1


If you don’t trust pinging a single IP address, you can setup multiple IP SLA operations (IP SLA 2, IP SLA 3)  and multiple tracks (Track 2, Track 3) and track all of them in a Track List.  Example:

track 100 list threshold percentage
object 1
object 2
object 3
object 4
threshold percentage up 25

ip route track 100

Each of the objects is a seperate IP SLA operation/track, and as long as =>25% of the objects are up, the list will be considered up.  This is good idea to implement in case one of your ping destination happens to go down or stops responding to pings.

ShoreTel VPN Concentrator NTP Vulnerability

The ShoreTel VPN Concentrator responds to NTP requests (for remote IP phones), which makes it vulnerable to the NTP DDOS attacks that have been happening lately.  To enable the WAN firewall on the ShoreTel VPN Concentrator (off by default) you must access a hidden menu.

  1. Log into your VPN concentrator, and click “Network” on the left-hand side
  2. Access the hidden firewall menu by changing the URL to page=5
    1. change to:
  3. Check the box “Enable WAN Firewall
    1. This allows you to manage the concentrator via HTTP or HTTPS
  5. Press submit (All current VPN sessions will drop for ~30 seconds)
  6. The concentrator will be unresponsive for 30-60 seconds while it applies, and you are done!

Cisco 3560-X Password Reset

1. Unplug Cisco 3560-X power
2. Hold down “Mode” button on front, and plug in power
3. Continue to hold “Mode” for about 15 seconds, until the SYST LED turns solid green, then release
4. Log into the switch via console, you should see the “switch:” prompt
5. Enter the command flash_init
6. Rename the config file with rename flash:config.text flash:config.old
7. Enter the command boot to load IOS
8. Enter No to abort system configuration dialog
9. Proceed to enable mode with the En command
10. Type rename flash:config.old flash:config.text
11. Then copy flash:config.text system:running-config
12. The configuration has now been loaded, go into conf t and change the user/enable password
13. Once changed, save the new passwords with wr mem

Cisco Show Command for ACL Interfaces

While browsing, I came across an interesting post regarding  easily finding interfaces in a config that have an ACL applied.  The thread is found here, and the command is as follows (credit goes to @Santino from the thread):

show ip interface | include line protocol|access list


802.1X Wireless Authentication in a Windows Environment

Recently came across an issue where users in an 802.1x wireless environment were logging into their laptops, and the AD logon script would run before they had wireless network connectivity.  This was a problem since their network drives would not map.

While the users blamed the wireless network, the problem is actually resolved in AD Group Policy by specifying the order of operations for wireless authentication/user logon/domain logon.  Technet has a great article on how to resolve this issue:

QoS DSCP/ToS Values

DSCP Class DSCP (bin) DSCP (dec) ToS (dec) ToS (bin) ToS Prec. (bin) ToS Prec. (dec)
none 000000 0 0 00000000 000 0
cs1 001000 8 32 00100000 001 1
af11 001010 10 40 00101000 001 1
af12 001100 12 48 00110000 001 1
af13 001110 14 56 00111000 001 1
cs2 010000 16 64 01000000 010 2
af21 010010 18 72 01001000 010 2
af22 010100 20 80 01010000 010 2
af23 010110 22 88 01011000 010 2
cs3 011000 24 96 01100000 011 3
af31 011010 26 104 01101000 011 3
af32 011100 28 112 01110000 011 3
af33 011110 30 120 01111000 011 3
cs4 100000 32 128 10000000 100 4
af41 100010 34 136 10001000 100 4
af42 100100 36 144 10010000 100 4
af43 100110 38 152 10011000 100 4
cs5 101000 40 160 10100000 101 5
ef 101110 46 184 10111000 101 5
cs6 110000 48 192 11000000 110 6
cs7 111000 56 224 11100000 111 7