WLC upgrade snippet

I was tired of trying to remember the syntax and all the commands to upgrade a Cisco WLC from the CLI, so I made a quick little TextExpander snippet to save time typing. It’s very simple, but it’ll save time. When you type the shortcut for this snippet, it will bring up a window that you fill in the blanks, then click OK to have the commands typed for you. Below is a screenshot of the dialog and the script. It’s hard coded for sftp and for a code update, but it’s easy to modify to your own ends.


transfer download filename %fill%
transfer download datatype code
transfer download path %fill%
transfer download serverip %fill%
transfer download mode sftp
transfer download username %fill%
transfer download password %fill%
transfer download start


A Strange, Unsolved WLAN Problem

I’m seeing strange behavior on the WLAN at one location. This location is one of many that are on the same controller and identically configured, save for AP locations and IP addresses. No other site is reporting this problem. Here’s my problem description:

Users report that many devices are unable to access “any site that requires a login”. From what I’ve seen, this really means most (but not all) SSL protected URLs. HTTP URLs work fine, HTTPS URLs timeout. This only happens on the open guest SSID. If one connects to the secured corporate SSID, everything works normally. Reports indicate that many, but not all devices are impacted. We couldn’t find a single Apple device that was impacted, but on-site staff believes it hits some of them, too. One of the staff owns an Android phone on which the problem is reproducible. I’m heading out there tomorrow with a suite of test devices to see if I can duplicate it with any of them. There is a possibility it only hits 802.11ac devices, but this is not the only 802.11ac site and it is the only one reporting this problem. I have connected with an 802.11ac laptop and had no issues. The 2.4GHz RF environment leaves something to be desired (and was the source of the Spectrum Analysis as Art post), but this problem also occurs on 5GHz. The APs are in FlexConnect mode, so I tried switching them to local mode and that did not change the behavior.

Does this sound like anything someone has seen? Any ideas what is going on?


What’s New In Cisco Wireless Software 8.0.110 – 8.1.102

Cisco Wireless Software 8.1.102 has been released and it’s time to bring my notes up to date.

These are abridged notes covering high points. Read the release notes for yourself and test your chosen build before deploying it in production.

8.0.110 (Release Notes)

Note: If you need to run 8.0.110 (or 7.6), please read this post: TAC Recommended AireOS 7.6 and 8.0 – 2Q CY15

  • If you have 3700P APs, don’t install this release. Contact TAC. This warning doesn’t apply to 3700i or 3700e.
  • Support added for the 1570 AP
    • Handful of features added to support 1570 specific features.
  • Support for priming universal APs (APs not locked to a regulatory domain) and auto setting the regulatory domain based on location. See Cisco Aironet Universal AP Priming and Cisco AirProvision User Guide for more info.
  • Enhancements to Express Setup for the 2500 WLC.
  • SSLv3 is now disabled by default.
  • Lots of resolved caveats. Lots of open ones, too…

8.0.115 (Release Notes)

  • Nothing new. Very short list of bug fixes, mainly for the 3700P.
  • The 8.0.110 special build (mentioned in the TAC Recommended link above) may be a better bet right now.

8.1.102 (Release Notes)

  • Virtual WLC now supported on KVM.
  • These APs retain feature parity with 8.0 and do not gain new features: 1050, 1140, 1260. Nice to see support not completely vanish, yet.
  • Support for WLC 5520 and 8540 added.
  • Dynamic Bandwidth Selection (DBS)
    • Chooses 20/40/80MHz channel width automatically. Tries to balance client needs with RF needs. I’ll be interested to hear what others think. I’m sticking with 40MHz.
  • Flexible DFS
    • Automatically adjust channel and width to avoid radar for more efficient channel usage. I presume this leverages DBS.
  • Enhanced Interference Mitigation
    • ED-RRM now also takes Wi-Fi interferers into account.
  • Optimized Roaming Extensions (802.11v BSS Transition Management)
    • Infrastructure helps clients make better roaming choices. Not sure what clients actually support this.
  • Defaults now implement best practices.
  • AVC added to FlexConnect APs
    • I’ve been told this isn’t supported on the 2504, but the release notes don’t say. YMMV.
  • SNMP MIB enhanced to allow monitoring of an HA WLC.
  • Support for Lync SDN API.
    • In short, Lync tells the WLC when a call is happening so the WLC can take QoS actions.
  • AVC updates
    • Per app, per client rate limiting
    • AVC based QoS markings
  • Inter controller roaming across IOS-XE and AireOS based controllers (8500 series, 5520, 5760)
  • AAA can override FlexConnect VLAN.
  • Stateful client switchover for mesh APs (RAPs and MAPs)
  • There is a decent list of caveats. Please go read them for yourself if you find this release of interest.

Notes at the end remind that 7.6 is still the recommended release for 802.11ac deployments, with 7.4 for 802.11n deployments.

Also, Field Upgradeable Software (FUS) 1.9 is recommended. You can do a “show sysinfo” to see what you have installed. Look for the Firmware Version line and the Field Recovery Image Version. If you have FUS 1.9, it will look like this:

Bootloader Version............................... 1.0.20
Field Recovery Image Version.....................


#vBrownBag Cisco Certification Series


The #vBrownBag folks over at ProfessionalVMware.com are currently running a Cisco Certification Track. They started last month and have had a several episodes covering related topics ranging from the Cisco Learning Network to setting up a virtual lab. Last week they had their first session talking about a specific certification, the CCENT.

The next two weeks for the US episodes (June 4 and June 10) will be Edward Henry (@NetworkN3rd) and myself talking about CCNA R&S prep. That will be followed by the infamous Tom Hollingsworth (@networkingnerd) leaving the #SDNicorn long enough to talk CCNA DC prep. Last on the schedule is Lawrence Kohan (@LawrenceKohan) with a three part series covering prep for the CCNP R&S exams.

It looks like a good lineup, so I hope you can join in live to ask questions Wednesdays at 730PM Central for the next couple months!


CCDA (640-864 DESGN)



While I was at Cisco Live 2013 I passed the CCDA (640-864 DESGN) exam. I took this towards the end of the conference using the on-site 50% discount. I had already attempted (and not passed) the CCIE R&S exam and really wanted to leave having passed an exam! There was a group of us that went to the testing center, and I think we were about 50/50 on passes vs fails. Either way, we all came away with good information (obviously without violating our NDAs) about the different exams. For example, we found out that the CCNA SP Ops exam is heavily an ITIL exam, but I’ll tell you a bit about what I learned about the CCDA.

Study Materials

First, I’m going to tell about my study materials. I had been studying using the Cisco Press CCDA 640-864 Official Cert Guide, Premium Edition eBook and Practice Test off and on for about a year. I would read a chapter or two occasionally when I grew tired of reading the CCIE Routing and Switching Certification Guide. The practice exams included in the Premium eBook Edition are very good for one that is included with a book, but it is not quite at the same level as a Boson practice exam. That said, the software does include all the “Do I know this already?” quizzes from the book, which I found convenient, and three different pools of questions for the actual practice exams. They exams include a fairly good representation of the kinds of questions you will see on the exam. They are all multiple choice questions (which is what you’ll see in this exam, anyway) and each question includes a short explanation of the correct answer and a link to the correct references in the eBook.

CCDA Practice Test Example

Example of the CCDA practice exam (click to see larger version)

The book itself is your typical Cisco Press eBook. It is broken up into logical sections that build upon each other and include the “Do I know this already?” quizzes to see if you already know the contents of that chapter. The book will prepare you for the exam. Unless you do a lot of Cisco pre-sales work, your experience may not help you as much. In fact, it may make things more difficult.

The Exam

Personally, I’ve not found any of our Cisco VARs to actually add value on the pre-sales side. I usually do my own research and my own designs, then have our local Cisco SE’s sanity check the design. Every time I try to leave it to the VAR, something is missed. This has caused me to ingest a lot of Cisco marketing literature, Cisco Validated Designs, product data sheets, etc. These are the kinds of things you need to know for the CCDA exam. CCDA is about cookie cutter designs based on scale. It uses rules of thumb that fit common situations. It’s about knowing the Cisco product lines, including less common components like WAAS. It’s about very basic design principles which most people who have doing this for a long time will know, but might disagree with the CCDA designated way. It’s really about being able to design a network quickly for pre-sales purposes.

In a nutshell, CCDA is a Cisco sales engineer certification. That’s why it’s #5 in the 15 Top Paying Certifications for 2013, because the CCDAs are getting commissions on sales.

That said, if you are new to networking, it’s still useful. Even if you aren’t, you may be forced to expand your knowledge of the Cisco product lines, which probably won’t hurt, and if you want to continue on to the CCDP, it’s a necessary hoop to jump through. However, this is not a very technical exam and I would argue that it is not really about network design.

To CCDP and Beyond!

The CCDP materials look to be more about actual design, and I look forward to getting a chance to dive into those materials, but what I’m really interested in reading is the book The Art of Network Architecture (which isn’t out, yet) by Russ White, Scott Morris, and Denise Donohue. That book is about how to think about network design. Go check out The Art of Network Architecture session from Cisco Live 2013 to get a taste. If you really enjoy that, you may be on the path to CCDE.


Password required, but none set

I had a strange thing happen today when I upgraded a 2960G from IOS 12.2 to 15.0. After booting the upgraded IOS, I logged in, entered the enable command, and was surprised to get this error:


Password required, but none set

Everything seemed to work fine, but it was a little odd. I did discover that the following command resolves the error:

aaa authentication enable default enable

Which is also odd, because the documentation states:

If the default list is not set, only the enable password is checked. This has the same effect as the following command:

aaa authentication enable default enable

I’ll go with the explicit statement because the error message, while it may be spurious, makes me uncomfortable.


No ISSU for you!

The Nexus 5000 series has the capability to do ISSU, an In-Service Software Upgrade. You can upgrade a vPC pair of Nexus 5000s without impacting any hosts (assuming everything is dual connected). Well, that is, unless you are using Windows Server 2012 with LACP. Let’s take a look at some show command output, shall we?

Nexus6# sh lacp issu-impact
For ISSU to Proceed, Check the following:
1. All port-channel member port should be in a steady state.
2. LACP rate fast should not be enabled on satellite member ports.

The following ports are not ISSU ready
Eth1/28     ,

OK, so Eth1/28 is going to prevent an ISSU because of an LACP issue, let’s check out the LACP details.

Nexu6# sh lacp interface e1/28
Interface Ethernet1/28 is up
  Channel group is 28 port channel is Po28
  PDUs sent: 21679
  PDUs rcvd: 750
  Markers sent: 0
  Markers rcvd: 0
  Marker response sent: 0
  Marker response rcvd: 0
  Unknown packets rcvd: 0
  Illegal packets rcvd: 0
Lag Id: [ [(0, 90-e2-ba-23-9e-8c, 0, 0, 100), (7f9b, 0-23-4-ee-be-2,
801c, 8000, 11c)] ]
Operational as aggregated link since Tue Jul 23 15:00:18 2013

Local Port: Eth1/28   MAC Address= 54-7f-ee-ef-cd-ab
  System Identifier=0x8000,54-7f-ee-ef-cd-ab
  Port Identifier=0x8000,0x11c
  Operational key=32796
  LACP_Timeout=Long Timeout (30s)
  Partner information refresh timeout=Short Timeout (3s)
Actor Admin State=(Ac-1:To-1:Ag-1:Sy-0:Co-0:Di-0:De-0:Ex-0)
Actor Oper State=(Ac-1:To-0:Ag-1:Sy-1:Co-1:Di-1:De-0:Ex-0)
Neighbor: 0x100
  MAC Address= 90-e2-ba-23-9e-8c
  System Identifier=0x0,90-e2-ba-23-9e-8c
  Port Identifier=0x0,0x100
  Operational key=0
  LACP_Timeout=short Timeout (1s)
Partner Admin State=(Ac-0:To-1:Ag-0:Sy-0:Co-0:Di-0:De-0:Ex-0)
Partner Oper State=(Ac-1:To-1:Ag-1:Sy-1:Co-1:Di-1:De-0:Ex-0)

You’ll notice that the LACP_Timeout values in bold do not match between the Local Port and the Neighbor. We need both ends set to the long timeout for ISSU to be happy. The LACP neighbor is a Windows Server 2012 box using switch dependent NIC teaming (LACP). We researched and were unable to find a way to tweak this timeout setting. This means you either run your Windows Server 2012 NIC team in switch independent mode (works the same as ESXi without LACP) or you don’t get to do ISSU… Somewhat defeats the whole point, no?

[UPDATE: Microsoft has since released a hotfix that allows you to change the timeout: https://support.microsoft.com/en-us/kb/3109099]

This problem also crops up with Linux hosts, but at least Linux lets you change the timeout. Getting your Linux admins to make the change may be another issue…