Cisco patch quells Duke's iPhone storms
Editor’s Note: This story is reprinted from NetworkWorld.
Cisco has just released a new security advisory that details what caused the address storms that recently afflicted Duke University’s wireless net.
The advisory, posted on the company’s Web site, says that Cisco’s wireless LAN controllers have “multiple vulnerabilities in the handling of Address Resolution Protocol (ARP) packets.” These vulnerabilities “could result in a denial of service (DoS) in certain environments.” The vendor is offering free software to patch this problem, and notes that “there are workarounds to mitigate the effects of these vulnerabilities.”
In keeping with Cisco’s standard format, the advisory makes no reference to the events at Duke, which were first reported a week ago. At the time, intermittent floods or storms of ARP requests were taking 20 to 30 WLAN access points off line for 10 to 15 minutes. The events involved the newly released Apple iPhone.
But a Cisco spokesman confirmed that the advisory deals with the problem uncovered at Duke. “To date, we have not seen widespread issues relating to Apple iPhone across our customers’ networks,” the spokesman wrote in an e-mail response.
The baffling problem, occurring at least nine times at Duke over about a week, triggered a wave of reader speculation, rants, and recommendations on Networkworld.com and other Internet tech sites.
The advisory finally makes it clear that the iPhone simply triggered the ARP storms that were made possible by the controller vulnerabilities. Any other wireless client device, moving from one subnet to another apparently could have done the same thing.
According to the advisory, the vulnerabilities are found in versions 4.1, 4.0, and 3.2 and earlier of the company’s Wireless LAN Controller software. Affected products include the 4100 and 4400 series of controllers, the earlier Cisco-Airespace 4000 series controller (introduced shortly after Cisco acquired Airespace ), the Catalyst 6500 series Wireless Services Module (WiSM, a single-board version of the controller), and the Catalyst 3750 Integrated Wireless LAN Controller.
Many other products are immune to these vulnerabilities, according to Cisco, including the 2000 and 2100 series controllers, various stand-alone access points, and the 3800, 2800, and 1800 series of Integrated Services Routers.
The identified vulnerabilities relate to a unicast ARP request which in certain circumstances can be flooded on the LAN links between a group of WLAN controllers (Cisco calls this a “mobility group”).
The advisory notes that IP Version 4 hosts use a method, specified in the IETF standard RFC 4436, to detect if they have re-attached to a network to which they had previously been attached. If so, the host may not have to request a new DHCP address lease if the current lease is still active, according to the advisory. To determine this re-attachment, the host host sends a unicast ARP request to the default gateway that it had previously used.
But the controller may mishandle this request, sparking the ARP storm. For this to happen, two vulnerable Cisco WLAN controllers, attached to the same set of Layer-2 VLANs, “must each have a context for the wireless client,” according to the advisory. That shared context can occur “after a Layer-3 (cross-subnet) roam by the client” or when a guest WLAN is in use.
“If the client sends a unicast ARP request with a destination MAC address that has not been learned by the Layer-2 infrastructure, that request will be flooded to all ports in the Layer-2 domain” after exiting the first WLAN controller. The second controller then reprocesses the ARP request and incorrectly re-forwards this packet back into the network.
If the ARP unicast features is enabled on the controller, the controller will re-forward broadcast ARP packets targeting the address of a known client context. “This creates an ARP storm if more than one [controller] is installed on the corresponding VLAN,” according to the advisory.
The trigger for the ARP storm or flood in a WLAN configured as described is apparently a wireless client that moves from one IP subnet to another. This was in fact the behavior noticed by Duke’s IT group, with a small number of iPhones. It was initially thought that the iPhone itself was generating the ARP flood.
Cisco will release software updates for versions 3.2 and 4.0 of the controller software on July 27. An update for version 4.1 apparently is now available from Cisco. Further, Cisco recommends that administrators require all clients to obtain their IP address from a DHCP server. To enforce this, all WLANs can be configured with a “DHCP required” setting. That will block any wireless client with a static IP address. The advisory notes this will “not be effective against deliberate attempts to craft packets that create an ARP storm.”
Duke CIO Tracy Futhey in a Web post last Friday revealed that the problem lay with Cisco equipment and not the iPhone itself, but did not provide details.