Cisco Gotchas - Max VLANs and STP Instances

Cisco switches have separate limitations on:

• The number of VLANs that can exist in the database.


• The number of Spanning-Tree Instances that can run.

Cisco kit tends to use per-VLAN spanning-tree in which case the two values will be the same.


To understand the problem and solution requires knowledge of the different types of spanning-tree available, this subject is huge but very briefly the types are:

1. Standard 802.1D Spanning-Tree


2. PVST, Per-VLAN Spanning-Tree (ISL trunking only)


3. PVST+, Per-VLAN Spanning-Tree plus (compatible with 802.1q trunks)


4. Rapid PVST - RSTP version of PVST+


5. MST - Multiple Spanning Tree

The limitations are as follows on switches running PVST, PVST+ or Rapid-PVST:

• 2950 SI: Maximum 64 STP instances, Maximum 128 VLANs.


• 2950 EI: Maximum 64 STP instances, Maximum 250 VLANS.


• 3550, 3560, 3750: Maximum 128 STP instances, Maximum 1005 VLANs.


• 6500: Based on logical ports. Article here.

If you exceed the number of VLANs then you'll get an error like this:

SPANTREE_VLAN_SW-2-MAX_INSTANCE: Platform limit of 64 STP instances exceeded. No instance created for VLANxxx

Solutions

There are a couple of workarounds:

• Delete some VLANs!


• Manually prune VLANs off the trunk links and set the switch to VTP mode transparent so it doesn't know about the extra VLANs.


• Use MST instead of PVST.

MST is my preferred option. Most networks will only require 2 or 4 Spanning-Tree topologies anyway so it is wasteful to have STP running in every VLAN.




Read more...

Cisco Gotchas - 16 Unique Standby Groups

HSRP is configured in standby groups, each one having an identifier value in the switch configuration.

The acceptable range of values is 0-255 and you can create as many as you want. Cisco don't recommended using more than 500 on a 6500 series or 64 on the 3550.

There are limitations on the number of unique ID values you can assign however.


Each instance is configured using the standby command:

interface FastEthernet 0/10
ip address 10.0.0.3 255.255.255.0
standby 10 ip 10.0.0.1
standby 10 preempt

The gotcha is that you can create as many groups as you want, but you can only use 16 different values for the ID.

This is hardware/software specific, but it's not clear what hardware it's specific to. The Cisco FAQ says it only affects PFC2, but their bug tracker doesn't mention the PFC and has a "fixed in" entry indicating that it's not an issue with newer software.

So feel free to create as many unique IDs as you like, but when adding the 17th check the logs to make sure it's actually working!

Or you could use VSS and get rid of HSRP entirely.

The Cisco FAQ for this issue on the 6500 is here.
There is a bug open on the Cisco tracker, CSCdx32496.


Read more...

Configure IOS by SNMP - Password Recovery

This is useful if you ever end up without a password for a router but you do know the SNMP read-write community values or usernames. It can save you the need to reboot a device and do a full password recovery, useful if the device is a thousand miles away and running some vital service.

You can load a configuration snippet into IOS from a TFTP server, triggered over SNMP.

The steps are:

Warning:

This can seriously b0rk your equipment and I accept no responsibility if it doesn't work for you. If your device is thousands of miles away, running critical services and you cannot login then your network has bigger issues than bodging the username on this one device! Don't do this unless you understand the possible consequences of blindly applying raw config to a Cisco device.

1. Create a text file containing the config snippet.


2. Run a TFTP server, on machine with IP 10.0.0.10 in this example.


3. Set the value of iso.org.dod.internet.private.enterprises.cisco.lo cal.lsystem.hostConfigSet.10.0.0.10 to a string containing the filename from #2

The config snippet could be something like:

username cisco priv 15 password cisco

The command line for NET-SNMP tools snmpset is:

snmpset <router> <community> .iso.org.dod.internet.private.enterprises.cisco.local.lsystem.hostConfigSet.10.0.0.10 s <filename>

You can do a write mem via SNMP too if you want (but I wouldn't recommend it):

snmpset <router> <community> .iso.org.dod.internet.private.enterprises.cisco.local.lsystem.writeMem.0 i 1

If you can't login to a device and have absolutely no idea why not then it may not be as simple as adding a new local account. The specifics may be guessed at depending on how standardized your network is and you may be able to guess at the extra bits (heed the warning above), e.g.

no aaa new-model
username cisco priv 15 password cisco
line vty 0 15
login local
password cisco

Read more...

VSS Introduction

VSS stands for Virtual Switching System and is a technology for use on 6500 series switches.

It works in a similar way to stackwise on the 3750s, you have two physical devices that end up with a single logical management plane.


In a large Cisco-style network you would have two core/distribution devices for redundancy. The network topology will typically need an active/standby topology. With Spanning-tree, only one uplink is forwarding for a given VLAN. To avoid unicast flooding the network should then have HSRP/VRRP and the IGP configured so that all traffic for that VLAN is routed via the same distribution (or core) switch.

Traffic is load balanced by alternating VLANs between the two distribution switches.

This topology is shown below for a single VLAN:



The basic VSS topology is shown below, the access device still has redundant uplinks but traffic flows over all links in etherchannels:




This topology it produces is similar to a network built around Nortel kit and the Split Multi-Link Trunking (SMLT) technology. The difference being that SMLT still has two separate management planes but VSS only has one.

There are a few benefits to this system:

Simplified Network Architecture.

• Single logical device at the core/distribution layer.
• Behaves as a single unit for management purposes.
• No need to fudge STP, HSRP and IGP per VLAN.
• No need of first hop redundancy protocols (FHRP).

Simpler Routing

• Half as many IGP neighbors.
• Half as many IGP routes.
• No IGP load balancing/weighting required.

ISSU.

• Software upgrades carried out without risky failovers and associated downtimes during convergence.
• Easier to roll back.
  

Faster Convergence.

• No STP convergence delays.
• Loss of redundant switch does not change logical topology, avoiding convergence entirely.

Reduced function of Spanning-Tree.

• No blocked ports to distribution/core from access layer.
• Reduces greatly the chance of bridging loops.
• Can use portfast trunk on access device uplinks.
  

The Cisco VSS Design Guide is well worth a read for more detailed information.



Read more...

3750 Stackwise Upgrades

Upgrading a Cisco switch is very easy:

1. Copy IOS image to the flash drive on the switch.
2. Point the boot statement at the new image.
3. Reboot the switch.

The 3750 stack is made up of several physical devices with just a single logical management interface so Cisco has implemented the archive-sw command to make upgrading it easy. It doesn't quite work as well as it should and I wouldn't recommend using it. The method below is much easier and more reliable:

1. Copy IOS image to the flash drive on the switch
Use the same command as usual, but run it once for each switch in the stack. The flash drives are numbered flash1: flash2: flash3: etc.

copy tftp flash1:
copy tftp flash2:

Delete any old images or html files to clear space as necessary.

2. Point the boot statement at the new image
This can be done with a one liner (replace image.bin with whatever file you want to use):

boot system all flash:image.bin

3. Reboot the switch
The "reload" command will restart the entire stack.

So what's wrong with archive-sw?
The upgrade process is just the 3 simple steps above. Last time I used archive-sw I had about 45 minutes of instability while various individual processes restarted on the switch, routing went down but basic IP services stayed up so HSRP couldn't fail over and the network was unreachable until archive-sw had finished doing whatever it was doing. Maybe it was a bug in the archive-sw version I was using, maybe the config on that device triggered it, maybe it's just not very good.

Either way, I'm sticking to the old method of upgrades because it works and only causes an outage for the time it takes the switch to reboot (assuming it comes up again, which doesn't always happen with stacks!).


ISSU - In Service Software Upgrades
ISSU allows devices to be upgraded without outage. Unfortunately it is not available for 3750 stacks.
It may be possible to carry out the instructions above and then just reboot one switch at a time in the stack, if the old and new versions are similar enough then you might get away with it and the stack might re-form, allowing you to reboot the other switch while maintaining connectivity. Personally I wouldn't try it, if you want maximum uptime then you need two separate logical switches or to forget about upgrading them.


This method was worked out with our partners are Cacti Networks Ltd, network and security providers in the Ross-on-Wye area.

Read more...

Rapid Spanning Tree Notes

Just been reading a bit on RSTP, there are plenty of excellent detailed resources around. Here's my summary.




PDF version available here

Rapid Spanning Tree - 802.1w

Port Roles

Role	Description	Detailed
Root	Path to root bridge	The port that is closest to the root bridge in terms of path cost.
Designated	Forwarding to a segment	The port that sends the best BPDU on the segment to which it is connected.
Alternate	Alternative path to root	Alternate route to the root bridge. This port receives more useful BPDUs from another bridge. State blocked. Can be activated if the root port fails (RSTP equivalent to uplinkfast).
Backup	Redundant port	This port receives more useful BPDUs from the local bridge, therefore is redundant. State is blocked.
Disabled	Shutdown	Administratively shut down.

Port States

State	Description
Discarding	Combines the 802.1D disabled, blocking and listening states.
Learning	Drops frames but learns MAC addresses.
Forwarding	Sends frames

Port Types

Type	Description
Edge	End host, selected when portfast is enabled.
Root	Path to root bridge.
Point-to-point	Designated port to another switch. Selected when non-edge port is in full duplex mode.

• It is assumed that a non-edge port in half-duplex mode may connect to a shared medium with multiple peer switches. These ports will fall back to 802.1D compatibility mode.

• A non-edge port transitioning to forwarding will generate a TCN.

BPDUs

• All switches send BPDUs out of all ports.

• Loss of 3 BPDUs from a neighbour is taken to mean the neighbour is down. All information on that port is aged out.

• If an 802.1D BPDU is received then that port falls back to 802.1D mode from RSTP.

Read more...

DSL Firewalls and PPP Half Bridges

A PPP Half Bridge lets you extend the public IP address of your network into the LAN. This is useful if you have a firewall that doesn't have it's own DSL modem and want to do VPNs.

It's also known as DHCP spoofing.


Half bridging is a hack where your DSL modem makes the PPPoA connection to your ISP and is issued a public address, then advertises that same IP back on the LAN with a very short DHCP lease time. It then goes into bridging mode and forwards all traffic to the LAN host. The LAN host is effectively directly connected to the internet.

This is useful if you want to avoid NAT for any reason, e.g. to terminate VPNs.

The topology is shown below:



There's a big cost benefit to connect a firewall device to the DSL connection without requiring a built-in DSL modem.

Cisco's firewall line is the ASA which has no DSL module available so the only option is to use an external ADSL router. The low-end box is the 5505 which can be picked up from around £250 depending what software license you want.
They also have the 857W (~£300) and 877W for around £400 which include both wireless, VPN capability and DSL ports. However these are routers, not firewalls. They may be able to run the IOS firewall software but the performance of that is not likely to be great.

Juniper used to make a 5GT-ADSL with wireless that did the job nicely, but has now been replaced by the SSG 5 at around £650 with wireless but no DSL modem available. The only all-in-one option is to buy an SSG 20 (~£800) and an ADSL expansion card (~£500). The 5GT are still available from ebay for around £2-300 if you don't mind no support.

The home-brew solutions such as pfsense, Smoothwall and IPCop will also benefit from using the half-bridge option. Run it on something like a PC Engines mini system which cost under £100 and you'd have a very powerful little box.


Compatible Modems
Not all ADSL routers support half bridging, it's probably fair to assume that most don't. I use an old Origo 8400 which aren't available any more, I think that company is now called Safecom and they do pop up on ebuyer as the value range from time to time.

The Zyxel P-660R-D1 supports half bridging and is available from around £25 online.

According to certain forums the Linksys AM200 supports it as does the Netgear DG834G (but not GT). Some Draytek models also support it.

Read more...