Monday 24 May 2010

SNMP Trap on VSS Failover

One of the big things that Cisco VSS is missing is the ability to clearly see when it's failed over.

You set up your spanking new 6500 with 10Gig supervisors, plug it into your network management, lose one of the boxes and get a few traps about routing problems. What you really want is a big in-your-face message saying "HELP ME THE VSS JUST FAILED OVER OH MY GOD THE SKY IS FALLING!!11!one" and it's not there. It'd be fairly easy to miss that anything happened at all.

One fix is to use the Embedded Event Manager (EEM) and hook it on one of the syslog messages. There's no obvious "WAAAH HELP VSS JUST DIED" message in syslog either, but it does have a few that pop up on failover such as this one (which also occurs on first VSS bootup but I'm not too fussed about an extra trap in that case):

%PFREDUN-SW2_SPSTBY-6-ACTIVE: Initializing as Virtual Switch ACTIVE processor


EEM lets you create an action that runs when this message appears in the log, I'll make it send an SNMP trap and write a clearer syslog. The following code is beerware, feel free to re-use and re-distribute. I'd appreciate a beer if you meet me (or at least link this site):

event manager applet VSS_active_state_change
event syslog pattern ".*%PFREDUN-.*_SPSTBY-6-ACTIVE.*"
action 1.0 snmp-trap strdata "VSS Failover - New Active VSS Supervisor"
action 1.1 syslog priority alerts msg "VSS Failover - New Active VSS Supervisor"


Now you get an SNMP trap and a SYSLOG saying:

*May 24 19:49:36.267: %HA_EM-1-LOG: VSS_active_state_change: VSS Failover - New Active VSS Supervisor


You can change the %HA_EM to something else with the "facility XXX" option in the syslog line, but it doesn't seem to be present on the 6500 with SXI3 IOS.

This does need EEM traps enabled, or you still won't have any alerts:
snmp-server enable traps event-manager