IBM Blade Centers - CIGESM Configuration

There are lots of docs from IBM explaining the blade center architecture but it's a lot of information to filter through if you're only interested in the network side.

The blade center in this example holds 14 blades which are best thought of as individual physical servers. It can use the Cisco switching module called a Cisco Systems Intelligent Gigabit Ethernet Switch Module, or CIGESM. They do run IOS but they don't behave in quite the same as normal Cisco switches.

Architecture

The CIGESM is a Cisco switch with 20 Gigabit Ethernet ports, these are:

• 4 Physical ports on the back of the chassis.


• 14 Ports connecting to the individual blades.


• 2 Ports connecting to the internal Management Module for managing the switch via the chassis web interface.

Each of the blades is dual homed into the two CIGESM modules.

This architecture of a single blade center is shown below, click for larger version (warning 400kb image). This one assumes two management modules and two CIGESMs.

Management

The blade center itself is configured via a web interface that runs on the management module (MM). To access this initially you connect a copper cable directly into the MM port (has a picture of a network next to it), give your laptop an address in the 192.168.70.0/25 range and browse to 192.168.70.126. The default login is USERID/PASSW0RD (zero).

The CIGESM is a layer 2 switch and has a defined management interface, with the command "management" under the SVI. The Blade Center Management Module (MM) will configure the IP address on this interface. You can change it on the Cisco CLI but it will probably get set back by the MM at some point in future, so set it to the same on both MM and CLI!

CIGESM Management SVI

If you want to change the management interface VLAN then you'll need to do the following:

1. If the IP is changing then update the network settings via the blade center web interface first.


2. Create the layer 2 VLAN (for applying config to ports 15/16).


3. Create the layer 3 SVI.


4. Run the command "management" under the layer 3 SVI.


5. Delete the old management interface.

The web interface screen is shown below (names changed to protect the innocent), the network settings are under "I/O Module Tasks" -> "Configuration". Click for larger image:




An example of changing the management VLAN from 1 to 2, note the auto updates to Gi0/15&16 (explained later):

BC-Sw01(config-if)#int vlan 2
BC-Sw01(config-if)#management
Port Gi0/15 allowed vlan list updated to include vlan 2
Port Gi0/16 allowed vlan list updated to include vlan 2

BC-Sw01(config-if)#desc Management Interface
BC-Sw01(config-if)#ip address 192.168.0.25 255.255.255.128
BC-Sw01(config-if)#do show run interface vlan 2
interface Vlan2
description Management Interface
ip address 192.168.0.25 255.255.255.128
no ip route-cache
management
end

Internal Connections - Gi0/15 & Gi0/16

As shown in the architecture diagram above ports 15 & 16 are internal links to the management modules. These are used for communications between the MM and the switch, for instance if you click the "telnet" link within the MM web interface.

The management VLAN needs to be allowed over the internal link to the MM. Some parts of the configurations on these ports cannot be configured manually, such as access or native VLAN. These entries are set automatically when you specify an SVI for management.

If you try to reconfigure them on the CLI you'll see errors as below:

BC-Sw01(config)#int range gi0/15 - 16
BC-Sw01(config-if-range)#switchport access vlan 2
User prevented from modifing Ethernet port Gi0/15 access vlan

Command rejected: not allowed on this interface.User prevented from modifing Ethernet port Gi0/16 access vlan

Command rejected: not allowed on this interface.
BC-Sw01(config-if-range)#switchport trunk native vlan 2
User prevented from modifying Native VLAN on Port Gi0/15

Command rejected: not allowed on this interface.User prevented from modifying Native VLAN on Port Gi0/16

Command rejected: not allowed on this interface.
BC-Sw01(config-if-range)#switchport trunk allowed vlan 2
BC-Sw01(config-if-range)#