Network Notes - IBM PowerHA / HACMP

Some info on the networking features of HACMP (High Availability Cluster Multiprocessing). This is now called PowerHA SystemMirror for AIX. It allows up to 16 nodes in a cluster. As of v7.1 the cluster can use multicast to communicate, previous versions used UDP broadcasts. The cluster heartbeats are sent both via LAN and SAN for redundancy.

Terminology:

Boot IP: The address bound to the physical interface (e.g. ifconfig blah x.x.x.x).
Service IP: The VIP to which clients connect to hit the actual service, can exist on any interface on any cluster member.
Persistent IP: Used to reach a host for management. Also called node VIP and can exist on any interface on a single cluster member.
HWAT - Hardware Address Takeover: MAC address follows the IP when failing over.
IPAT - IP Address Takeover: Moves the service IP between interfaces and cluster members.

There are two methods of doing IPAT, via replacement and by alias.

IPAT via Replacement

This is the older method, it uses HWAT so no gratuitous ARP is required as the MAC address fails over with the service IP. However clearly port security cannot be used! You need two interfaces in the same VLAN, 1 configured with a real IP address (boot IP) and 1 with any IP (standby IP) that need not be routable. When HACMP starts it replaces the real IP address on NIC 1 with a VIP in the same subnet. The failover moves both VIP and MAC onto NIC 2. You can only have 1 service VIP per adapter pair.

IPAT via Aliasing.

The newer and recommended method, it requires a network that can support gratuitous ARP as HWAT is not used. The service IP is the only routable address needed. The 2 NICs are configured with IP addresses on different subnets that need not be routable. The service VIP is an alias address on the interface and fails over as an alias. You can have as many service VIPs as you want on an interface.

Heartbeats.

The boot IPs seem fairly pointless, however network heartbeats are broadcast/multicast from the boot IP so they should be allocated from the same subnet, an example allocation is:

Node service IP 10.0.0.10

Node 1
NIC1 boot IP 192.168.0.1
NIC2 boot IP 192.168.10.1
Persistent IP 10.0.0.101

Node 2
NIC1 boot IP 192.168.0.2
NIC2 boot IP 192.168.10.2
Persistent IP 10.0.0.102

Node 3
NIC1 boot IP 192.168.0.3
NIC2 boot IP 192.168.10.3
Persistent IP 10.0.0.103

Routing.

Any routes should be configured via the service IP subnet and persistent/node IP subnet. You should not use the boot addresses as they may not always be reachable (e.g. if NIC failover). You can use the service IP to manage the system but it might not be on that node if the cluster has failed over so better to use a persistent IP. Service and Persistent IPs can be on same subnet or different ones. If different then you'll either need multiple IPs configured on the VLAN interface or static routing configured on the AIX box as they'll both be in the same VLAN. I would KISS and have both on same VLAN & same subnet.

Read more...

Cisco ASA 8.4 - Global Access Lists

Handy new feature in version 8.4 of the ASA software is the ability to do global access lists.


The Cisco ASA allows security levels to be applied to interfaces, traffic is automatically allowed from a high to low security level interface but not vice versa. It's probably designed for the fairly common use case of a perimeter device between a LAN and the internet. The internet link is set to security level 0 and the inside interface to 100. All LAN traffic is then allowed to flow out. This is shown below:



Prior to version 8.3, access lists (ACLs) had to be applied on an interface and in a direction, e.g.

access-list MYACL extended permit tcp any any eq www
access-group MYACL in interface outside

As soon as an ACL is applied to an interface, it will pass traffic based on the ACL rather than based on security levels. However it gets complicated as traffic coming in another interface that would previously have been allowed is now still allowed, in the example above if you permitted port 80 in from the internet, all outgoing LAN traffic is still allowed.


Now in version 8.4, Cisco have added the ability to have a single global ACL that applies to all traffic regardless of which interface it uses. This is how most other firewalls work so a welcome change. To do this you create the ACL then apply it with:

access-list MYACL extended permit tcp any any eq www
access-group MYACL global

When a global ACL is applied, it removes all behaviours based on security levels from ALL interfaces. So in the original example, you would need a rule in your global ACL that permits LAN hosts access to the internet. The any/any rule is a good example of what not to do as this now globally means "any address" rather than specific to any particular interface.

Update 2015 It appears that not quite "ALL" security level behaviours are removed, you still need the same-security-level command to allow traffic to flow between interfaces regardless of ACL.

Read more...

Evaluation Assurance Levels - EAL

EAL stands for evaluation assurance level and is a certificate of security for IT products measured against a set of common security criteria. The main source of information on EAL levels is the common criteria portal where you can find details of approved products and information on the criteria used for the EAL certifications.

Who uses it?

Your average network bod may not come across EAL very often. It tends to crop up in areas that are regulated by government bodies such as CESG who will often require EAL4 certified products for certain secure environments. However you don't just buy EAL4 kit and be government approved, it fits into a much larger security framework such as ISO27k dealing with everything from who gets into the building to how you manage changes to IT systems.

How does a product get EAL certified?

It is assessed against a set of common criteria by an approved agency. The developer of the system produces a security target (ST) document containing a list of features to be assessed.The ST is based on the criteria here. The process is long and expensive, according to wikipedia vendors were spending $1 - $2.5million to gain EAL4 certification in the 1990s.

What do you get when EAL certified?

Certified products are listed on the common criteria portal along with the rating granted, the ST it was assessed against and the assessment report. e.g. here (PDF) is the ST for the Cisco ASA as a firewall and here (PDF) is the assessment report. Interesting to note that the EAL4 VPN certificate was issued separately, so an ASA acting as both firewall and VPN endpoint is not a valid EAL4 solution, strictly speaking you would need two in series performing each task.

So what does it mean in to a network engineer?

Probably not a lot, it's a policy requirement for many places but the assessment is only against the device, not against the specific implementation of it. You could deploy an EAL4 firewall with a policy of "permit any any" and it's still an EAL4 device! At that point the other security mechanisms should have stopped you from putting it on the network.

If you are involved in hardware selection for a regulated organization then you may need to use EAL4 devices in certain situations.

What is required to meet the various levels?

The EAL process is broken down to cover the following aspects of a system:
Development, documentation, life-cycle support, security target evaluation, testing, vulnerability assessment.

Each EAL level goes into slightly more detail, for example the "development" area at EAL1 requires a basic functional specification to be provided by the developer. EAL2 requires that same functional specification but expanded to include details of security inforcement. It also requires a security architecture description and a basic design. The specifics of those items are detailed here.

How long does it take to get EAL4?

It seems to vary from a very long time to aeons, certainly it's measured in years rather than months. A look on the NIAP CCEVS evaluation and evaluated list for firewalls shows a few examples:
Checkpoint R65 HFA01 on IPSO recorded as submitted Oct 2005 although R65 was released in 2007 so the process was started early during development. It passed March 2009. So that's 4 years to get certified and the product went EOL in March 2011, 2 years later.
Cisco ASA 8.3 as a VPN submitted November 2009 still not passed, predicted June 2011.
Palo Alto submitted various devices in December 2009 and still running.

What exactly is certified?

The certification is issued against a specific software release and hardware platform.

A specific version of the software you say? As in....minor version??

That is how the cert is written. The Cisco ASA obtained EAL4 for firewall purposes on version 7.0(6) of it's OS which was released in August 2006. Cisco have been patching and updating that for 5 years! The ASA is now up to release 8.4, which has been submitted again to CCEVS (scheme run by NIST and NSA) for evaluation.

In reality there will be a security assessor on the ground who will review the solution and hopefully be sensible about using a modern patched version of the OS and judge it acceptable to meet an EAL4 requirement, even if it's not strictly what's on the EAL4 certificate.

I don't know anyone who would tell you with a straight face that using a 5 year old OS on a firewall is going to increase your security!

What about high end firewalls?

There is a bit of a gap, if you need an EAL4 firewall with 10gig throughput then you're out of luck as the only one that's passed assessment is Checkpoint Power-1 on the 5075/9075, however that went end of life last month (March 2011). The closest is the Cisco 5580 which has been submitted for EAL4, due November 2011 and is arguably similar enough to the 5540 to be acceptable, however it's recently announced as being binned in preference of the 5585 so after August 2011 you can't buy one any more!

The security market moves quickly compared to the EAL assessments and it proves tricky.

The top end Cisco firewall platform is the 5585, not even showing as submitted for EAL evaluation yet.
Checkpoint has R71 under assessment now, predicted result in November 2011.
Palo Alto has various items aiming for November 2011, but their flagship model the PA-5000 is not listed as under assessment, it only recently hit market in the UK so EAL certification may not have been discussed yet.
Juniper have EAL4 for their ScreenOS platform the SSG, which goes EOL in 2013. They have EAL3 for Junos 9.3 on the SRX platform, the current version is 10.4. There doesn't appear to be any indication that the SRX security platforms have been submitted for EAL4 certification, although it would be surprising if that were the case as governments would be ditching Juniper en-masse before 2013.

So until November 2011 there are no EAL4 10gig firewalls. You'll have to build a farm of 1gig ones instead!

What alternative schemes are there?

FIPS-140 from NIST.
CAPS, the CESG Approved Product Scheme.

Is it worth me buying EAL4 products?

If you have to ask then probably not. If your business is regulated and the agencies setting those policies define EAL4 as a requirement then you have no choice.

For companies with the option I would say it's a helpful indicator but I would certainly use other aspects above the EAL status when selecting a device:

• Performance.


• Price.


• Published security tests and exploits.


• Staff familiarity.


• Internal testing.

An EAL4 certificate does indicate that the product was developed following good practices and has a well defined and documented architecture. These are clearly good things in terms of stability and security. However not having EAL4 doesn't necessarily mean the product hasn't followed a good development process and isn't secure, it just means the manufacturer hasn't paid for it to be assessed.

Read more...

Legacy FRTS & Subinterfaces

FRTS and subinterfaces. This page follows on from the previous article on legacy FRTS configuration here and shows the default behaviour of FRTS with subinterfaces.

The legacy frame-relay traffic shaping has to be enabled on a physical interface. Any subinterfaces will then inherit the configuration, which is 56kbps by default. The network is shown below:



In the example below FRTS is turned on but not configured, both subinterfaces are then shaped to 56kbps (using screenshots as the output to "show traffic-shape" doesn't like this sites layout).

R1#show run | begin interface Serial0/0
interface Serial0/0
no ip address
encapsulation frame-relay
no fair-queue
clock rate 2000000
frame-relay traffic-shaping
!
interface Serial0/0.102 point-to-point
ip address 192.168.12.1 255.255.255.0
snmp trap link-status
frame-relay interface-dlci 102
!
interface Serial0/0.103 point-to-point
ip address 192.168.13.1 255.255.255.0
snmp trap link-status
frame-relay interface-dlci 103

As shown below, the target rate is 56000b/s



This config sets a map on one of the subinterfaces shaping it to 2mbit:

map-class frame-relay TEST_MAP
frame-relay traffic-rate 2000000 2000000


interface Serial0/0.102
frame-relay class TEST_MAP

The remaining subinterface remains at 56kbps:



You can apply the map to the physical interface, the sub-interfaces then inherit these settings:



Applying other maps to the subinterfaces overrides any inherited settings:

map-class frame-relay TEST_MAP_2
frame-relay traffic-rate 128000 128000


interface Serial0/0.103
frame-relay class TEST_MAP_2

Read more...

Frame Relay Traffic Shaping - Legacy Configuration

This is a basic lab to play around with frame-relay traffic shaping, FRTS. It uses the legacy configuration method rather than MCQ. INE have a great article here describing the other options.


This article assumes some knowledge of QoS terms such as CIR, Bc, Be and Tc.

The lab used looks like this:



I'll use the GNS3 built in frame switch to make life easier, the config is below:



The basic router configurations are:

hostname R1
!
interface Serial0/0
ip address 192.168.0.1 255.255.255.0
encapsulation frame-relay
clock rate 2000000


hostname R2
!
interface Serial0/0
ip address 192.168.0.2 255.255.255.0
encapsulation frame-relay
clock rate 2000000

In this mode no shaping is enabled, WFQ is the default for serial interfaces below E1 size (2.048mbps).

R2#show int s0/0
Serial0/0 is up, line protocol is up
Internet address is 192.168.0.2/24
Encapsulation FRAME-RELAY, loopback not set
Queueing strategy: weighted fair
Output queue: 0/1000/64/0 (size/max total/threshold/drops)
Conversations 0/1/256 (active/max active/max total)
Reserved Conversations 0/0 (allocated/max allocated)
Available Bandwidth 1158 kilobits/sec

To turn on FRTS use the commands as below:

R1(config)#int s0/0
R1(config-if)#frame-relay traffic-shaping

This gives the interface a default configuration, which is 56kbps and has Bc set to 7000bits. This can cause problems with subinterfaces as they'll end up at 56k unless configured otherwise. The queuing method is also changed to FIFO.

R1#show int s0/0
Serial0/0 is up, line protocol is up
Internet address is 192.168.0.1/24
Encapsulation FRAME-RELAY, loopback not set
Queueing strategy: fifo
Output queue: 0/40 (size/max)

R1#show traffic-shape

Interface Se0/0
       Access Target    Byte   Sustain   Excess    Interval  Increment Adapt
VC     List    Rate     Limit   bits/int bits/int  (ms)      (bytes)  Active
102            56000     875    7000      0         125       875       -

The actual configuration is done in a class map:

R1(config)#map-class frame-relay TEST_MAP

The options are configured using the frame-relay command:

R1(config-map-class)#frame-relay ?
adaptive-shaping Adaptive traffic rate adjustment, Default = none
bc Committed burst size (Bc), Default = 7000 bits
be Excess burst size (Be), Default = 0 bits
cir Committed Information Rate (CIR), Default = 56000 bps
congestion Congestion management parameters
custom-queue-list VC custom queueing
end-to-end Configure frame-relay end-to-end VC parameters
fair-queue VC fair queueing
fecn-adapt Enable Traffic Shaping reflection of FECN as BECN
fragment fragmentation - Requires Frame Relay traffic-shaping to be
configured at the interface level
holdq Hold queue size for VC
idle-timer Idle timeout for a SVC, Default = 120 sec
interface-queue PVC interface queue parameters
ip Assign a priority queue for RTP streams
mincir Minimum acceptable CIR, Default = CIR/2 bps
priority-group VC priority queueing
tc Policing Measurement Interval (Tc)
traffic-rate VC traffic rate
voice voice options

There are a couple of ways to shape traffic, the traffic-rate command sets the rate & peak rate, IOS then calculates Bc and Be based on a time interval of 125ms. To set the rate to 128kbps and the peak rate to 256kbps:

R1(config-map-class)#frame-relay traffic-rate 128000 256000
R1(config-if)#^Z
R1#show traffic-shape
Interface Se0/0
       Access Target    Byte   Sustain   Excess    Interval  Increment Adapt
VC     List    Rate     Limit   bits/int bits/int  (ms)      (bytes)  Active
102            128000  18000   128000    128000    125       2000 -

Note that Tc (interval) is still 125ms.

IOS then calculates Be as being Tc * (PIR - CIR), which is .125 * (256000 - 128000) = 16000.



You can also specifically configure the committed information rate (CIR) and Burst Excess (Be) in the map-class, this allows you to change the value of Tc which is calculated as Bc/CIR as below on R2:

map-class frame-relay TEST_MAP_R2
frame-relay cir 128000
frame-relay bc 12800

R2#show traffic-shape

Interface Se0/0
       Access Target    Byte   Sustain   Excess    Interval  Increment Adapt
VC     List    Rate     Limit   bits/int bits/int  (ms)      (bytes)  Active
201            128000    1600   12800     0             100       1600 -

You can also see the shaping configuration by looking at the PVC:

R2#show frame pvc 201

PVC Statistics for interface Serial0/0 (Frame Relay DTE)

DLCI = 201, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0

input pkts 8 output pkts 7 in bytes 622
out bytes 588 dropped pkts 0 in pkts dropped 0
out pkts dropped 0 out bytes dropped 0
in FECN pkts 0 in BECN pkts 0 out FECN pkts 0
out BECN pkts 0 in DE pkts 0 out DE pkts 0
out bcast pkts 2 out bcast bytes 68
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
pvc create time 00:27:55, last time pvc status changed 00:27:55
cir 128000 bc 12800 be 0 byte limit 1600 interval 100
mincir 64000 byte increment 1600 Adaptive Shaping none
pkts 1 bytes 34 pkts delayed 0 bytes delayed 0
shaping inactive
traffic shaping drops 0
Queueing strategy: fifo
Output queue 0/40, 0 drop, 0 dequeued

Read more...

Zone Based Firewall & Port Forwarding

This article covers setting up port forwarding with Cisco Zone Based Firewall (ZBF) on a typical home connection.

There are a couple of steps:
1 - Give your LAN host a static IP.
2 - Set up NAT to handle the port forwarding
3 - Set up ZBF rules to allow the traffic

1 - Static IP

You can either manually configure the client or use a DHCP reserved address.

DHCP reservation is a royal pain on the Cisco 800 series, make sure there are no existing bindings when you try to configure it (show ip dhcp bindings), the basic configuration is:

ip dhcp pool MATT_PC
host 192.168.0.10 255.255.255.0
client-identifier 01aa.aabb.bbcc.cc
client-name Matt-PC

The client-identifier is the VLAN ID prepended to the MAC address. If that fails then you can try using "hardware-address aaaa.bbbb.cccc", strangely for my dual boot system Linux only picks up the reserved address using hardware-address config and Windows 7 only works using the client-identifier option. Use whatever works!

2 - NAT forwarding

You need several bits of information, the name of the external interface, the static IP used internally, the port and protocol (TCP/UDP) that you wish to forward. The format is:

ip nat inside source static <protocol> <LAN-IP> <port> interface <external-interface> <port>

e.g. a DSL router using Dialer0 interface to forward UDP traffic on port 88 to 192.168.0.10 is:
ip nat inside source static udp 88 192.168.0.10 interface dialer0 88

3 - Set up ZBF rules to allow the traffic

This is the actual Zone Based bit. You'll need to understand the setup to tweak it, no simple guides I'm afraid.

You need to know the name of your zones, I'm using the default setup which has called the outside/internet "out-zone" and internal/LAN "in-zone". It will show up in the interface configuration under the "zone-member security XXXX" option (e.g. found with "show run interface dialer0")

You basically make a policy, then apply that policy to traffic going between two zones.

You can use CBAC with the "inspect " command but in this case I'll use an access-list instead. ACL may be your only choice if the protocol you are forwarding is not one supported by CBAC. In this case it's Xbox-live or Games for Windows so one port may be kerberos (88) but the other is not known so an ACL is used. The steps are:

3a) Set up the ACL

I'm doing XBL so I need UDP/88, UDP/3074 and TCP/3074:

ip access-list extended GFW_Incoming
permit udp any host 192.168.0.10 eq 88
permit tcp any host 192.168.0.10 eq 3074
permit udp any host 192.168.0.10 eq 3074

3b) Set up a class-map to match the ACL

class-map type inspect match-any Incoming-Traffic
match access-group name GFW_Incoming

3c) Create a policy saying what to do to the traffic

policy-map type inspect incoming-policy
class type inspect Incoming-Traffic
pass
class class-default
drop

Note that "pass" means let it through. Inspect means run CBAC on it, but it must be a recognised protocol. "Drop" should be clear enough, you could include "drop log" but it will never log anything (see note * at end).

3d) Tell ZBF where the policy applies, specifically between which zones.

zone-pair security Outside-to-Inside source out-zone destination in-zone
service-policy type inspect incoming-policy

And that should be it!

To help troubleshoot, you can do show ip access-list GFW_Incoming and see if any packets are being matched.


Final note - why drop log doesn't log anything in this case.
ZBF creates a firewall policy for traffic going between two zones. In this case our policy is for traffic going between the out-zone (WAN) and the in-zone (LAN). The traffic is "routed" between those zones by the NAT rules doing the actual port forwarding.

Traffic that comes from the internet (out-zone) and hits the firewall without triggering a NAT rule is not going anywhere near the in-zone. The relevant policy would be out-zone to self-zone (the router itself).

So the only time our out-zone to in-zone policy can ever drop traffic is if the NAT rules are forwarding but our policy does not match (the ACL in this example), i.e. if it's configured wrong! If you want to log dropped traffic then you need to specify a policy for out-zone to self-zone and use logging on that.





Read more...

MST - Multiple Spanning Tree - Don't change the mappings!

MST allows you to create spanning-tree instances and map VLANs into them.

Combined with VTP version 3 means you can advertise the MST mappings automatically, as shown here.

MST has the concept of regions.

Whether a switch is a member of a particular region depends on three things:

• The configured MST region name.


• The configured MST revision number.


• The VLAN to MSTI mappings.

The entire VLAN to MSTI mapping isn't advertised in each BPDU but a checksum of the mapping table is.

So why do I care?

Because if you change the mappings, you change the region.

If you change the region that the root bridge belongs to, it's a topology change and you trigger a total STP re-convergence.


So if you're in a live network and you tweak the VLAN-to-MSTI mappings then you'll cause a complete outage. If it's running VTPv3 then the outage will be longer as the change ripples through the network and switches "re-join" the region.

The solution

The solution in this case is fairly easy, set up all your mappings on day 1 and stick to them!

If you've two switches then you're best off finding a suitable way to distribute the VLANs, e.g.

MSTI 1 - VLANS 1 to 1999 - Root bridge SW1
MSTI 2 - VLANS 2000 to 4096 - Root bridge SW2

Then make sure you set up HSRP/VRRP in future to match where the MSTI is rooted for each particular VLAN.

Read more...