Sunday, 22 October 2023

Linux Shortcuts and Auto Updating AppImages

A minor annoyance with software deployed as AppImages that auto update is each update breaks any shortcuts you've created.

The trick is not to create the shortcut directly to the file, e.g.

  /home/me/software/myappname-v32831.38474-linux-x86_64.AppImage 
  


Instead create the shortcut to run the "find" command so it locates any file in that folder starting with the same text and launches it, regardless of the version number.

find /home/me/software/ -name myappname-* -exec {} \; 


Could introduce problems so there are a few assumptions/caveats:
  • The app is saved/run from /home/me/software/ in the above example
  • You're aware this is just launching any file in that folder with the right name, so if other users can access it then they could replace it and you'll launch their file
  • The AppImage update process needs to delete the old file, otherwise you'd have to modify the find command to sort by date

Read more...

Saturday, 22 January 2022

3rd Party Firewalls in Azure

You can use 3rd party firewalls in Azure but there are some differences in how High Availability works.

Standard firewall H/A works via lower level network communication to move IP addresses between devices (e.g. gratiutous ARP), but the underlying network in Azure/AWS/etc won't support that approach. There are APIs to inform Azure that an IP address has moved to a different device, but at time of writing this approach results in very slow failover (1min+).

The current vendor pattern architectures use a load balancer and two separate active firewalls to provide resilience. In Azure there are two main types of Load Balancer:
  • "Public" which resembles load balancer on a stick (or one-armed load balancing) without SNAT, just changes the destination IP.
  • "Private" or "Internal" which is a sort of software defined version of routed load balancing, traffic is forwarded to the backend pool members but the destination IP is not changed.

Inbound Flows

For an inbound connection to a public IP address that fronts a single service, use a Public load balancer with the firewalls set as the backend. The load balancer will then re-write the destination address to whichever firewall it decides to use. This needs the firewall to DNAT traffic to the actual destination, and SNAT to ensure return packets go via the same firewall. This looks like the below, red text refers to IP addresses:
This is fairly limited because not many networks would deploy a pair of firewalls to front a single service. Multiple services are more complicated and there are a few big constraints:
  • The firewall normally uses destination IP address to direct traffic, but in the above scenario the load balancer has set the destination IP as the firewall itself.
  • With typical one-armed load balancers you can have a different SNAT address for each service, but Azure Load Balancer doesn't do SNAT.
  • A "private" type load balancer would maintain the original destination IP address, but you can't apply public IP addresses to them, Microsoft's use-case for those is strictly internal.
  • A "public" type load balancer could be put infront of a private one to have separate load balancers for service and firewalls, but the backend pool hosts need to be on the local subnet with the Azure Load Balancer.
The only option for non-web traffic is to use the "public" type load balancer with either a separate IP addresses on every firewall for every service, or a separate port on every firewall for every service, which will get complicated fairly quickly so will become problematic at scale:
For web traffic a nicer solution is to use an Application Gateway on the outside to load balancing the service (backend pool = actual servers) and the inside a private/internal load balancer doing the firewall H/A (backend pool = firewalls). The destination IP address for the entire flow is the backend server, so no DNAT is required and neither is one IP on every firewall per service.
There is also a feature "gateway load balancer" that seems to do away with the need for SNAT on the firewalls but I've not played with it yet: https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/dmz/nva-ha.

Outbound Flows

Outbound traffic can be load balanced using a "private" load balancer sat on the "inside" of the network, which maintains the original destination while providing H/A via the available firewalls. This looks the same as the first diagram.

References

Vendor Model Architectures
https://docs.fortinet.com/document/fortigate-public-cloud/6.0.0/use-case-high-availability-for-fortigate-on-azure/224311/basic-concepts
https://www.paloaltonetworks.com/resources/reference-architectures
https://blogs.cisco.com/security/secure-cloud-for-azure

F5 docs on cloud failover, Azure seems slower than other platforms: https://clouddocs.f5.com/products/extensions/f5-cloud-failover/latest/userguide/performance-sizing.html MS docs on public/private load balancer in Azure:
https://docs.microsoft.com/en-us/azure/load-balancer/components

Read more...

Thursday, 20 May 2021

GDPR and Appropriate Security Controls

GDPR article 32 requires "appropriate" controls to protect personal data, but what exactly is appropriate? The ICO has published various cases that can be used to gauge their expectations.

Having a risk assessment helps to qualify the approach and show due dilligence, but it's a subjective process so results will vary and if the ICO disagree with the outcome then the financial penalties can be large. Marriot claimed a risk based approach supported their decision to focus database monitoring and encryption on card holder data, however the ICO disagreed with that conclusion and held them liable for not giving personal data the same level of protection.

Industry regulations such as PCI can help indicate that controls are "appropriate".

Conclusion

The ICO appear to take the state of the art as their baseline and seem to have a fairly idealistic view on implementing enterprise security. They look at recommendations from bodies such as NCSC, NIST, etc alongside industry regulations and consider any deviation from those as an indicator of negligence which increases the liability. Sensible efforts will be considered such as Marriot's MFA implementation, which ultimately turned out to be incomplete but that was not considered in the judgement as an independent audit had informed them the control was in place.

A few particular controls were called out in multiple cases and should be on any organisations radar:
  • Application Whitelisting.
  • Multi-factor authentication.
  • Detection of configuration change.
  • Privileged access management (PAM) and implementation of least privilege.
  • Risk Assessment of personal data storage and processing.
  • Awareness of good practice and current issues with technologies in use.
  • Strict control over remote access.
  • Compliance with internal security policies and relevant industry regluation.
Network segregation was also discussed in some cases, but highlighting that segregating the IP network is not the whole story if the same Active Directory is permitted to all network areas. This is an important consideration for organisations who may be implementing segregation, particularly post the Maersk NotPetya incident.

In terms of specific items called out with some big cases that indicate expectations of controls are:

Ticketmaster

https://ico.org.uk/media/action-weve-taken/2618609/ticketmaster-uk-limited-mpn.pdf
  • Hacked via 3rd party Javascript chat bot they'd included on their website, 3rd party was then compromised.
  • The 3rd party had ISO27001, this was not considered relevant by the ICO as it's not a software security standard.
  • The ICO used blog posts and Stackoverflow questions about risks from including 3rd party Javascript on website as evidence that this was a recognised issue, combined with supply chain articles by NIST and NCSC.

BA

https://ico.org.uk/media/action-weve-taken/mpns/2618421/ba-penalty-20201016.pdf
  • Referenced CPNI GPG from April 2015 supply chain guidance on assessing risk
  • Mentioned various NCSC and NIST documents recommending MFA
  • Highlighted that BA's own internal policy mandated use of MFA.
  • However their implementation of Citrix did not apply MFA to all access.
This highlights the need to test the implementation of security controls to ensure they are working and effective, or at least audit their configuration.
  • Did not have a risk assessment of the Citrix solution or the applications accessed through it.
  • Had not locked down the services available by Citrix.
  • Lack of app whitelisting.
  • Lack of server hardening.
  • Limited restrictions on apps being opened by stopping clicking the icon, but still able to run via file->open.
  • Environment was pen tested, but scope appears to have been limited so many issues were not detected.
  • Called out use of hardcoded passwords.
  • Suggested logging access to certain files containing hardcoded passwords would be a suitable control.
  • Lack of implementation of "least privilege" principles.
  • Lack of monitoring of unexpected (e.g. guest) account logins.
  • No use of PAM.
  • Limited monitoring.
  • Used PCI DSS but were not compliant with it.
  • Left debug logging in place on live systems, increasing data available to attackers.
  • Lack of File Integrity Monitoring (FIM).
  • No ability to detect changes to the website code.

Marriot

https://ico.org.uk/media/action-weve-taken/mpns/2616891/dsg-mpn-20200107.pdf
Marriot thought MFA was implemented and had even audited it, but there were undiscovered gaps. The ICO accepted this and did not include that in their assessment.

  • Insufficient monitoring of privileged accounts - not logging access to systems, noting from other cases that logging alone is of little value unless someone is checking the logs or being alerted.
  • Insufficient monitoring of databases - Guardium deployed but only on cardholder storing tables, so they had done a risk based approach to choose where to monitor, but was not deemed adequate by the ICO. SOC/SIEM was not logging the user access to databases. Boundary controls were not enough without internal monitoring.
  • Control of critical systems - app whitelisting, monitoring/alerting.
  • Encryption - no justification/risk assessment of data held without encryption.

DSG

https://ico.org.uk/media/action-weve-taken/mpns/2616891/dsg-mpn-20200107.pdf
  • It was deemed that that PAN (Primary Account Number - i.e. Card data) does constitute personal data, so be wary of this as data is considered PII if people can be indirectly identified by it, phone numbers being a common example that many may not initially consider as being PII. See: https://www.gdpreu.org/the-regulation/key-concepts/personal-data/
  • PCI DSS is not indicative of appropriate security for PII but certifications like this can be helpful in deeming what level is considered appropriate, it sounds like DSG had some issues with PCI compliance.
  • Segregation was considered as both network/IP and Active Directory, the inference being that segregating your network but not your AD is probably not appropriate.
  • Not having host based firewall was called out, despite that it wouldn't have prevented this attack. Also the ability to detect changes to the configuration of these local firewalls was called out as a requirement.
  • Inadequate patching on domain controllers.
  • No logging/monitoring in place to detect and respond to attacks.
  • Outdated versions of Java.
  • Not strictly controlling privileged access rights - i.e. no PAM.
  • Not using standard builds with hardening built in.
  • Patching of devices was not compliant with their own policy.
  • Case notes state that application whitelisting is considered "appropriate" control

Read more...

Monday, 25 November 2019

Powershell for AD Querying

Powershell commands for mucking about with AD:

Basic info on the user:
Get-ADUser username

List all groups a user is in:
Get-ADPrincipalGroupMembership username | select name


List all users in a group
Get-ADGroupMember "Groupname" | select name


List all groups in the AD
Get-ADGroup -searchbase "OU=GROUPS_OU,DC=domain,DC=com" -Property member -Filter * | select-object name, @{n='count';e={$_.member.count}} | sort-object descending


Batch file to run powershell:
@echo off & setlocal
set batchPath=%~dp0
powershell.exe -ExecutionPolicy ByPass -file "%batchPath%file.ps1"

Read more...

Thursday, 12 May 2016

Automated Install for OpenStack Kilo on Ubuntu Server

I've been messing around with OpenStack lately, there is the excellent Devstack system for building test deployments but I wanted something to survive a reboot which meant needing a full OpenStack. There's some great docs on OpenStacks website for installing Kilo on Ubuntu 14.04.

To automate things I've scripted the process above with a few tweaks, available on github:

https://github.com/unixhead/mattstack/.

It's meant for installing on a single host and playing around with, there's no telemetry or orchestration module and it just uses the basic legacy nova networking.

How to use it!

  • Download and install the Ubuntu Server 14.04 (Trusty) image. It won't work on any other version! You don't need to specify any software components during the install, although SSH server might be handy.
  • Download the build-kilo.sh script from https://raw.githubusercontent.com/unixhead/mattstack/master/build-kilo.sh
  • Edit the script ("nano build-kilo.sh") and set the variables at the top, they have descriptive comments explaining what needs doing.
  • Run "chmod +x build-kilo.sh" to make it executable.
  • Run the script as root. "sudo su -" and then "./build-kilo.sh"
  • Reboot at the end of the install and you should have a working OpenStack Kilo build.

If you want to rebuild then you're probably best off re-installing Ubuntu server and starting from scratch.

There are a few niggles with the original build process such as getting error 500 denied messages when trying to perform various operations, resolved by changing Keystone not to use memcached. Also had some issues with Qemu due to not having the nova-compute-qemu package installed and the /etc/nova/nova.conf not quite right, needed for deploying Openstack in something like Virtualbox without KVM support. This script should sort those problems out.


Read more...

Saturday, 2 January 2016

Vagrant Lab for HAProxy

This article is about setting up a lab using Vagrant to play with the HAProxy load balancer.

If you want the TLDR version where you just copy/paste a few lines and the lab gets created then this will do the job, it's explained in more detail below. It's great with tools like Virtualbox and Vagrant that such a lab can be set up so easily, this would have taken days to build prior to virtualization!

#install software
sudo apt-get install virtualbox vagrant git

#configure host-only subnet address in virtualbox
VBoxManage hostonlyif create
VBoxManage hostonlyif ipconfig vboxnet0 --ip 172.28.128.1 --netmask 255.255.255.0
VBoxManage dhcpserver modify --ifname vboxnet0 --ip 172.28.128.1 --netmask 255.255.255.0 --lowerip 172.28.128.100 --upperip 172.28.128.250

#install the lab files, for some reason the box fails to auto-download so install it manually, should be fixed in future
git clone https://github.com/unixhead/haproxy-basic-lab && cd haproxy-basic-lab 
vagrant box add hashicorp/precise32 https://vagrantcloud.com/hashicorp/boxes/precise32/versions/1.0.0/providers/virtualbox.box


#run the lab
vagrant up


Now there is a slight caveat in that I use Linux Mint and the current versions of Vagrant & Virtualbox aren't quite right, so I had to manually install Vagrant from the website, but never let the truth get in the way of a good story.

My Virtualbox host-only network uses the range 172.28.128.0/24 and the network to be created is shown below, a simple load balancer infront of two web servers. The client is also the hypervisor hosting the VMs. It's very similar to the configuration of the Vagrant tutorial system. Virtualbox by default uses 192.168.0.0/24 for the host-only networks but that overlaps with a few places I work so had to change it.



The files needed are listed below, save them all to the same directory and run "vagrant up" in it. You can download them all in one go from github with the command:

git clone https://github.com/unixhead/haproxy-basic-lab

Vagrantfile - The configuration for Vagrant itself providing 3 VMs using the Ubuntu 32 bit image. One is the HAProxy load balancer and two web servers.

Vagrant.configure(2) do |config|
  config.vm.box = "hashicorp/precise32"

 config.vm.provider "virtualbox" do |v|
   v.memory = 1024
    v.cpus = 1
 end

 config.vm.define "lb" do |config|
  config.vm.hostname = "lb"
    config.vm.network "private_network", ip: "172.28.128.10"
   config.vm.provision :shell, path: "bootstrap-haproxy.sh"
 end

 config.vm.define "web1" do |config|
  config.vm.hostname = "web1"
    config.vm.network "private_network", ip: "172.28.128.11"
   config.vm.provision :shell, path: "bootstrap-apache.sh"
 end

 config.vm.define "web2" do |config|
  config.vm.hostname = "web2"
    config.vm.network "private_network", ip: "172.28.128.12"
   config.vm.provision :shell, path: "bootstrap-apache.sh"
 end

end


bootstrap-apache.sh - This script runs on the webservers after Vagrant has built them. It installs Apache with PHP, then sets the web root to the current Vagrant project directory.

#!/usr/bin/env bash

apt-get update
apt-get install -y apache2 php5
if ! [ -L /var/www ]; then
  rm -rf /var/www
  ln -fs /vagrant /var/www
fi


bootstrap-haproxy.sh - This runs on the load balancer after build, it installs HAProxy and copies the provided configuration file.

#!/usr/bin/env bash

apt-get update
apt-get install -y haproxy hatop
cp /vagrant/haproxy.cfg /etc/haproxy
echo "ENABLED=1" > /etc/default/haproxy
service haproxy start


haproxy.cfg - Basic HAProxy configuration for load balancing port 80 between two web servers.

frontend http
  bind *:80
  mode tcp
  option tcplog

  default_backend web-backend

backend web-backend
   balance roundrobin
   mode tcp
   server web1 172.28.128.11:80 check
   server web2 172.28.128.12:80 check


index.php - A basic web index to show which web server was accessed by printing the servers hostname. Both web-servers will load the same file.

<?php
echo gethostname() . "\n";
?>


To test it simply run Curl a few times against the IP address of the load balancer, the replies show that the web sessions are being balanced across both hosts:

matt@client haproxy-lab $ curl 172.28.128.10
web1
matt@client haproxy-lab $ curl 172.28.128.10
web2
matt@client haproxy-lab $ curl 172.28.128.10
web1
matt@client haproxy-lab $ curl 172.28.128.10
web2
matt@client haproxy-lab $ curl 172.28.128.10
web1
matt@client haproxy-lab $ curl 172.28.128.10
web2
matt@client haproxy-lab $ curl 172.28.128.10
web1
matt@client haproxy-lab $ curl 172.28.128.10
web2
matt@client haproxy-lab $ curl 172.28.128.10
web1
matt@client haproxy-lab $ curl 172.28.128.10
web2
matt@client haproxy-lab $ curl 172.28.128.10
web1

Read more...

Tuesday, 10 February 2015

Cisco IOS TCL - Reset Interface if DHCP Fails

I've got some devices where DHCP doesn't always work properly for a number of reasons, running shut/no shut on the Cisco router seems to fix it. To automate that I've knocked up a TCL script:

The script itself:


#script to check if an interface has an IP address and reset it if not.
#copy to flash via TFTP or write it using the technique described here:
# http://sabotage-networks.blogspot.co.uk/2013/02/applying-ciscos-new-licenses-without.html
#
#set as cron job to be run every 60 minutes with:
#
# kron policy-list checkInteface
# tclsh flash:/checkIP.tcl
# exit
#
# kron occurence checkInterface in 60 recurring
#


#set this to name of WAN interface
set interface "fa0/0"

set output [exec "show interface $interface | include Internet address"]
if {[regexp (DHCP) $output]} {
#no ip found, reset interface
puts "no ip address found, restarting interface $interface"
ios_config "interface $interface" "shutdown";
after 5000;
ios_config "interface $interface" "no shutdown";
}

Read more...

Sunday, 17 February 2013

Applying Cisco's New Licenses Without Network Servers

Cisco have a new licensing method that involves installing an XML license on the end device. The license you buy is a code but rather than just entering that onto the device you have to go to Cisco.com and associate the code with a device using part and serial number. Then they generate an XML license file which you are supposed to download and install on the device.

The ways they support doing this are FTP, SCP, TFTP, HTTP, which is no use if you're in a locked down environment, especially working remotely. Luckily as most of their boxes now include TCL so you can fudge it to paste the license straight on via a terminal. Thanks muchly to http://www.internetworkpro.org/wiki/Edit_files_using_TCL The license looks something like this:
<?xml header stuff?>
<SomeStuff></SomeStuff>
<SomeMoreStuff></SomeMoreStuff>
<license><![CDATA[loadsandloadsofrandomgarbagethatisfartoolongtofitonasinglelineofxmlsoyouneedtosplitthislineupintoseveraldifferentvariablesthisfieldcontainsabinaryloadofgunkpretendingitsopenandinteroperablebecauseitsxml]]></license>
<EvenMoreStuff></EvenMoreStuff>

The trick is to use TCL. You create a TCL variable containing the license file data and write it to a text file on the flash memory. The problem is that the license file contains a blob in a CNAME field that is longer than the maximum TCLSH line length. One way around this is to break the file down into multiple lines, store each as a separate variable and write the lot into the same file without any line returns in between.

Several things to watch out for:
  • Don't put extra carriage returns in as the license will not be valid
  • Don't paste carriage returns as it seems to mess up the TCL shell - paste one line at a time then hit enter
  • The +> prompt means TCL is still accepting input for the same variable.
The commands are:
Router#tclsh
Router(tcl)#set file [open "flash:keyfile.lic" w+]

Router(tcl)#set line1 {
+><?xml header stuff?>
+><SomeStuff></SomeStuff>
+><SomeMoreStuff></SomeMoreStuff>
+><license><![CDATA[loadsandloadsofrandomgarbagethatisfartoolongtofitonasinglelineofxml}
Router(tcl)#set line2 {<soyouneedtosplitthislineupintoseveraldifferentvariablesthisfieldcontainsabinary>}
Router(tcl)#set line3 {<loadofgunkpretendingitsopenandinteroperablebecauseitsxml]]></license>
<EvenMoreStuff></EvenMoreStuff>}


Router(tcl)#puts -nonewline $file $line1
Router(tcl)#puts -nonewline $file $line2
Router(tcl)#puts -nonewline $file $line3
Router(tcl)#close $file

Router(tcl)#tclquit
Router#license install flash:keyfile.lic


Now you have the license in place, so all is great! Except that you need to reboot it to activate, hope you weren't running any live services on this box!

Read more...

Wednesday, 31 October 2012

Bluecoat Terminal Length

The Bluecoat SGOS equivalent of term len 0 is line-vty in config mode:

Bluecoat#(config) line-vty
Bluecoat#(config line-vty) length ?
(0 for no pausing)
Bluecoat#(config line-vty) length 0


Handy for grabbing the text config.

Read more...

Wednesday, 27 June 2012

EIGRP RTP Unicast Fallback

Having just started studying for ROUTE to refresh a variety of Cisco exams I had a look at EIGRP and got far too involved in RTP. Probably all you need to know for the ROUTE exam is that it's Reliable Transport Protocol in the context of EIGRP and that it's used to ensure reliable delivery of updates. But to dig a little deeper....

RTP (not the same as real-time-protocol) can use both unicast and multicast. On an ethernet LAN, routing information is transmitted via Multicast (unless the neighbours are defined as unicast ones with neighbour statements). The RTP feature adds it's own reliability with the addition of sequence numbers and a state table on the updating router which keeps track of the acknowledgements from neighbours. If any do not respond then RTP falls back to trying unicast transmission.

To test it I built this flat network with 3 EIGRP neighbours on the same subnet:




The addresses used are:
  • R1 - 192.168.0.1
  • R2 - 192.168.0.2
  • R3 - 192.168.0.3


In this scenario the routing update messages are sent using multicast. For removing a route the "query" type message is used. I'll shut down a loopback interface on R3 which wil generate an EIGRP query. The packet dump below shows the query being multicast (to 224.0.0.10). The two neighbours then acknowledge this via unicast.



On R3 you see the following in the output of "debug eigrp packet", it shows the process:
  1. R3 sending the query messages
  2. Both R1 and R2 responding via unicast.
*Mar 1 00:20:35.567: EIGRP: Enqueueing QUERY on FastEthernet0/0 iidbQ un/rely 0/1 serno 27-27
*Mar 1 00:20:35.571: EIGRP: Enqueueing QUERY on FastEthernet0/0 nbr 192.168.0.1 iidbQ un/rely 0/0 peerQ un/rely 0/0 serno 27-27
*Mar 1 00:20:35.571: EIGRP: Enqueueing QUERY on FastEthernet0/0 nbr 192.168.0.2 iidbQ un/rely 0/0 peerQ un/rely 0/0 serno 27-27

*Mar 1 00:20:35.575: EIGRP: Sending QUERY on FastEthernet0/0
*Mar 1 00:20:35.575: AS 1, Flags 0x0, Seq 34/0 idbQ 0/0 iidbQ un/rely 0/0 serno 27-27

*Mar 1 00:20:35.587: EIGRP: Received ACK on FastEthernet0/0 nbr 192.168.0.1
*Mar 1 00:20:35.591: AS 1, Flags 0x0, Seq 0/34 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/1

*Mar 1 00:20:35.603: EIGRP: Received ACK on FastEthernet0/0 nbr 192.168.0.2
*Mar 1 00:20:35.607: AS 1, Flags 0x0, Seq 0/34 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/1

[snip]


Now to test the unicast failback by blocking the multicast updates on R1, this is quite tricky as these multicast packets are required to keep the EIGRP neighbour relationships up. My cunning plan is to increase the EIGRP hold timer so that I can drop multicast without disrupting the neighbours.

Because the hold timer is not a local setting but an "advertised value", I actually need to set it on R2 & R3 which will then tell R1 not to worry if it doesn't see any hellos for the next ten minutes.

R3(config)#int f0/0
R3(config-if)#ip hold-time eigrp 1 600

R1(config)#int f0/0
R1(config-if)#ip access-group DENYEIGRP in

R1#show ip access-list DENYEIGRP
Extended IP access list DENYEIGRP
10 deny ip any host 224.0.0.10 log (4 matches)
20 permit ip any any (27 matches)


At this point EIGRP neighbours are all up and R1 is not expecting to hear from R3 for the next ten minutes. Now I'll shut down the interface on R3 again to generate an EIGRP query message. The wireshark output is shown below:



The debug output on R3 shows as below, you can see the phases of the RTP mechanism:
  1. R3 multicasts a query to 224.0.0.10.
  2. R2 responds via unicast (you can see the text peerQ un/rely 0/1 indicating a unicast message).
  3. R1 does not respond as it has not seen the message.
  4. Meanwhile R2 completes the exchange with R3 via unicast.
  5. R3 then realises there is an outstanding response from R1 and retries the query again via unicast showing
    *Mar 1 00:09:13.995: EIGRP: Sending QUERY on FastEthernet0/0 nbr 192.168.0.1, retry 1, RTO 3321
    *Mar 1 00:09:13.995: AS 1, Flags 0x0, Seq 18/18 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/1 serno 18-18
  6. R1 now responds via unicast and exchange completes as normal. This is shown in bold.


The complete debug output is:
R3(config-if)#shut
R3(config-if)#
*Mar 1 00:09:11.775: EIGRP: Enqueueing QUERY on FastEthernet0/0 iidbQ un/rely 0/1 serno 18-18
*Mar 1 00:09:11.779: EIGRP: Enqueueing QUERY on FastEthernet0/0 nbr 192.168.0.1 iidbQ un/rely 0/0 peerQ un/rely 0/0 serno 18-18
*Mar 1 00:09:11.779: EIGRP: Enqueueing QUERY on FastEthernet0/0 nbr 192.168.0.2 iidbQ un/rely 0/0 peerQ un/rely 0/0 serno 18-18

*Mar 1 00:09:11.783: EIGRP: Sending QUERY on FastEthernet0/0
*Mar 1 00:09:11.783: AS 1, Flags 0x0, Seq 18/0 idbQ 0/0 iidbQ un/rely 0/0 serno 18-18

*Mar 1 00:09:11.799: EIGRP: Received ACK on FastEthernet0/0 nbr 192.168.0.2
*Mar 1 00:09:11.799: AS 1, Flags 0x0, Seq 0/18 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/1

*Mar 1 00:09:11.811: EIGRP: Received REPLY on FastEthernet0/0 nbr 192.168.0.2
*Mar 1 00:09:11.811: AS 1, Flags 0x0, Seq 17/18 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/0

*Mar 1 00:09:11.815: EIGRP: Enqueueing ACK on FastEthernet0/0 nbr 192.168.0.2
*Mar 1 00:09:11.815: Ack seq 17 iidbQ un/rely 0/0 peerQ un/rely 1/0
*Mar 1 00:09:11.819: EIGRP: Sending ACK on FastEthernet0/0 nbr 192.168.0.2
*Mar 1 00:09:11.819: AS 1, Flags 0x0, Seq 0/17 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 1/0

*Mar 1 00:09:13.995: EIGRP: Sending QUERY on FastEthernet0/0 nbr 192.168.0.1, retry 1, RTO 3321
*Mar 1 00:09:13.995: AS 1, Flags 0x0, Seq 18/18 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/1 serno 18-18

*Mar 1 00:09:14.019: EIGRP: Received ACK on FastEthernet0/0 nbr 192.168.0.1
*Mar 1 00:09:14.019: AS 1, Flags 0x0, Seq 0/18 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/1

*Mar 1 00:09:14.027: EIGRP: Received REPLY on FastEthernet0/0 nbr 192.168.0.1
*Mar 1 00:09:14.031: AS 1, Flags 0x0, Seq 19/18 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/0

*Mar 1 00:09:14.031: EIGRP: Enqueueing ACK on FastEthernet0/0 nbr 192.168.0.1
*Mar 1 00:09:14.031: Ack seq 19 iidbQ un/rely 0/0 peerQ un/rely 1/0

*Mar 1 00:09:14.035: EIGRP: Sending ACK on FastEthernet0/0 nbr 192.168.0.1
*Mar 1 00:09:14.035: AS 1, Flags 0x0, Seq 0/19 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 1/0



Read more...

Wednesday, 2 May 2012

WCCP Redirect ACLs and Masks


This article is about WCCP redirect ACLs, masks and how they relate to TCAM usage on Cisco switches. It's quite important to understand if doing WCCP as you want to ensure forwarding is done in hardware which runs at wire speed and not software which will cause considerable CPU usage and potentially performance issues.

This is quite a difficult subject to explain and I'm not entirely sure I've done it that well here, the info has been pulled in from a variety of sources and I'm also not entirely sure it's correct as a few bits don't quite tie together. It's been re-written several times and I'm still not entirely happy, however here is the info warts n all.

A very basic recap on WCCP.

WCCP redirects traffic as it passes through a switch or router which act as a WCCP server. This is for things like proxy servers or WAN optimisers which are the WCCP clients. The server has redirect ACLs that specifies what traffic will be sent to the WCCP client device. On Cisco routers/switches these ACLs are not stateful and you have to capture traffic flows going in both directions.
This diagram shows the example setup, the Proxy server is the WCCP client, the switch is the WCCP server. 

For example to grab HTTP from LAN to WAN you would have:

ip wccp 100 redirect-acl HTTP_LAN_TO_WAN
ip access-list extended HTTP_LAN_TO_WAN
 permit tcp 10.0.0.0 0.0.0.255 any eq 80


Then to grab the return traffic:

ip wccp 200 redirect-acl HTTP_WAN_TO_LAN
ip access-list extended HTTP_WAN_TO_LAN
 permit tcp any eq 80 10.0.0.0 0.0.0.255


These are then configured on interfaces to capture traffic, Cisco supports both ingress and egress however the switches will only do hardware forwarding for ingress WCCP sessions.

int gi0/1
 description LAN
 ip wccp 100 redirect in


int gi0/2
 description WAN
 ip wccp 200 redirect in


With this configuration alone nothing will happen. You need to add a WCCP client and tell it to communicate with the WCCP server. i.e. you need to configure WCCP on the proxy server and tell it to talk to the switch, it will then start chatting and negotiate certain parameters.


Once that is done the WCCP server will start redirecting traffic. If no WCCP clients are active then the server will just forward traffic as per normal. If one or more WCCP clients are active then the switch will load balance traffic between them depending on configuration.

TCAM

Stands for Ternary Content-addressable Memory. It is used for hardware forwarding, packets are compared against the TCAM table and it tells the switch or router how to forward them. If an entry isn't found in the TCAM table then the packet must be software routed which is not desirable.
Ternary means there are three values, 0, 1 and don't care. "don't care" is represented by an x in this doc and just to really confuse you I'll use 0x to prefix any hex values.


Redirect ACLs

The redirection ACL tells the WCCP server what traffic to intercept and divert to the WCCP client/s, any traffic not matching is passed as normal. As this ACL is likely to be applied on an interface seeing a lot of traffic (probably all transit traffic for the network) then you want it to run entirely in hardware and be as fast as possible. There are a couple of rules with regards to TCAM usage and this ACL:

  • Each permit statement in the ACL requires at least one TCAM entry.
  • Each load balanced path requires at least one TCAM entry.
  • The number of load balanced paths can be calculated with the number of bits in the assignment mask (see below).
  • In all cases except where the mask is 0x0, Deny statements use less TCAM entries than Permit statements. This is because you don't need to load balance traffic being dropped so a Deny statement will only take up 1 TCAM entry.




The Mask.

Cisco switches only support hardware forwarding for WCCP mask based assignments, not using the hash method.The mask is a hexadecimal value that does several things:
  • Restricts how many WCCP clients can be part of the load balancing arrangement.
  • Affects the TCAM usage by WCCP.
  • Defines what IP addresses are load balanced to which WCCP clients.
The last point is critical for WAN optimisers which work in pairs by forming shared byte caches, if you have a farm of WAN optimisers, e.g. in a data centre, then you want remote sites to always speak to the same member of the farm to avoid having to maintain multiple shared caches. i.e. all hosts within a certain subnet will be load balanced to the same WCCP client.

The masks are written in hex and usually configured in hex, but I found to make sense of them it's best to convert them to binary. Also convert the IP addresses to binary and think of the mask being applied bit-by-bit. 

The mask is configured on the WCCP client, e.g. the WAN Optimiser or Proxy Server, which then informs the server during the WCCP session negotiation.

How the Switch Uses the Mask

On Cisco switches all combinations of bits in the mask are used to create different values. These values are applied to the IP addresses in the redirect ACL to create entries in the forwarding table (TCAM), which the switch uses to forward the traffic to WCCP clients.

For example a mask of 0x10 in binary is represented as 0001 0000.
The available combinations of bits are: 0000 0000 and 0001 0000
Because it's a ternary mask we are only interested in the specific bit used in the original mask, the other zero's all become "don't care" values, so the two masks the forwarding table will end up using are:
xxx1 xxxx
xxx0 xxxx

These masks would be applied against the ACL and used to create the TCAM forwarding paths for the traffic, any IP address with a 1 in the 5th position would match the first mask and any with a 0 in the 5th position would match the second. If you configure this mask then look at the WCCP session it appears as below:

Switch#show ip wccp 100 detail
WCCP Client information:
   WCCP Client ID:    192.168.0.100
   Protocol Version:   2.0
   State:     Usable
   Redirection:    L2
   Packet Return:    L2
   Packets Redirected:   0
   Connect Time:     00:01:07
   Assignment:     MASK

   Mask SrcAddr DstAddr SrcPort DstPort
   ---- ------- ------- ------- -------
   0000: 0x00000010 0x00000000 0x0000 0x0000

   Value SrcAddr DstAddr SrcPort DstPort CE-IP
   ----- ------- ------- ------- ------- -----
   0000: 0x00000000 0x00000000 0x0000 0x0000 0xC0A80064 (192.168.0.100)
   0001: 0x00000010 0x00000000 0x0000 0x0000 0xC0A80064 (192.168.0.100)



Mask Load Balancing.

The number of bits in the mask determines how many devices you can load balance traffic between.

A mask of 0x0 does not allow load balancing and will give a single path only (useful if you only have a single WCCP client and are short on TCAM).

A mask of 0x1 allows for load balancing between 2 WCCP clients only. The binary mask values can be either 0 or 1.

A mask of 0x3 allows for up to 4 WCCP clients as it's made up from 2 bits and available mask values can be 00, 01, 10 and 11.


The default mask is 0x1741. In binary that is 0001 0111 0100 0001. 6 bits are used. That allows for 2^6 WCCP clients. I have no idea why Cisco chose this number, even their own WAAS troubleshooting guide recommends you don't use it. Because it has a bit in the leftmost position it will load balance alternating every single IP address and if they wanted a 6 bit mask then 0011 1111 would make more sense, 0x3F. Possibly there is some mathematical significance I haven't seen, possibly it works best with their hardware, possibly it was made up at random or possibly this entire article is wrong and I don't understand the masks at all. Take your pick.

Mask IP Address Matching.

The simplest example is a mask of 0x1. As these masks are used against IP addresses the value would be converted to 32 bits and represented in TCAM as xxxxxxxx.xxxxxxxx.xxxxxxxx.xxxxxxx1


The ACL is "permit tcp 10.0.0.0 0.0.0.255 any eq http". With a mask of 0x1 it would produce two forwarding paths which will match IP traffic as follows:
Path 1  - mask 0 - 10.0.0.2, 10.0.0.4, 10.0.0.6, 10.0.0.8....
Path 2 - mask 1 - 10.0.0.1, 10.0.0.3, 10.0.0.5, 10.0.0.9...


With two WCCP clients, one would receive HTTP traffic from hosts with IPs matching path 1, the second client would receive path 2 clients.


With a mask of 0x10, the binary value is 10000 (in TCAM this would be xxxxxxxx.xxxxxxxx.xxxxxxxx.xxx1xxxx). This will load balance clients in "chunks" of 16 addresses.

If the ACL is "permit tcp 10.0.0.0 0.0.0.255 any eq http" then this will create two groups and distribute traffic as follows:
Mask 0 - 10.0.0.0 to 10.0.0.15, 10.0.0.32 to 10.0.0.47, 10.0.0.64 to 10.0.0.79.....
Mask 1 - 10.0.0.16 to 10.0.0.31, 10.0.0.48 to 10.0.0.63, 10.0.0.80 to 10.0.0.95......

If there were two active WCCP clients then you'd see the traffic distributed as above.

For large solutions you may want to distribute using a different pattern, with WAN optimisers you want the same optimisers to speak to each other rather than have a branch office device communicate with several different data centre devices as it'd either have to maintain several different copies of the byte caching tables or you'd end up with the optimiser cluster forwarding traffic internally to keep the same device peerings. For a system where you wanted to split subnets on a /21 boundary and have up to 4 WCCP clients in your farm then you'd choose a mask as follows:

/21 in binary would look like this: 11111111.11111111.11111xxx.xxxxxxxx
The WCCP mask could be xxxxxxxx.xxxxxxxx.xxxx1xxx.xxxxxxxx
But this would only allow for 2 possible mask values, so only 2 WCCP clients.
To allow 4 WCCP clients you need 2 bits, the mask becomes xxxxxxxx.xxxxxxxx.xxx11xxx.xxxxxxxx
In hex that is shown as 0x1800

This would give four available combinations/masks of:
xxxxxxxx.xxxxxxxx.xxx00xxx.xxxxxxxx shortened to mask 00
xxxxxxxx.xxxxxxxx.xxx01xxx.xxxxxxxx shortened to mask 01
xxxxxxxx.xxxxxxxx.xxx10xxx.xxxxxxxx shortened to mask 10
xxxxxxxx.xxxxxxxx.xxx11xxx.xxxxxxxx shortened to mask 11


With an ACL of "permit tcp 10.0.0.0 0.255.255.255 any eq http" the split would be:

00 - 10.0.0.0 - 10.0.7.255, 10.0.32.0 - 10.0.39.255...
01 - 10.0.8.0 - 10.0.15.255, 10.0.40.0 - 10.0.47.255...
10 - 10.0.16.0 - 10.0.23.255, 10.0.48.0 - 10.0.55.255...
11 - 10.0.24.0 - 10.0.31.255, 10.0.56.0 - 10.0.63.255...


Weighted Load Balancing.

I've said above that each available forwarding path equals a single WCCP client. This is not necessarily the case as you can weight WCCP clients. Consider a case with two WAN optimisers (A and B) of different specifications where A can process twice as much traffic as B. In that event you would want at least 3 forwarding paths, 2 of them pointing to A and 1 to B. Your mask needs to use at least 2 bits. This is another area I'm a bit hazy on, I would think you'd need a multiple of 3 to make this work properly but you can only ever have an even number of forwarding paths...

TCAM Usage

The equation for working out TCAM usage is defined as:

2^<mask bits> * <acl entries>

To include all entries in the ACL the full definition would be:
( 2^<mask bits> * <number of permit statements in redirect acl> ) +   <number of deny statements in redirect ACL>


On a 3750 the WCCP TCAM is shared with the ACL TCAM. You have to run the router SDM template to support WCCP and it supports a maximum of 1024 entries. So if using the default mask you can have up to 1024/6 = 170 entries in the redirect ACL and no other ACLs on the switch.

If you wanted to capture HTTP and HTTPS traffic, split the network by /24 and allow for 8 forwarding paths in your farm then your ACL may be:

permit tcp 10.0.0.0 0.255.255.255 any eq 80
permit tcp 10.0.0.0 0.255.255.255 any eq 443

And your mask may be 0x70 (xxxxxxxx.xxxxxxxx.xxxxxxxx.x111xxxx)


This would result in 8 forwarding paths, each being created for both of the ACL entries, a total usage of 16 TCAM entries. If you are matching traffic in both directions it's a total of 32 TCAM entries used for WCCP.



Read more...

Tuesday, 17 January 2012

Network Notes - IBM PowerHA / HACMP

Some info on the networking features of HACMP (High Availability Cluster Multiprocessing). This is now called PowerHA SystemMirror for AIX. It allows up to 16 nodes in a cluster. As of v7.1 the cluster can use multicast to communicate, previous versions used UDP broadcasts. The cluster heartbeats are sent both via LAN and SAN for redundancy.


Terminology:


Boot IP: The address bound to the physical interface (e.g. ifconfig blah x.x.x.x).
Service IP: The VIP to which clients connect to hit the actual service, can exist on any interface on any cluster member.
Persistent IP: Used to reach a host for management. Also called node VIP and can exist on any interface on a single cluster member.
HWAT - Hardware Address Takeover: MAC address follows the IP when failing over.
IPAT - IP Address Takeover: Moves the service IP between interfaces and cluster members.

There are two methods of doing IPAT, via replacement and by alias.

IPAT via Replacement


This is the older method, it uses HWAT so no gratuitous ARP is required as the MAC address fails over with the service IP. However clearly port security cannot be used! You need two interfaces in the same VLAN, 1 configured with a real IP address (boot IP) and 1 with any IP (standby IP) that need not be routable. When HACMP starts it replaces the real IP address on NIC 1 with a VIP in the same subnet. The failover moves both VIP and MAC onto NIC 2. You can only have 1 service VIP per adapter pair.


IPAT via Aliasing.


The newer and recommended method, it requires a network that can support gratuitous ARP as HWAT is not used. The service IP is the only routable address needed. The 2 NICs are configured with IP addresses on different subnets that need not be routable. The service VIP is an alias address on the interface and fails over as an alias. You can have as many service VIPs as you want on an interface.

Heartbeats.


The boot IPs seem fairly pointless, however network heartbeats are broadcast/multicast from the boot IP so they should be allocated from the same subnet, an example allocation is:

Node service IP 10.0.0.10

Node 1
NIC1 boot IP 192.168.0.1
NIC2 boot IP 192.168.10.1
Persistent IP 10.0.0.101

Node 2
NIC1 boot IP 192.168.0.2
NIC2 boot IP 192.168.10.2
Persistent IP 10.0.0.102

Node 3
NIC1 boot IP 192.168.0.3
NIC2 boot IP 192.168.10.3
Persistent IP 10.0.0.103



Routing.


Any routes should be configured via the service IP subnet and persistent/node IP subnet. You should not use the boot addresses as they may not always be reachable (e.g. if NIC failover). You can use the service IP to manage the system but it might not be on that node if the cluster has failed over so better to use a persistent IP. Service and Persistent IPs can be on same subnet or different ones. If different then you'll either need multiple IPs configured on the VLAN interface or static routing configured on the AIX box as they'll both be in the same VLAN. I would KISS and have both on same VLAN & same subnet.

Read more...

Wednesday, 17 August 2011

Cisco ASA 8.4 - Global Access Lists

Handy new feature in version 8.4 of the ASA software is the ability to do global access lists.


The Cisco ASA allows security levels to be applied to interfaces, traffic is automatically allowed from a high to low security level interface but not vice versa. It's probably designed for the fairly common use case of a perimeter device between a LAN and the internet. The internet link is set to security level 0 and the inside interface to 100. All LAN traffic is then allowed to flow out. This is shown below:



Prior to version 8.3, access lists (ACLs) had to be applied on an interface and in a direction, e.g.

access-list MYACL extended permit tcp any any eq www
access-group MYACL in interface outside
As soon as an ACL is applied to an interface, it will pass traffic based on the ACL rather than based on security levels. However it gets complicated as traffic coming in another interface that would previously have been allowed is now still allowed, in the example above if you permitted port 80 in from the internet, all outgoing LAN traffic is still allowed.


Now in version 8.4, Cisco have added the ability to have a single global ACL that applies to all traffic regardless of which interface it uses. This is how most other firewalls work so a welcome change. To do this you create the ACL then apply it with:

access-list MYACL extended permit tcp any any eq www
access-group MYACL global

When a global ACL is applied, it removes all behaviours based on security levels from ALL interfaces. So in the original example, you would need a rule in your global ACL that permits LAN hosts access to the internet. The any/any rule is a good example of what not to do as this now globally means "any address" rather than specific to any particular interface.

Update 2015 It appears that not quite "ALL" security level behaviours are removed, you still need the same-security-level command to allow traffic to flow between interfaces regardless of ACL.


Read more...

Wednesday, 13 April 2011

Evaluation Assurance Levels - EAL

EAL stands for evaluation assurance level and is a certificate of security for IT products measured against a set of common security criteria. The main source of information on EAL levels is the common criteria portal where you can find details of approved products and information on the criteria used for the EAL certifications.

Who uses it?


Your average network bod may not come across EAL very often. It tends to crop up in areas that are regulated by government bodies such as CESG who will often require EAL4 certified products for certain secure environments. However you don't just buy EAL4 kit and be government approved, it fits into a much larger security framework such as ISO27k dealing with everything from who gets into the building to how you manage changes to IT systems.

How does a product get EAL certified?


It is assessed against a set of common criteria by an approved agency. The developer of the system produces a security target (ST) document containing a list of features to be assessed.The ST is based on the criteria here. The process is long and expensive, according to wikipedia vendors were spending $1 - $2.5million to gain EAL4 certification in the 1990s.


What do you get when EAL certified?


Certified products are listed on the common criteria portal along with the rating granted, the ST it was assessed against and the assessment report. e.g. here (PDF) is the ST for the Cisco ASA as a firewall and here (PDF) is the assessment report. Interesting to note that the EAL4 VPN certificate was issued separately, so an ASA acting as both firewall and VPN endpoint is not a valid EAL4 solution, strictly speaking you would need two in series performing each task.

So what does it mean in to a network engineer?


Probably not a lot, it's a policy requirement for many places but the assessment is only against the device, not against the specific implementation of it. You could deploy an EAL4 firewall with a policy of "permit any any" and it's still an EAL4 device! At that point the other security mechanisms should have stopped you from putting it on the network.

If you are involved in hardware selection for a regulated organization then you may need to use EAL4 devices in certain situations.

What is required to meet the various levels?


The EAL process is broken down to cover the following aspects of a system:
Development, documentation, life-cycle support, security target evaluation, testing, vulnerability assessment.

Each EAL level goes into slightly more detail, for example the "development" area at EAL1 requires a basic functional specification to be provided by the developer. EAL2 requires that same functional specification but expanded to include details of security inforcement. It also requires a security architecture description and a basic design. The specifics of those items are detailed here.

How long does it take to get EAL4?


It seems to vary from a very long time to aeons, certainly it's measured in years rather than months. A look on the NIAP CCEVS evaluation and evaluated list for firewalls shows a few examples:
Checkpoint R65 HFA01 on IPSO recorded as submitted Oct 2005 although R65 was released in 2007 so the process was started early during development. It passed March 2009. So that's 4 years to get certified and the product went EOL in March 2011, 2 years later.
Cisco ASA 8.3 as a VPN submitted November 2009 still not passed, predicted June 2011.
Palo Alto submitted various devices in December 2009 and still running.

What exactly is certified?


The certification is issued against a specific software release and hardware platform.

A specific version of the software you say? As in....minor version??


That is how the cert is written. The Cisco ASA obtained EAL4 for firewall purposes on version 7.0(6) of it's OS which was released in August 2006. Cisco have been patching and updating that for 5 years! The ASA is now up to release 8.4, which has been submitted again to CCEVS (scheme run by NIST and NSA) for evaluation.

In reality there will be a security assessor on the ground who will review the solution and hopefully be sensible about using a modern patched version of the OS and judge it acceptable to meet an EAL4 requirement, even if it's not strictly what's on the EAL4 certificate.

I don't know anyone who would tell you with a straight face that using a 5 year old OS on a firewall is going to increase your security!

What about high end firewalls?


There is a bit of a gap, if you need an EAL4 firewall with 10gig throughput then you're out of luck as the only one that's passed assessment is Checkpoint Power-1 on the 5075/9075, however that went end of life last month (March 2011). The closest is the Cisco 5580 which has been submitted for EAL4, due November 2011 and is arguably similar enough to the 5540 to be acceptable, however it's recently announced as being binned in preference of the 5585 so after August 2011 you can't buy one any more!

The security market moves quickly compared to the EAL assessments and it proves tricky.

The top end Cisco firewall platform is the 5585, not even showing as submitted for EAL evaluation yet.
Checkpoint has R71 under assessment now, predicted result in November 2011.
Palo Alto has various items aiming for November 2011, but their flagship model the PA-5000 is not listed as under assessment, it only recently hit market in the UK so EAL certification may not have been discussed yet.
Juniper have EAL4 for their ScreenOS platform the SSG, which goes EOL in 2013. They have EAL3 for Junos 9.3 on the SRX platform, the current version is 10.4. There doesn't appear to be any indication that the SRX security platforms have been submitted for EAL4 certification, although it would be surprising if that were the case as governments would be ditching Juniper en-masse before 2013.

So until November 2011 there are no EAL4 10gig firewalls. You'll have to build a farm of 1gig ones instead!

What alternative schemes are there?


FIPS-140 from NIST.
CAPS, the CESG Approved Product Scheme.

Is it worth me buying EAL4 products?


If you have to ask then probably not. If your business is regulated and the agencies setting those policies define EAL4 as a requirement then you have no choice.

For companies with the option I would say it's a helpful indicator but I would certainly use other aspects above the EAL status when selecting a device:

  • Performance.

  • Price.

  • Published security tests and exploits.

  • Staff familiarity.

  • Internal testing.



An EAL4 certificate does indicate that the product was developed following good practices and has a well defined and documented architecture. These are clearly good things in terms of stability and security. However not having EAL4 doesn't necessarily mean the product hasn't followed a good development process and isn't secure, it just means the manufacturer hasn't paid for it to be assessed.

Read more...

Monday, 14 February 2011

Legacy FRTS & Subinterfaces

FRTS and subinterfaces. This page follows on from the previous article on legacy FRTS configuration here and shows the default behaviour of FRTS with subinterfaces.

The legacy frame-relay traffic shaping has to be enabled on a physical interface. Any subinterfaces will then inherit the configuration, which is 56kbps by default. The network is shown below:



In the example below FRTS is turned on but not configured, both subinterfaces are then shaped to 56kbps (using screenshots as the output to "show traffic-shape" doesn't like this sites layout).

R1#show run | begin interface Serial0/0
interface Serial0/0
no ip address
encapsulation frame-relay
no fair-queue
clock rate 2000000
frame-relay traffic-shaping
!
interface Serial0/0.102 point-to-point
ip address 192.168.12.1 255.255.255.0
snmp trap link-status
frame-relay interface-dlci 102
!
interface Serial0/0.103 point-to-point
ip address 192.168.13.1 255.255.255.0
snmp trap link-status
frame-relay interface-dlci 103


As shown below, the target rate is 56000b/s



This config sets a map on one of the subinterfaces shaping it to 2mbit:

map-class frame-relay TEST_MAP
frame-relay traffic-rate 2000000 2000000


interface Serial0/0.102
frame-relay class TEST_MAP


The remaining subinterface remains at 56kbps:



You can apply the map to the physical interface, the sub-interfaces then inherit these settings:



Applying other maps to the subinterfaces overrides any inherited settings:

map-class frame-relay TEST_MAP_2
frame-relay traffic-rate 128000 128000


interface Serial0/0.103
frame-relay class TEST_MAP_2





Read more...

Sunday, 13 February 2011

Frame Relay Traffic Shaping - Legacy Configuration

This is a basic lab to play around with frame-relay traffic shaping, FRTS. It uses the legacy configuration method rather than MCQ. INE have a great article here describing the other options.


This article assumes some knowledge of QoS terms such as CIR, Bc, Be and Tc.

The lab used looks like this:



I'll use the GNS3 built in frame switch to make life easier, the config is below:



The basic router configurations are:
hostname R1
!
interface Serial0/0
ip address 192.168.0.1 255.255.255.0
encapsulation frame-relay
clock rate 2000000


hostname R2
!
interface Serial0/0
ip address 192.168.0.2 255.255.255.0
encapsulation frame-relay
clock rate 2000000


In this mode no shaping is enabled, WFQ is the default for serial interfaces below E1 size (2.048mbps).


R2#show int s0/0
Serial0/0 is up, line protocol is up
Internet address is 192.168.0.2/24
Encapsulation FRAME-RELAY, loopback not set
Queueing strategy: weighted fair
Output queue: 0/1000/64/0 (size/max total/threshold/drops)
Conversations 0/1/256 (active/max active/max total)
Reserved Conversations 0/0 (allocated/max allocated)
Available Bandwidth 1158 kilobits/sec



To turn on FRTS use the commands as below:

R1(config)#int s0/0
R1(config-if)#frame-relay traffic-shaping


This gives the interface a default configuration, which is 56kbps and has Bc set to 7000bits. This can cause problems with subinterfaces as they'll end up at 56k unless configured otherwise. The queuing method is also changed to FIFO.

R1#show int s0/0
Serial0/0 is up, line protocol is up
Internet address is 192.168.0.1/24
Encapsulation FRAME-RELAY, loopback not set
Queueing strategy: fifo
Output queue: 0/40 (size/max)

R1#show traffic-shape

Interface Se0/0
       Access Target    Byte   Sustain   Excess    Interval  Increment Adapt
VC     List    Rate     Limit   bits/int bits/int  (ms)      (bytes)  Active
102            56000     875    7000      0         125       875       -


The actual configuration is done in a class map:

R1(config)#map-class frame-relay TEST_MAP


The options are configured using the frame-relay command:

R1(config-map-class)#frame-relay ?
adaptive-shaping Adaptive traffic rate adjustment, Default = none
bc Committed burst size (Bc), Default = 7000 bits
be Excess burst size (Be), Default = 0 bits
cir Committed Information Rate (CIR), Default = 56000 bps
congestion Congestion management parameters
custom-queue-list VC custom queueing
end-to-end Configure frame-relay end-to-end VC parameters
fair-queue VC fair queueing
fecn-adapt Enable Traffic Shaping reflection of FECN as BECN
fragment fragmentation - Requires Frame Relay traffic-shaping to be
configured at the interface level
holdq Hold queue size for VC
idle-timer Idle timeout for a SVC, Default = 120 sec
interface-queue PVC interface queue parameters
ip Assign a priority queue for RTP streams
mincir Minimum acceptable CIR, Default = CIR/2 bps
priority-group VC priority queueing
tc Policing Measurement Interval (Tc)
traffic-rate VC traffic rate
voice voice options

There are a couple of ways to shape traffic, the traffic-rate command sets the rate & peak rate, IOS then calculates Bc and Be based on a time interval of 125ms. To set the rate to 128kbps and the peak rate to 256kbps:
R1(config-map-class)#frame-relay traffic-rate 128000 256000
R1(config-if)#^Z
R1#show traffic-shape
Interface Se0/0
       Access Target    Byte   Sustain   Excess    Interval  Increment Adapt
VC     List    Rate     Limit   bits/int bits/int  (ms)      (bytes)  Active
102            128000  18000   128000    128000    125       2000 -

Note that Tc (interval) is still 125ms.

IOS then calculates Be as being Tc * (PIR - CIR), which is .125 * (256000 - 128000) = 16000.



You can also specifically configure the committed information rate (CIR) and Burst Excess (Be) in the map-class, this allows you to change the value of Tc which is calculated as Bc/CIR as below on R2:
map-class frame-relay TEST_MAP_R2
frame-relay cir 128000
frame-relay bc 12800

R2#show traffic-shape

Interface Se0/0
       Access Target    Byte   Sustain   Excess    Interval  Increment Adapt
VC     List    Rate     Limit   bits/int bits/int  (ms)      (bytes)  Active
201            128000    1600   12800     0             100       1600 -


You can also see the shaping configuration by looking at the PVC:
R2#show frame pvc 201

PVC Statistics for interface Serial0/0 (Frame Relay DTE)

DLCI = 201, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0

input pkts 8 output pkts 7 in bytes 622
out bytes 588 dropped pkts 0 in pkts dropped 0
out pkts dropped 0 out bytes dropped 0
in FECN pkts 0 in BECN pkts 0 out FECN pkts 0
out BECN pkts 0 in DE pkts 0 out DE pkts 0
out bcast pkts 2 out bcast bytes 68
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
pvc create time 00:27:55, last time pvc status changed 00:27:55
cir 128000 bc 12800 be 0 byte limit 1600 interval 100
mincir 64000 byte increment 1600 Adaptive Shaping none

pkts 1 bytes 34 pkts delayed 0 bytes delayed 0
shaping inactive
traffic shaping drops 0
Queueing strategy: fifo
Output queue 0/40, 0 drop, 0 dequeued

Read more...

Tuesday, 21 December 2010

Zone Based Firewall & Port Forwarding

This article covers setting up port forwarding with Cisco Zone Based Firewall (ZBF) on a typical home connection.

There are a couple of steps:
1 - Give your LAN host a static IP.
2 - Set up NAT to handle the port forwarding
3 - Set up ZBF rules to allow the traffic


1 - Static IP



You can either manually configure the client or use a DHCP reserved address.

DHCP reservation is a royal pain on the Cisco 800 series, make sure there are no existing bindings when you try to configure it (show ip dhcp bindings), the basic configuration is:

ip dhcp pool MATT_PC
host 192.168.0.10 255.255.255.0
client-identifier 01aa.aabb.bbcc.cc
client-name Matt-PC


The client-identifier is the VLAN ID prepended to the MAC address. If that fails then you can try using "hardware-address aaaa.bbbb.cccc", strangely for my dual boot system Linux only picks up the reserved address using hardware-address config and Windows 7 only works using the client-identifier option. Use whatever works!



2 - NAT forwarding



You need several bits of information, the name of the external interface, the static IP used internally, the port and protocol (TCP/UDP) that you wish to forward. The format is:

ip nat inside source static <protocol> <LAN-IP> <port> interface <external-interface> <port>


e.g. a DSL router using Dialer0 interface to forward UDP traffic on port 88 to 192.168.0.10 is:
ip nat inside source static udp 88 192.168.0.10 interface dialer0 88



3 - Set up ZBF rules to allow the traffic



This is the actual Zone Based bit. You'll need to understand the setup to tweak it, no simple guides I'm afraid.

You need to know the name of your zones, I'm using the default setup which has called the outside/internet "out-zone" and internal/LAN "in-zone". It will show up in the interface configuration under the "zone-member security XXXX" option (e.g. found with "show run interface dialer0")

You basically make a policy, then apply that policy to traffic going between two zones.

You can use CBAC with the "inspect " command but in this case I'll use an access-list instead. ACL may be your only choice if the protocol you are forwarding is not one supported by CBAC. In this case it's Xbox-live or Games for Windows so one port may be kerberos (88) but the other is not known so an ACL is used. The steps are:


3a) Set up the ACL



I'm doing XBL so I need UDP/88, UDP/3074 and TCP/3074:
ip access-list extended GFW_Incoming
permit udp any host 192.168.0.10 eq 88
permit tcp any host 192.168.0.10 eq 3074
permit udp any host 192.168.0.10 eq 3074



3b) Set up a class-map to match the ACL



class-map type inspect match-any Incoming-Traffic
match access-group name GFW_Incoming


3c) Create a policy saying what to do to the traffic



policy-map type inspect incoming-policy
class type inspect Incoming-Traffic
pass
class class-default
drop

Note that "pass" means let it through. Inspect means run CBAC on it, but it must be a recognised protocol. "Drop" should be clear enough, you could include "drop log" but it will never log anything (see note * at end).


3d) Tell ZBF where the policy applies, specifically between which zones.



zone-pair security Outside-to-Inside source out-zone destination in-zone
service-policy type inspect incoming-policy


And that should be it!

To help troubleshoot, you can do show ip access-list GFW_Incoming and see if any packets are being matched.


Final note - why drop log doesn't log anything in this case.
ZBF creates a firewall policy for traffic going between two zones. In this case our policy is for traffic going between the out-zone (WAN) and the in-zone (LAN). The traffic is "routed" between those zones by the NAT rules doing the actual port forwarding.

Traffic that comes from the internet (out-zone) and hits the firewall without triggering a NAT rule is not going anywhere near the in-zone. The relevant policy would be out-zone to self-zone (the router itself).

So the only time our out-zone to in-zone policy can ever drop traffic is if the NAT rules are forwarding but our policy does not match (the ACL in this example), i.e. if it's configured wrong! If you want to log dropped traffic then you need to specify a policy for out-zone to self-zone and use logging on that.





Read more...

Thursday, 27 May 2010

MST - Multiple Spanning Tree - Don't change the mappings!

MST allows you to create spanning-tree instances and map VLANs into them.

Combined with VTP version 3 means you can advertise the MST mappings automatically, as shown here.

MST has the concept of regions.

Whether a switch is a member of a particular region depends on three things:

  • The configured MST region name.

  • The configured MST revision number.

  • The VLAN to MSTI mappings.


The entire VLAN to MSTI mapping isn't advertised in each BPDU but a checksum of the mapping table is.

So why do I care?


Because if you change the mappings, you change the region.

If you change the region that the root bridge belongs to, it's a topology change and you trigger a total STP re-convergence.


So if you're in a live network and you tweak the VLAN-to-MSTI mappings then you'll cause a complete outage. If it's running VTPv3 then the outage will be longer as the change ripples through the network and switches "re-join" the region.

The solution


The solution in this case is fairly easy, set up all your mappings on day 1 and stick to them!

If you've two switches then you're best off finding a suitable way to distribute the VLANs, e.g.

MSTI 1 - VLANS 1 to 1999 - Root bridge SW1
MSTI 2 - VLANS 2000 to 4096 - Root bridge SW2

Then make sure you set up HSRP/VRRP in future to match where the MSTI is rooted for each particular VLAN.

Read more...

Wednesday, 26 May 2010

Bundling Frame Relay Links

Frame-relay study time!

There are several ways to bundle links together in frame-relay:

  1. Frame-relay Multilink - FRF.16

  2. PPP Multilink


FRF16 requires configuration all along the path, so the service provider must support it. PPP multilink can be used on any frame-relay links and doesn't require anything from the SP.

Frame-relay Multilink - FRF.16


I'll be using the incredibly complicated topology shown below. To keep it simple the two routers are directly connected.


By default the "encapsulation frame-relay" command configures a device as a frame-relay DTE and two DTE's back-to-back won't talk. They'll both be sending LMI status enquiries but with no DCE to answer to tell them which DLCI is active, neither router will send any traffic. Using the "no keepalive" command brings the links up and stops the LMI messaging (which transmits every 6 keepalive cycles by default).


The configs are:


hostname R1
!
interface MFR1
ip address 192.168.0.1 255.255.255.0
no keepalive
frame-relay interface-dlci 100
!
interface Serial0/0
encapsulation frame-relay MFR1
clock rate 2000000
!
interface Serial0/1
encapsulation frame-relay MFR1
clock rate 2000000

hostname R2
!
interface MFR1
ip address 192.168.0.2 255.255.255.0
no keepalive
frame-relay interface-dlci 100
!
interface Serial0/0
encapsulation frame-relay MFR1
!
interface Serial0/1
encapsulation frame-relay MFR1


You can check the multilink status as below:

R1#show frame-relay multilink
Bundle: MFR1, State = up, class = A, fragmentation disabled
BID = MFR1
Bundle links:
Serial0/1, HW state = up, link state = Up, LID = Serial0/1
Serial0/0, HW state = up, link state = Up, LID = Serial0/0


PPP Multilink


In this case I've added a frame-relay switch in the middle.


It's providing two separate VCs and they are being bundled together at either end using PPP multilink. Configs are below:


hostname R1
!
interface Serial0/0
encapsulation frame-relay
frame-relay interface-dlci 102 ppp Virtual-Template1
!
interface Serial0/1
encapsulation frame-relay
frame-relay interface-dlci 304 ppp Virtual-Template1
!
interface Virtual-Template1
ip address 192.168.0.1 255.255.255.0
ppp multilink



hostname R2
!
interface Serial0/0
encapsulation frame-relay
frame-relay interface-dlci 201 ppp Virtual-Template1
!
interface Serial0/1
encapsulation frame-relay
frame-relay interface-dlci 403 ppp Virtual-Template1
!
interface Virtual-Template1
ip address 192.168.0.2 255.255.255.0
ppp multilink


Don't need to disable the keepalives now as there's a FR switch responding to LMI status queries.

To test the bundle, use "show ppp multilink". The Virtual-Access interfaces are randomly generated by PPP from the virtual-template and may not always be the same.

R1#show ppp multilink

Virtual-Access4, bundle name is R2
Endpoint discriminator is R2
Bundle up for 00:05:50, total bandwidth 200000, load 1/255
Receive buffer limit 24384 bytes, frag timeout 1000 ms
0/0 fragments/bytes in reassembly list
0 lost fragments, 0 reordered
0/0 discarded fragments/bytes, 0 lost received
0x2 received sequence, 0x2 sent sequence
Member links: 2 (max not set, min not set)
Vi1, since 00:05:50
Vi2, since 00:05:50
No inactive multilink interfaces

R1#show int vi1
Virtual-Access1 is up, line protocol is up
Hardware is Virtual Access interface
Internet address is 192.168.0.1/24
MTU 1500 bytes, BW 100000 Kbit, DLY 100000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation PPP, LCP Open, multilink Open
Link is a member of Multilink bundle Virtual-Access4
PPPoFR vaccess, cloned from Virtual-Template1
Vaccess status 0x44
Bound to Serial0/0 DLCI 102, Cloned from Virtual-Template1, loopback not set

R1#show int vi2
Virtual-Access2 is up, line protocol is up
Hardware is Virtual Access interface
Internet address is 192.168.0.1/24
MTU 1500 bytes, BW 100000 Kbit, DLY 100000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation PPP, LCP Open, multilink Open
Link is a member of Multilink bundle Virtual-Access4
PPPoFR vaccess, cloned from Virtual-Template1
Vaccess status 0x44
Bound to Serial0/1 DLCI 304, Cloned from Virtual-Template1, loopback not set






Read more...

Monday, 24 May 2010

SNMP Trap on VSS Failover

One of the big things that Cisco VSS is missing is the ability to clearly see when it's failed over.

You set up your spanking new 6500 with 10Gig supervisors, plug it into your network management, lose one of the boxes and get a few traps about routing problems. What you really want is a big in-your-face message saying "HELP ME THE VSS JUST FAILED OVER OH MY GOD THE SKY IS FALLING!!11!one" and it's not there. It'd be fairly easy to miss that anything happened at all.

One fix is to use the Embedded Event Manager (EEM) and hook it on one of the syslog messages. There's no obvious "WAAAH HELP VSS JUST DIED" message in syslog either, but it does have a few that pop up on failover such as this one (which also occurs on first VSS bootup but I'm not too fussed about an extra trap in that case):

%PFREDUN-SW2_SPSTBY-6-ACTIVE: Initializing as Virtual Switch ACTIVE processor


EEM lets you create an action that runs when this message appears in the log, I'll make it send an SNMP trap and write a clearer syslog. The following code is beerware, feel free to re-use and re-distribute. I'd appreciate a beer if you meet me (or at least link this site):

event manager applet VSS_active_state_change
event syslog pattern ".*%PFREDUN-.*_SPSTBY-6-ACTIVE.*"
action 1.0 snmp-trap strdata "VSS Failover - New Active VSS Supervisor"
action 1.1 syslog priority alerts msg "VSS Failover - New Active VSS Supervisor"


Now you get an SNMP trap and a SYSLOG saying:

*May 24 19:49:36.267: %HA_EM-1-LOG: VSS_active_state_change: VSS Failover - New Active VSS Supervisor


You can change the %HA_EM to something else with the "facility XXX" option in the syslog line, but it doesn't seem to be present on the 6500 with SXI3 IOS.

This does need EEM traps enabled, or you still won't have any alerts:
snmp-server enable traps event-manager



Read more...