OSPF Reference Bandwidth Table

Comparison of costs that are returned for various size links when the reference bandwidth is changed.

		OSPF Interface Cost
Ref Speed	Ref BW Value	10mbit link	100mbit link	1gbit link	10gbit link
100mbit	100	10	1	1	1
1gbit	1000	100	10	1	1
10gbit	10000	1000	100	10	1
20gbit	20000	2000	200	20	2
100gbit	100000	10000	1000	100	10

Read more...

PPPoA Password Retrieval on Netgear DG834

I wanted to install a new firewall on my home DSL connection recently, nothing against the Netgear as it works great, but I had a Juniper box that I wanted to try out. The UK uses PPPoA for home DSL connections so you need a username and password to get onto the network.


The DG834 has a web interface for management, Netgear's own FAQ has help with retrieving that password if you've lost it. Once on there, you can get the basic settings which looks as follows:



The password is starred out.

It's hidden using an HTML password field, which is meant to stop someone looking over your shoulder from seeing what password you type. However that is all it aims to protect against, it's then down to the individual web application as to whether it actually strips the password from the source code...and this Netgear doesn't.

You can use your browsers "view source" option (or "this frame" -> "view source") to read the PPPoA login from the HTML code:



Read more...

Fixing VirtualBox VRMMRO.r0 Error on Ubuntu

I'd just rebuilt my eeepc recently and decided to install Virtualbox. Unfortunately after installing it from the Ubuntu repository it didn't quite work, giving the error "Failed to load VMMR0.r0 (VERR_SYMBOL_NOT_FOUND)".


From googling around it quickly became apparent that there's an easy fix, to uncomment a single line in a Makefile and re-compile the Virtualbox kernel modules. Unfortunately the package on Ubuntu won't recompile due to a mismatch of paths from the package installer and the actual build script.

The suggested steps for a standard install are:

Edit src/vboxdrv*/Makefile and uncomment the line: # VBOX_USE_INSERT_PAGE = 1

Then run the script /etc/init.d/vboxdrv setup

You'll need some extra steps to create various paths that the build script expects on Ubuntu as follows:

Edit /usr/src/vboxdrv*/Makefile and uncomment the line: # VBOX_USE_INSERT_PAGE = 1

cd /usr/src
ln -s vboxdrv-2.1.4 vboxdrv
ln -s vboxnetflt-2.1.4 vboxnetflt
cd /usr/lib/virtualbox
ln -s /usr/src .

Run /etc/init.d/virtualbox-ose setup

You can work out which links are needed by looking at the startup file /etc/init.d/virtualbox-ose, locate the setup section:

setup()
{
    stop

    [snip]

    begin_msg "Recompiling VirtualBox kernel module"
    if ! $BUILDVBOXDRV \
        --save-module-symvers /tmp/vboxdrv-Module.symvers \
        --no-print-directory install > $LOG 2>&1; then
        failure "Look at $LOG to find out what went wrong"
    fi
    if ! $BUILDVBOXNETFLT \
        --use-module-symvers /tmp/vboxdrv-Module.symvers \
        --no-print-directory install >> $LOG 2>&1; then
        failure "Look at $LOG to find out what went wrong"
    fi
    rm -f /etc/vbox/module_not_compiled
    succ_msg
    start
}

BUILDVBOXNETDRV is a variable pointing to the actual build script, it is the install path in the config file /etc/vbox/vbox.cfg, the value ends up being: "/usr/lib/virtualbox/src/vboxdrv/build_in_tmp", which doesn't exist.

If you look in /usr/lib/virtualbox then there is no src path, so create one of those to /usr/src (or you could change the INSTALL_DIR in vbox.cfg but I'm not sure what else that breaks). Then look in /usr/src, no "vboxdrv" directory, so make a link to that. Later in the build you'll run into the same thing for vboxnetflt.



Footnote:
Virtualbox is a great piece of software from Sun, although ESX may be master of the type 1 hypervisor, Virtualbox is (in my opinion) the best type 2 hypervisor around, especially for networking labs. The ability to bind physical NICs to seperate VMs is very handy and being able to do it all from a straightforward GUI in a few minutes is great. It allows Linux and XP to run in parallel on my eeepc and still get 5+ hours battery life.

Read more...

ASA Site-to-Site VPN Using Certificates from OpenSSL

To provide increased security over pre-shared keys you can authenticate VPN endpoints using certificates. Ideally this would be using a full PKI solution but a simpler method is available if you don't need revocation.

This guide is for the Cisco ASA. As with most things it's a doddle in the ASDM because you can take full advantage of the GUI and be prompted for each step, however I'll show the CLI commands.


I'll use OpenSSL as a CA, following on from the previous article. There is a Cisco guide for using Microsofts CA (search for "asa site-to-site vpn certificate microsoft" if the link dies), but OpenSSL works just as well. This guide pretty much mirrors the Microsoft one above except for the editing of certificates and OpenSSL commands used to sign the certs.

The ASA can run it's own CA, however it seems aimed squarely at supporting remote user VPNs, it doesn't support signing CSRs and the user certificates it outputs are passcode protected so not usable by another ASA in a site-to-site VPN. If Cisco include that functionality it'd be very useful, if they add SCEP features to allow enrollment of devices then it'll be very handy indeed.


The procedure is:

1. Install the CA public key onto the ASA.
2. Generate a Certificate Signing Request (CSR) on the ASA.
3. Sign the CSR using the CA.
4. Install the signed certificate onto the ASA.
5. Configure the VPN.
   

1. Install the CA public key onto the ASA

If you've followed the directions from the previous article then the public key is called cacert.pem, open this file in a text editor (wordpad rather than notepad if using Windows). On the ASA configure a CA trust point for terminal enrollment (pasting the key into the terminal), the authenticate command will then prompt you to paste the key:

crypto ca trustpoint Test_CA
enrollment terminal

crypto ca authenticate Test_CA
* At this point you'll be prompted to paste the CA public key (from cacert.pem) *

That's it, the ASA will now trust keys signed by your CA.

2. Generate a Certificate Signing Request (CSR) on the ASA.

You could just use the general keys for this but I'd make a specific key for each VPN. You then need a trustpoint where you can fine tune the certificate options. The enroll command will output the key to console.

!Generate the RSA key pair
crypto key generate rsa label FW1-Key modulus 1024

!Configure a trustpoint for the signed certificate
crypto ca trustpoint FW1-Key-trustpoint
enrollment terminal
fqdn 10.0.0.1
subject-name CN=10.0.0.1
keypair FW1-Key
crl configure

!output the CSR to console
crypto ca enroll FW1-Key-trustpoint

Copy this key into a text editor. It'll need the format changed slightly, by default the ASA outputs keys like:

MIIBnicE+P+jvc8JQBSbElTZUH8e7ULzXt8KWZQH6M/0U70wp6lqH6caHeCfDgKW
QIDAQABoDMwMQYJKoZIhbvHyAplT4ToJAATL7GwRhXqcmIpxL8k3xjP3XMkzorMZ
mLZdV8X4byX7Z9MlIEU44iWF+nePEJc60RQRgqTiLPqZcyO8besQxh74FVLW5H40
FE1/FceBzgz4ImgXMcC+0DxrNNToS2wZQDoYo7uL5I6hc9KBiV8Ki0drPe7ZOu4G
Pwdq+Ah9axuXtDua8xPe6G1bRQ9dGCgE4NSeQPnSz380WNqv6TZNddZhERlTNDoX
lbDQG0v+k5zJpCktRYa+f18IvJJxY9pVlpD4WKvyRHdaQydtudPejObY9prIg3rU
Gn1aJyCv0mjgF/JH4wnbhXgNbxBk0J1alpIEupq6Xo=

---End - This line not part of the certificate request---

But you need to edit the key so it looks like this:

-----BEGIN CERTIFICATE REQUEST-----
MIIBnicE+P+jvc8JQBSbElTZUH8e7ULzXt8KWZQH6M/0U70wp6lqH6caHeCfDgKW
QIDAQABoDMwMQYJKoZIhbvHyAplT4ToJAATL7GwRhXqcmIpxL8k3xjP3XMkzorMZ
mLZdV8X4byX7Z9MlIEU44iWF+nePEJc60RQRgqTiLPqZcyO8besQxh74FVLW5H40
FE1/FceBzgz4ImgXMcC+0DxrNNToS2wZQDoYo7uL5I6hc9KBiV8Ki0drPe7ZOu4G
Pwdq+Ah9axuXtDua8xPe6G1bRQ9dGCgE4NSeQPnSz380WNqv6TZNddZhERlTNDoX
lbDQG0v+k5zJpCktRYa+f18IvJJxY9pVlpD4WKvyRHdaQydtudPejObY9prIg3rU
Gn1aJyCv0mjgF/JH4wnbhXgNbxBk0J1alpIEupq6Xo=
-----END CERTIFICATE REQUEST-----

3. Sign the CSR using the CA.

To sign the key, copy it onto the system you've installed OpenSSL as per the previous guide. Use the following command to sign the CSR, mine's saved in a text file as "FW1-req.txt":

openssl ca -in FW1-req.txt -out FW1-signed.txt

4. Install the signed certificate onto the ASA.

To copy this certificate back to the ASA, import it for the trustpoint created earlier:

crypto ca import FW1-Key-trustpoint certificate
* Paste in the edited key as per above *

5. Configure the VPN.

The authentication method is configured in the crypto map, this will appear as follows:

crypto map outside_map 1 match address outside_1_cryptomap
crypto map outside_map 1 set pfs group2
crypto map outside_map 1 set peer 10.0.0.2
crypto map outside_map 1 set transform-set ESP-AES-128-SHA
crypto map outside_map 1 set trustpoint FW1-Key-trustpoint
crypto map outside_map interface outside

The rest of the VPN configuration is the same as pre-shared keys. Congratulations, you've got a certificate based VPN! If it's not working then try with a PSK to verify that the VPN is configured correctly, if that doesn't work then welcome to the wonderful world of debugging VPNs.

Read more...

OpenSSL CA for VPN Certificates

The aim of this article is to provide some pointers to getting a certificate authority (CA) up and running with OpenSSL and provide a few handy commands for reference. This will be used in some future articles about certificate based VPNs.


To do certificate based VPNs you need a couple of things:

• A Certificate Authority (CA) that your end devices trust.
• Host certificates signed by the CA.

I'll use the manual signing procedure which is done in several steps:

1. Install the CA public key on the end devices.
2. Create a Certificate Signing Request (CSR) on each end device.
3. Use the CA to sign the CSR.
4. Install the signed certificate back onto the device.
5. Configure the VPN to use that certificate for authentication.
   

I've used the Ubuntu docs project guide to setting up a CA, that can be found here.

Update - August 2012 - Because Ubuntu are redirecting that site to another page with a different domain name the SSL cert is being rejected by strict filtering systems, so I'm copy/pasting some of it here for reference. The following is taken from the link above:

Creating the Certificate Authority
First, create an initial working environment, for example within your home directory by issuing the following command from a terminal prompt:


cd && mkdir -p myCA/signedcerts && mkdir myCA/private && cd myCA
The above command will place you in a newly-created subdirectory of your home directory named myCA, and within this subdirectory, you should have two additional subdirectories named signedcerts and private.

Within this initial working environment, the significance of the subdirectories, and their contents is as follows:

~/myCA : contains CA certificate, certificates database, generated certificates, keys, and requests

~/myCA/signedcerts : contains copies of each signed certificate 

~/myCA/private : contains the private key

Next, create an initial certificate database in the ~/myCA subdirectory with the following command at a terminal prompt:

echo '01' > serial  && touch index.txt

Now create an initial caconfig.cnf file suitable for the creation of CA certificates. Using your favorite editor, edit the file ~/myCA/caconfig.cnf, and insert the following content into the file:

sudo nano ~/myCA/caconfig.cnf

# My sample caconfig.cnf file.
#
# Default configuration to use when one is not provided on the command line.
#
[ ca ]
default_ca      = local_ca
#
#
# Default location of directories and files needed to generate certificates.
#
[ local_ca ]
dir             = /home//myCA
certificate     = $dir/cacert.pem
database        = $dir/index.txt
new_certs_dir   = $dir/signedcerts
private_key     = $dir/private/cakey.pem
serial          = $dir/serial
#       
#
# Default expiration and encryption policies for certificates.
#
default_crl_days        = 365
default_days            = 1825
default_md              = md5
#       
policy          = local_ca_policy
x509_extensions = local_ca_extensions
#       
#
# Default policy to use when generating server certificates.  The following
# fields must be defined in the server certificate.
#
[ local_ca_policy ]
commonName              = supplied
stateOrProvinceName     = supplied
countryName             = supplied
emailAddress            = supplied
organizationName        = supplied
organizationalUnitName  = supplied
#       
#
# x509 extensions to use when generating server certificates.
#
[ local_ca_extensions ]
subjectAltName          = DNS:alt.tradeshowhell.com
basicConstraints        = CA:false
nsCertType              = server
#       
#
# The default root certificate generation policy.
#
[ req ]
default_bits    = 2048
default_keyfile = /home//myCA/private/cakey.pem
default_md      = md5
#       
prompt                  = no
distinguished_name      = root_ca_distinguished_name
x509_extensions         = root_ca_extensions
#
#
# Root Certificate Authority distinguished name.  Change these fields to match
# your local environment!
#
[ root_ca_distinguished_name ]
commonName              = MyOwn Root Certificate Authority
stateOrProvinceName     = NC
countryName             = US
emailAddress            = root@tradeshowhell.com
organizationName        = Trade Show Hell
organizationalUnitName  = IT Department
#       
[ root_ca_extensions ]
basicConstraints        = CA:true
 IMPORTANT: Make sure to adjust the obvious site-specific details in the file, such as the two instances of /home// under [ local_ca ] and [ req ]. Also change commonName, stateOrProvinceName countryName etc under [ root_ca_distinguished_name ] to personalize for your environment. For more information on the directives contained within this configuration file, use the man config command.

When you've edited the file to match your environment, save the file as ~/myCA/caconfig.cnf.

Next, you need to generate the Certificate Authority Root Certificate and Key, by issuing a few commands. First, do this:

export OPENSSL_CONF=~/myCA/caconfig.cnf
The previous command sets an environment variable, OPENSSL_CONF, which forces the openssl tool to look for a configuration file in an alternative location (in this case, ~/myCA/caconfig.cnf).

Now, generate the CA certificate and key with the following command:

openssl req -x509 -newkey rsa:2048 -out cacert.pem -outform PEM -days 1825
You should be prompted for a passphrase, and see output similar to this:


Generating a 2048 bit RSA private key
.................................+++
.................................................................................................+++
writing new private key to '/home/bshumate/myCA/private/cakey.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
Do not forget the passphrase used with the command above! You'll need it every time you want to generate and sign a new server or client certificate!

The above process will create a self-signed certificate using PEM format and RSA public/private key encryption. The certificate will be valid for 1825 days. The location, and purpose of the resultant files is as follows:

~/myCA/cacert.pem : CA public certificate

~/myCA/private/cakey.pem : CA private key

Optional Step
Strip the certificate from all its text to keep only the -CERTIFICATE- section to create a crt

openssl x509 -in cacert.pem -out cacert.crt
Creating a Self-Signed Server Certificate
Now that you have a Certificate Authority configured, you may use it to sign self-signed certificates. Prior to beginning the steps below, you may wish to encrypt the certificate's private key with a passphrase. The advantages of encrypting the key with a passphrase include protection of the certificate in the event it is stolen. 

The certificate cannot be used with SSL-enabled applications without entering the passphrase every time the SSL-enabled application is started. This condition, while being most secure, can present a problem: If the server must be started in an unattended manner as in the case of a computer restart, then no one will be available to enter the passphrase, and subsequently the server will not start. One way to eliminate this condition involves a trade-off in security: The key may be decrypted, to remove the passphrase necessity; thus SSL-enabled applications will start automatically, without a need for you to enter a passphrase.

To actually generate a self-signed certificate for use with an SSL application, follow this process:

Create the server configuration file, by editing ~/myCA/exampleserver.cnf with your favorite text editor. Add this example content:


#
# exampleserver.cnf
#

[ req ]
prompt                  = no
distinguished_name      = server_distinguished_name

[ server_distinguished_name ]
commonName              = tradeshowhell.com
stateOrProvinceName     = NC
countryName             = US
emailAddress            = root@tradeshowhell.com
organizationName        = My Organization Name
organizationalUnitName  = Subunit of My Large Organization
Be sure to change the values under server_distinguished_name especially the commonName value. The commonName value must match the host name, or CNAME for the host you wish to use the key for. If the commonName does not match the intended hostname, then host / certificate mismatch errors will appear in the client applications of clients attempting to access the server.

Once you've edited the file appropriately, save it as ~/myCA/exampleserver.cnf. Generate the server certificate, and key with the following commands:

export OPENSSL_CONF=~/myCA/exampleserver.cnf

The previous command sets an environment variable OPENSSL_CONF which forces the openssl tool to look for a configuration file in an alternative location (in this case, ~/myCA/exampleserver.cnf).

Now generate the certificate, and key:

openssl req -newkey rsa:1024 -keyout tempkey.pem -keyform PEM -out tempreq.pem -outform PEM

You should be prompted for a passphrase, and see output similar to this:


Generating a 1024 bit RSA private key
...++++++
...............++++++
writing new private key to 'tempkey.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
Don't forget the passphrase!

Next, you may translate the temporary private key into an unencrypted key by using the following command:

openssl rsa < tempkey.pem > server_key.pem

You should be prompted for the passphrase used above, and see the following output:


Enter pass phrase:
writing RSA key
If you wish to leave the key encrypted with a passphrase, simply rename the temporary key using the following command, instead of following the step above:

mv tempkey.pem server_key.pem

 Remember: If you use a server key encrypted with a passphrase, the passphrase will have to be entered each time the server application using the encrypted key is started. This means the server application will not start unless someone, or something enters the key.

Now you need to sign the server certificate with the Certificate Authority (CA) key using these commands:

export OPENSSL_CONF=~/myCA/caconfig.cnf

The previous command modifies the environment variable OPENSSL_CONF which forces the openssl tool to look for a configuration file in an alternative location (in this case, ~/myCA/caconfig.cnf to switch back to the CA configuration).

Then sign the certificate as follows:

openssl ca -in tempreq.pem -out server_crt.pem

You will be prompted for the passphrase of the CA key as created in the Certificate Authority setup from above. Enter this passphrase at the prompt, and you will then be prompted to confirm the information in the exampleserver.cnf, and finally asked to confirm signing the certificate. Output should be similar to this:


Using configuration from /home/bshumate/myCA/caconfig.cnf
Enter pass phrase for /home/bshumate/myCA/private/cakey.pem:
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :PRINTABLE:'tradeshowhell.com'
stateOrProvinceName   :PRINTABLE:'NC'
countryName           :PRINTABLE:'US'
emailAddress          :IA5STRING:'root@tradeshowhell.com'
organizationName      :PRINTABLE:'Trade Show Hell'
organizationalUnitName:PRINTABLE:'Black Ops'
Certificate is to be certified until Jan  4 21:50:08 2011 GMT (1825 days)
Sign the certificate? [y/n]:y


1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
Remove the temporary certificate, and key files with the following command:

rm -f tempkey.pem && rm -f tempreq.pem

Congratulations! You now have a self-signed server application certificate, and key pair:

server_crt.pem : Server application certificate file

server_key.pem : Server application key file

END OF COPY/PASTE

The instructions should work for any Unix system with OpenSSL.

The only changes I've made to their configuration were the hostname/path/DN changes and to alter policy about which fields needed to be supplied in the CSRs. This is just to save filling out every field each time I want to change certificates in the lab.

[ local_ca_policy ]
commonName              = supplied
stateOrProvinceName     = optional
countryName             = optional
emailAddress            = optional
organizationName        = optional
organizationalUnitName  = optional

With your directory structure created as per that article, config file created and appropriate environmental variables set you have a functional CA.

Handy OpenSSL Commands

Generate the CA public and private keys:

openssl req -x509 -newkey rsa:1024 -out cacert.pem -outform PEM -days 1825

Show information on a certificate:

openssl x509 -text -in <CERT>

Show information in a CSR:

openssl req -text -noout -in <CSR>

Verify the signature of a CSR:

openssl req -text -noout -in <CSR> -verify

Sign a CSR:

openssl ca -in <CSR> -out <SIGNED-CERT>

Read more...

Checkpoint Traffic Sniffing

There are a couple of handy commands for sniffing traffic on Checkpoint.

Tcpdump and fw monitor.


The following runs a tcpdump capture on IPSO, snagging the entire packet to a file:

tcpdump -s1700 -w output.cap -ni <interface> host <IP-addr>

Without the -s1700 it'll just grab the first part of each packet and not the full contents. If you're running it on a newer platform (e.g. SPLAT) then you can probably use -s0 instead, it's to specify the size of the packets that are recorded.

The problem with tcpdump is that running it on a device designed to filter traffic means you may not be seeing exactly what you expect. So it's probably better to use "fw monitor", the Checkpoint tool that lets you specify exactly where in the stack you are sniffing packets from.

The official guide is here.

Not much more to say about it except to give a simple example for reference:

fw monitor -e 'accept src=<src-ip> or dst=<dst-ip>;'

Just remember to terminate each line with a semi-colon.

Read more...

MPLS: EIGRP as CE-PE Routing Protocol

Following the OSPF sham links and OSPF as PE-CE routing protocol articles, this entry shows how to use EIGRP as the PE-CE routing protocol.

The network topology is as before.


EIGRP is fairly easy to configure in this case. The CE router is just configured as per a vanilla EIGRP network with no special entries needed, on router CE1:

CE1#show ip int brief | inc up
FastEthernet0/0 10.0.255.1 YES NVRAM up up
FastEthernet0/1 172.16.1.1 YES NVRAM up up
CE1#show run | section eigrp
router eigrp 1
network 10.0.0.0
network 172.16.0.0
no auto-summary

On the PE router it is fairly straightforward, EIGRP is configured with an "address-family ipv4 vrf VRFNAME" command. The bit to note is the "autonomous-system X" line which tells the router what EIGRP AS is running in the VRF.

PE1#show run | section eigrp
router eigrp 1
auto-summary
!
address-family ipv4 vrf VPN_ONE
redistribute bgp 65001 metric 10000 20 255 1 1500
network 10.0.255.0 0.0.0.3
no auto-summary
autonomous-system 1

Any show commands on the PE router must include the VRF statement, e.g.:

PE1#show ip eigrp vrf VPN_ONE neighbors
IP-EIGRP neighbors for process 1
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 10.0.255.1 Fa0/0 11 00:06:48 6 200 0 5

As with OSPF (solved by sham links), there is potential for problems with EIGRP if you have backdoor links between sites. The administrative distance of EIGRP internal routes is 90 whereas the iBGP routes AD is 200. To avoid this the pre-bestpath cost BGP extended community value is used.

The Cisco article on the cost community is here. Basically it allows the BGP routing decision process to be overridden by the cost community value which is split into two parts, the first being called the POI (Point-of-Insertion) which is the position in the routing process (e.g. before the normal BGP decision hooks such as weight, local pref, route origin) and the second part being a value that is copied from the EIGRP composite metric. This allows routes to be chosen based purely on the EIGRP metric and BGP factors are ignored.

Unlike sham links this is automatically enabled, here is the route to 172.16.2.0 as seen by BGP:

PE1#show ip bgp vpnv4 vrf VPN_ONE 172.16.2.0
BGP routing table entry for 1:1:172.16.2.0/24, version 11
Paths: (1 available, best #1, table VPN_ONE)
Not advertised to any peer
Local
10.255.255.52 (metric 129) from 10.255.255.52 (10.255.255.52)
Origin incomplete, metric 100, localpref 100, valid, internal, best
Extended Community: RT:1:1 Cost:pre-bestpath:128:307200 0x8800:32768:0
0x8801:1:51200 0x8802:65281:256000 0x8803:65281:1500
mpls labels in/out nolabel/23

The route is then received on CE1 with an advertised metric of 307200:

CE1#show ip eigrp top 172.16.2.0 255.255.255.0
IP-EIGRP (AS 1): Topology entry for 172.16.2.0/24
State is Passive, Query origin flag is 1, 1 Successor(s), FD is 332800
Routing Descriptor Blocks:
10.0.255.2 (FastEthernet0/0), from 10.0.255.2, Send flag is 0x0
Composite metric is (332800/307200), Route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 3000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 2

Read more...

OSPF Sham Links

If you install a backup link between sites in an MPLS VPN then you can run into problems as shown below.

This article follows on from OSPF as a PE-CE routing protocol and uses the same network layout.

I've set everything into area 0 to simplify things a bit, however there is now a serial link between the two customer sites. Topology is shown below:


Before the link is brought up the routing table on CE1 looks as follows, the important route is the 172.16.2.0 (CE2 site LAN) which is currently reached via the MPLS WAN:

CE1#show ip route

    172.16.0.0/24 is subnetted, 2 subnets
C       172.16.1.0 is directly connected, FastEthernet0/1
O IA    172.16.2.0 [110/20] via 10.0.255.2, 00:01:15, FastEthernet0/0
    10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
O E2    10.254.254.52/32 [110/10] via 10.0.255.2, 00:01:10, FastEthernet0/0
O IA    10.0.255.4/30 [110/20] via 10.0.255.2, 00:01:15, FastEthernet0/0
C       10.0.255.0/30 is directly connected, FastEthernet0/0

The link is brought up using 192.168.255.0/24 and OSPF is enabled across it. As it's a backup link the cost is set accordingly.

CE1#show run int s0/0
Building configuration...

Current configuration : 142 bytes
!
interface Serial0/0
description Backup Link to Site 2
ip address 192.168.255.1 255.255.255.0
ip ospf cost 30000
clock rate 2000000

CE1#show ip route          
[snip]
    172.16.0.0/24 is subnetted, 2 subnets
C       172.16.1.0 is directly connected, FastEthernet0/1
O       172.16.2.0 [110/30010] via 192.168.255.2, 00:00:51, Serial0/0
    10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
O E2    10.254.254.51/32 [110/10] via 192.168.255.2, 00:00:51, Serial0/0
O E2    10.254.254.52/32 [110/10] via 10.0.255.2, 00:00:51, FastEthernet0/0
O       10.0.255.4/30 [110/30010] via 192.168.255.2, 00:00:51, Serial0/0
C       10.0.255.0/30 is directly connected, FastEthernet0/0
C    192.168.255.0/24 is directly connected, Serial0/0

The problem is that the remote site is now being routed via the backup link, which is a 2mbit line, when there is a 100mbit ethernet link available. The reason for this is due to the administrative distance.

There are two routes being advertised to CE1 for the 172.16.2.0 network, one via iBGP and one via OSPF. The administrative distance of iBGP is 200 and OSPF is 110 so the OSPF route is chosen. The bandwidth of the link isn't compared because of the order of route selection:

1. Prefix Length.
2. Administrative Distance (configured then default).
   
3. Routing Protocol Metric.
   

The AD is compared before the metric so the routing process doesn't get as far as looking at the bandwidth of the link or the configured OSPF cost, it's already decided that OSPF is preferable to iBGP and will use it whatever.

The way around this is to use a Sham link. This is basically fudging the routing decision process by running an OSPF link across the MPLS WAN so that you have matching ADs and then can use the metric to decide which route is preferred.


The configuration is simple enough, you need a loopback with a /32 address on each PE router. That address must be advertised into the VRF across BGP. We've already got this with loopback 1 so the configuration is simply:

PE1#show run int loop1
Building configuration...

Current configuration : 96 bytes
!
interface Loopback1
 ip vrf forwarding VPN_ONE
 ip address 10.254.254.51 255.255.255.255
end
!
PE1#show run | section ospf
router ospf 100 vrf VPN_ONE
 router-id 10.254.254.51
 area 0 sham-link 10.254.254.51 10.254.254.52 cost 10

With the opposing configuration on PE2. The OSPF neighbors on PE1 then show an additional neighbor across the Sham Link:

PE1#show ip ospf 100 nei   

Neighbor ID     Pri   State           Dead Time   Address         Interface
10.254.254.52     0   FULL/  -           -        10.254.254.52   OSPF_SL0
172.16.1.1        1   FULL/DR         00:00:34    10.0.255.1      FastEthernet0/0

PE1#show ip ospf 100 int
OSPF_SL0 is up, line protocol is up 
  Internet Address 0.0.0.0/0, Area 0 
  Process ID 100, Router ID 10.254.254.51, Network Type SHAM_LINK, Cost: 10

Now the routing table on CE1 shows that the MPLS WAN is being used for all traffic:

CE1#show ip route
[snip]
     172.16.0.0/24 is subnetted, 2 subnets
C       172.16.1.0 is directly connected, FastEthernet0/1
O       172.16.2.0 [110/40] via 10.0.255.2, 00:07:56, FastEthernet0/0
     10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
O E2    10.254.254.51/32 [110/10] via 10.0.255.2, 00:07:56, FastEthernet0/0
O E2    10.254.254.52/32 [110/10] via 10.0.255.2, 00:07:56, FastEthernet0/0
O       10.0.255.4/30 [110/30] via 10.0.255.2, 00:07:56, FastEthernet0/0
C       10.0.255.0/30 is directly connected, FastEthernet0/0
C    192.168.255.0/24 is directly connected, Serial0/0

Read more...

MPLS: OSPF as PE-CE Routing Protocol

This article shows a basic configuration for using OSPF as the PE-CE routing protocol.

It follows on from the basic VRFs entry and uses the same network topology, with a couple of networks added to represent each sites internal LAN.

OSPF uses a hierarchical network structure where normally all areas would be connected directly to area 0. In the case of MPLS VPNs, there is always a redistribution to and from BGP in the middle of the network. The way MPLS is implemented avoids these routes being seen as external by using the concept of an MPLS VPN Superbackbone above area 0.

The OSPF network looks as below:


In a vanilla network assuming redistribution was set up everywhere, 172.16.1.0/24 (sourced at CE1) would be redistributed back to OSPF by PE2 as a type-5 external route.

In this case is PE1 redistributes the route from OSPF into BGP and adds extended communities to inform it's peer about the OSPF attributes of the route. The image below shows a capture of the BGP update message from PE1 advertising the 172.16.1.0 route (CE1 LAN). Note the extended attibutes in the update message, you can also see the label mappings advertised in the NRLI section.




PE2 can then take this information and rebuild the route advertisement as an OSPF type-3 (summary) LSA. The output below shows how PE2 receives the route from MP-BGP, containing the domain ID (corresponds to the OSPF process ID), route type (format area:type:option) and the advertising router ID.

PE2#show ip bgp vpnv4 vrf VPN_ONE 172.16.1.0
BGP routing table entry for 1:1:172.16.1.0/24, version 32
Paths: (1 available, best #1, table VPN_ONE)
Flag: 0x820
  Not advertised to any peer
  Local
    10.255.255.51 (metric 129) from 10.255.255.51 (10.255.255.51)
      Origin incomplete, metric 10, localpref 100, valid, internal, best
      Extended Community: RT:1:1 OSPF DOMAIN ID:0x0005:0x000000640200 
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.254.254.51:0
      mpls labels in/out 23/26

The OSPF metric is copied to/from the BGP MED value.

This route ends up in CE2 as an OSPF Summary (type-3 LSA):

CE2#show ip ospf database summary 172.16.1.0

            OSPF Router with ID (172.16.2.1) (Process ID 100)

  Summary Net Link States (Area 0)

  Routing Bit Set on this LSA
  LS age: 618
  Options: (No TOS-capability, DC, Downward)
  LS Type: Summary Links(Network)
  Link State ID: 172.16.1.0 (summary Network Number)
  Advertising Router: 10.254.254.52
  LS Seq Number: 80000001
  Checksum: 0x5167
  Length: 28
  Network Mask: /24
 TOS: 0  Metric: 10 

The relevant configurations for one half of the network is below (CE1/PE1), the other half is configured in the same way but with different IP addresses. No configuration is needed on the P routers as they purely switch based on labels arranged by the PE routers, P routers don't care about the customer VPNs.

It's worth noting that all the funky business is going on at the PE router which would be managed by the service provider and not accessible to the customer.

PE1

ip vrf VPN_ONE
 rd 1:1
 route-target export 1:1
 route-target import 1:1

interface Loopback1
 ip vrf forwarding VPN_ONE
 ip address 10.254.254.51 255.255.255.255

interface FastEthernet0/0
 description To CE1
 ip vrf forwarding VPN_ONE
 ip address 10.0.255.2 255.255.255.252

router ospf 100 vrf VPN_ONE
 router-id 10.254.254.51
 redistribute bgp 65001 metric 10 subnets
 network 10.0.255.2 0.0.0.0 area 0

router bgp 65001
[snip - see previous articles for full BGP config]
 !
 address-family ipv4 vrf VPN_ONE
  redistribute connected
  redistribute ospf 100 vrf VPN_ONE metric 10 match internal external 1 external 2
 exit-address-family



CE1
interface FastEthernet0/0
 description To PE1
 ip address 10.0.255.1 255.255.255.252

interface FastEthernet0/1
 ip address 172.16.1.1 255.255.255.0

router ospf 100
 log-adjacency-changes
 network 10.0.255.1 0.0.0.0 area 0
 network 172.16.1.1 0.0.0.0 area 101

Read more...

MPLS Lab #3 -Simple VRFs

This follows on from the previous article part 2.

In this article I'll get the customer sites connected up in a very simple VRF.


The first step is to create a VRF for the customer sites to use. This is done by naming it on each PE router, assigning a route distinguisher (RD) and setting route targets (RT) for BGP to use. The simplest way to do this is to allocate the RD in the format AS:nn where AS is the customers autonomous system number, then to assign the same value to the RT.

In this case the customers AS will be 1 so the configuration is:

ip vrf VPN_ONE
rd 1:1
route-target export 1:1
route-target import 1:1

This creates the VRF on the two PE routers, to make use of them the interfaces leading to the CE routers have to be put into the VRF:

PE1#show run int fastEthernet 0/0
Building configuration...

Current configuration : 153 bytes
!
interface FastEthernet0/0
description To CE1
ip vrf forwarding VPN_ONE
ip address 10.0.255.2 255.255.255.252
end

PE2#show run int fastEthernet 0/0
Building configuration...

Current configuration : 153 bytes
!
interface FastEthernet0/0
description To CE2
ip vrf forwarding VPN_ONE
ip address 10.0.255.5 255.255.255.252
end

At this stage the CE routers can contact their directly connected PE router, but they cannot contact each other. They are also completely segregated from the backbone of the service providers network. They are given a default route via the PE router:

CE1#show ip route
[snip]
S*   0.0.0.0/0 [1/0] via 10.0.255.2


CE1 to PE1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.255.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/5/8 ms

CE1 to CE2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.255.6, timeout is 2 seconds:
U.U.U
Success rate is 0 percent (0/5)
CE1#ping 10.0.0.9

CE1 to P1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.9, timeout is 2 seconds:
U.U.U
Success rate is 0 percent (0/5)

To allow communication between the two customer sites (the two CE routers), we need to have MP-BGP advertising the vpnv4 routes between the two PE routers. Because there aren't really any customer sites yet, just the links that are directly connected between CE and PE, we can "redistribute connected" routes on the PE routers so that MP-BGP advertises them with the RD for this customers VRF.

This is done as follows:

PE1#config term
Enter configuration commands, one per line.  End with CNTL/Z.
PE1(config)#router bgp 65001
PE1(config-router)#address-family ipv4 vrf VPN_ONE
PE1(config-router-af)#redistribute connected

With the same on PE2. Now we can look at the BGP routes specific to this VPN and see the connected interfaces are being advertised:

PE1#show ip bgp vpnv4 vrf VPN_ONE
BGP table version is 17, local router ID is 10.255.255.51
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
             r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

  Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf VPN_ONE)
*> 10.0.255.0/30    0.0.0.0                  0         32768 ?
*>i10.0.255.4/30    10.255.255.52            0    100      0 ?

As the VRF instances on the two PE routers can now see each other, they are able to forward traffic between the two CE routers. The customer sites are now linked up:

CE1#ping 10.0.255.6     

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.255.6, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/11/16 ms

So a very basic network setup, two sites able to communicate over a WAN. The PE router configuration is given below, snipped for brevity. Points to note are:

• VRF VPN_ONE defined.
• OSPF running internal to the Service Provider WAN, not visible to CE1 attached to f0/0.
• BGP running between PE1 and PE2, advertising vpnv4 routes to allow the VRF communications.
  

hostname PE1
!
ip vrf VPN_ONE
rd 1:1
route-target export 1:1
route-target import 1:1
!
interface Loopback0
ip address 10.255.255.51 255.255.255.255
!
interface FastEthernet0/0
description To CE1
ip vrf forwarding VPN_ONE
ip address 10.0.255.2 255.255.255.252
!
interface Serial0/0
description To P1
ip address 10.0.0.1 255.255.255.252
mpls ip
!
interface Serial0/1
description To P2
ip address 10.0.0.5 255.255.255.252
mpls ip
!
router ospf 1
router-id 10.255.255.51
network 10.0.0.0 0.255.255.255 area 0
!
router bgp 65001
bgp router-id 10.255.255.51
neighbor 10.255.255.52 remote-as 65001
neighbor 10.255.255.52 update-source Loopback0
!
address-family ipv4
 neighbor 10.255.255.52 activate
exit-address-family
!
address-family vpnv4
 neighbor 10.255.255.52 activate
 neighbor 10.255.255.52 send-community extended
exit-address-family
!
address-family ipv4 vrf VPN_ONE
 redistribute connected
exit-address-family

To try and show a bit more of the segregation that is now in place, look at the routing table on PE1. Note that it does not contain any information from the VRF VPN_ONE on router PE2 (such as the CE2 IP address of 10.0.255.6), also note that PE2 is not able to directly contact CE2, although it can contact it via the VRF "VPN_ONE":

PE1show ip route 
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     10.0.0.0/8 is variably subnetted, 8 subnets, 2 masks
O       10.0.0.8/30 [110/128] via 10.0.0.2, 00:26:46, Serial0/0
O       10.0.0.12/30 [110/128] via 10.0.0.6, 00:26:46, Serial0/1
C       10.0.0.0/30 is directly connected, Serial0/0
C       10.0.0.4/30 is directly connected, Serial0/1
C       10.255.255.51/32 is directly connected, Loopback0
O       10.255.255.52/32 [110/129] via 10.0.0.6, 00:26:46, Serial0/1
                         [110/129] via 10.0.0.2, 00:26:46, Serial0/0
O       10.255.255.102/32 [110/65] via 10.0.0.6, 00:26:46, Serial0/1
O       10.255.255.101/32 [110/65] via 10.0.0.2, 00:26:46, Serial0/0
PE1ping 10.0.255.6

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.255.6, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
PE1#ping vrf VPN_ONE ip 10.0.255.6

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.255.6, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/6/8 ms

The following diagram should help make several things clear:


The first thing is that I'm a network engineer and not an artist for good reason!
The second is that we've now managed to overlay a customer network onto the service provider network, the two customer sites are segregated IP networks from the SP backbone. The network VPN_ONE is shown by the grey shaded areas and bounded by red.


Read more...

Minimal Downtime Checkpoint Upgrade

This is an old process I worked out for an R55->R65 upgrade on a Nokia VRRP cluster to provide minimal downtime. In lab tests this worked with under a second of lost packets. In reality I wouldn't bet on a busy cluster being upgraded without losing traffic under any circumstances!

My preferred method is to wipe everything completely and re-build from config backups rather than try to do software upgrades.


Step 0 is to make sure you have printed copies of all documents (incase you kill the internet), copies of both old and new IPSO, Checkpoint, HFAs any CDs you need and all passwords.


First steps are to upgrade the Server, a newer version server can manage old enforcement modules.

1 - Run upgrade_export on the Smart Center Server (SCS) and save the file remotely.
2 - If you have a backup SCS then run cpstop on it, keep it for backout!
3 - Run cpclean on the SCS to remove all Checkpoint software. Run the CD install.
4 - Use upgrade_import to load the config exported in step 2.
5 - Sort out the licenses (test this thoroughly in advance or allow hours/days/weeks for this step)
6 - Check the server is talking to the enforcement modules correctly.
7 - Backup server can be updated at any point from here on, personally I'd leave it until much later to be sure we don't need to roll back to it.

- SCS now updated -

8 - Run the IPSO backups on all enforcement modules (firewalls).
9 - Turn off "Firewall Monitoring" in the VRRP options on all firewalls.
10 - Set the cluster version to be R65 in Smart Dashboard.
11 - Reboot a backup firewall and run the IPSO install from the boot manager, also install the R65 wrapper.
* This may not work with anonymous FTP, best to set a username/pass on FTP server *
12 - When the box is rebuilt, log in via SSH, edit the file $FWDIR/lib/webgui_client.def and add the IP address of the PC you want to use to access Voyager initially.
13 - Run comp_init_policy -g to regenerate the initial firewall policy with the new client definition.
14 - Reboot the box.
15 - Install any HFAs, check whether the client.def file needs editing again each time (HFA may overwrite it).
16 - Log into the system on voyager, load the IPSO backup file from earlier. Turn off VRRP monitor firewall.
* Check the package and IPSO image configuration, this may have reverted back to the old versions, you may need to re-install Checkpoint packages to get these settings back to sensible values *
17 - Push policy from Smart Dashboard, it should install to the updated firewall only.

* OUTAGE COMING UP!! *
You can use cphaprob state to see whether you have a cluster working but it's unlikely due to the version differences

18 - Force VRRP to fail over, either by editing the priority values or by disabling/disconnecting a monitored interface on the master unit.

Make sure you edit the priorities anyway on the upgraded unit so it remains the master when you restore the IPSO config later onto the next unit.

Don't just turn the priority down on the (old) master as the IPSO restore will reset them and you may end up with a VRRP master that has no firewall software loaded and just acts as a router!

19 - Run any traffic tests you have, decide whether the new version is working properly and whether you want to go past the point of no return! You can still fail back to the old version at this stage.

20 - Run steps 11-17 on the remaining firewall.

21 - Push policy to the entire cluster, verify all units accept it. Check that the cluster is talking with "cphaprob state"
22 - Set VRRP values back to the original state, re-enable "monitor firewall".

23 - Test out the new firewalls. Include

Hopefully it's all worked and you're now on the new versions.

Read more...

Useful Checkpoint Commands

Some useful Checkpoint commands for reference:

fw unloadlocal - Unload the local policy.
fw stat - Show the policy version that is currently installed.
fw log - Show the log file.
fw ver - Show the installed Checkpoint version.
fw lslogs - List all log files available.
fw logswitch - Force a log cycle.


cp_conf sic state - Show SIC (secure internal comms) status.
cp_conf ha enable/disable - Enable/Disable HA (not sure what difference is to cphastop/cphastart)
cp_conf lic get - Show installed licenses - same as "cplic print"

cpstat - Show module status
cpstat -f routing os - Show routing table.
cpstat -f ifconfig os - Show interface configurations.
cpstat -f accelerator vpn - Show VPN hardware status.

cpwd_admin - Checkpoint process command.
cpwd_admin list - Show running processes.

vpn debug - Turns on VPN debugging, logs to $FWDIR/log/vpnd.elg
vpn debug ikeon - Turns on debugging to $FWDIR/log/IKE.elg (needs IKEView application to read)

vpn tu - VPN Tunnel Utility

vpn ver - Shows VPN version installed (same as fw ver in any NG/NGX).

iclid - Basic shell to provide IOS like commands.


Clustering / HA commands (These also work for VRRP HA)

cphaprob state - Show clustering status (this should list all firewalls in the cluster or something isn't working).
cphastart / cphastop - Start/Stop the High Availability/Clustering.
cphaconf - HA configuration from command line.
cphaconf set_ccp broadcast/multicast - Set the Cluster Control Protocol to use broadcast or multicast

Read more...

MPLS Lab Part 2

Follows on from part 1.

Now to get MP-BGP up and running on the MPLS lab. BGP-4 (as described in RFC 1771) can only carry IPv4 prefixes. RFC 2858 adds multiprotocol capability to BGP-4, which is needed to work with the vpnv4 routes that MPLS uses.


vpnv4?
In an MPLS network you may have customers with overlapping IP address space. In order to provide unique addressing, a vpnv4 prefix is used consisting of a Route Distinguisher (RD) followed by the IP prefix. The RD is 64 bits and typically in format AS:nn (or IP:nn). For example if you use an RD of 65001:1 and your IP prefix is 10.0.255.0/24 then the vpnv4 prefix is 65001:1:10.0.255.0/24.


The reason things are like this is because you have many customers sharing the same WAN, they are seperated by an MPLS VPN between their sites. This is done by using MP-BGP relationships as shown in the diagram below:



In this case I'm just setting up a single customer so the network looks like this:




Firstly I'll set up vanilla BGP-4 between PE1 and PE2, specifying to use the loopback interfaces for the peer relationships:

PE1#sh run | section bgp
router bgp 65001
 no synchronization
 bgp router-id 10.255.255.51
 bgp log-neighbor-changes
 neighbor 10.255.255.52 remote-as 65001
 neighbor 10.255.255.52 update-source Loopback0
 no auto-summary


PE1#show run | section bgp
router bgp 65001
 no synchronization
 bgp router-id 10.255.255.51
 bgp log-neighbor-changes
 neighbor 10.255.255.52 remote-as 65001
 neighbor 10.255.255.52 update-source Loopback0
 no auto-summary

When watching the debugs this gives:

*Mar  1 00:15:42.659: BGP: 10.255.255.52 passive open to 10.255.255.51
*Mar  1 00:15:42.659: BGP: 10.255.255.52 went from Active to Idle
*Mar  1 00:15:42.659: BGP: 10.255.255.52 went from Idle to Connect
*Mar  1 00:15:42.659: BGP: 10.255.255.52 rcv message type 1, length (excl. header) 26
*Mar  1 00:15:42.659: BGP: 10.255.255.52 rcv OPEN, version 4, holdtime 180 seconds
*Mar  1 00:15:42.659: BGP: 10.255.255.52 went from Connect to OpenSent
*Mar  1 00:15:42.659: BGP: 10.255.255.52 sending OPEN, version 4, my as: 65001, holdtime 180 seconds
*Mar  1 00:15:42.659: BGP: 10.255.255.52 rcv OPEN w/ OPTION parameter len: 16
*Mar  1 00:15:42.659: BGP: 10.255.255.52 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
*Mar  1 00:15:42.659: BGP: 10.255.255.52 OPEN has CAP
PE1#ABILITY code: 1, length 4
*Mar  1 00:15:42.659: BGP: 10.255.255.52 OPEN has MP_EXT CAP for afi/safi: 1/1
*Mar  1 00:15:42.659: BGP: 10.255.255.52 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
*Mar  1 00:15:42.659: BGP: 10.255.255.52 OPEN has CAPABILITY code: 128, length 0
*Mar  1 00:15:42.659: BGP: 10.255.255.52 OPEN has ROUTE-REFRESH capability(old) for all address-families
*Mar  1 00:15:42.659: BGP: 10.255.255.52 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
*Mar  1 00:15:42.659: BGP: 10.255.255.52 OPEN has CAPABILITY code: 2, length 0
*Mar  1 00:15:42.659: BGP: 10.255.255.52 OPEN has ROUTE-REFRESH capability(new) for all address-families 
BGP: 10.255.255.52 rcvd OPEN w/ remote AS 65001
*Mar  1 00:15:42.659: BGP: 10.255.255.52 went from OpenSent to OpenConfirm
*Mar  1 00:15:42.659: BGP: 10.255.255.52 send message type 1, length (incl. header) 45
*Mar  1 00:15:42.663: BGP: 10.255.255.52 went from OpenConfirm to Established
*Mar  1 00:15:42.663: %BGP-5-ADJCHANGE: neighbor 10.255.255.52 Up 

The neighbor relationship is up

PE1#sh ip bgp nei
BGP neighbor is 10.255.255.52,  remote AS 65001, internal link
  BGP version 4, remote router ID 10.255.255.52
  BGP state = Established, up for 00:01:42
  Last read 00:00:41, last write 00:00:41, hold time is 180, keepalive interval is 60 seconds
  Neighbor capabilities:
    Route refresh: advertised and received(old & new)
    Address family IPv4 Unicast: advertised and received

Two important bits are missing, the neighbor does not show capability "Address family VPNv4 Unicast" and the afi/safi values in the debug output don't show support for VPNv4.

*Mar  1 00:15:42.659: BGP: 10.255.255.52 OPEN has MP_EXT CAP for afi/safi: 1/1

The Address Family Identifier (AFI) and Subsequent Address Family Identifier (SAFI) values can be found on the IANA website: AFI and SAFI. An AFI of 1 is IPv4, SAFI of 1 is NRLI for unicast forwarding and SAFI of 128 is for labelled VPN forwarding.

To activate the vpnv4 support, a bit more configuration is required:

PE1#show run | section bgp
router bgp 65001
 bgp router-id 10.255.255.51
 bgp log-neighbor-changes
 neighbor 10.255.255.52 remote-as 65001
 neighbor 10.255.255.52 update-source Loopback0
 !
 address-family ipv4
  neighbor 10.255.255.52 activate
  no auto-summary
  no synchronization
 exit-address-family
 !
 address-family vpnv4
  neighbor 10.255.255.52 activate
  neighbor 10.255.255.52 send-community extended
 exit-address-family





router bgp 65001
 bgp router-id 10.255.255.52
 bgp log-neighbor-changes
 neighbor 10.255.255.51 remote-as 65001
 neighbor 10.255.255.51 update-source Loopback0
 !
 address-family ipv4
  neighbor 10.255.255.51 activate
  no auto-summary
  no synchronization
 exit-address-family
 !
 address-family vpnv4
  neighbor 10.255.255.51 activate
  neighbor 10.255.255.51 send-community extended
 exit-address-family

Now the debug line in question shows:

*Mar  1 00:21:31.183: BGP: 10.255.255.52 OPEN has MP_EXT CAP for afi/safi: 1/128

And the neighbor capabilities are listed as:

PE1#show ip bgp neigh
BGP neighbor is 10.255.255.52,  remote AS 65001, internal link
  BGP version 4, remote router ID 10.255.255.52
  BGP state = Established, up for 00:00:54
  Last read 00:00:54, last write 00:00:54, hold time is 180, keepalive interval is 60 seconds
  Neighbor capabilities:
    Route refresh: advertised and received(old & new)
    Address family IPv4 Unicast: advertised and received
    Address family VPNv4 Unicast: advertised and received

This gives us working MP-BGP between the two PE routers, but nothing is actually being advertised over BGP just yet because there are no redistribute or network statements.


Read more...

MPLS Lab Part 1

I'm currently studying MPLS for the CCIP qualification so will be putting up a series of articles on building a basic service-provider network to test various MPLS configurations.

The first article puts together a simple WAN running OSPF.


The network will look as below:



(These diagrams are all created using the excellent open source software dia)


The idea here is simply to get the provider part of the network up and running. The links are all using /30 masks and the core links are all using serial lines in this case (because I have plenty of WIC cards and no ethernet cards). It's actually a very simple OSPF network with everything in area 0 and MPLS enabled on all of the interfaces.

Because there are diverse routes across the network from PE-PE, there are two entries in the routing table, e.g. 10.255.255.52 (PE2) from PE1:

PE1#sh ip route 10.255.255.52                 
Routing entry for 10.255.255.52/32
  Known via "ospf 1", distance 110, metric 129, type intra area
  Last update from 10.0.0.6 on Serial0/1, 00:00:54 ago
  Routing Descriptor Blocks:
    10.0.0.6, from 10.255.255.52, 00:00:54 ago, via Serial0/1
      Route metric is 129, traffic share count is 1
  * 10.0.0.2, from 10.255.255.52, 00:00:54 ago, via Serial0/0
      Route metric is 129, traffic share count is 1

The LFIB is populated on each router:

PE1#sh mpls forwarding-table 
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop    
tag    tag or VC   or Tunnel Id      switched   interface              
16     Pop tag     10.0.0.8/30       0          Se0/0      point2point  
17     Pop tag     10.0.0.12/30      0          Se0/1      point2point  
18     17          10.0.255.4/30     0          Se0/1      point2point  
       19          10.0.255.4/30     0          Se0/0      point2point  
19     21          10.255.255.52/32  0          Se0/1      point2point  
       21          10.255.255.52/32  0          Se0/0      point2point  
20     Pop tag     10.255.255.101/32 0          Se0/0      point2point  
21     Pop tag     10.255.255.102/32 0          Se0/1      point2point   

And the basic connectivity is working, PE1 can ping the loopback on PE2:

PE1#ping 10.255.255.52

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.255.255.52, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/8 ms

This is almost CCNA level simplicity, which is good, KISS is a great principle to follow.

Question

So on this network, will the pings from PE1 to PE2 be load balanced across the two P routers?

Answer (highlight text to read):

No!
If you look at the PE1 route table, there are two routes to 10.255.255.52 and there are two MPLS labels which will allow load balancing of traffic, however the question was specifically about pings from PE1 to PE2.

So to find the answer you need to look at how the router decides which path to send traffic, which is done by CEF. The default CEF load balancing algorithm is per-destination (which is actually source & dest). As the source/dest are identical on all of the ICMP packets then they are not load balanced.

They would be load balanced if you turned on per-packet load balancing.

The relevant configuration bits are as follows (left out loopbacks for brevity):

PE1

interface Serial0/0
 description To P1
 ip address 10.0.0.1 255.255.255.252
 mpls ip
!
interface Serial0/1
 description To P2
 ip address 10.0.0.5 255.255.255.252
 mpls ip
!
router ospf 1
 router-id 10.255.255.51
 network 10.0.0.0 0.255.255.255 area 0

P1

interface Serial0/0
 description To PE1
 ip address 10.0.0.2 255.255.255.252
 mpls ip
!
interface Serial0/1
 description To PE2
 ip address 10.0.0.9 255.255.255.252
 mpls ip
!
router ospf 1
 router-id 10.255.255.101
 network 10.0.0.0 0.255.255.255 area 0

P2

interface Serial0/0
 description To PE1
 ip address 10.0.0.6 255.255.255.252
 mpls ip
!
interface Serial0/1
 description To PE2
 ip address 10.0.0.13 255.255.255.252
 mpls ip
!
router ospf 1
 router-id 10.255.255.102
 network 10.0.0.0 0.255.255.255 area 0

PE2

interface Serial0/0
 description To P1
 ip address 10.0.0.10 255.255.255.252
 mpls ip
!
interface Serial0/1
 description To P2
 ip address 10.0.0.14 255.255.255.252
 mpls ip
!
router ospf 1
 router-id 10.255.255.52
 network 10.0.0.0 0.255.255.255 area 0

Read more...

Checkpoint Software Blade Architecture

I received an email this morning from Checkpoint advertising their new software blade architecture.

Checkpoint already provide a hardware virtualization platform in the VSX-1, so could this be a blade-center version? No, the blades here are not actual blades in the sense that the rest of the industry uses the term, they are infact software modules.

So what is it?
It's a "new" architecture to Checkpoint systems that allows you to choose which software modules you want installed on your gateway/smart center servers.

So whats the difference between this and R65's cpconfig software install section?
It has templates (and possibly a GUI).

Is there anything new here?
Kind of, if you dig a bit deeper there are some interesting bits to R70, but they're nothing to do with blades or this software blade architecture.

The Checkpoint blade architecture seems mainly to be a new model for licensing, it'd be nice to avoid the confusion of the past. The old licenses seem so simple in theory, but in practice quickly become painful, especially when you have upgraded a complex system through several releases (and who runs Checkpoint on anything other than a complex system). I'll be watching with interest!

I do wish they'd called it something else though, blade is a fairly well known term and using it in this context is likely to cause confusion.

So what is new?
R70 replaces smart defence with IPS, URL filtering, anti-spam and anti-virus modules. These are software only modules, not like the Cisco ASA IDS/IPS units, and they run on the same hardware. It'll be interesting to see how well these perform in the wild as I don't currently have any data on them.


Nokia/IPSO support in R70
It may (or may not) be news to some but R70 now is being released for Nokia/IPSO platforms. I'm a bit suprised that Checkpoint are still supporting the Nokia boxes now that they have their own line of hardware running SPLAT.

Nokia is no longer a supported platform for Sourcefire and I can't see Checkpoint supporting them indefinitely. Both SPLAT and IPSO are Unix-based systems but SPLAT is a Linux kernel and IPSO is FreeBSD based so they won't be using identical code and it must be expensive for Checkpoint to develop and maintain both lines.

Nothing has been announced that I'm aware of but it would seem like a good idea to start thinking about migrating away from the Nokia boxes if you use them currently.

Read more...

Caribbean?

The image below comes from a beach on the west coast of Scotland, taken while visiting one of my clients sites recently.

A slight departure from the technical nature of this site, but you've got to stop and enjoy moments like this when they occur:


Read more...

Locating a host

Something that often comes in handy is the ability to physically locate hosts on a large campus network from their IP address. This article assumes the device is correctly configured on the network.

The steps are:

1. Find the devices MAC address.
2. Locate the STP root bridge.
   
3. On the root bridge follow the path to the MAC address.

1. Find the devices MAC address.


The MAC address can be found in the ARP cache of the hosts default gateway. To find the default gateway you need to run traceroute from any host on the same network (tracert on windows) which will give you something like the following:

matt@host:~$ traceroute 10.10.10.10
traceroute to 10.10.10.10 (10.10.10.10), 30 hops max, 60 byte packets
1  10.0.0.1 (10.0.0.1)  2.598 ms  2.875 ms  3.363 ms
2  10.10.10.10 (10.10.10.10)  3.908 ms  4.684 ms  4.363 ms

If the host is running a firewall then the last line may just show stars.

Connect (telnet/ssh) to the last hop before the device itself and check the ARP cache for the device. Remember that ARP entries time out after 4 hours of inactivity, the traceroute will refresh it.

Router# show ip arp 10.10.10.10

Protocol  Address Age(min)  Hardware Addr  Type   Interface
Internet  10.10.10.10 2 001e.1234.4321 ARPA   VLAN 10

This output is from an MSFC card so the interface is the VLANs SVI. You may get a physical interface so check the CDP neighbors to ensure a switch is attached and not another router, if it is the a router then connect to it and keep tracing the address until you find a switch or VLAN interface. If you can't find it then check the NAT setup.

You now need to hop onto any switch associated with the interface shown above (VLAN 10 in this case). You may need some local knowledge or a bit more CDP investigation to find a suitable device. You may already be on it if it's a 6500 in native mode.

2. Locate the STP root bridge.
If you are working on a nicely designed network with combined routing/switching devices (e.g. 6500) then chances are good you're already logged onto the STP root bridge for that VLAN.

If not then find the VLAN you need by running

show mac-address-table address 001e.1234.4321

The commands needed here are:

show spanning-tree vlan 10
show cdp neighbors x/y

Check the spanning tree output, find the root port for that particular VLAN, check CDP neighbors for that port, connect to that device and repeat until you are sitting on the device that is the root bridge. It will either say clearly "this is the root bridge" or you'll be able to tell because the device MAC and root bridge MAC are identical.

3. On the root bridge follow the path to the MAC address.

In IOS/native mode use

show mac-address-table address 001e.1234.4321

In CatOS/hybrid mode use:

show cam dynamic 001e.1234.4321
or
show cam dynamic 00-1e-12-34-43-21

(the latter if it's really old CatOS and doesn't like new format MAC addresses)

This should give you a port, e.g. in IOS:

Router# show mac-address-table address 001e.1234.4321

vlan   mac address     type    learn qos            ports

------+----------------+--------+-----+---+--------------------------

10  001e.1234.4321   dynamic     yes  --  2/6

or in CatOS the output looks like the following (if you get no output read the note at the end of this article):

coresw1> (enable) show cam dynamic 001e.1234.4321
* = Static Entry. + = Permanent Entry. # = System Entry. R = Router Entry.
X = Port Security Entry $ = Dot1x Security Entry

VLAN  Dest MAC/Route Des    [CoS]  Destination Ports or VCs / [Protocol Type]
----  ------------------    -----  -------------------------------------------
10     001e.1234.4321             2/6 [ALL]

In this case the MAC is found on port 2/6. Assuming that isn't a copper port run "show cdp neighbors 2/6". Connect to that device and repeat this step until you get an answer pointing to a copper port.

You've now found the device!


Hopefully someone was thoughtful enough to either fill out the SNMP location field, the exec banner, the port description or to create a document telling you where on campus that switch is. It's now a simple matter to trace the cable and locate the device in question.


If you don't have any information beyond the switch name then it's time to start tracing cables and creating the documents yourself.



*note: CatOS will not show any output for "show cam dynamic 001e.1234.4321" if that MAC address is configured for port security on the local device as those addresses are not entered into the dynamic CAM table. In those cases you'll be able to find the port by running "show run | include 001e.1234.4321"

Read more...

SNMPv3 in CiscoWorks

A brief article on how I've got SNMPv3 working in older versions of CiscoWorks. This follows on from the SNMPv3 intro.

Ciscoworks only allows you to associate a single SNMP user account with each device. The same SNMP user account is used to poll the device and receive traps.

The new version of Device Fault Manager apparently has full SNMPv3 support. The older ones don't seem to, but they support polling devices with AuthPriv and you can use AuthNoPriv to send SNMPv3 traps to CiscoWorks.

So to get CiscoWorks to talk with an IOS device using SNMPv3 you need:

• An AuthPriv account for SNMP polling.
• That account configured in CiscoWorks in the devices properties.
• Traps configured to use the same SNMP username, but with AuthNoPriv.

This may seem a bit unusual, defining an SNMPv3 user with a privacy key and then defining the same user without the key but it seems to work. The configuration I use is:

snmp-server group <group> v3 priv
snmp-server user <user> <group> v3 auth sha MYPASSWORD priv des PRIVPASS
snmp-server host <ip> traps version 3 auth <user>

With this configuration, you need to supply the encrypted password when polling the device, however when the device sends traps doesn't encrypt the packet contents (AuthNoPriv).


This article comes with a large dose of YMMV, it works on the version of CiscoWorks I've tested it with, it may not work on yours.
Read more...

SNMPv3

Quick article on SNMP version 3, focused on Cisco kit.

Previous versions of SNMP were configured using community strings and single-line commands in IOS.

SNMPv3 management is more like user/account management and the configuration does not appear in "show run" but is instead hidden in the private config. You access SNMPv3 configuration information via "show snmp x" commands.

SNMPv3 Communication.

There are three ways to configure SNMPv3 communication:
NoAuthNoPriv (called noauth in IOS) – username only authentication, packet sent cleartext.
AuthNoPriv (called auth in IOS) – HMAC authentication (SHA or MD5), packet sent cleartext.
AuthPriv (called priv in IOS) – HMAC authentication, encrypt entire packet.


SNMPv3 Groups.

Groups are configured as follows:

snmp-server group <groupname> v3 noauth|auth|priv [access X]

(access X is an ACL restricting hosts where accounts in the group can be accessed from).


SNMPv3 Users.

SNMP Users are configured as follows:

snmp-server user <user> <group> v3 auth <sha|md5> <pass> priv des <privpass> [access x]

Note: The password should be at least 8 letters long according to the RFC.



SNMPv3 Views.

These are used to control which MIBS the users can browse or which traps are allowed to be sent. There are three types:

Read views – Values you can look at by SNMP polling the device.
Write views – Values you can set via SNMP polling.
Notify views – Values that can be sent as SNMP traps.

I don't specifically use groups so the default settings are:

If you create a new group then it is given the read view "v1default".

If you create an SNMP trap configuration then the group associated with it has a notify view automatically generated by IOS. These views have names beginning with *tv.FFF


I've written a brief dit on getting older versions of CiscoWorks to support SNMPv3 here
Read more...

Checkpoint to Cisco VPNs #2

This is part 2 of the article started here. I'll be creating traditional mode VPN rules, because they are less abstracted than VPN communities and a bit easier to understand (in my opinion). There are some complications*.

To follow a similar methodology to that used on the Cisco router, the steps are as follows:

1. Define the ISAKMP (phase 1) policy.
2. Define the IPsec (phase 2) policy.
3. Create the firewall rule specifying traffic to pass over the VPN.
   
4. Define the encryption domains.
   

Checkpoint doesn't really have a concept of applying the crypto map to an interface as with Cisco. The Cisco concept of a crypto map is close to the encryption domains on Checkpoint.


1. Define the ISAKMP (phase 1) policy.

In Checkpoint the ISAKMP configuration is applied to an "interoperable device" object created for the Cisco VPN gateway. Edit your interoperable device and select "traditional mode configuration" on the "VPN" page to open the ISAKMP properties as shown here:

You configure encryption and integrity, specify to use a pre-shared secret (although you won't be able to enter one just yet) and in the "Advanced" options you can specify DH and lifetimes.

Be careful as the Cisco and Checkpoint default lifetimes do not match.

In practice, ISAKMP can negotiate new keys when the old ones are due to expire but best to make them match anyway to keep things tidy.



2. Define the IPsec (phase 2) policy.
3. Create the firewall rule specifying traffic to pass over the VPN.

IPSec policy is defined under the actual firewall rule which also specifies the interesting traffic. Create a rule allowing the VPN traffic through the firewall (using pre-NAT addresses) and set the action to be "Encrypt"*. Double click on "Encrypt" and you can edit the IPSec settings for the connection.

In this case we're using 3DES/MD5, DH group 2 and specifying which interoperable device the VPN is talking to.

The traffic being allowed is between 10.0.0.0/24 (local) and 172.16.0.0/24 (remote) in either direction and Any protocol.




4. Define the encryption domains.

The encryption domains on Checkpoint are defined as properties of the gateway objects. Edit the interoperable device and open the "Toplogy" page, you can either manually set the topology or use another object to represent the encryption domain.

Here I am using the Remote_172.16.0.0_24 network object to define the Cisco end of the VPN encryption domain.

If you do destination NAT then the encryption domain is the post-NAT address as opposed to the firewall rules which must contain the pre-NAT address.




So that's the remote encryption domain defined, the local encryption domain is specified in the Checkpoint gateway object in the same way. The best way to do this is with a group, so that you can easily add further subnets in future if needed.



There is just one thing missing from the above, the shared key. This is actually defined on your local Checkpoint object rather than the remote peer's object. The reason I put this last is that the peer name won't show up until you've configured the interoperable device properties.

Edit your own Checkpoint enforcement module and open the VPN "Traditional mode configuration", then use the "Edit Secrets" button and you should see an entry for each of your VPN peers that have the "Pre-Shared Secret" box ticked.

Simply select the peer and enter the secret, "cisco" in this case.








Now having said all of the above, I will also say that making these systems interact nicely is a tricky business and you may find that it doesn't work even if you've followed these instructions to the letter.

Part 3 of this series of articles will include some troubleshooting methods to try and help fix any problems.

---

* Note: Checkpoint firewall policies are either in traditional VPN mode or simplified VPN mode which uses communities rather than the rules shown above.

If you don't have the "Encrypt" option in the Action column or if you have a "VPN" column then you're using a simplified VPN policy and cannot create traditional mode VPNs. Unfortunately you cannot convert a policy from simplified mode to traditional so your options are either to create a new policy from scratch, specifying "traditional" when you create it, or to use VPN communities instead.
Read more...

Checkpoint to Cisco VPNs #1

Article #1 - Intro & Cisco Setup.

To show some of the finer points of Checkpoint VPNs I'll rig up a test lab with a site-to-site VPN linking a Cisco IOS router and a Checkpoint R65 splat box.

This article is not intended to be a general VPN introduction, rather the specifics of Checkpoint/Cisco interaction.


The network will look like this:




The local end is using 10.0.0.0/24, the Smart Center sits in this subnet.

The VPN traffic is shown by the red arrow. I'm using a transit network, VLAN 150 - 192.168.0.0/24 but this will probably be the internet in most cases.

The first VPN is going to be very simple, no NAT involved anywhere.








Cisco IOS Setup

In this case we need a static route configured on the IOS router to ensure traffic for the remote LAN goes out of the correct interface:

ip route 10.0.0.0 255.255.255.0 192.168.0.2

In reality you may not need this as for internet-based site-to-site VPNs the routers default route often does the job.

Now the five steps to create a VPN, handy bit of ISCW revision!

1. Define the ISAKMP (phase 1) policy.
2. Define the IPsec (phase 2) policy.
3. Create the crypto ACL specifying interesting traffic.
4. Make the crypto map to bind it all together.
5. Apply the crypto map to an interface.

1 - ISAKMP policy

crypto isakmp policy 10
encr aes 256
hash sha
authentication pre-share
group 2
crypto isakmp key cisco address 192.168.0.1

2 - IPsec policy

crypto ipsec transform-set TESTSET esp-aes 256 esp-sha-hmac

3 - Crypto ACL
Define here what traffic should be encrypted.

ip access-list extended VPN_INTERESTING_TRAFFIC
permit ip 172.16.0.0 0.0.0.255 10.0.0.0 0.0.0.255 log

4 - Crypto Map.

 crypto map VPN_MAP 10 ipsec-isakmp
set peer 192.168.0.1
set transform-set TESTSET
set pfs group2
match address VPN_INTERESTING_TRAFFIC

5 - Apply to an interface

Cisco_Router(config)#interface fastethernet 0/0
Cisco_Router(config-if)#crypto map VPN_MAP
*Mar  1 00:25:33.187: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON

If you see the "ISAKMP is ON" message then it's looking good. If you get errors then check whether your hardware and software is supported, as an example some of the 8xx series routers cannot do AES encryption and I don't think IPsec is supported on any of the standard IOS images.

Part 2 goes over configuration of the Checkpoint end.
Read more...