A quick refresh for all:

1. PE physical link(s), ties one VRF to another using one subnet for each VRF. Commonly deployed using vlans across a DOT1q trunk (anyone still using ISL? ok, ok a tagged trunk but don’t tell me I’m not allowed to say “trunk”). Possible to use routing protocols between each individual VRF, most commonly staticly routed as it’s the safest model, the other side isn’t trusted…
2. PE MP-eBGP, ties the two MPLS clouds together exchanging VPN labels. BGP can be used to exchange the vpn labels, no need for tag-switching of IP traffic. Requires the inter- link to be known in both IGPs as the next-hop changes, only the vpn label is preserved end-to-end.
3. P tag-switching, joins the two domains together at the lowest level. The inter-link can be between P devices all labels are preserved end-to-end. Even the IGPs must have ‘full’ connectivity, of course I mean PE-to-PE…

Type-2 seemed to make most sense due to the amount of vrfs involved, no need to go for type-3 and type-1 would add way too much complexity to be able to support a multi month long migration.

Now for the MTU issue. The old MPLS cloud uses 3600’s as PE’s in the relevant sites and the new network uses 7600’s. Admittedly a bit slow on the ball -mind you I was not the original designer- we implemented an mtu of 1536 on the sup720 GE based infrastructure links on the new network. However the vpn label on the 4 FE inter-AS/MP-eBGP links was forgotten untill a service using TCP keepalives was partially migrated. Sessions flapped due to the keepalives being 1500 bytes in size and having their df-bit set. I wonder why they used TCP but as of yet I’m left guessing.

The moment we noticed the error of our ways we found that IOS threw us a few curve balls:

1. A 3600 running 12.3 doesn’t accept anything over 1500 as mtu on a FastEthernet interface
2. A 7600 with 6748-GE (LAN interfaces) doesn’t allow a larger than 1500 mtu size either (routed interface)

The short answers are the following:

1. On the 3600: tag switching mtu 1508
2. On the 7600: mpls mtu 1508

fyi, 1504 would have done fine as an mpls label is just 4 bytes we just threw in another 4 bytes for luck

Some links to back things up:

• Cisco MTU config for 7600 12.2SR & layer3 ports
• Cisco IOS hints and tricks: The tale of three MTUs
• You’ll have to take my word for this quote from Cisco: “I can confirm that using tag-switching mtu at 1508 on the 3600 should resolve the issue from that side” (despite not being able to set the interface mtu higher than 1500)

Command to verity mpls mtu:

sh mpls [intf] det

My first google tonight led me to an article on the excellent ios hints and tricks site (ioshints.info). Though this is for a full public DNS server it was a little overkill for me, please read this article as well if you do plan to go that route.

My goal was to have my DSL router serve local IP’s for a few public and local domain names. For example this allows me to use sip.djerk.nl as my proxy address in my soft-phone both at home and away.

The following code is what I ended up with.

# These hostnames will be served locally
# As such they overrule their public dns
# clients do need to use this router as a dns server
ip host view VoIP sip.djerk.nl 192.168.100.130
ip host sip.djerk.nl 192.168.100.130
ip host c877 192.168.100.1

interface Vlan10
description *** VoIP LAN ***
ip dns view-group VoIP

# For my Voice vlan
ip dns view VoIP
domain name djerk.nl
# search list for this view
domain list djerk.nl
domain name-server x.x.x.x
domain name-server y.y.y.y
domain round-robin

# Default dns handling
ip dns view default
domain name geurtscass.com
domain list geurtscass.com

ip dns view-list VoIP
view VoIP 1
view default 100

# Enable local dns server (proxy dns if no domains are configured here)
# I do not allow inbound dns requests (intf ACL) so no need to worry about abuse
ip dns server

Most interesting is the usage of views and a view-list. According to CCO the view-list is needed to bind a view to an interface, views can’t be assigned directly. Also noteworthy is that I’ve not found a way to reduce the sip.djerk.nl host mapping to a single line, as adding the default view to the VoIP view-list was not enough. Maybe I should add the VoIP view to the default view-list.

Well just tried that and it didn’t work either. Anyone out there who knows how to do the same in IOS but with less lines?

Well my working config turned out to be:

crypto keyring Daniel
pre-shared-key address 1.2.3.4 key s3cr3t
!
crypto isakmp policy 1
encr 3des
authentication pre-share
group 2
lifetime 28800
!
crypto isakmp client configuration group Daniel-grp
key s3cr3t
crypto isakmp profile Daniel
keyring Daniel
match identity address 1.2.3.4 255.255.255.255
crypto isakmp profile Daniel-ez
match identity group Daniel-grp
!
crypto ipsec transform-set tset1 esp-3des esp-sha-hmac
!
crypto map Daniel 10 ipsec-isakmp
set peer 1.2.3.4
set transform-set tset1
set isakmp-profile Daniel
match address 102
!
Interface Dialer 1
crypto map Daniel
!
access-list 102 permit ip 192.168.100.0 0.0.0.255 192.168.1.0 0.0.0.255
!
ip route 192.168.1.0 255.255.255.0 1.2.3.4 name Daniel

Also I had to prevent traffic destined for the ipsec tunnel being natted, this meant adding a simple deny line in the NAT ACL. 1.2.3.4 is the remote site IP address, 192.168.100.0/24 my local LAN, 192.168.1.0/24 the remote LAN. As opposed to the ezvpn tunnel I tried earlier this tunnel is dynamic and a static route is required rather than routes being added dynamically. Personally I think ezvpn can be quite cool for remote access (read client access) but I have sslvpn working fine for that, which was much easier to set up to boot.

It appears that Debian Lenny by default only binds to 127.0.0.1 (localhost). I had to add the following two lines to snmpd.conf to get snmpd to listen to external requests.

interface eth0
agentaddress 217.195.248.251:161

I eventually found the solution at debianhelp.org.

I debugged and pinged, even installed hping3 to do UDP pings. I don’t want to run cacti as root, especially not on a vhost. So the UDP ping had to work. The pings arrived but still no replies.

Getting sidetracked I noticed that the one router that did work was being hit by SSH login attempts and it’s cpu was spiking. An ACL took care of the break-in attempts but then I noticed that directed broadcasts were made to my server’s segment. So I nailed that down plus proxy-arps when I noticed that the router which had worked before now was causing errors in Cacti as well.

Tracking back I noticed that the UDP ping ‘replies’ were unreachables rather than ICMP replies (doh, how obvious!) . I enabled IP unreachables on both routers again and I was done. It’s amazing how blind one can be at times to the blatantly obvious…

ip route prefix mask {ip-address | interface-type interface-number [ip-address]} [dhcp] [distance] [name next-hop-name] [permanent | track number] [tag tag]

Static routes that point to an interface on a connected router will be advertised by way of Routing Information Protocol (RIP) and EIGRP regardless of whether redistribute static commands are specified for those routing protocols.

This situation occurs because static routes that point to an interface are considered in the routing table to be connected and hence lose their static nature. Also, the target of the static route should be included in the network (DHCP) command. If this condition is not met, no dynamic routing protocol will advertise the route unless a redistribute static command is specified for these protocols

The following is also good to know, I’ve noticed it’s source of common misunderstanding:

Specifying a numerical next hop that is on a directly connected interface will prevent the router from using proxy ARP. However, if the interface with the next hop goes down and the numerical next hop can be reached through a recursive route, you may specify both the next hop and interface (for example, ip route0.0.0.0 0.0.0.0 ethernet 1/2 10.1.2.3) with a static route to prevent routes from passing through an unintended interface.

Source  (UniverCD)

I’m counting on being there, though untill my manager gives approval and it’s been booked I will not know fur sure…

If you’re going as well and want to meet me then drop me a line, with a suggestion of when you’re free. I don’t think I’ll be able to intentionally run into anyone there, my schedule is way too busy for that.

Suggestions for surviving Networkers are welcome as I’ve never been before.

A little background

A remote router upgrade failed for some strange reason, ok I was stupid enough not to verify the downloaded image while I noticed the download had finished quicker then expected. But the file size was ok and a colleague saw the image increase in size on flash:. Anyway the router did not return to a normal state.

Recovery setup
A connected switch which had been connected into the (ethernet) uplink path earlier today provided connectivity to a test machine connected to the console of the switch. Oh how I love backup paths, as long as I can use them. I was able to SSH to my test machine’s secondary interface and access the switch’s console. Noticed that the router’s interfaces were up so it was powered on. Luckily someone was found who was able to move the console cable from the switch to the router.

ROMMON

And as expected the router was happy to see me in ROMMON, issuing the boot command revealed a magic cooky to be missing/wrong/… . So I put my first bet on tftpdnld and installed it through 2 SSH hops on another server in the same Ethernet segment. Setting the right settings and firing up tftpdnld… It failed: monitor: command “tftpdnld” not found, time to try something else.

Xmodem

Minicom is capable of sending files, and lucky me it supports xmodem (others too but I wasn’t interested in those). Changed the console speed to 115200 instead of the default 9600. Alt-Z, S, file not found? Pulled the IOS image from the earlier installed tftp server to the ‘console’ machine and tried again. Als-z, S, select file, “Retry 0: Got 01 for sector ACK”. Right something clearly was not going well. Client was waiting, server was trying to send but no go.

CRC-16

The solution was to add -r to the command in ROMMON: xmodem -c . It adds a CRC check which minicom sets by default on sx -used for x/ymodem (etc) transfers. So now I’m waiting for this 27Mb file to finish and see if it will boot. Ahh, it’s at 20Mb now so only 7Mb to go and then I’ll still have to wait for the erasing and programming of the flash. Oh well time to read a book maybe.

Someone please wake me up when this router is done…