Archive for the 'Main blog' category

Cisco LACP config for Aruba AP

 | 8 Jul 2015 17:30

Aruba LogoDon’t we all love it when we find that a standard requirement states one thing and what to date is implemented elsewhere doesn’t comply? Dual active uplinks for a premium office standard is one of those requirements I found. Now I haven’t seen the standard Cisco wireless deployment for premium sites, but in light of vendor ‘diversity’ Aruba is deployed instead of Cisco.

Motivation aside, the dual uplink raises an interesting question for lightweight access points (LWAPs). Aruba (by default) GRE tunnels all client traffic to the wireless LAN controller (WLC) for processing, filtering and forwarding there, like Cisco and common in corporate environments. The alternatives are split tunneling or no tunneling, which normally comes at the cost of losing corporate controls. The QoS trade-off and headaches of tunneling WLAN traffic to WLCs is food for another post entirely.


Using AP225 APs, I found I had LACP at my disposal. Cheaper models (< AP220) don’t do LACP and only have STP for redundancy. Some of my first concerns:

  • Standard Cisco LACP is mostly configured unconditional, which means the ports don’t come up if LACP isn’t detected on the link. How is an AP meant to get its profile from a WLC if it can’t get there. Remember I don’t want to reconfigure the switch ports after an AP has connected and obtained its profile (configuration) from the WLC.
  • Aruba documentation and forums (Airheads) didn’t list much configuration about Cisco switch port configuration. What I did find was that LACP is supported and needs switch configuration for it to work.
  • A single GRE tunnel using 2 etherchannel members?! LACP uses an IP hash table to select which member link to forward packets on. An AP only has a single IP address and without LACP the WLC also only has a single IP address for termination of LWAP GRE tunnels. Surely all GRE tunnels would only use a single LACP bundle-member, restricting maximum throughput to 1 Gbps. If so, what’s the point?

Reading up I found the following helpful information:

  • Aruba solves the LACP IP hash table problem by using a second WLC IP address to terminate a second GRE tunnel. This second tunnel uses the 2nd member-link. Each GRE tunnel serves a radio, 2.4GHz and 5GHz, this does not enable more than 1 Gbps for 5GHz but at least 2.4GHz traffic won’t eat into the uplink speed available to 5GHz traffic. The Aruba config for LACP centres around “AP LACP GRE striping IP” (see Google for more info).
  • “no port-channel standalone-disable”, this port-channel configuration gem permits link members to come up as individual links. This allows a LWAP to connect to the network, get an IP via DHCP, find the WLC and pull its configuration. Once provisioned by the WLC LACP kicks in.


Beware of the LACP hash algorithm, Cisco switch default is src-mac. In an edge-routed design the source-mac will be the mac of the switch SVI towards the WLC. The Switch terminating the LWAPs is the same as the one terminating the WLC and the WLC also uses LACP to connect to the LAN. For my deployment the solution was src-ip as the GRE sessions towards the LWAPs have a distinct WLC IP address (must be odd/even). Traffic destined for the WLC is also src-ip based, which is good as the load-balancing will then be based on the targets of the clients whether internet or LAN based it works as long as corporate clients don’t all hit the same target at the same time. I think is most situations the resulting total bandwidth restriction of a single LAN source towards wireless clients at 1 Gbps is beneficial to the fair sharing of bandwidth between LAN based services.

The AP225 only pulls PoE over a single link. If the link providing PoE goes down it will reboot and come up one the remaining link.

Though the dual links provide extra bandwidth, if the your NOC doesn’t monitor these links either via WLC management or switch trap/port monitoring, a single link failure won’t be noticed. I think this is no different to the issue of APs losing their physical link and continuing in mesh connectivity, which is great as a last resort but not when the situation isn’t resolved before things get really bad.

Cisco config

This is the LWAP switch port config that worked for me:

WLAN-SW01(config)#int range g1/0/1,g2/0/1
 description WLAN-AP01
 switchport access vlan 4
 switchport mode access
 channel-group 1 mode active
WLAN-SW01(config)#int po1
 description WLAN-AP01
 switchport access vlan 4
 switchport mode access
 no port-channel standalone-disable
WLAN-SW01#sh eth 1 sum
Flags: D - down P - bundled in port-channel
       I - stand-alone s - suspended
       H - Hot-standby (LACP only)
       R - Layer3 S - Layer2
       U - in use f - failed to allocate aggregator

       M - not in use, minimum links not met
       u - unsuitable for bundling
       w - waiting to be aggregated
       d - default port
Group  Port-channel  Protocol    Ports
1      Po1(SU)         LACP      Gi1/0/1(P) Gi2/0/1(P)

When the LWAP hasn’t fetched it’s configuration the Flags show either (D) for down or (I) when the port is up but LACP is inactive. As long as LACP is inactive the APs MAC address will hop between the two ports and a MAC flap warning is reported by the switch.

Jul  8 2015 08:33:59.259 UTC: %SW_MATM-4-MACFLAP_NOTIF: Host 94b4.0f50.47f0 in vlan 4 is flapping between port Gi2/0/1 and port Gi1/0/1

Another error I’ve seen is about PoE. What happens is that both member ports offer PoE but the AP only signals acceptance on a single port. The switch doesn’t seem to understand the lack of response, calls the AP rude, turns off PoE on that port and logs the ‘error’.

Jul 8 2015 17:08:39.030 UTC: %ILPOWER-7-DETECT: Interface Gi1/0/2: Power Device detected: IEEE PD
Jul 8 2015 17:08:41.202 UTC: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi2/0/2: PD removed
Jul 8 2015 17:08:41.203 UTC: %ILPOWER-3-CONTROLLER_PORT_ERR: Controller port error, Interface Gi2/0/2: Power given, but Power Controller does not report Power Good
Jul 8 2015 17:08:41.885 UTC: %ILPOWER-7-DETECT: Interface Gi2/0/2: Power Device detected: IEEE PD
Jul 8 2015 17:08:42.995 UTC: %ILPOWER-5-POWER_GRANTED: Interface Gi2/0/2: Power granted
Jul 8 2015 17:08:50.035 UTC: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/2, changed state to up
Jul 8 2015 17:08:50.187 UTC: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/2, changed state to up
Jul 8 2015 17:08:55.025 UTC: %ILPOWER-5-IEEE_DISCONNECT: Interface Gi1/0/2: PD removed

WLAN-SW01#sh power inline
Module Available Used Remaining
 (Watts) (Watts) (Watts)
------ --------- -------- ---------
1 1110.0 200.2 909.8
Interface Admin  Oper       Power   Device              Class Max
--------- ------ ---------- ------- ------------------- ----- ----
Gi1/0/1   auto   on         15.4    Ieee PD             4     30.0
Gi2/0/1   auto   off        0.0     n/a                 n/a   30.0

Check LACP from the WLC

Some great LACP related WLC CLI tools I found on Airheads:

Check if GRE striping IP has been set: “show ap system-profile ”

(WLAN-WLC01) #show ap system-profile LACP

AP system profile "LACP"
Parameter Value
--------- -----
RF Band g
RF Band for AM mode scanning all
Backup LMS IP N/A
Backup LMS IPv6 N/A
LMS Preemption Disabled
LMS Hold-down Period 600 sec
LMS ping interval 20
GRE Striping IP

Check the if an APs LACP has come up: “show ap debug lacp ap-name ”

(WLAN-WLC01) #show ap debug lacp ap-name WLAN-AP01

AP LACP Status
Link Status  LACP Rate  Num Ports  Actor Key  Partner Key  Partner MAC
-----------  ---------  ---------  ---------  -----------  -----------
Up           slow       2          17         1            88:90:8d:d9:b8:00
Slave Interface Status
Slave I/f Name  Permanent MAC Addr  Link Status  Member of LAG  Link Fail Count
--------------  ------------------  -----------  -------------  ---------------
eth0            94:b4:0f:c2:83:b2   Up           Yes            0
eth1            94:b4:0f:c2:83:b3   Up           Yes            0

Check if GRE tunnels are being created to both the switch IP address and the GRE stripping IP address configured in the AP system profile: “show datapath session | include ”

(WLAN-WLC01) #show datapath session | include "30.29"
...   17   4500  4500   0/0  0    0   0   pc1         119  70         71872      FC     47   0     0      0/0  0    0   1   pc1         c    0          0          FC           

There you have it, LACP between an Aruba AP and a Cisco switch. Kudos to Abi’s over at Airheads for this article about LACP on the Aruba AP225 and AirOS 6.3. I was working on 6.4, ymmv with different versions.

One year down the road…

 | 14:37

A year post jumping ship

Seems like a good time to look back a little and wonder why it took me so long. June last year my time at Deloitte (London office) came to an end, with the customer transitioning their IT workforce I was the last from nscglobal to leave the account. I had a good time working in a small project team, always on site but then the people and the interesting work made up for the hassle and cost of the daily commute. So Long, and Thanks for All the Fish.

Life without two wheels sucks

But then again, if I’d still had my dear BMW, I would have gone back the O2 account in Slough. Despite the daunting prospect at the time. It was the right time to jump ship and join my wife’s limited company. This is where it pays off to leave a wealth of happy customers and (ex)colleagues, that and God’s provision.  Either way, I’m grateful. May last year I was an employee living in a 2 bedroom ground floor apartment with mould growing in my boys’ bedroom. Now by the grace of friends and benefit of being self-employed; I’m happy with my work, we live in an amazing house and are practically debt free since moving to the UK in 2008. Oh, and I’ve started running again, properly this time…

3.5 hour commute

A what?! Yes, you read that right. I have a 3.5 hour commute, each way. Blessed as I am, I only rarely have to make the trek to Macclesfield. It’s funny for these things work out, but it does work and it works very well. We now have a great accountant, contact me if you need his details (you really should!). He’s just finished our 14/15 numbers and I can’t tell you great it is to get on with the person dealing with our business finances.

I’m eager to find out what the new financial year will bring. First up is another contract extension, it’s great having happy clients, there’s no better advertisement than that. Second, longer bound by the restrictions of an employer on what I blog about. I’m now able to write a lot more about the interesting challenges faced at my customers and the solutions found to them.

Temporary London CCIE LABs

 | 25 Jun 2014 15:23

Just received an email about Cisco’s mobile CCIE LAB coming to London:

Mobile CCIE Lab Available in London, United Kingdom, from October 6 to October 14, 2014

To address the urgent need for certified IT professionals, and to offer more convenient testing, Cisco has developed the Mobile CCIE Lab for qualified candidates who are ready to take their CCIE Routing and Switching exam or CCIE Security exam.

We encourage you to take advantage of the mobile lab scheduled in London, United Kingdom, from October 6 to October 14, 2014. The CCIE mobile testing lab will allow qualified candidates to more easily and quickly take the exam, reducing the waiting time, effort, and costs accrued by having to travel to take the exam. The eight-hour lab exam tests your ability to configure actual equipment and get the network running in a timed-test situation. There will be 42 seats available for the CCIE Routing and Switching exam and 7 seats available for the CCIE Security exam.

Apart from the Cisco Certified Architect (CCAr) certification, the Cisco CCIE certifications are the highest level of achievement for network professionals. Less than 3 percent of all Cisco certified professionals earn their CCIE certification.

Click here to register for the CCIE Routing and Switching exam or CCIE Security exam in London.

For information on registering for a Mobile CCIE Lab event or for additional information about the Mobile CCIE Lab program, visit the Cisco Learning Network.

SSL Intercept headaches

 | 17 Jun 2014 22:50

BlueCoat Logo

A recent proxy upgrade, has seen me working many hours – fixing things that weren’t broken before. It was intended to be a drop-in replacement, but somebody couldn’t resist the opportunity to specify ‘a few minor’ new requirements.

  • 1 year log retention of all traffic
  • SSL interception to enable data leakage protection for all traffic types

The first doesn’t sound like a big issue, however it turned our we underestimated the logging volume for 8000 concurrent users. Additionally the reseller hadn’t flagged the issue either, I’m ‘sure’ they’ll pay more attention next time… As for SSL interception. It broke a host of things. Some lessons learned:

  • Bluecoat ProxySG devices come with root CA certificates installed. Many site-admins using SSl fail to install the intermediate certs which slows down session set-up but also means we had to install many intermediates as the proxy does not go looking for them. This means manually finding and installing certs based on users calling the help-desk because they weren’t allowed to access sites with untrusted certificates.
  • Commercial sites using self signed certificates. Bad practise, but sadly it’s not always up to engineers/consultants whether or not such a site should be honoured with a business critical status or not.
  • Applications tunneling proprietary protocols over TCP:443. Some encrypted, some not so much. The ProxySG was configured to detect the protocol and to deny all unrecognised traffic. This breaks Adobe Creative Cloud for example. Skype is another hot-potato.

Skype in particular proved to be a big time-waster. As you may well know Skype uses proprietary protocols and tries very hard to remain hidden from prying eyes. As Skype was an application that was in use before the migration and the ‘as-is’ rule lingered, there was some pressure to get Skype working. The short is that I got it working without globally turning off SSL Intercept, well – to a degree anyway…

Show me more… »

Comments now working (fixed)

 | 27 Jan 2014 23:08

Not that they weren’t working before but it’s kinda hard to see white text against a white background… This has been fixed. Just in case you weren’t aware, there’s no need for a local account if you want to comment, you can use your FB, twitter or whatever account…

Where are the real (business) men…?!

 | 22 Jan 2014 08:00

What is a “real man”?

What I do not mean is Gordon Gekko’s “Greed is good“. James Bond figures, martial arts black belts, famous actors and bearded men don’t classify by default either in my opinion. Which brings me to my next point.

What defines the public opinion?

No-one would argue that this not important, apart from those who profuse not to care about public opinion or that it shouldn’t matter. If it matters or not is irrelevant to my current argument; we all seem to be blind to the fact that we don’t practise what we preach. We reason one way yet act another, so we’re all hypocrites…? Well, I would dare to state, … yes.

How can I be so harsh? For one I know myself and in addition to that it is one of the fundamental psychological pins that many fiction and non-fiction books hinge on. Take a person who’s made some bad choices, stack the odds against him and watch as he struggles to come out on top. We like an underdog, why? Because we identify with them, we all struggle we all make mistakes. We’re all human right?

So why does public opinion matter then? It does because it shapes our thinking, colours our emotions and as a result affects how we behave, talk, interact and work.

Get to the point already…!

What I’m getting at is that public opinion often subconsciously(?) turns out to be our ethical thermometer. Some local and recent examples relevant to the UK.

Tax avoidance is for the rich, they can afford the financial advisors and accountants that find the loopholes. We all hate it but can’t change it because the rich are in government and our vote doesn’t count because of the political system. I won’t say this too loud but I think it’s good on them though, politicians and the government in any shape or form can’t be trusted so if you can get away with it…


I don’t vote because the system is sick to the core. If enough people stop voting it will send a signal that the system has to change. If nobody would vote the system would break down and a better one would have to be put in its place.

These are popular opinions today, there may be variations but at the core the same message resounds. Many know it’s wrong, and why? Because (many) people got hurt. If nobody (excluding government) gets hurt then good on you for getting away with it, we all know we all need to look after ourselves. Nobody’s going to do it for you, we’re all grown-ups here, work hard and don’t let anyone take your hard earned cash.

It’s a lie that not voting constitutes a vote against the system. What it is is an abdication of your opinion to the general opinion, effectively saying “I’m for whatever everyone else says”. Which is the exact opposite of what most people I know who don’t vote think they’re doing. The way to vote against the system is called a ‘protest vote‘ or ‘blank vote’. This is relevant due to the false sense of security by means of “security is bliss”. Hiding from the truth does not make us less guilty or responsible, in fact it’s plain and simple immature.

Problem #1: Don’t trust anyone, just don’t…

We all know that bankers can’t be trusted, they pay themselves huge bonuses after getting their banks bailed out for billions. CEO are up there too, giving themselves massive bonuses while sacking people kicking them out the back door.

Again it seems to harmless, but it’s a generalisation hidden by layers of veneer of so called ‘truth’. Guilty as charged, yes indeed I hold the view that there are a lot of people out there who fit this stereotype. But I refuse to believe that ALL bankers and CEO’s are like this, sadly I haven’t made friends yet with those who do break the mould.

Our mistrust of government is deeply rooted, nurtured by decades of democratic childish behaviour, scandals and broken promises. Let our yes be yes, and our no be no. So shall we start here? Rather than tell others to do a better job, how about we ourselves model how it should be done. I believe this is applicable to everything we do, not just politics or banking. I admit this is a vulnerable position to put yourself in, but it’s the only option available in order to break with hypocrisy. So make up your mind, be truthful yourself, in every way you can. Or hold your tongue about others, who are we to judge when the root of the crime is the same?

Real men don’t lie, instead they build and maintain integrity

Problem #2: Take what you can get, fend for yourself

We hate to admit it but most of us default to this mode. What we fail to see is that we do so under the pre-tense  of maturity and responsibility. Surely looking after our family comes before everything else? I won’t argue with you there, but will it really hurt that much to keep an eye out for those around you. And those people around you include your boss and those placed under your authority at work.

Now this is where it gets interesting, as in my real point for writing this post. Please excuse my french, but I like this quote and it’s apt:

Shit gets promoted upward

Funny Business Flow-chartIn corporate life we often see that good people are held back and bad people are promoted in order to get rid of them. This highlights the following problems: narrow focus on a managers own department, lack of care for the larger business and lack of care for those entrusted to the manager.

Personally I’m of the opinion that the head of the company, be it the CEO, management team or one or more partners, set the business ethics of their company. In a positive sense this is the exact reverse of the image on the right or the line I quoted above. I truly believe that if a CEO sets out to honour his staff and invests in them he or she will yield a much higher return on their investment. I hope you are wondering what exactly I mean by ‘investment’.

By investment I mean time, money (salary, bonus, secondary package etc) and vision. Hiring someone and not spending any of the above other than just money, bears the risk of just paying for a job done without the desire to see growth of the employee. An I do use growth in a very broad sense; maturity, skill, confidence and trust just to name a few.

Having spent just over 6 years in the UK, I’ve come to see a trend where investment is something that an employee is expected to do himself. The problem is that this causes the workplace to become a confusing and fragmented place where individuals work hard to appease their bosses both at work and at home. Family life is sacrificed as all hope is lost for change, this is just the way it is and will be. Now why does that remind me about the general public opinion of UK politics?! Don’t answer that… Most people I know are disillusioned with work, going through the motion like the proverbial grindstone. Mouths to feed and mortgages to pay. Some manage to escape the extremes of London but I suspect many will find that though the pace may be slower the work ethics are similar.

The silver lining

Luckily for the UK’s suicide statistics, this country is incredibly social. To the point where I’d say that The Netherlands has no work social life to speak of in comparison. I would dare to say that this may well be the key binding factor for employees to stay put, despite the abysmal lack of job security and the demands on our ‘out of hours’ efforts to maintain ones working position. Don’t get me wrong, I’m all for people who invest in themselves. However (for example) businesses that require employees to be 100% billable while also demanding study for accreditation in their own time for the sake of the wellness and continuity of the business can not count on my favour. Yet this seems to be the norm in the UK, why do people put up with it. Again the parallel to the political system is obvious, the task too big to even attempt or contemplate setting change in motion. In my book this is national apathy at it’s worst and it’s rearing its head in the two most important aspects of day to day life.

Where are the real men?

Where are the CEO’s that invest in their biggest assets, their work force? That conduct their business with integrity, paying bills on time. Ensuring their sales force doesn’t sell pie in the sky and effectively managing their business ethics all the way down to the cleaners and caterers. If you’re a CEO and your personnel turn over is above 20%, I dare say you are not managing your people well. My advice to you? Stop using thumb screws on your managers and start focusing on fostering a culture of trust, integrity, loyalty and value. Most people would value their family as top priority, so why don’t you start there? Ensure all your employees can grow within working hours, let working hours be working hours and give back time to the families under your extended care.

I’m not proposing a lax approach to efficiency, but rather a change in priorities and means to an end.

We all have ethical responsibilities

  • To our families to lead by example and not follow the herd of public opinion.
  • To those in our care, those who depend on us for their livelihood.
  • To people who depend on those in our care

Whether we like it or not, when we lead we replicate ourselves. What legacy do you leave in your wake?

Some relevant articles:

Ubuntu Studio, Rakarrack and some hardware

 | 21 Jan 2014 01:27

So I’ve had some hardware lying around, that was screaming at me to be used. It was initially meant for some fancy HTPC build but I never managed to get that done, I had at the time (prematurely) invested in the following:

  • ASUS AT5IONT-I deluxe (mini ITX w dual core Atom D525)
  • 2GB RAM (for said board)
  • Blue-ray slot loading drive (does CD and DVD too, in case you were wondering)
  • 16GB SSD
  • USB 16×32 LED matrix display (, twitter:@TiertexDesign)
  • Arduino ATmega2560 (as of yet unused)

Now that we’re living in an apartment I’m a little more conscious of noise when playing guitar and I’ve always wanted to dabble with effects but never had the guts to just try out effects by buying and selling them (is there another way?). Pre-built pedal boards seem to just be beyond me, I think I lack the patience to come to understand them… I’d seen stuff about virtual pedal boards, recording software and real-time simulation. So after a little research I came up for the following.

Ubuntu Studio
Queue Linux and Rakarrack; Ubuntu is still my OS (Operating System) of choice these days and though there are contenders out there I figured I should have a go with the familiar first. It’s unusual for me but hey maybe getting older is making me that little wiser these days. Ubuntu Studio claims to sport a real time kernel and comes with Jack and Rakarrack by default. Sorry this is not a review of the software I used, Google is your friend – do your own home work.

  • Ubuntu Studio (13.10)
  • snd-aloop loopback sound driver (VU meters on the LED matrix using python, I needed an ALSA instance to poll)
  • Mbox2 hardware from eBay (lacks Windows7 support but natively supported in Linux kernel 3.x, so ‘cheap’ to get, has two high-Z/microphone/line inputs and two line outputs all at 48khz sample rate. I figured it was good enough for my needs)
  • Python to drive the LED matrix display so see in- & output VU levels, CPU, memory, app load and preset-name at a distance.
  • Touch screen. In order to prevent needing to use a keyboard and/or mouse for on the fly changes. Keeping a close eye on eBay helped again as they can go for £100+, but I managed to get a 17″ screen for £26.
  • Cheap PCIe VGA card. The touch screen was VGA and the AT5IONT only has HDMI and DVI-D; no analog, I found this out the hard way.
  • Panel mount USB port (Neutrik of course)
  • USB serial adapter, as the touch screen was serial not USB. Another reason I got the touch screen for just £26…
  • Old stuff I had lying around
  • Old amp (3U 19″)
Rakarrak build using scavenged parts

Rakarrak build using scavenged parts

The end product
I gutted an old amplifier, added some perspex, case modding style LED’s, some gaffer tape (had to be done) and there you have it… The flight case is a quick botch job, I’d like to build a custom case with flip up touch screen lid and pedal board front lid, but haven’t got the money for the needed hardware at the moment. Additionally a laptop with touch screen may just prove to be more powerful and portable, this beast is just too heavy to carry.

My Python script
For those of you interested or doing similar stuff. My Python script for the ledmatrix can be found here, I’m sure I’ve committed many sins in coding this. Feel free to comment your improvements etc there.

Do note however that I struggled with alsaaudio as it returned [-31, 0x0] every other time I tried to grab for a VU level reading. I ended up testing for this and grabbing again. Effectively doubling the cpu load involved. Someone on IRC suggested Pure-data which looks great for grabbing VU levels but having to then take a numeric value and convert it into the code needed to drive the LED’s when I already had a working script based on microphone input level, this proved too much of a hurdle. I tried several Python libraries but settled on the one that worked best for me with the smallest impact on CPU. The displayed text is read from a file.

The plan was to read this from Rakarrack, but as this isn’t yet supported. What I’ll most likely end up doing is building a (USB) midi pedal board with the Arduino so I can control Rakarrack and Guitarix while playing. Hopefully I can then use the same midi process to send the preset name to the LED matrix display.

Intel Atom & real time audio processing?
The Atom D525 isn’t powerful enough for all that Rakarrack can throw at it but there are quite a few that are very usable. My Jack buffer is I think 256 so latency is around 20 msec but I’ve not had any problems with that being too slow for me. Think of me what you will… The 2GB ram is barely used and boot time due to the SSD is a mere few seconds. One significant thing is however that Guitarix is a lot less CPU hungry than Rakarrack. Despite this I do prefer Rakarrack due to it being easier to use. Guitarix tries too hard to be simple effectively making it difficult to set up for effects (imho). Maybe time will change my mind, I use both at the moment.



32x16 LED matrix

32×16 LED matrix



Mbox2 has native support in Linux

Mbox2 has native support in Linux

Jack Patchbay

Jack Patchbay

Jack Connections

Jack Connections

HTOP output

HTOP output

Cisco Voice-VLAN (VVLAN) inconsistencies

 | 12 Nov 2012 12:41

First off I’d like to say that this is just a minor issue, more relating for routers versus switch, I’m still a lot happier at how Cisco implements config and features as opposed to most if not all of their competitors…

At a customer I’ve recently had to commit a grave operational sin; to connect a small switch at the end of a floor patch. These things are normally operational nightmares as they have a tendency to quickly bring an entire LAN environment down to its knees when such a ‘switch’ is connected to the network twice. Always by accident but having management kick you for something someone else did is not anyone’s idea of fun. I won’t go into the underlying principles here as I’m assuming most who frequent my blog will know about broadcast storms, their causes and the tools and solutions available to mitigate the risks.

Our justification to operations was that we wanted a few more local LAN ports to test VoIP devices on than we had available through floor patches. As such I reasoned with Operations that this was a calculated choice to segregate our testing from the rest of the LAN but still make it as realistic as possible. Using the means available meant that I had to make do with a Cisco 1801. Single routed and 8 switched interfaces. Think of it as a router with one Ethernet interface and an 8 port HWIC-ESW nailed to it. Didn’t need the ATM or WiFi it has.

So I set out, disabling IP routing, admin down all non-Ethernet ports. set up the vlan database -old style, remember?-; I did not want this baby to participate in VTP, in fact I don’t think it even can! It’s limited to 8 vlans. Pulled two cables to it. One switched port as trunked with some data and voice vlans and configured the routed interface for management access.

All sweet and dandy, tested the BPDU-guard functionality prior to installation by connecting an access-port to the LAN. Clunk! it went down as desired, result I thought… Then when installing the LAN wouldn’t bring up the LAN port. Doh! I’d missed that the 1801 doesn’t send BPDU’s until a VLAN becomes active. I’d checked if spanning-tree was operational, and it wasn’t until I brought an interface up. So I disabled STP for all vlans in the VLAN database. Now my laptop received an IP address and the data VLANs all worked.

So, time to connect a Mitel phone. No dice, it received it’s first DHCP response with VLAn information, then it would just sit ennuncing it was waiting for a DHCP response. Dang, I’d configured the voice vlan so why did the switch not detect the phone, enable trunking so that the phone could send it’s DHCP request on the voice VLAN?

It was only when I started reading up on HWIC-ESW voice-VLAN config I noticed that Cisco hasn’t implemented the auto enable of dot1q trunking when a phone is detected… The solution is to add two lines of code; “switchport truck native vlan xyz” and “switchport mode trunk”. The crux is that this platform is at heart a router, not a native switch…

Cisco documentation

Twittertools is dead

 | 14 Jun 2012 16:12

Long live Social

Testing automatic posting to Twitter and Facebook from my blog, sorry if you consider this spam.

To CCIE or not to CCIE?

 | 15:30

This one has been coming for a while… Last time I went for my CCIE was Q3 2007 and it’s been pretty quiet here since.

In the mean time I’ve moved country, changed jobs and moved again, though not as far. So how is my CCIE doing? Well to be honest, it’s not. You see, I came from a sponsored certification track -which incidentally I forged with my employer at the time- and thus had to sort out a transfer sum from my new employer. No problem for the Dutch employment market, at least at the time. But here in the UK I had no such luck, suffice to say that financially it was not a good idea to move country. My new employer expressed that they would be happy to support attaining CCIE status. Little did I know that we had completely different concepts of ‘support’.

Loving a new challenge I quickly settled into my new position, only to find two years had gone before the question of attaining CCIE popped up again. No wonder really when you have a 100% billable target and there’s plenty of over time involved in the projects at hand. Then when things settled down and I moved to another customer I only found myself further and further away from (CCIE related) technology.

Yes, I still design and do have a consulting role but I’m in no way challenged and kept on my toes protocol wise. It took a few technical interviews to fully realise the impact of this. Which brought me to this point: Either, I work and retrain myself towards CCIE in my own time, hope my employer will pay for the lab and favour me with a mere week off for full-time study. Or do I let go of the desire to attain CCIE and trust that there will be employers out there who are smart enough not to be blind sided by the highly praised numbers of my peers.

My choice has been to no longer pursue CCIE. I can’t ask my family for a long and hefty sacrifice once again, we’ve been there done that. I’m now one kid and two cancers ( /@maizymoo tweet) further and value my own time more than my career, if this means UK employers don’t like me any more than so be it. I know what I’m capable of, just wish I was better at convincing prospective employers…

[Above edited, below added - Jan 27th 2014]

Sad thing is that the CCIE we find ourselves hiring. often do not have the consultancy skills needed to satisfy our customer’s needs. To the HR managers out there: Hiring a CCIE does not mean, you get good communication skills, customer facing skills or even basic networking experience for that matter. Trust me I know, I’ve had to replace a number of (single, dual and a quad) CCIEs on various projects where things had gone so bad we were about to lose all future business.

Don’t get me wrong, I value certifications and see them as a good means for career progression. I do have issue though with the UK market putting so much pressure on individuals to develop themselves, alienating them completely from a corporate drive to improve through a shared responsibility. UK employer – employee relationships have become too one sided… For more on this see a recent article of mine.

Alcatel 7210 port mirroring

 | 12:52

Recently I’ve been doing more on Alcatel as I’m working in O2′s test-bed down in Slough, slaving away at testing aspects of their new LLU broadband core and new BT 21CN wholesale connectivity. Although I’ve not been able to write a lot in recent years due working for an integrator rather than an ISP; I’m mostly not allowed or it’s unwise for me to divulge what I’m working on…

However, it’s common knowledge that many providers use Alcatel and they seem to do pretty well in the ‘booming’ broadband market. Hence I thought I’d share a little snippet of an annoyance I recently encountered.

When using an Alcatel 7210 to sniff traffic and interconnect different media; 1Gbps copper and 10Gbps fibre. I found that sniffing is counter intuitive to people only trained on Cisco. A few pointers:

  1. Port mirror destinations are defined in configuration
  2. Port mirror sources are set through debug commands
  3. When mirroring VPLS ports (I needed an e-pipe/Layer-2 tunnel) I found that egress sources did not work, only ingress did and only one ingress port can be set per mirror session. It did not matter if I use the port or the SAP as source.

I was left to sniff in two places to capture both up- & down-stream traffic. YMMV as a 7750 will be different, but I don’t have one available to me to test on…

Commands used:

echo "Mirror Configuration"
    mirror-dest 4 create
      sap 1/1/4 create
      no shutdown
    mirror-dest 11 create
      sap 1/1/11 create
      no shutdown

And the debug command:

*A:<hostname># debug mirror-source 4 port ?
- no port ...
- port <port-id> egress ingress
- port <port-id> egress
- port <port-id> ingress
- port lag ...

*A:<hostname># debug mirror-source 4 sap ?
- no sap <sap-id> [ingress]
- sap <sap-id> {[ingress] }

As can be seen above capturing by SAP is only supported at ingress. Using port and SAP yielded the same result, only ingress packets were ever sent to the destination port. Despite show mirror stating both Egr & Ing.

*A:<hostname># show mirror mirror-dest 11
Mirror Service
Service Id       : 11                   Type          : Ether
Description      : (Not Specified)
Admin State      : Up                   Oper State    : Up
Forwarding Class : be                   Remote Sources: No
Slice            : 0
Destination SAP  : 1/1/11               Egr QoS Policy: 1
Local Sources
Admin State      : Up
-Port                                   1/1/26                          Egr Ing

Hacked, but not for long

 | 23 Oct 2009 03:25

Two days ago my wife was notified by one of her customers of a defacement of their site. Obviously this kinda stuff always happens when I’m ill and the last thing I want to be doing is dealing with some random hacker. This post is about what I found and how this should motivate people to at all times update their websites with the latest security fixes and practices.

As far as I’m aware no sensitive data was access nor was any serious damage done other than a few index.html files over written. The only real loss is the joomla site that was broken into as the owner is on holiday and hadn’t updated it for a while, suffice to say it’s down for the time being.

Show me more… »

Ada Lovelace Day, to the one I love

 | 24 Mar 2009 15:58

Today is Ada Lovelace Day, an international celebration of women in technology that centres around the use of blogs.

Launched by Suw Charman-Anderson, a freelance software consultant, Ada Lovelace Day is a day of blogging designed to draw attention to women who are “excelling in technology”.

Seeing this initiative made me think of my wife who has spent the last year carving out her own piece of the IT industry. Some may call me biased, true. Call it what you like, I’m taking this opportunity to put my wife in the limelight and honour her for her efforts and achievements after we relocated to the UK.

Show me more… »

Python rocks

 | 2 Mar 2009 11:21

Doing a network upgrade of 28 sites, 60 services and roughly 160 old devices to just 60 devices. Python has become my friend for reading configs, creating csv files and verification. Will be posting my scripts later. I’m sure there are clever people out there who can tell me where I went wrong or what I could be doing better. However time fails me to post them right now.

mod_rewrite = voodoo()

 | 20 Feb 2009 19:10

Been hacking away as I found out I couldn’t edit a page nor did the page show up. My about page linked to the gallery instead. Spilt some blood and danced around on the keyboard loads. the result:

Note to self: DO NOT alter permalinks for pages! my about page was linked as who-am-i and this caused major issues. I also turned off the automatic XHTML corrections in the write-options page, not sure what it does so I’d rather get an error when writing a page than have my site go down. Priorities they say…

next to that my gallery now runs mod_rewrite in safe mode which works fine and I’ve re-linked an image on my about page. I recon the link was old style from WP 2.3 and older versions of WPG2 and Gallery2.

Please let me know if you find any issues, particularly with missing images. I hope all is well now and my server won’t die another terrible dead due to rewite/permalink hell.

Finally I have found the real culprit: /.htaccess was messing with things. What made it hard to find was that this file has been in place for a few years now. Upgrading WordPress to v2.7 as well as WPG2/Gallery2 tipped the balance.

What worked?
RewriteRule ^$ /wordpress/ [R=301,L]

Instead of:
RewriteCond %{REQUEST_URI} !^/.+
RewriteRule ^(.*)$ /wordpress/ [R=301,L]

%d bloggers like this: