(The other method of combining VLAN-tagging and bridging is to bridge the physical interface first, then create VLANs on the bridge.  I couldn’t work out how to get VLAN-unaware guests attached to this kind of setup, and it didn’t work for me even to give IP access to the host using a br0.100 for example.  Still, it must work for someone as it’s written about a lot…)

I realised that from particular virtual machines I needed to get access to the VLAN tags — I needed VLAN-awareness.  Now I knew up-front that the way I could do this was to just throw another NIC into the machine and either dedicate it to the virtual guest or set up a bridge with VLAN tags intact.  I really wanted to exhaust all possible avenues to solve the problem without throwing hardware around (as I’ve been doing a bit of that recently, I have to admit).

First, I tried to use standard Linux bridges as a solution, but discovered that an interface can’t belong to more than one bridge at a time, which put paid to my plan to have one or more VLAN-untagging bridges and a VLAN-tagged bridge.  I figured it could be done with bridges, but I envisaged a stacked mess of bridge-to-tap-to-bridge-to-tap-to-guest connections and decided that wasn’t the way to go.

Next I checked out VDE, which I had first seen a couple of years ago — but something gave me the impression that VDE either wasn’t really going to give me anything more than bridging would, or was not flexible enough to do what I needed.  I like the distributed aspect of VDE (the D in the name) but I’d rarely use that capability so it wasn’t a big drawcard.  I widened my search, and found two interesting projects — one that eventually became my solution, and another that I think is quite incredible in its scope and capability.

First, the amazing one: ns-3, “a great network simulator for research and education”.  As the name suggests, it simulates networks.  It is completely programmable (in fact your network “scripts” are actually C++ code using the product’s libraries and functions) and can be used to accurately model the behaviour of a real network when faced with network traffic.  The project states that ns-3 models of real networks have produced libpcap traces that are almost indistinguishable from the traces of the real networks being modelled…  I’ll take their word for that, but when you get to configure the propogation delay between nodes in your simulated network it seems to me it’s pretty thorough.  Although the way that I found ns-3 was via a forum posting from someone who claimed to have used it to solve a similar situation as me, and ns-3 does provide a way to “bridge” between the simulated network and real networks, the simulation aspect of ns-3 seems to be more complexity than I’m looking for in this instance.  It does look like a fascinating tool however, and one I’ll definitely be keeping at least half-an-eye on.

To my eventual solution, then: Open vSwitch.  Designed with exactly my scenario in mind–network connection for virtualisation–it has at least two functions that make it ideal for me:

• a Linux-bridging compatibility mode, allowing the brctl command to still function
• IEEE 802.1Q VLAN support (innovatively at that)

The Open vSwitch capability can be built as a kernel module (there’s a second module that supports the brctl compatibility mode), or very recent versions have the ability to be run in user-space (with a corresponding performance drop).

On the surface, configuring an OvS bridge does seem to result in something that looks exactly like a brctl bridge (especially if you use brctl and the OvS bridging compatibility feature to configure it), but its native support for VLANs really brings it into its own for me.  In summary, for each “real” bridge you configure in OvS, you can configure a “fake” bridge that passes through packets for a single VLAN from the real bridge (the “parent” bridge).  This is exactly what I needed!

For the guest interfaces that needed full VLAN-awareness, I simply provided the name of my OvS bridge as the name of the bridge for libvirt to connect the guest to–OvS bridge-compatibility mode took care of the brctl commands issued in the background by libvirt.  The VLAN-unaware guest interfaces presented a bit of a challenge–the OvS “fake” bridge does not present itself like a Linux bridge, so it doesn’t work with libvirt’s bridge interface support.  This ended up being moderately easy to overcome as well, thanks to libvirt’s ability to set up an interface configured by an arbitrary script–I hacked the supplied /etc/qemu-ifup script and made a version that adds the tap interface created by libvirt to the OvS fake bridge.

The only thing I might want from this now is an ability for an OvS bridge to have visibility over a subset of the VLANs presented on the physical NIC.  The OvS website talks about extensive filtering capability though, so I’ve little doubt that the capability is there and I’m just yet to find it.  From a functionality aspect, OvS is packed to the gills with support for various open management protocols, including something called OpenFlow that I’d never heard of before (but I hope that some certain folks in upstate New York have!) but is apparently an open standard that enables secure centralised management of switches.

Detail of exactly how I pulled this all together will come in a page on this site; I’ll make a bunch of pages that describe all the mucky details of my KVM adventures and update this post with a link, so stay tuned!

N’s school recently decided to distribute the weekly school newsletter via e-mail, and had allowance for one e-mail address per family.  Not wanting the additional overhead of having to have either S or me receive it and then having to forward it to the other, I thought it would be neat to have a single common address that, when items arrived, distributed the mail to multiple boxes.  Of course I took the stupid path of providing the school with a yet-to-be-created e-mail address, foolishly trusting my ability to set the system up before they tried to send anything to it…  but in the end it was not so foolish after all, as unbeknown to me I already had everything I needed to achieve my objective.

Unfortunately the first thing I did was assume that I needed mailing list software.  I installed Mailman, and started to read-up on the process to get it working.  I did this on my yet-to-be-commissioned KVM-hosted mail server (a blog post for another day), and started trying to diagnose why mail wasn’t getting delivered.  I had set up Postfix on this mail server to point to my existing LDAP to test, and thought that there was a problem there (but also started to work out if there was a way to use the LDAP server to manage the Mailman aliases).  I re-found the Postfix LDAP HOWTO, and stumbled over the section entitled “Example: expanding LDAP groups”.  Et voila: multidrop incoming mail without the need for a mailing list manager!

I had always assumed that e-mail aliases were a one-to-one mapping of alias address to real destination.  Not the case: an alias can have multiple destinations.  It doesn’t just apply to LDAP alias support, either: as per the “aliases” man page you can do

name: value1, value2, ...

In my LDAP situation, all I need to do is list the alias in the “mailLocalAddress” attribute of which ever users need to receive mail for that alias.  Done!

I may have to keep Mailman, however.  Shortly after this success, I wondered how cool it would be to have the notification SMS messages for voicemail received at home, that currently go only to S, come to me as well.  I’m using a hosted email-to-SMS gateway service for this, so the “alias” would have to expand to multiple external e-mail addresses.  I’m not sure if you can alias mail addresses that are not in your domain…  I’ll have to try and see–might be easier to do that than subscribing to a Mailman list via SMS-to-email! 

Since I’ve been back home now for almost a month, it seems silly to call these posts “travel updates”. 

With the experience of visiting le Viaduc de Millau still buzzing in my head, I pointed my trusty Peugeot back toward Montpellier for the journey to Germany.  The run down the mountain back toward the coast was a really nice drive, but by the time I was back in Montpellier it was back to nasty busy city driving.  I think I made a little bit of an error: instead of following the path that Google found for me to get to the A9 (which was more-or-less back through the middle of town), I followed the first sign I saw that said “A9 NIMES”.  This ended up taking me on a Cooks Tour of bypass roads around the south outskirts of the city, past industrial estates and the consequent heavy workaday traffic.  The city path was very likely to have been quicker and easier.  Oh well.

Once I made it to the A9 for the trip north, I was able to settle in and enjoy the drive again.  The autoroutes in France are excellent, with a great smooth driving surface (in spite of the heavy-vehicle traffic they carry) and plenty of visibility and clearance for cars to be able to carry the 130km/h speed limit (again, in spite of the heavy-vehicle traffic, which is only permitted to do about 90km/h).  Mind you I ended up paying around 50€ in tolls while I was in France!  If it’s a demonstration of how tolling a road can lead to better quality, I don’t mind at all.

The traffic bogged down a bit going through Lyon, but soon opened up again.  I was starting to get a bit worried about the time: I’d left Montpellier three or four hours before, yet seemed to be only a third of the way there!  Night was starting to fall as I turned east onto the A36 — the car was at last actually pointing toward Germany!  A short while after that, I stopped for some dinner before making the last part of the drive.   I was not far from the border by this time, and it looked like I was making good time after all.

I hadn’t planned for my first drive on an autobahn to be at night, but that’s how it worked out.  About the only indication that I’d actually crossed into Germany was the change in the road signage!  The speed limit dropped to 120km/h, but a little while later I saw a sign that showed the 120 crossed-out.  This, I eventually worked out, was the only indication I would get that I was on one of the famous speed-unlimited autobahnen (well, the Mercs and Beemers and Audis rocketing past me were another indication).  Because it took me so long to work out what was going on, I almost didn’t get to go for a rocket myself — I had wound the Peugeot up to about 140-150 and was still getting passed like I was stationary, so I decided to give it a run.  In a few seconds the little Pug was at 195km/h, and seemed like it could have gone a bit higher, but slower traffic ahead meant I had to back off.  As it turned out, I didn’t get another chance to wind it out because we were in and out of roadworks for the last part of the run to Stuttgart.

Eventually I found the last motorway exit I had to take, and I was on the streets of Sindelfingen.  I had made it all the way from Montpellier, without a single wrong turn!  Before congratulating myself too heartily though, I had to find my hotel…  and this was a bigger challenge than I had thought.  I found it, eventually, but not before I’d driven up the same street three times (at least) and done at least one U-turn in front of the place without realising it…

As I was planning my drive from Montpellier to Stuttgart, I suddenly remembered “that stonking-great bridge somewhere in France that those pommie tossers drove those cars over”.  I really had no idea where it was — I couldn’t even remember the name of it.  Somehow, however, I managed to locate it — and found that it was only a bit over an hour’s drive from Montpellier.

So Google Maps told me at least, and my record with that site was not great.  When first I consulted the Googleplex for how to get from Montpellier to Stuttgart, I’m sure it said it would take 3-4 hours.  Just before I’d found le Viaduc de Millau, though, I asked it again and it said more like 8 hours.  More on that later…  but now I was contemplating making my 8-plus hour trip to Stuttgart into at least 11.  I was seriously considering giving up on the tentative plan to see the bridge.  Then I thought: how would I feel if I went home, knowing that I was so close and didn’t bother going?  I made my mind up: I was going to Millau.

I planned my departure the following morning to be a little earlier than originally scheduled, and packed the bags the night before.  The next day I got moving nice and early, right in the middle of Montpellier weekday-peak morning traffic!  It didn’t take long for that to clear, though, and I was on the A750 heading west.  The A750 joined the A75, and then I was heading up into higher altitude.  The diesel Peugeot I was driving ate up the twisting climb with no trouble, and before long the road had levelled- and straightened-out a bit.

I saw a tourist sign saying “Viaduc de Millau”, and realised I was almost there.  Then, I was there!

Darned windscreen wiper! Actually it doesn't matter really, since there's no way a photo from a moving car could do it justice.

You can see the towers of the bridge pylons in the distance: the seventh (and most distant) one is still over two kilometers away! The sign in this photo is for the tourist stop on the north side of the valley, which is three kilometers down the road, and the bridge starts just past the sign…

I tried to take a couple of photos as I was going over the bridge to get a sense of the height and distance involved, but it was a wasted effort.  Not only was the camera unable to focus on anything but the blurring side barrier of the bridge, but the valley floor below was probably too far away for a camera to be able to convey the scene from a car.  So I concentrated on driving the rest of the way over, and trying to enjoy some of the view.

On the north side (as the signpost said) there is an information kiosk and observation area, so I pulled off the road and stopped there.  The observation point turned out to be the peak of a hill accessed by a very steep climb up a bitumen path… but when I made it to the top, the pain of the climb was soon forgotten.

The bridge actually looked to me like it was from another world: it is so big, so high, so amazing and different, that it just doesn’t seem like it could have been made here.  It was truly an amazing thing to see, and it didn’t matter about the lung-bursting climb up the hill or the finger-numbing-face-freezing wind blowing up the Tarn valley or the drizzle of rain that just refused to go away — I could not bear the thought of having to leave there.

Le Viaduc de Millau. I'm surprised I got these photos, I was beginning to wonder about my chances of frostbite thanks to the wind and rain!

I took a stupid number of photos, and stood for a while and just gazed.  I realised it was still (just) daytime in Australia and phoned home, but must have sounded like an idiot just banging on about a bridge.

Eventually I realised that I would have to leave in order to get to Stuttgart in a reasonable time, so reluctantly I set off back down the hill.  I went through the souvenir shop and picked up a trinket or two, along with a brochure or two that N might take an interest in.  Then, with even more reluctance, I got in the car and departed.  I wasn’t able to avoid the toll plaza — 12 euro (6€ each way) in tolls!  It was a small price to pay though — besides, I got to drive over it again!

The Millau Viaduct is a wonder of the modern world, and I am so glad that I didn’t talk myself out of driving up to see it.

I took a few photos then loaded my gear on the train (big bag in the luggage space at the end of the carriage, smaller stuff in the overhead rack), then went back onto the platform to get a few more photos.  I’m sure I was still acting like a stunned mullet as I wandered around the station!

As departure time drew closer, I headed back to my train and got comfortable.  I faintly heard the sound of the doors closing and then, without a sound, the train started moving.  It picked up speed as it started to snake along the lines heading out of Paris: there were a couple of curves where I could see the front of the train as we went.  Even though we were still in the suburbs and the tracks were eight-wide, the TGV was moving at quite a pace as we headed south.

Some breakfast came by, and the next time I looked out I noticed that the other tracks were gone and we were moving a lot faster now.  At no time had I felt any great acceleration, I suppose for comfort’s sake they let the train wind up gradually.

Then we got faster still.  And faster.  And faster.  And faster.

Again I have to reiterate: if you’re not a train-fan, you probably won’t appreciate how exciting, exhilarating and mildly terrifying it was for me.  I realised that I was actually on the ground at 300+km/h, and that if I was in a plane I’d be airborne by then!  In the dark the night before, I hadn’t been able to appreciate going through tunnels or passing under bridges at that speed.  The line ran near a highway at one stage, and I just couldn’t get my head around seeing the cars that I knew were going in the same direction as I was moving backward!

I could see trackside distance markers, and did a rough timing of our travel over one kilometre: “one-onethousand-two-onethousand … 12-onethousand”.  Math it out: that’s 300km/h.

I expected that the train would stop a couple of times, but there was only one stop (Nimes, about 100km from Montpellier).  The remaining run from Nimes down to Montpellier was fast, but not TGV-fast.  As we pulled into Montpellier, I gathered up my gear and got ready to leave the train.  My first TGV journey was over!

When the train did arrive, it was three minutes late.  I was amazed: over all those hundreds of kilometres, we only accrued a delay of three minutes.

I used a map in the Montpellier railway station to find that my hotel was literally a stone’s throw away.  I hauled my bags up the street and into what seemed like a dingy alley to the hotel and checked in.  My room had a dodgy double doorway onto the dingy alley, and I looked out at the street and watched a few cars go by.  I also got my first spectator view of French contact-parallel-parking!  That evening I met up with my residency colleague and a couple of his workmates over a couple of Belgian beers, and went for a stroll through the city after taking a slightly wrong turn when I was dropped off near the station.

The next day, since the plan to go to IBM didn’t work out, I had a chance to look around.  First order of business was to do some planning for the drive to Germany the next day, so I did some internetting before going to pick up my car.  The car was a diesel Peugeot 308, and I went for a bit of a drive to familiarise myself.  Thankfully the streets of Montpellier are a bit more forgiving than metropolitan Paris!  I managed to get lost a couple of times, but did my usual Zen navigation to get back on track (thank-you, Douglas Adams).

After the car adventure, I went for a bit of a walk around the old part of the city and took a couple of photos along Esplanade Charles de Gaulle.  Once again I saw that although large cities around the world are starting to become more and more alike (town square, shopping mall, etc.), European cities still have the charm of the “old town”.  I really like the narrow cobbled streets with people walking along seemingly day or night, and the food stalls and shops every couple of doors — real food shops, like a patisserie or coffee shop, not your chain-of-the-week like Starbucks or McDonalds.  Yes, I could really get the hang of Europe: I need to put more effort into learning more of the local language though.  I found myself too cautious about my inability to order from those patisseries and coffee shops to be able to enjoy them.  Dinner one night in Montpellier was Subway, and as I walked back to the hotel to eat I found myself looking at the local shops and regretting that I wasn’t confident enough to try.

The time came for me to leave Montpellier though, and start my journey to Sindelfingen in Germany.  My research on the route yielded an interesting fact: the Millau Viaduct is only a little over an hour’s drive from Montpellier…

While my Thalys journey to Paris was coming to an end, I got a phone call.  I wasn’t going to answer it for the combined reasons of being in a foreign country, being on a train in a foreign country, and being in a small room not normally associated with telephone communication while on a train in a foreign country.  Something made me answer the damned thing though.

It was someone claiming to be from my bank in Australia, asking me if I’d just used my credit card to buy something from the Apple Store in the US.  Now I did very nearly say yes, and to stop bothering me with such stuff: this was the card that had been associated with my Apple ID, and there was a chance that N had picked up my iPod and stumbled through and found the App Store and bought something.  They’d never contacted me before about App Store purchases however, and then I remembered that card was not on my Apple ID any more.  So I replied non-committally (and very helpfully, in hindsight)…

“Well, maybe.”

“This was only in the last couple of minutes,” said the bank.

“Oh,” said I.  That changed things.  Knowing that the card was not linked to my Apple ID any more, there was very little chance that N might have done something.  It definitely wasn’t me either, given where I had been during the few minutes in question.  ”No, then,” I replied.

At this, the VISA machinery sprang into action.  Within seconds I had been recited the standard dialogue about how my card(s) had now been cancelled and that I would soon receive new card(s) and PIN(s), so on and so on.  Being none too happy about having to re-arrange scheduled charges to the account (the only use that particular card gets, as it turns out) I started to think about how the number had got into the wild.

In spite of knowing that there are card number generators that the bad guys use to generate valid card numbers to try on unsuspecting e-commerce sites, something gave me the thought that it was more likely I had lost the small wallet that card was kept in.  I started thinking about the other cards that were in that case.  Hotel/car loyalty cards: painful, but not a problem.  Unused AMEX: cancellation drama only.  Travel-backup credit card: hmm, that might be a problem.  What else…

Oh, wait a minute…

OH CRAP.

Last I saw that wallet it was wrapped around…

O. M. F. G.

My PASSPORT.

Instantly I understood the feeling described by the term “heart in my mouth”.  There I was, standing on a train pulling into Paris Gare du Nord with my knees buckling contemplating the possibility that my passport was lost.

I started to look through the bags I was carrying, the places where I knew the wallet should have been.  Nothing.  By this time the train had stopped, and I alighted the train with the other passengers and took my frantic search to the Gare du Nord platform.  Still nothing.

My mind was racing.  Do I continue my journey to Montpellier as planned, and sort out the passport later?  Maybe ironically, the thing I was most upset about was having lost all the stamps in my passport!

I decided that I couldn’t think properly standing on a train platform and that I had to get to my hotel and sort it out there.  I managed to find the subway that links the SNCF station to the RER, but halfway through the subway I realised that I couldn’t go any further without having a proper search.  So in the middle of a railway station subway in Paris I started rifling my luggage like a sniffer-dog looking for the stash (and it wasn’t until later that I realised how much trouble that might have got me into).

FOUND.

The wallet, all cards secure and still encasing my passport, had worked its way into the lowest portion of the wheeled laptop bag I use.  I suffered the joyous feeling of my heart returning to its rightful place, combined with the return of my ability to breathe.  As I put stuff back into my bags and resumed my journey, I tried to concentrate on the task of getting the right ticket, the right RER line and the right train to get me to Gare de Lyon (thanks to the signage in the station, this was made very easy).

Once I was on the RER, with other commuters around me and me trying to marshall my luggage, I realised how I could not have done the trip thinking that I had lost my stuff.  I also realised that that was the closest I ever want to come to actually losing my passport while overseas.

Oh, and the credit card?  Like I said, it was in the wallet all along.  The card hasn’t been out of that same wallet for over twelve months, and the regular deductions (and my automatic payment to cover them) have been the only transactions on the account for at least that long, so I guess a card generator just happened to get lucky with my number.

So if you happen to get that phone call from your bank, think seriously about your card’s whereabouts and recent activity… and for heaven’s sake don’t do what I did and jump to the conclusion that the card was lost or stolen — your imagination might just take you someplace you really don’t want to go.

Update: Before I left Australia I had asked the bank to reissue the card for a promotion they were having, but the new one didn’t reach me before I left the country. When I got home, I had a look at the card that had been issued — the one that got hacked.  I think I know now why I got pinged: the CVV number (the three-digit printed number on the back of the card that is supposed to increase security) was the last digit of the card number followed by “00″ — I’d have to think that would be about the weakest CVV number the card could possibly have had! I feel much better now that this was simply a random selection by a card-number generator, facilitated by a stupidly-insecure CVV.

I’m not going to talk about the work stuff in these updates: for one, this is not actually a work blog so I probably shouldn’t anyway.  Secondly, it’s a bit on the boring side of things and I’d rather talk about the travel.  So, with that decided, let’s continue…

So Monday arrived and I did my presentation, then picked up my bags from the hotel and went to Schiphol.  I made use of the NS HiSpeed lounge (a little bit like an airline club lounge, but on a smaller scale) to have a refreshment before heading down to the platform for my train.  My final destination was Montpellier, France, but because of the time I thought I had to be there I had to overnight in Paris: so it was Thalys to Paris on Monday, then TGV to Montpellier on Tuesday morning.

I feel the need here to reiterate what I mentioned in the previous post: I’m a rail-fan.  When TGV was introduced in the 1980s, I made it one of my life’s goals to make a TGV journey one day.  I marked the goal halfway complete when a colleague and I travelled on Thalys in 2006: half-complete because we only went from Amsterdam to Brussels, which is not true high-speed (although I saw that it has been upgraded, and Thalys will run high-speed to Amsterdam from December 2009).

As I boarded Thalys for Paris that Monday night, I realised that my goal was about to become fully-complete.  I settled in as the train departed into the Dutch night, and started to enjoy the comforts of Thalys “Comfort 1″.  I hooked up to the Wi-Fi and made a couple of silly Facebook updates, and saw a nice little map feature they provided on their portal page:

My train was just south of Antwerp at this time... Cool, eh!

Due to the dark outside, it was difficult to get a sense of how fast the train was moving: the only way to know for sure was the occasional lit-up building or car that went by.  As I said, having travelled on Thalys before I knew that the best was yet to come (in other words, after we went through Brussels).

Eventually we pulled into Brussels, and my excitement built a little more.  The wait in Brussels-Midi station was almost unbearable!  Finally though, we got moving again.

I read an article by a UK travel reviewer when the TGV first ran.  He described a dramatic surge of acceleration as the 1k5V standard French pantograph was lowered and the 25kV circuit was activated on the high-speed line to Lyon.  I didn’t experience any such hard surge, but as we picked up speed out of Brussels I just knew that something was different.  I guess I was seeing enough points-of-reference outside to know that we were moving much faster than before, but whatever it was I could tell that now we were really moving.

I sat and enjoyed it all for a while: the surreal feeling of approaching the continuous lights of a stream of traffic on a road or highway impossibly fast, and realising that the train was actually going to pass over it…  and then the lights were gone as the train flashed over the highway.  The thrilling hum and vibration of the train itself: not disturbing at all, just the feeling of being on board a piece of machinery that was working hard.  After a while I checked back on the ThalysNet map, and realised that the map was clickable…  I clicked, and was rewarded with an enlarged view, with a speedometer! I refreshed the view a couple of times to make sure it wasn’t something static…

I did refresh and get one that said 300km/h, but it looked a bit staged

I refreshed each time I felt a large change in speed (and before anyone asks, no at 300km/h you can’t tell a change of 5km/h), and saw enough change in the display to be confident that it was a real representation of the train’s speed.

Unfortunately the journey had to come to an end.  I’ll write a separate post about the terrible experience I had as I arrived in Paris, but once I got over that I worked on the task of getting myself from Gare du Nord (where Thalys operates from) across town to Gare de Lyon (where the southbound TGVs run from, near which I’d booked my room for the night).  I ended up managing very easily to find the way to the RER station, buy my ticket, find the right train — a direct train, where my research had told me I’d need to change trains — and hop off at Gare de Lyon.  After a little mixed-up street navigation (unbeknownst to me I’d left the station from the back entrance, and ended up walking all the way around to the front) I made it to my hotel, checked in, and negotiated an old-style elevator (with a swinging outer door!) to my floor and my room.

The next morning I went for a little walk.  I realised I was quite close to the River Seine, so thought I couldn’t go home without seeing it.  What can I say: yes, it’s a river.  I thought I’d be able to see perhaps just the top of the Eiffel Tower, but there were too many buildings in the way.  Back to the hotel then, to check out and go to the station.

When I got to the plaza in front of the station, I had to pause.  There I was, actually standing in front of Paris Gare de Lyon!  Okay, a railway station… but which railway station!  This is where TGV basically started it’s first passenger services.  I was having another one of those dream-about-to-come-true moments.  Then I went inside and saw a real TGV! If you’ve seen the movie Cars, you’ll know the scene at the end when the Michael-Schumacher-Ferrari drives into Luigi’s Casa Della Tires and Luigi ends up fainting (“a REAL FERRARI!”).  For me, seeing not one but at least five TGVs was much like that.  Okay, they aren’t the old TGV Orange that I knew when I was a kid, and the design is a bit updated, but they’re TGV and they’re where modern high-speed commuter rail began.

Next update I’ll describe more of my TGV experience, as well as my first European drive!

I arrived in Amsterdam on Sunday morning, and decided that rather than get bounced by the hotel trying to check in too early that I’d fill in some time at Schiphol Airport.  While I was there I had a chance to reflect on the flight over from Hong Kong.

My stop in Hong Kong turned out to be just long enough to create a risk of me missing the flight.  I was sitting in a comfortable chair in the lounge and getting occasional snacks and drinks, and successfully dozing for ten or fifteen minutes at a time.  Our boarding time was delayed by about an hour, and whether I became just a bit more relaxed because of the delay or because the noodle dinner I had settled in my stomach I ended up falling into a proper sleep.  Like many airlines overseas, Cathay Pacific doesn’t make boarding announcements in their lounges…  I awoke to find the departure monitor flashing “Final Call” next to my flight.

The terminal building of Hong Kong’s Chek Lap Kok Airport is shaped a bit like a big letter-Y with an upside-down capital-T overlaid.  You guessed it, to get to my flight I had to go from the tip of one arm of the T (where the Cathay Lounge is located) to the fork of the Y (where my departure gate was).  Doesn’t sound like far, does it: perhaps I didn’t mention that this munged-up T-Y shape got stretched out in the middle and is about three-times longer than it is wide…  As I fast-walked along travelators, I kept looking at monitors and expecting the dreaded “Gate Closed” to appear.  I drew closer to the gate, and could see no queue of people waiting to board–now I knew I was pretty late.  I saw someone in a green uniform starting to walk away from the gate toward the main terminal building (toward me), and as I got closer I saw that she was carrying a card with my flight number on it in large letters.  At this point I relaxed: they’re still looking for the latecomers.  I waved at her when I realised this, and when I reached her she asked my seat number.  It looked like I would be okay.

When I finally got to the gate I saw that I did actually have a bit of time available: the queue was actually on the aerobridge waiting to actually get on the plane.  It took at least five minutes in the bridge before I was on the aircraft, so it wasn’t like I was keeping them from closing the doors.  Still, a bit of panic to teach me to be careful about being inattentive in an airline lounge.

The flight was bad.  Actually I take that back: in the context of other long-distance flights I’ve taken it compares badly, but really it was probably just your normal Cathay Pacific long distance economy-class flight.  Cathay’s seat in Economy though is weird: it’s actually a little enclosure, a bit like a discount version of the Qantas Business-class cocoon.  What’s weird is that the seat doesn’t recline, instead the seat cushion slides forward and back.  The bottom of the seat back is attached to the back edge of the seat cushion, and the top of the seat back slides in runners in the seat frame.  So the angle of the seat back does in fact change, but for someone tall like me instead of being able to actually recline with a straight back I got curled up by the crazy thing.  The only good thing about it is that it eliminates that feeling of intrusion you get when you want to sit fairly straight but the person in front wants to recline all the way back.

The arrival into Amsterdam’s Schiphol Airport was uneventful.  The reason we were delayed from departing Hong Kong was actually so that we didn’t arrive too early to be allowed into Schiphol; the flight time was about an hour shorter than scheduled.  So even though we left late, we arrived on-time (if not a bit early).  The arrival hall in the airport was a mess however: I’m sure that they put us into the smallest passport hall in the place, and our full 747 from Hong Kong arrived at the same time as a United flight from the US.  The place was packed–they had to switch off the escalator into the hall to stop people from getting jammed into this room like sardines.

After I made it out to pick up baggage I waited another 10 minutes or more.  I had thought that the length of time it took to get through passport control would have given plenty of time for the bags to get off the plane, but it wasn’t so.  It took almost an hour to get landside–perhaps the messiest arrival at Schiphol I’ve ever had.

I decided to save some time for the next day and went to pick up my ticket for the train to Paris.  The airport terminal is actually built behind the railway station–if you were to arrive at Schiphol by taxi for the first time, you’d be tempted to think that the taxi driver misunderstood and took you to the train station instead of the airport.  Entering through the front doors, you have to walk past all the railway kiosks and ticket machines and timetables before you get to anything that even remotely looks like something to do with air travel.  I’m a train-guy though, so I like it that way.

I am so much a train-guy in fact that after I had some breakfast I decided to throw my bags in a locker and hop on a train to fill in some time.  I went to Utrecht, which I thought would be an hour or so away and would fill in a nice chunk of time, but actually only turned out to be 30 minutes away.  On this little trip I saw some Dutch countryside and was stunned at how green it looked.

When I got back to Schiphol I picked up my bags and got the airport shuttle to the hotel.  ”Yes, sir, your room is ready”–how wonderful those words can sound when you’re tired and filthy from a long plane ride!

I’d love to break my blogging drought now, as I have about five hours before I board my next flight, but I have a splitting headache which I’m sure you understand is not conducive to effective computer usage (which is a shame, as the Wi-Fi here is excellent).  Maybe later.

By the way, what brings me to Hong Kong?  I’m going to Europe for my remaining ITSO Workshop presentations.  Amsterdam on Monday, then Montpellier (France) on Tuesday.  I make some things up for a few days, then London next Monday followed by Milan on Thursday, then flying home via Rome and HK (again, three fortnights in a row).

I’m in Poughkeepsie for a couple of ITSO projects. For the last week I’ve been working on the development of material for the ITSO’s Workshop World Tour. I’m helping out with the “System z Virtualization” stream, and I’m lucky enough to be presenting in a half-dozen cities around the world. Only problem with that is that the first one is only a few days away, and the material probably needs a bit longer in the oven (so to speak). Still, I’m looking forward to the event as a way to help build the Linux on System z community (and to do a bit of travel to some new places).

Starting this week I’ll be doing a residency with the ITSO. The book will be “Security on z/VM” — I don’t know if it was made so open-ended on purpose, but there’s a lot that could go into a book with that title so I’m looking forward to a busy time as the project develops.

More on my US travel as I get settled (I’ve been here a week and my sleep is still all wrong).

I’ve been a user of Cisco IP phones, buying 7960s and a couple of 7970s through a well-known internet site (maybe it starts with an “e”, not sure).  The phones have been excellent, and I’ve even written a few XML apps to supplement their use here.  The 7960s are getting a bit dated now, however, and I found myself contemplating buying 7971s (or even something newer, like the 7965 or 7975).  Before I committed myself further into the relationship with Cisco, though, I thought about what I was really getting out of using Cisco phones.

Like many users of second-hand Cisco gear, I only purchased the hardware.  I do occasionally succumb to a nagging feeling of being an “outlaw” (at least in the eyes of Cisco), but admittedly that feeling usually only comes when I find out that Cisco has released another new version of SIP software that I can’t get because I haven’t paid for SmartNet.  The last time I had this thought though, I had a realisation: even if I did pay for SmartNet, the only thing I’d get would be the firmware: Cisco will only support their phone software when connected to their CallManager server (yes, even the SIP firmware).  Anyone running Cisco phones against anything other than CUCM gets no support from Cisco in the event something doesn’t work–and based on the information floating around, the problems are many.

So basically I would be paying Cisco to allow me to run one of the worst SIP implementations in embedded existence, with no opportunity to report problems with it in my environment.  Hmm, let me think about that for a minute…

At around the same time, I happened across the NerdVittles site, and in particular the post where NerdUno nominated the Aastra 57i as the “World’s Best Asterisk Phone“.  I started to do some research into it, and was astounded at the level of support the manufacturer (a Canadian company which a few years ago acquired the telephony business of a little mob called Nortel) and the community provide for this phone and Asterisk.  Looking through the phone manual, I found functions that only work with Asterisk! I found a full set of integration scripts that provide XML applications, right through to automatic provisioning tools.  Possibly the best thing was that on the product page for their phones — right there on the page that descibes the product — are links to current versions of firmware, documentation, XML application development guides, even a Linux-based application to encrypt the phone configuration files.  Not hidden in some obscure hard-to-find portal, or behind a registration-only support site.

I started to think of the possibilities…  I’d be able to freely modify the phone configuration (even via a HTTP interface if I so chose), without having to make trial-and-error changes to a cryptic and totally undocumented configuration file.  I’d be able to write XML apps without having to do laborious debugging to cater for why the parser was choking on XML that was perfectly okay according to the documentation but apparently tripped over an undocumented field length restriction or character encoding limitation.  I could get access to things like Visual Voicemail, BLF, integration with Asterisk functions like day/night mode and call parking.  I could keep the phones up-to-date for new functions and bug fixes.  With a click of a mouse I could get proper Australian tones!

So, I decided to give one a try.  Finding nothing on that “e” site I went looking for a vendor locally, and found several places that would sell one to me (legitimate e-tailers, no less!  Zounds!  A VoIP phone with a warranty?  You jest!).  It took a while for my chosen vendor to source it for me, but I’ve had it now for a couple of weeks.  It’s probably going to take a while for it to live up to it’s full potential in my installation, but since that potential is so much greater than what I have been able to do with the Ciscos I think I’m already ahead.

More in the coming weeks as the Aastra settles in.

I figured this would be an ideal way to make use of an old Nokia 6230 with a broken speaker.  Somewhat foolishly, on the assumption that it would Just Work (and that all the troubles experienced by others would not beset me) I went and bought a two-pack of prepaid mobile SIM cards and went through the adventure of activating them.  One of these SIMs I threw into the 6230, the other I kept on hand for after I got everything working.  The plan, you see, was to be able to take advantage of free calls between the two accounts by taking one of the phones with me when travelling and leaving the other strapped to Asterisk at home.

I think it’s probably fair to say that I’ve had more success with it than a lot of other folk have.  The process of configuring Asterisk to use the Bluetooth dongle is quite straightforward, and it’s even quite easy to configure the chan_mobile driver to have calls enter your Asterisk system in a routable way.  When I dialled the “tethered” mobile from another phone, I was rewarded with the ringing of my desk phone–and at this point, I think I gave myself the kiss-of-death.  “Wow, that was easy,” I thought…

When I picked up the desk phone, I was rewarded with silence.  Not just the silence of the phone not ringing any more, but also the silence of no audio being passed either way over the call path.  Nothing put the pure, desolate sound of FAIL.

Things actually went downhill from there, believe it or not.  I have tried a total of four different Bluetooth dongles, with results ranging from the aforementioned signalling-but-no-audio to why-the-@#%$-won’t-this-thing-pair.  The three different phones I’ve tried elicited a similar spectrum of results.  “Make sure your dongle has a Cambridge Silicon radio, they definitely work” say the forum experts…  Sorry guys, one of the biggest failures I had–failure of Asterisk to pick up the call–was on the last dongle I tried and, yes, it was a CSR.  I’ve even had two different versions of the bluez stack and (I think) two different asterisk-addons versions.

The one thing that I’ve distilled from all the experiences I read through is that there is a ridiculously high level of sensitivity to particular phone and dongle features.  For example, great success has been reported with the Nokia 6230i.  I figured that I was lucky and that a 6230 would be close enough…  Doesn’t look like it.  There is one model of D-Link Bluetooth device–no longer in production, by the way–generally reported to give the most success.  Tweaking the device class reported by the bluez stack in the Linux host is said to give success too, but led to me being unable to get a connection to Asterisk.  Unfortunately, I have neither the time nor the patience to spend too much time trying to go through the motions of getting it working.  I tell you, if it really is that difficult to get two Bluetooth devices to talk to each other it’s no wonder that the majority of folks still use wired headsets!

Luckily all this little experiment has cost me so far is time.  The two-pack of SIM cards cost me the grand total of $2, and they had enough start-up credit on them to allow me to receive calls without a top-up.  The handsets are from that ever-growing pile of GSM hardware that just about every modern household is accumulating now (well, at least the ones that house a gadget-freak who can’t even bear to part with a broken one).  The kernel version I’m running on the system could be an issue, since I get ugly error messages from the btusb module when I take a call, so a kernel update might help.  After that though it’s likely to cost real money–buying a new/different Bluetooth dongle, for example.

If anyone out there has suggestions on something else to try, I’m listening (reading? watching?).  I don’t mean to complain, after all I am one that usually subscribes to the “it’s Open Source, it’s the hard work and dedication of others, you got it for nothing, you’ve got no right to complain” philosophy.  It is really frustrating to come away from a couple of days’ effort with nothing to show for it, though.

I’m implementing a router box on an old low(-ish)-power PC that will be backed up by a virtual machine on my main virt-box.  I’ve already done most of the preparation of using keepalived to implement VRRP, and a colleague has given me some pointers in using the Linux-HA tools like Heartbeat and DRBD to make services like e-mail and Samba redundant.

I’ve had a soft spot for LDAP for ages; I’ve always thought that putting as much backend data into LDAP as you can would be a really good way to get failover and redundancy.  Instead of having to deal with every single server’s different way of doing replication and failover, just bung everything into LDAP and get that replicating.  Sounds good in theory, but in a nutshell it’s not working out that way for the two least-celebrated but most important components of my (arguably any) network: DNS and DHCP.

There are a number of LDAP-backed DNS projects out there.  If I’m willing to go to the bleeding edge with BIND on my Gentoo build I can get access to the two most talked-about ones (bind9-sdb-ldap and the BIND DLZ LDAP driver), and other solutions like PowerDNS and ldapdns are available.  But none of them offer integration with DHCP, and I’m currently using dhcpd’s “interim DDNS update method” to make sure that hostnames are seen in my DNS when a lease is granted (okay, there’s a Perl daemon that goes with bind9-sdb-ldap, but it seems like a sort-of clunky afterthought).

Speaking of DHCP, LDAP backends for it are virtually non-existent.  The only LDAP-enablement I’ve found for ISC DHCP involves putting the config file into LDAP, not the leases…  I actually used that for a few days a while ago and pulled it out because it was actually more work to do it that way (and for no benefit in failover).

It seems to me it would be a project ripe for the picking: take an integrated DNS/DHCP server like dnsmasq and make it write into LDAP instead of to a file.  If I had more free time I’d probably have a go at it, except for the fact that no-one really seems to be that interested in storing DNS and DHCP in LDAP: that it hasn’t been done says to me that there’s no demand for it, and it’d end up being a big waste of time and effort.

Over to you, lazyweb…  Is this a yawning chasm of unfulfilled networking dreams, or a case of me trying to make something more complex than it needs to be?  After all, the rest of the world gets by with DNS master-slave and DHCP failover, they should be good enough for me too, right? 

Nerd Vittles linked to this blog, which alludes to the possibility of mobile carriers putting pressure on Nokia to remove “free” calling capability (i.e. VoIP) from their phones.  Within the comments on that blog post comes a link to a post on Nokia Conversations (I’ve never seen that site before, but it seems to simply be a bit of a PR site…).

“Charlie” from Nokia Conversations tries to spin the changes to Nokia’s SIP support.  Firstly, in what seems to be almost believable at first he says “no, the SIP stack is still there, in fact it is actually better in FP2 than previous versions”.  Apparently, the improvements meant that the integrated VoIP client had to be dropped because it wasn’t ready.  This explanation loses credibility, however, when you see that Charlie’s blog post was made on 27 August 2008: nearly one year ago! And folks are still commenting on that thread, saying “where’s my VoIP client?”.  I cannot believe that it would take Nokia a full year to update the VoIP client and package a firmware update for these phones–especially given that two other S60 3rd-ed FP2 phones released after the N78 and N96, namely the N79 and N85, apparently do have the VoIP client!

On 8 December 2008, Charlie posts a follow-up on Nokia Conversations.  In it he says “well we made some folks unhappy, but we’ve made a fix”.  He points to something called the “SIP VoIP Settings” application that was supposed to bring back what people were asking for.  Problem is, it’s not a VoIP client at all: it’s simply a configuration tool allowing more detailed control over the configuration of a SIP profile.

In the final insult it appears that the new N97, Nokia’s current flagship also has no VoIP client.  The N97 is based on S60 5th edition and not 3rd edition, but 5th is supposedly just 3rd updated for touch-screen anyway (not a significant change in technology).

Looking more closely at the specifications pages for these N-series phones, the tiny-tiny text that says “VoIP” is missing.  It’s probably arguable therefore that Nokia never advertised the phones as having VoIP capability[1], so anyone who bought one without checking has created their own situation.  However, Nokia, why is the “upgrade” to the N95 missing one of that phone’s most popular features?

At one point Nokia’s story changes… it seems that VoIP is a function that doesn’t fit the product direction of N-series and belongs in the E-series phones (indeed both the E75 and the soon-to-be-released E72, reportedly S60 3rd-ed FP2 phones, list VoIP capability).  Why, then, do other S60 3rd-ed FP2 phones like the N79 and N85 have VoIP?

This whole “affair” seems to have been handled really poorly by Nokia.  Firstly, claim a technical limitation.  When that fails (because you discover that your users actually know something about tech), claim that your third-party providers have developed a solution.  When it turns out that the third-party products are steamers that don’t even use the infrastructure your OS provides (something you didn’t know before either), claim that the product has been “realigned” and doesn’t service that market any more–while simultaneously marketing a product in the same series with the same technology that still has the disputed feature.

I must admit to being a lot less angry about this after researching this post than when I started it.  I’m more angry about the survey I completed earlier today when I visited the Nokia website–I was very complimentary about .  My shopping-slash-wish list just lost an item–not that I was seriously contemplating buying the N97, but it’s nice to have a technical reason not to buy it rather than the boring can’t-really-justify-it line.

[1] Of course it’s easy to make this statement based on what the product pages look like now…

Unless, of course, I don’t get it.

(The first result got fixed, this version (courtesy Google’s cache) has the goodness.)

I have an ISDN phone line coming into my Asterisk system.  One of the indials is for our home number, the other is one I use for work.  Before I found FreePBX, I had manually worked the Asterisk dialplan to have calls made from my work phone(s) appear from the work phone number–useful not just for Caller-ID, but also required for the long-distance provider I use to bill calls for work.  With FreePBX I was able to use a custom context to pre-select a route that dialled the provider override prefix to send the calls through the other provider, but it was a bit of a hack using hand-written dialplan code and a bit of luck.

Before I changed to device-and-user, I naively assumed that FreePBX would allow a user to be associated with a context in the same way a device/extension can be.  This is not the case, and the context of the device is still used for routing.  This meant that I could not use a device for either work or personal calls without having to log onto FreePBX and change the device context (logging on as the work user was not enough).  I was back to square one…

I did a little research.  Firstly, I rediscovered how I was making the existing routing work.  The ISDN interface I use (driven by chan_capi in Asterisk) simply uses the outgoing caller-id of the call to select from available MSNs[1].  So I had one route that had the “normal” MSN set as the outbound caller-id, another route with the “work” MSN (plus the rewrites to add the long-distance override code at the front), and a custom context for the work devices that made only the route with the work MSN available.

Looking more closely at the user definition page, I saw that there is an “Outgoing Caller-ID” field.  By using this field, I was able to do away with the separate route and the custom context and set the work MSN there instead.  This gives me just what I need: a way to control my outbound MSN on a per-user basis!  This got me half-way there, as I still needed a way to set the long-distance override codes for work calls.  A bit more research turned up a predialling macro hook that the FreePBX folks made available.  With a bit of code to test the caller-id and the number dialled (the long distance company doesn’t handle free calls, for instance) I get just what I need.

Here’s the macro hook (this is added to extensions_custom.conf):

[macro-dialout-trunk-predial-hook]
exten => s,1,NoOp(Trunk is ${OUT_${DIAL_TRUNK}}, CallerID ${CALLERID(num)} calling ${OUTNUM} )
exten => s,n,GotoIf($["${CALLERID(num)}" != "xxxxxxxxx"]?endit)
exten => s,n,GotoIf($["${OUT_${DIAL_TRUNK}:4:11}" != "CAPI/contr1"]?endit)
exten => s,n,GotoIf($["${OUTNUM}:0:4}" = "1800"]?endit)
exten => s,n,Set(OUTNUM=1xxx${OUTNUM})
exten => s,n(endit),MacroExit()

So I’m now a happy device-and-user user!

Last time I looked at this I thought there were some problems with the way it was implemented that meant it didn’t work for my installation, so I ended up hacking together a mess of fake extensions, ring groups and queues that more-or-less reimplemented the good parts of device-and-user mode and still letting me use things from extensions mode.  Doing this however meant that FreePBX always complained about what it called “invalid destinations”, and I had to use some custom logic for doing something simple like a common voicemail access number.

What won me over to device-and-user mode again though was the ability to log a device on and off from a number.  I have a couple of Nokia handsets that have SIP clients now, and it’s handy to just have one device that all my work calls (for example) arrive on.  After-hours though, I didn’t want that device to still be tied up to the work line; it made more sense to be able to use that device for home calls.  To do that with extensions mode and my ring-group hack would mean reprogramming the ring-group (and one other change, which I’ll talk about later) when I wanted to switch over.  Presumably I could write some script or AGI logic that I could tie to a feature code in FreePBX, but I’d simply be making more custom modifications for little real gain.

In the end, I realised that the main thing keeping me in extensions mode–the ability to call a device by it’s “extension” number regardless of who’s logged on to it–wasn’t something I used often enough to warrant all the work I’d have to do to make extensions mode do what I needed.  With that in mind, I edited amportal.conf and made the all-important change:

AMPEXTENSIONS=deviceanduser

I had to reload the FreePBX admin page a couple of times, but eventually the “Extensions” tab changed into two tabs, “Devices” and “Users”.  True to the description of extension and device-and-user modes given in the FreePBX doco, the Devices and Users tabs had the same number of entries.  All I needed to do was delete the users that were no longer required (i.e. almost all of them) and the devices that belonged to the voicemail extensions from my original setup.  I then ran through each of the device definitions and correctly assigned them as either “Fixed” (statically allocated to a user) or “Ad-Hoc” (able to be logged on to a user).

This was the point at which I worked out a solution to my original dial-a-device-directly problem.  I realised that the majority of times I need that functionality is when testing.  So, for those devices that I use for testing, I left the user definition in place and made it the “default” user for that device.  This means that when I log out of the real user from that device it is reachable by the default user number, and I can dial it directly for testing.  The other use for direct-connection to a device, the intercom, requires a separate SIP endpoint anyway (due to the Cisco phones not adhering to the SIP command for remote off-hook) so I need to keep those as separate users too.

I’m quite happy with how it’s turned out–at least, I was once I’d overcome the showstopper I found!  Read about that in part two.

I imported all the entries from the old engine (I’m even gradually working through them to categorise and tag them).  I even found some Perl and SQL mojo I never knew I had to make nice redirections from the old blog to the new.

As I said when I first started this blog, none of this coolness is likely to make me write more…  But here’s hoping! 

I’d recently updated the machine that provides the transparent web proxy function for the network; one of the updates took Squid up to version 3.0 (from 2.6). This was the first thing I was suspicious of.

There’s an option in Squid that controls how it handles an “If-Modified-Since” request from a client. The default is for Squid to respond based on the age of the item in the cache, not based on the real item on the source web page. The comments in the Squid config file indicate that some clients use an IMS when requesting a reload — looks like APT is one of those clients.

Setting this option to “on” (from the default of “off”) in squid.conf fixed the issue for me:

refresh_all_ims on

The first thing was obviously to get hold of the audio file. This turned out to be surprisingly easy, thanks to Google pointing me to a piece of software called MacTracker. MacTracker is actually a reference guide for Apple products (computers all the way back to the Macintosh XL, the MessagePads, printers, displays, even iPods and mice), but part of the information it holds about the computers is their startup and death chimes.

There’s no option in MacTracker to export the audio files, but by opening the app package (“Show Package Contents” in Finder) it’s possible to navigate to where the chime sound files are stored. Then from Finder, all I had to do was zap the file to the phone via Bluetooth. On the phone, opening the Bluetooth message gave me an option to save the “music” file, which I did — this adds the file to the Music Player, but importantly makes it easily selectable in the configuration of the alert tones.

So now when I receive an SMS I hear the death chime of a Macintosh LC, and the startup sound of the Twentieth Anniversary Macintosh alerts me to incoming e-mail. I’m going to apply similar configuration to my desktops: on-and-off for the last ten years I’ve been using a Homer Simpson soundbite to advise incoming mail, and it’s a bit tired now…

Next task will be to replace the startup sound on my N810 with something a bit retro-Mac!

I've turned off the comment capability, until I can get something in place to bring the rubbish under control (a recent update to PolarBlog helped a bit, in that the crap doesn't display on the site any more, but when I log on I get to see the mess). I'm thinking of a new site, where I can discuss technical stuff a bit more and thoroughly while keeping the private stuff separate if I need to.

The site has had a bit of downtime recently, due to my non-existent monitoring of what's happening on my hosted server. This will change shortly, and I'm looking forward to things returning to the stability they had when I was self-hosting.

S's and my respective creative sides are being adequately satisfied by the iLife suite on the Mac, but there are times when we need to get the pictures out of the silver tower and onto other media—on this occasion paper, for albums and so on. A large retailer here has part of their floor space in each store set aside for those photo printing kiosks, and I introduced S to the art of putting photos onto a USB stick so that she could print some photos when next she went there…

On her return from the shop, she reported that we hadn't successfully put the photos she wanted onto the stick. When she'd plugged the stick in, she'd found only less than half of the photos we'd stored there. Sure enough, when I plugged the stick in all the files were there safe and sound. Strange thing was I could find nothing in common about the files (uppercase/mixedcase filename, long or 8.3 filename, datestamp, etc) that would have yielded the number of photos that the kiosk had found on it.

Annoying, but life is too short to worry about it. After all, this same retailer was plastering adverts of their new web-based photo printing service… S could submit the photos online for printing and pick them up from the store later.

<sarcasm>This is where the fun really started.</sarcasm>

Their app is Flash-based but seems to have some Java involved as well. While it loaded quickly enough, the app portion of the web page had an incongruous grey background that just looked dodgy. S had to create an account and sign onto the site just to get this far though, which was a bit annoying.

The workflow seemed to be to create an album, upload pictures to the album, then select photos from the album for processing. Creating the album went fine, but when the upload function was selected there were no action buttons visible to complete the operation! S was using Safari, but Firefox made no difference.

Then I suggested she use her laptop, which runs Ubuntu 8.04. The situation actually seemed a bit better to start with, as instead of the upload function showing an embedded file selection dialog like it did on the Mac we got a "normal" GNOME file dialog box. However, only some of the photos showed again: this time, it was because they had hard-coded a non-modifiable filename filter for the dialog that was only picking lower-case file extensions!

Trying to work around this, I mounted the stick manually with different mount options. I succeeded in getting all but one of the files showing with a lowercase name, and a rename fixed that one. Back in the web page however, it still didn't like us: any file chosen from the dialog box resulted in a nonsensical error message followed by a "You have selected no files to upload" dialog.

S was beyond caring by this stage (she has a very low threshold for being stuffed around by technology). She went to Snapfish after a friend's recommendation, and found a well-designed and easy to use WEB site that required no downloads or other junk.

So why did this wind me up to the point of spending all this time blogging it? Because nowhere on Big-W's site is there any mention of browser or operating system compatibility. Not even a "we've tested only on Windows, Mac users may experience difficulty"[1]. Not a blessed thing. Their Help page has a single paragraph about trouble uploading, blaming "your IT Department" for "setting certain network properties that inhibit the upload tool from working".

I wonder if the developers of the app were just so blind to believe that their gunk would just work wherever it was run, or whether they really think that it's a Windows world. Of the two I hope it's the former.

So Snapfish gets a recommendation for being not just an application hosted on the web but a web application. They do good photos too!

[1] I never expect to see Linux mentioned on these things and get pleasantly surprised on the occasions it is; even if it says "Linux is not supported", someone there at least knows enough to mention it.

While the screen of the 7970 looks nice on it’s own, and is certainly a drastic improvement over the 7940/7960, it doesn’t have the resolution to be able to display anything useful other than text and simple images. The BOM images are 524×564, while the largest image the Cisco can show is 298×168.

Here’s an example of my app in action:

Because the phone’s image viewer can’t scroll a large image I’ve had to scale and crop it, losing a fair amount of detail in the process (I did try just scaling it to fit the entire image, but it was totally useless to view). Plus, one of the key attractions of the BOM radar site is the animated images, and the phone has no way to display animations.

So what do you think, lazyweb? I’ve learned a bit more about coding Cisco XML apps, but other than that have I wasted my time? I will put it up on my Projects site eventually (once I’ve put in some more error handling etc), but let me know with a comment here…

I think what finally did it for me was the App Store. I love being able to simply go to an app on the device and easily look for software, installing what I like with no fuss. I especially like the fact that my downloads are synced with my computer, so that I don't have to keep track of all the individual items I've installed (unlike my phone; I can't think where all the sis and sisx files for different stuff I've installed might be).

My Facebook friends will know that I'm much more active there suddenly. Why? The Facebook app on the Touch — I no longer have to start up a computer or open a browser to update my status or reply to comments. I had a bit of this function with Fring's Facebook interface on my phone, but the large screen of the Touch makes things like this much more friendly.

I came very close to getting an iPhone actually — but not to use as a phone. This was after I'd realised that it's just as valuable as an Internet-connected device as an actual phone. The cost of iPhone service is still a bit prohibitive to me though, especially for an occasional-use device.

One of the things that had turned me off was the closed nature of the iTunes ecosystem (iPod, iPhone, Apple TV, iTunes). People sometimes ask me about Skype, and I say that the worst thing about it is that it Just Works. I mean, it's a closed system with no interconnections other than those provided by Skype themselves — by this nature it should fail, and yet because it works (arguably) better than any other desktop VoIP product it enjoys immense success. Same goes for Apple's stuff: the iTunes ecosystem Works And Works Bloody Well.

I've been thinking for ages about sync for calendar and contacts and stuff; I've been hunting for services and software and tools for ages. I could build something myself, and indeed started to (I've looked at Google Apps, used Chandler, checked out Ovi, and played with Sync4J before it was called Funambol). I could spend time and effort coming up with something myself…

Or I could just buy an iPod.

The problem is that Synergy and the VMware console don’t play well together (I could have sworn that when I first started using Synergy I had no trouble with it, but there are a few hits around that describe problems like I’ve now hit). The problems people are reporting are that keys like Shift and Ctrl are not passed to the VM (some described here and here).

My problem is slightly different: the screen of my Synergy client (the one that’s running VMware) locked while a VMware guest had focus. Now, the Shift and Ctrl keys are not picked up by gnome-screensaver to unlock the screen. Even the real keyboard attached directly via USB doesn’t work. Big problem, for the following reasons:

* Thanks to password strength rules enforced on the Linux build I use, my password now has a Shift-obtained punctuation character.
* I can’t switch to a virtual console, since that requires Ctrl (e.g. Ctrl-Alt-F1).

Okay, so the keyboard doesn’t work. This client machine just happens to be a tablet PC, and I had hacked gnome-screensaver (to display the onscreen keyboard to allow the screen to be unlocked in tablet mode). I grabbed the pen and tapped out my password, but it *still* didn’t work: even the output of the virtual keyboard gets the Shift modifier dropped. Hmm… Starting to fume now.

Never mind, I’ll connect via the network…

* Fedora does not start SSH by default (okay, yes, and I didn’t make sure it gets started after I’d finished the install).
* There is no remote desktop (VNC server, XDMCP) configured.
* The shiny web-based management interface on VMware Server 2.0 only listens on 127.0.0.1 (or is being blocked by the Fedora firewall).

So with no way to get access to the machine to try and fix it, a power-off is the only solution. Some readers are probably thinking “boo-hoo, diddums had to kill-switch his widdle poota, how tewwible,” but I hate having to do that; not because the system doesn’t recover, but it’s “problem resolution, Windows-style”.

Even though the real problem was between Synergy and VMware, I’m blaming the (perceived) need for security since without that I wouldn’t have a cryptic password that I can’t enter without Shift and a system I can’t administer over the network. Red Hat and Fedora doing everything in their power to ensure I don’t fall prey to nasty Internet fiends (rich analogies to governmental nannying, but that’s probably over-thinking things).

So in summary: Synergy is great, just as long as you’re not using VMware console and have a password with punctuation or uppercase… Remember to have your SSH or other network access enabled before you play!

The post I made here last night was going to be a comment about how ironic it was that we didn’t want to know our baby’s gender and yet the time of the birth was known. Well as fortune would have it, I would have been wrong on both points!

We were due to arrive at hospital at 1:30pm today for a 2pm induction, but our baby had different plans! S went into labour spontaneously at about 1:30am this morning, so we had the dash to the hospital that we never thought we’d have. By 2:30am we were in the birthing suite, and just over one hour after that our baby girl K arrived!

As for the gender thing, although we were obviously going to be happy to have a healthy baby of either gender we’d both been hoping for a girl. This time, something was telling me that it was in fact a girl–I guess you’d say I was very confident. So confident in fact, that S was quite angry at me about a week ago for not committing to a name for a boy.

N met his baby sister this morning… he has a very proud-looking smile on his face whenever he looks at her! He’s a wakeup to our grownup tricks though–we’d bought him a present to take home with him “from the baby”. When given the present, he reportedly (and in his best “hang-on-a-minute-you-can’t-trick-me” voice) said “that’s not from the baby, babies can’t go shopping!”

PS: What’s Clinker? That’s the nickname that S’s work friends gave to her baby-bump!

Every so often I found that some random junk would show up in comments to my blog posts. When I saw it I’d just delete it, and it didn’t occur often so I didn’t really think much of it.

This was until I spied a comment that I actually needed to reply to, and found I couldn’t. I started looking at why the record number of the comment was so high, and found that my blog of little-more than 100 entries had become home to over 13000 items of blog-spam.

I blame myself, obviously, as the software I use had introduced spam-filtering techniques a couple of versions ago and I hadn’t kept up.

In cleaning up the garbage, behind the red mist of rage I saw at having my blog being violated so, I noted something interesting. The issue had been going on for some time, and I realised that in front of me, in my humble little blog, I had a snapshot of the evolution of blog-spam.

The early stuff was primitive, and easily identified by querying for the names of erectile dysfunction drugs and other medications. The later stuff was harder and harder to detect until I was virtually picking it record-by-record out of the database. Some of it made absolutely no point to me at all: strings of random alphabetics with not even a URL in sight; maybe this was a worm just looking for the kudos of a DOS.

The thought occurred to me that perhaps I should have kept it, in much the same way as someone I know keeps copies of PC viruses and worms in a little (hopefully isolated) folder. Then I realised two things:

* Preserving something, or putting it in a museum, gives it some legitimacy. I don’t want to legitimise blog-spam; and

* The art (if any) in blog-spam is in the code that generates it, not in the crap it leaves behind.

As for all the hits on my ClustrMap, I figure 80% are the spambots infecting the blog and about 19% are the poor folk that got drawn to my site as a result of the spam. I had been thinking of a different blog platform, perhaps this episode shows that I need something a little harder.

Of course another way to fight blog-spam is to get your network disconnected from the ‘Net, and my ever-so-unfriendly ISP went out of their way to do that for me this weekend. Unsolicited, of course, which is even better. On a Friday afternoon, too — better still, as if you do actually manage to get someone on the phone it’s too late for them to find anyone who can do anything about it (apparently).

Recommendations of a good ADSL ISP accepted: although keep it to yourself if your ISP’s called wwkjukhkkjlpuggh or qjkdfsdfaksjkulkfhg…

The first was an update to the spine resource poller, part of the Cacti project but installed separately (it used to be called cactid). Turns out that somewhere between 0.8.7a and 0.8.7b, bugs were introduced that made spine unreliable on 64-bit systems. The update brought in a SVN version of spine which, while still labelled 0.8.7a, must have been somewhere after one or more of the bugs came in. The symptom was that every data value obtained via SNMP was garbage and ignored.

The second issue was strange — graphs were getting generated (even those for which there was no data) but there was no text on them! Titles, margins, legend, axes, all were blank. Some posts pointed to a problem accessing the TTF font file provided with rrdtool, but the actual problem turned out to be the upgrade to rrdtool 1.2.28 which introduced different parameters for the configuration of text attributes in graphs — and a corresponding “feature” that suppressed any text output if the new parameters were missing.

So what does “~” have to do with this? The software on your system is built according to the architecture of your machine. In Gentoo, this is called your “arch” (for architecture) and is usually “x86″ or “amd64″. Gentoo implements a “testing branch” in an arch which starts with “~”; if a pre-release version of a package exists in portage you can bring it in with the “~x86″ keyword. The nice thing about this is that you don’t have to enable a testing repository across your whole system — you can enable the ~ keyword for specific packages on your system, and everything else stays stable.

Unfortunately, this flexibility has a cost. The “amd64″ arch seems to lag a bit behind “x86″ in terms of packages being marked stable or just simply having packages available. This means that just to get things installed, it’s necessary to flag packages with “x86″, “~amd64″ or even “~x86″. This flagging is easily done — almost too easy in fact, as it creates a problem later on when the package you actually set the keyword for eventually becomes stable and you don’t need the keyword set any more. It’s a manual process to revisit the keywords you’ve set and verify that they are still needed (and you know how well manual processes work).

Some time ago I started adding comments to the Portage config file where keywords are set, trying to explain why I set the flag: “to bring in version 1.2.34″ for example. That way, if I ever do get around to manually auditing the package.keywords file, I’ll be able to check if some of the keywords are still needed. Still a manual review though.

So in the case of rrdtool and spine, I had set the “~” keyword some time in the past for some reason, possibly to get early access to a bug-fix ebuild. With no established method to revisit the keywords, I continued to pull in unstable versions of packages long after the packages I really needed had been marked stable. Eventually, it bit me.

The pre- and post-upgrade chacklist grows some more…  :)

I’m trying to get over the silly habit I’ve developed of bringing home stacks of coins from overseas, and it looks like I’ve had a bit of success this time. Somehow I’ve managed to come home with almost no coins! I brought a stack of coins I had collected on previous trips, and not only have I got rid of all them I’ve collected hardly any more.

I indulged my gadget addiction to the tune of an Archos 605 Wi-Fi media player. Yes, I know that there is a new series of devices released by Archos, but they are not generally available and may not be for a while (in this geography at least). Besides, the 605 has what I need (especially since the supplier over here includes the key plugins that I need) and is available now. The store recently reduced the price too — admittedly, probably to clear stock in advance of the new models coming in a few months.

Here’s hoping I get some sleep on the plane, as I want to have a good day with N and S before I have to do the Canberra thing all over again.

The weather here is, obviously, hot and humid. It’s been far from unbearable though, in fact I’d almost say it’s comfortable (which is quite something from someone who usually can’t stand hot weather). I’ve rediscovered the transport system, the excellent MRT train system with its regular services and its cheap fares, and I’m using it to go to and from work.

I’ll make some further notes as the week goes on. Wish me luck with the training!

Some months ago I was in an IRC channel with a group of folks in the team I was working with at the time. The conversation had come around to green electricity, what deals our respective electricity companies were offering, and whether we were “doing the right thing” and selecting green energy.

I was a nay-sayer. “It’s a scam,” I railed. “Why should I pay extra for green power when the electricity companies know they should be doing that anyway?” The conversation turned to subsidies for installing solar power systems, and soon after that we actually got back to work.

Months later I recalled that conversation while listening to a podcast. The presenter was discussing climate change and the need for urgent action, whatever the cost. Which is when it hit me: green energy and it’s friends are like an early-adopter tax for a sustainable future.

In the early 90s, I remember models of the IBM ThinkPad would cost A$12k and more. Twelve THOUSAND dollars! Over time however, the developments in the technology have led to such remarkable improvements that a modern laptop can be had for a fraction of that amount, and projects like OLPC becoming viable. None of it wold have happened, however, if early-adopters had not backed the IBMs, Compaqs, and Toshibas (and the Osbornes before that, bless them) and supported the idea.

In 1978, when Mercedes-Benz first fitted ABS to the S-Class[1], I expect they would have wanted to make it at least an option on all their vehicles. That they didn’t, when the cost of doing so would have been astronomical, ensured that they were able to viably continue research and development on the technology and bring the cost down over time. Together with other car makers who progressively did the same, they ensured that even a modern $10k car can have access to such technology, but again it wouldn’t have happened if not for those S-Class buyers validating the idea and stumping-up the cash.

I’ve realised that businesses don’t have a conscience, and that the current economic model cannot reward a company for “knowing what it should be doing”. In quite a real way, companies need their customers to be their conscience by supporting those products that make a contribution to society, and rejecting products that are damaging or harmful. Longer-term, those companies that “get it” will thrive while those that don’t will fail.

So my consideration on things like green electricity changed to, simply, “can we afford to?”. Knowing that in around three months I’ll be meeting my second child (all going well), and becoming maudlin about the state of the world that a new person is being brought into (as new parents sometimes are wont to do), perhaps the question should be “can we afford NOT to?”…

[1] Other manufacturers fitted ABS systems to cars earlier than 1978, but they seemed to be one-off decisions that were inconsistently implemented or met with commercial failure. Mercedes-Benz, once the decision was made, stuck with it.

I’ve had this phone for just on a week now, and it’s certainly one of the best phone purchases I’ve ever made. In a nutshell, the key things about it are:

* QWERTY keyboard, in a form factor not much larger than the N70. Importantly, it’s much smaller than the E61 that preceded it (now there’s a phone that was just MADE of ugly). Despite it’s size the keyboard is amazingly easy to type on, although I may have to update this after I give my thumbnails a trim.

* Symbian OS. Maybe I’m biased, as the owner of a Psion 5, but to me Symbian has an edge over other phone OSes. Not only with the functions in the handset and Nokia’s Series 60 interface, but the range of third-party apps for Symbian (or Series 60 specifically) is great. Almost straight after charging the battery I downloaded PuTTY (SSH client) and “vejotp” (S/Key one-time-password utility). Plus, the recent news that Nokia intends to open-source Symbian is a great thing.

* Nokia Maps and A-GPS. While the iPhone glitterati download the entire UBD or Melways every time they walk down the street thanks to Google Maps, I get quick GPS mapping for zero download (the last few times I’ve used it, the download counter has stayed stuck on “0.0kb” even though A-GPS is supposed to cost a bit of data every startup). It’s not the most accurate GPS ever made, for sure, but it’ll do me for now at least.

* Built-in podcast support. I was getting more and more frustrated with the way that Amarok and iPod fought with each other over my podcasts. It never seemed to work as well as it did on iTunes! Now, I can use the device I download the podcasts with to listen to them as well. It’s self-contained, tidy (no more podcasts mixed in with the music library and causing havoc), elegant.

* Wi-fi capability and SIP client. Being able to connect to the home network obviously means that I can do things like update my podcasts without having to second-mortgage the house to pay for HSDPA data. The SIP client is very cool too: I’ve connected it to my Asterisk box, and now have a cordless home phone and mobile in one device.

* Solid construction. It’s got to be the most sturdy-feeling phone I’ve ever owned. The case is metal, and it has a nice weight to it. The buttons feel solid, almost like real keyboard keys.

* Drop-dead gorgeous. I got the grey version, the metal casing looks like titanium and has a glossy finish (which is a little prone to fingerprints, but cleans easily). The screen is just amazing, usable in daylight, bright and colourful and incredibly high resolution.

I’ll mention more as time goes on, but for now I am very happy!

I’ve been doing the three-day-per-week thing now for nearly two months and it’s fair to say that I’m over it. It’s not the weather so much: okay, it’s cold, but at the moment I’m breezing in and out so I don’t have to live with it. It’s not the loneliness; I’ve been working at home (and away) for long enough that solitude doesn’t bother me at all. I have a bit of an idea of things it might be, though…

* The inconsistency of accommodation: one week I’ll be staying five-plus stars, the next I’ll be in a little fleabag place with barely adequate heating and hundred-year-old floors (which is quite a feat when the city itself has only been there eighty years[1]). One week I’ll have a kitchen and be making myself healthy meals, the next I’ll be eating lukewarm takeaway noodles using toothbrush handles as chopsticks.

* Trying to do the right thing by work by saving money: this has me riding my motorbike to the airport at 5am to save a night’s accommodation and a couple of taxi fares. I’m DEFINITELY over that.

* Separation from N: Although he’s coping really well now (much better than me, I think), it worries me that I’ll be one of those absent fathers that ends up having only a passing influence on the lives of their children. A bit dramatic, sure, as this is not meant to last for long, but how long is too long?  How long is long enough to do damage? It seems like I do little else but spend time with N when I get back, which is probably making things worse.

* Separation from S: I think we’ve changed. We used to do the living-apart thing really well (when I was working in Melbourne and Auckland, we really seemed to be living proof that “absence makes the heart grow fonder”), but now things are different. She is looking after N, and is in full-time work, so she has a lot more on her plate than before, and I barely get wound-down from one trip than I’m having to gear up for the next.

* Not being able to “connect” with the place: When I have worked away from home in the past, I went to special effort to do things that would familiarise me with the locality. In Melbourne I went on tram rides “just because”, and in Auckland I’d ride my motorbike here and there. Day trips, intended to not only help me know my way around but to give me a feel for where my career had sent me. With Canberra, however, these work trips have never been across weekends and I’ve not had any opportunity to do my “connecting”. If not for a colleague of mine who was living in Canberra about eight years ago, and whom I visited one weekend while I was working in Melbourne, I would never have visited places like the Australian War Memorial, Parliament House, Mt Stromlo Observatory, and the comms tower on Black Mountain. After all the visits I’ve made I’ve never seen the National Gallery, The Royal Australian Mint, and the dozens of other things worth visiting.

However, probably the number-one thing that’s getting to me:

* Having little (nothing) to do when I get there: I’m a technical person, and I feel most useful doing technical things. From that perspective, there is no need for me to be in Canberra to do what I’m doing. My presence there at the moment is merely a perception thing, building trust and a relationship; intangible things that probably are tremendously significant to the client and their perceptions, but don’t register with me at all.

I mentioned how well N is coping with me being away: he’s doing so well that I’m in awe. It seems like he’s keeping me sane. He got upset the first couple of times, but now he almost packs my bag for me! I took him to preschool the other day and this is the conversation that took place:

I said “now, Mummy’s going to come and get you this afternoon, mate.”

“Why?” said N.

Oh great, I thought, I almost made it without mentioning anything. “Well I’m going on the plane this afternoon,” I said.

“Are you going to Sydney, or Canberra?” he asked.  I was taken-aback: no tears, no don’t-go-daddy-I-need-you…

“Canberra,” I told him, “back to Canberra.”

“Oh, okay Daddy,” said N, “have a good trip. Bye!”, and off he ran to play with his friends.

Four. Years. Old.

So on Canberra… I think I’ve worked out which are the roads not to be on when I need to get from place to place (not having to spend much time in, near, or passing through Civic helps in that regard). I used to take a drive around after work at least once a trip (the closest I’ve been able to come to my “connecting”), but I’ve stopped doing that now as I think I’ve been to most of the districts. I have a bit of an idea what are the good food places, and which restaurants one can go to (as a male person dining alone) without ending up feeling like a total outcast.

Next update I might talk a bit more about the places I work and stay. Don’t expect anything too soon, though, as I have about 18 more months in which to do it…

[1] Okay, my little joke. The floors obviously aren’t a hundred years old. They do, however, sound like they belong on the sound-stage of a cinematic re-enactment of the voyage of the First Fleet.

At the moment it’s still running Windows XP Tablet Edition, but that’s only because a Hardy Heron install DVD is quite some way away from me right now.

I just trained Microsoft speech recognition and this paragraph has been dictated using speech recognition.  The recognition rate is actually quite good for the amount of training I’ve done, although I am having to go back and correct quite a few minor errors.

Back to handwriting recognition, which is outstanding: it’s only just now, as I write on the screen of this device, that I realise how poor my handwriting has become! Almost certainly this is due to lack of practice and under-use! I can tell as I’m writing that its recognition is somewhat dictionary-based, as if I pause or lift the pen it will make a guess abort what I’ve written which it will change as I complete the word.

Credit where credit is due, I think tablet Windows looks pretty good. Having used it before just putting Ubuntu on, I’ll be expecting a bit more from Linux…

On the hardware side, the machine looks quite sturdy and solid — so much so it’s a bit or the weighty side. I did install the additional battery pack though, and I’m sure that if I was adopting a proper tablet-PC posture it wouldn’t feel as heavy as it does. It has a 3G modem built-in, which works fine with my Telstra USIM (that’s how I’m posting this now), and the usual complement of Wi-Fi, Bluetooth, PC-Card, SD  and infra-red. It even has a place to stow the pen-stylus.

As I said, I don’t have a Linux disc handy to start setting up properly so the big Linux question will have to wait for a while. Some early Googling shows that there is support for most of the hardware components in it (the digitiser is Wacom, the Wi-Fi is Intel 3945, I’ve even seen support for the 3G adapter. It’s the interface and application level that I’m worried about — the state of handwriting recognition, whether XRandR lives up to the promise, and so on. I’ll post more as I get used to it and see what it can do…

In the past when I’ve done this kind of recovery I copied the details of the partition table and copied each partition in turn off the failed drive. This time however in my haste (or something) I imaged the entire 80GB in a single file by using dd pointing at the disk device node (/dev/sda, say) instead of each partition.

When it came to restore, this brought me undone. I tried the reverse of what I’d done to create the image, just dd to the disk device node, but at around the point where the first partition would have ended it failed with an I/O error.

Assuming that I did actually have a valid image of the entire drive, I needed to find a utility that would treat a disk file as an image of an entire drive, partition table and all. Then, theoretically, I could copy each partition out of the source file in the same way I was used to.

As usual, the lazyweb came through for me. I found a beaut utility called kpartx, apparently part of multipath-tools. I also found some blog posts describing how to use the tool for what I needed. What kpartx does by itself is scan a block device or normal file (significant, as I’m sure when I first used it I had to use losetup) and create device-mapper mappings to the partitions it finds. kpartx was just what I needed!

This blog post describes the process I went through (and no losetup here either). For me, I let kpartx create device-mapper nodes for the partitions in the disk image, and then used dd to copy from each of those partitions to the real partitions in the new disk (carefully created using fdisk to match the originals). Then I set up grub on the new disk, and installed it into the laptop. It worked first go!

So now on its third hard drive, this Sony Vaio (same one with the weird wireless issue in Hardy) gets yet another lease on life. The battery is almost expired though, so I think it won’t be long before it goes to the Old Hardware Shelf…

Edit: I realised my confusion about losetup. kpartx uses losetup under the covers if it detects that you’re trying to work on a disk file instead of a block device. When you use the -a or -l switches to kpartx, if it needs to it straps up a loop device for you and automatically works on that; running kpartx with the -d removes not only the device-mapper nodes but the loopback node as well.

I’ve got a system here that is a host for a virtualisation environment I run. I dedicated a couple of network cables to the adapters owned by the virtualised system, and a third one was attached to the host’s IP stack. To get connectivity for another system, I had to steal the host’s cable though–which wasn’t a problem as the operation of the system works more-or-less entirely from the console rather than over the network. Just for grins, however, I decided to set up connectivity to the host by routing through the virtualised environment it hosts.

Having established the tunnel connection between the virtualiser and the host stack, I set about configuring the special details required to support routing through this system. After a few tries at getting it right, I was rewarded with successful pings between the systems on my LAN and the hosts system on its routed connection. So I jumped onto the console of the machine and light up Firefox, but got an error page. I realised I hadn’t set DNS resolution–on the LAN, the machine was having resolv.conf configured by DHCP, so now I had to do it manually.

Okay, so DNS resolver now correctly set, let’s see Firefox WIN! Oh. Fail.

When I hit Try Again or Reload, the page would instantly refresh. This was starting to look like no routing problem. I used dig to test name resolution, and it told me it was being rejected. I looked at my dns.conf… Nope, so subnet restrictions coded there…

So I hit the lazyweb, and it didn’t take too long before I found a forum post that led me to this. In BIND 9.4.1-P1, ISC basically changed the default behaviour of a couple of query filtering settings. This had the effect of rejecting some requests that were previously accepted, such as those from non-local subnets. A reconfiguration of my DNS server gave me success at last.

Hooray for persistence! Now, someone hand me some Cat-5 so I can make a cable and plug this thing back in properly.

The onboard wireless on this thing is an ipw2100, hence only 802.11b, and I had a PCMCIA 802.11g NIC lying around (actually it came from the Lifebook, liberated from there after I bought it a Mini-PCI 802.11g card on eBay). On Gutsy, I used the hardware kill-switch to disable the onboard adapter to make double-sure that it wouldn’t try and drag the network down to 11Mbps.

This laptop was the last machine I upgraded to Hardy, and I was playing with KDE 4 on it so I was looking forward to seeing what KDE4-ness made it into Hardy. While the upgrade was taking place the wi-fi connection dropped out, but I didn’t think anything of it since Ubuntu upgrades try and restart the new versions of things and I figured NetworkManager had fallen and couldn’t get up. After the reboot, however, KNetworkManager (still the KDE3 version, don’t get me started there) could find no networks — could find no adapters, in fact.

I logged back into KDE3 and poked. Still no wireless (as if the desktop would make a difference, but I had to make *some* start on pruning the fault tree). The Hardware Drivers Manager was reporting that the Atheros driver was active (for the PCMCIA card), and an unplug-plug cycle generated all kinds of good kernel messages.

On a whim, I flicked the hardware kill-switch for the onboard wifi[1]. Almost instantly, KNetworkManager prompted to get my wallet unlocked — it had found my network and wanted the WPA passphrase. I provided it, and got a connection: via the PCMCIA NIC.

“That’s odd”, I thought, and flicked the switch. A few seconds passed, and the link dropped. Flicked the switch on, link came back. Flicked the switch off again: this time a few minutes went past, but again the link failed. Tried it several times again, and the same thing happened. The state of the kill-switch for the onboard NIC was influencing the other NIC too!

It seems that this is altered behaviour in NetworkManager, applying the state of the hardware switch to all wi-fi adapters. If it annoys me significantly I’d like to think I’ll trawl changelogs, or even better lodge something on Launchpad… more likely though I’ll forget all about it having found a kludgy workaround.

I’ve now added ipw2100 to the module blacklist and things work okay (presumably because the state of the onboard switch can’t be reported any more). I’ll also have a think about whether a few dollars for another g-capable Mini-PCI NIC will be throwing good money after bad, as this laptop really is quite long-in-the-tooth.

Oh yes, that’s right… KDE 4. Next time perhaps.

[1] I can’t think why I did this. I knew that I’d disabled 802.11b in my access point, to make triple-sure an 802.11b device wouldn’t slow my network down… The onboard 802.11b NIC would never successfully get a connection.

Odd whales

We're seeing a number of whales pop up around the site, especially on profile pages. We're aware of the issue and working on it now.

Update: site back up and mostly whale free.

Bizarre — but maybe if I was a more regular user it would make more sense.

Unfortunately my visibility of their problems started when they shut off their IM functionality — unfortunate because at around the same time I had done an upgrade of my Jabber server and blamed the upgrade for not being able to connect to Twitter. Hours of fruitless config checking and Googling finally led me to the aforementioned Twitter Status Blog and the news that IM had been disabled. Ever since then (two or three weeks ago) IM function has been the “number one priority to get restored”…

I’m not complaining. It’s a service I barely use, and one I’ve never paid for. I can appreciate people getting upset however; as someone who does have trouble keeping in touch with friends and family, if I had gotten more used to it I’m sure I’d be one of those complaining.

Doom-and-gloom time… Part of the flak Twitter has been copping is around the design of their systems (virtually no redundancy or scalability) — if this is systemic to Web 2.0 startups and the investors and venture-capitalists catch on, things could get ugly FAST. Are we set for a repeat of Dot-Bomb? Could Twitter be the first victim of an impending burst of the Web 2.0 bubble?

I hope it comes back. Twitter is a nice way to see what folks are doing, and it’s a reminder of the fact that there are people out there. Perhaps I’ll even make a bit more use of it, if it can get its IM groove back on. Good luck folks, and watch out for the whales…

Edit: Technically I have flown economy to the United States: when Susan and I went to Hawaii we were on economy. I should have said that I haven’t flown economy to the mainland US before. Or maybe I shouldn’t have said anything… Anyway, it was far-and-away the longest economy class flight I’ve ever done and I must say it wasn’t anywhere near as bad as I was imagining it.

The fact that it hasn’t taken centre-stage is possibly as much to do with VMware’s bogus clock-drift problems as anything, as I haven’t dedicated hardware to my Zeroshell instance yet (I could keep it running virtual, but some of the things I want to do with it will make more sense if it’s a separate machine). VMware Server takes another barb for its handling of VLAN tagging (but to be fair that might be the Linux 8021q module works). It seems that if you have any VLAN definitions on a network card, VMware won’t get to see any VLAN tags on that NIC. You can get a guest attached to a bridged interface to see the real VLAN tags, but only if Linux has not got any VLAN awareness over that NIC.

Alright, so enough ragging on VMware. I have Zeroshell attached to the networks it needs and all is fine. Except that I can’t actually change anything! The web interface that I spoke so highly of originally is actually very restricted in some areas. One of these is in the RADIUS server, and it bit me badly when I decided I’d use Zeroshell’s RADIUS server to authenticate access to the Web interface of my Linksys switch. Turns out that the Linksys firmware expects a particular attribute to appear in the response from the RADIUS server.

The fact that Linksys don’t document this anywhere is not Zeroshell’s fault, but that there is no interface allowing me to do updates to the records above what Zeroshell uses for its own applications is a bit of an issue. It means that instead of a Zeroshell box potentially becoming the hub of administration functions, it is in danger of becoming just another little vertical application server that doesn’t integrate.

Having said that, the backend for most (all?) authentication data is LDAP so a tool like PHPLDAPAdmin might be usable to extend the base records. But, arguably, I shouldn’t have to do that! It is still beta software though, so improvements and enhancements will be made.

The other area that it’s a bit lacking in is monitoring/graphing. Okay sure, I’d probably integrate Zeroshell into the rest of my Cacti setup, but it would be nice if Zeroshell did like other router distos and had a pre-built statistics/graphing page.

Zeroshell is still my pick (I revisited pfSense and fixed the problem updating, but to me it doesn’t have enough function to justify running its own hardware), but it’s just not quite the bees-knees it was when I first saw it.

I’ve become fairly complacent about system updates. All the distros I use now have got excellent tools for keeping everything up-to-date, and for making sure that things don’t go wrong in the process. It’s all just software, however, and it’s all too easy for something to get missed or for a bug to creep in. One such bug that did exactly that is this one. Unreported at the time I did my update, it rendered my Slug unbootable after the update I gave it.

It took me a day to realise that the Slug was off the network. The failure of the nightly backups was my first clue. Next was the inability to stream any of the media files stored on it. For the next week, on-and-off, I tried a dozen things in an attempt to get it working again. I finally arrived at a process that used the Debian Installer firmware image as a way to get a running system onto the device, allowing me to then access the hard disk and try and reflash earlier kernel and initrd images to it.

I started trying to work on the boot disk, but I couldn’t see it for some reason. Then I discovered that the power supply of the USB2 disk enclosure that holds it was playing up! Now, I had two problems–was one related to the other? Was my boot problem just a hard disk problem all along? Turns out that the power supply failure was a coincidence–replacing the power supply got the disk working again but made no improvement in the bootup scenario.

The NSLU2 boots differently to a PC. On a PC, the BIOS locates some boot code on a storage device and executes that, which usually is a program like LILO or GRUB that has more intelligence and (in the case of GRUB) a way to interact with it. These boot loader programs then load in the kernel and start executing it. With the NSLU2, however, the kernel and the “initial root device” are written into the flash memory of the device–they more-or-less are the BIOS.

On a PC, if there’s a problem with the kernel or initrd you can generally select another one from a list. Worst-case would have you installing the hard-disk in a different PC and fixing the problem from there. On a NSLU2, however, any problem with the kernel or initrd can’t be fixed by changing the hard disk because the kernel and initrd aren’t read from the hard disk but from the flash memory instead. There’s also no option for selecting another kernel, since the NSLU2 is a “headless” device with no console (besides, there’d be no room in the flash memory for two copies of kernel and initrd).

Once I’d been able to get my Slug booting (by writing out a previous version of a kernel and initrd) I was going to leave it alone… but curiosity got the better of me. I’d suspected a bad update to the utility that generates the initrd, and sure enough an “apt-get update && apt-get upgrade” revealed a pending update to the initramfs-tools package. Google led me then to the above bug report. With fingers crossed I did the update, reflashed, and rebooted… successfully!

The Slug is now back in its usual place, quietly going about its business of entertaining us and keeping critical data safe. I might at least think twice before doing a kernel update on the poor beast in future though!

Sure enough, I put a different disk in the laptop and it worked, and the original disk in a USB caddy failed (but only after working successfully a couple of times, leading me to think it was a transient problem and reassemble the laptop, at which point it failed again… sigh).

Through persistence and determination (and a couple of goes in the freezer) I managed to get a copy of the disk onto another drive. I then went shopping, but decided to check the warranty on the dud drive: lo-and-behold, it still had nearly four years of a five year warranty to run. Better yet, unlike the Western Digital I had to send at my own cost to Singapore for replacement, Seagate have an address in Australia that can be used.

Sod it, I said, anything more than the original 80GB (since for less than what I paid for the 80GB a year ago I’m looking at 160GB or more!) is wasted on this particular machine, so I completed the RMA, found a box to pack the drive in, and sent it off.

The address in Australia is a mail forwarder to Seagate in Singapore. I had to keep that in mind when I checked their order status page, which a week later was still showing “awaiting your return”. Nevertheless, it wasn’t long before the page changed to “shipped”. Looking a bit closer I could see that my 80GB drive must have put on a bit of weight on the way to its birthplace, as Seagate was sending me a 100GB drive in return!

Having left Singapore last Thursday the drive arrived on Monday, but due to work commitments (plus having to fix the Slug first) I wasn’t able to do anything with it until today. Stay tuned for the recovery exercise…

First I found pfSense, a FreeBSD-based distro that seemed to fit the bill–indeed the very first question the Live-CD asked me on bootup was “do you want to use VLANs?”. It also promised a very extensive set of additional packages that extend it’s capability into areas like file/print, WWW proxying, and a host of other features. However, even though it has a very nice web-based configuration facility, due to what looks like a problem on their web site I was unable to even look at what packages are available. Since some of the basic function I would like is provided by these packages, I’ve had to move on–but pfSense gets an honourable mention because of its easy installation and excellent configuration interface.

I looked again at Smoothwall, but soon remembered why I discounted it at the time I chose IPCop. For me, the level of function I think I’d use is a bit too close to the threshold of function in the “community” (read, “free”) version. Astaro would go in this category too, except that I was too dense to be able to even find much clear information about the level of function you get in their community version. So no recommendation on either of these, as I’ve never used either–I do work with a fellow who happily uses Smoothwall though.

Then, I came across Zeroshell. The lead developer describes it as “a small Linux distribution for servers and embedded devices aimed at providing the main network services a LAN requires”. And does it ever! It’s a veritable Alladin’s Cave of features and functions. It certainly does everything I was looking for, from VLAN tagging through QoS to VPNs, from an SPI firewall to multi-zone DNS and multi-subnet DHCP servers, but also has Certificate Management (using a self-signed CA certificate or one you import), a RADIUS server, WiFi access-point capability with multiple SSID and VLAN mapping, captive portal or “normal” HTTP proxying, 802.1d bridging, clients for Dynamic DNS, a Kerberos 5 server, plus a raft of other capabilities. Zeroshell–named because the author wanted to provide a system that was extremely flexible and powerful yet did not require users to access a shell prompt–is remarkably feature rich, and yet the download for the ISO image is only around 100MB (a bit beefier than pfSense, admittedly, which weighed in at around 60MB).

There are a couple of downsides, however. Until very recently, installing to a hard disk was not supported. The distro is designed to boot from a CD only, but can use an installed hard disk (if available) for what it calls “databases”, where configuration and other data is kept. With the latest release, however, the developers have created a “1GB USB drive” download (the size of the download isn’t 1GB), which is designed to be copied to a USB pendrive or hard disk.

The other downside (and it’s not fair to say that, as will become clear) is the web interface. Not because it’s ugly or not functional: it is neither of those. It’s clean and well laid out, and fairly consistent. It’s very technical, however. Where other distros tackle the “SOHO divide” by hiding details such as protocol numbers or port ranges, Zeroshell uncovers all this stuff in its gory detail. So it’s great for someone like me, who looks at the interfaces on other systems and pines for the knobs I can’t fiddle with, but it’s not for newcomers.

It looks to be a fairly new project (current release is 1.0beta9), but the forums look good and there does seem to be a bit of activity around it. I’m running Zeroshell in a VMware guest at the moment while I kick the tyres–the VMware download is also available from the project’s mirrors–but I reckon this one will be a keeper!