I’ve been doing a lot of mucking around with KVM with libvirt (I keep promising an update here, don’t I).  In my desktop virtualisation requirements I had a need for presenting VLAN traffic to guests: simple enough, and I’ve done it before.  You can do what I usually do, and configure all your VLANs against the physical interface then create a bridge for each VLAN you want to present to a guest.  The guest then attaches to the bridge appropriate to the VLAN it wants access to, with no need to configure 8021q.

(The other method of combining VLAN-tagging and bridging is to bridge the physical interface first, then create VLANs on the bridge.  I couldn’t work out how to get VLAN-unaware guests attached to this kind of setup, and it didn’t work for me even to give IP access to the host using a br0.100 for example.  Still, it must work for someone as it’s written about a lot…)

I realised that from particular virtual machines I needed to get access to the VLAN tags — I needed VLAN-awareness.  Now I knew up-front that the way I could do this was to just throw another NIC into the machine and either dedicate it to the virtual guest or set up a bridge with VLAN tags intact.  I really wanted to exhaust all possible avenues to solve the problem without throwing hardware around (as I’ve been doing a bit of that recently, I have to admit).

First, I tried to use standard Linux bridges as a solution, but discovered that an interface can’t belong to more than one bridge at a time, which put paid to my plan to have one or more VLAN-untagging bridges and a VLAN-tagged bridge.  I figured it could be done with bridges, but I envisaged a stacked mess of bridge-to-tap-to-bridge-to-tap-to-guest connections and decided that wasn’t the way to go.

Next I checked out VDE, which I had first seen a couple of years ago — but something gave me the impression that VDE either wasn’t really going to give me anything more than bridging would, or was not flexible enough to do what I needed.  I like the distributed aspect of VDE (the D in the name) but I’d rarely use that capability so it wasn’t a big drawcard.  I widened my search, and found two interesting projects — one that eventually became my solution, and another that I think is quite incredible in its scope and capability.

First, the amazing one: ns-3, “a great network simulator for research and education”.  As the name suggests, it simulates networks.  It is completely programmable (in fact your network “scripts” are actually C++ code using the product’s libraries and functions) and can be used to accurately model the behaviour of a real network when faced with network traffic.  The project states that ns-3 models of real networks have produced libpcap traces that are almost indistinguishable from the traces of the real networks being modelled…  I’ll take their word for that, but when you get to configure the propogation delay between nodes in your simulated network it seems to me it’s pretty thorough.  Although the way that I found ns-3 was via a forum posting from someone who claimed to have used it to solve a similar situation as me, and ns-3 does provide a way to “bridge” between the simulated network and real networks, the simulation aspect of ns-3 seems to be more complexity than I’m looking for in this instance.  It does look like a fascinating tool however, and one I’ll definitely be keeping at least half-an-eye on.

To my eventual solution, then: Open vSwitch.  Designed with exactly my scenario in mind–network connection for virtualisation–it has at least two functions that make it ideal for me:

• a Linux-bridging compatibility mode, allowing the brctl command to still function
• IEEE 802.1Q VLAN support (innovatively at that)

The Open vSwitch capability can be built as a kernel module (there’s a second module that supports the brctl compatibility mode), or very recent versions have the ability to be run in user-space (with a corresponding performance drop).

On the surface, configuring an OvS bridge does seem to result in something that looks exactly like a brctl bridge (especially if you use brctl and the OvS bridging compatibility feature to configure it), but its native support for VLANs really brings it into its own for me.  In summary, for each “real” bridge you configure in OvS, you can configure a “fake” bridge that passes through packets for a single VLAN from the real bridge (the “parent” bridge).  This is exactly what I needed!

For the guest interfaces that needed full VLAN-awareness, I simply provided the name of my OvS bridge as the name of the bridge for libvirt to connect the guest to–OvS bridge-compatibility mode took care of the brctl commands issued in the background by libvirt.  The VLAN-unaware guest interfaces presented a bit of a challenge–the OvS “fake” bridge does not present itself like a Linux bridge, so it doesn’t work with libvirt’s bridge interface support.  This ended up being moderately easy to overcome as well, thanks to libvirt’s ability to set up an interface configured by an arbitrary script–I hacked the supplied /etc/qemu-ifup script and made a version that adds the tap interface created by libvirt to the OvS fake bridge.

The only thing I might want from this now is an ability for an OvS bridge to have visibility over a subset of the VLANs presented on the physical NIC.  The OvS website talks about extensive filtering capability though, so I’ve little doubt that the capability is there and I’m just yet to find it.  From a functionality aspect, OvS is packed to the gills with support for various open management protocols, including something called OpenFlow that I’d never heard of before (but I hope that some certain folks in upstate New York have!) but is apparently an open standard that enables secure centralised management of switches.

Detail of exactly how I pulled this all together will come in a page on this site; I’ll make a bunch of pages that describe all the mucky details of my KVM adventures and update this post with a link, so stay tuned!

Mark isgetting an internal server error on http://tinyurl.com/n5fgd9.  I think I amgetting an internal server error too.

Unless, of course, I don’t get it.

(The first result got fixed, this version (courtesy Google’s cache) has the goodness.)

I’ve been extremely slack at blogging recently. It’s ironic that what has been keeping me from blogging has probably been the very thing that I could be blogging plenty about: the constant travel to Canberra. I’m planning on writing a bit more about my experiences of and in our nation’s capital, so for now here is the first edition.

I’ve been doing the three-day-per-week thing now for nearly two months and it’s fair to say that I’m over it. It’s not the weather so much: okay, it’s cold, but at the moment I’m breezing in and out so I don’t have to live with it. It’s not the loneliness; I’ve been working at home (and away) for long enough that solitude doesn’t bother me at all. I have a bit of an idea of things it might be, though…

* The inconsistency of accommodation: one week I’ll be staying five-plus stars, the next I’ll be in a little fleabag place with barely adequate heating and hundred-year-old floors (which is quite a feat when the city itself has only been there eighty years[1]). One week I’ll have a kitchen and be making myself healthy meals, the next I’ll be eating lukewarm takeaway noodles using toothbrush handles as chopsticks.

* Trying to do the right thing by work by saving money: this has me riding my motorbike to the airport at 5am to save a night’s accommodation and a couple of taxi fares. I’m DEFINITELY over that.

* Separation from N: Although he’s coping really well now (much better than me, I think), it worries me that I’ll be one of those absent fathers that ends up having only a passing influence on the lives of their children. A bit dramatic, sure, as this is not meant to last for long, but how long is too long?  How long is long enough to do damage? It seems like I do little else but spend time with N when I get back, which is probably making things worse.

* Separation from S: I think we’ve changed. We used to do the living-apart thing really well (when I was working in Melbourne and Auckland, we really seemed to be living proof that “absence makes the heart grow fonder”), but now things are different. She is looking after N, and is in full-time work, so she has a lot more on her plate than before, and I barely get wound-down from one trip than I’m having to gear up for the next.

* Not being able to “connect” with the place: When I have worked away from home in the past, I went to special effort to do things that would familiarise me with the locality. In Melbourne I went on tram rides “just because”, and in Auckland I’d ride my motorbike here and there. Day trips, intended to not only help me know my way around but to give me a feel for where my career had sent me. With Canberra, however, these work trips have never been across weekends and I’ve not had any opportunity to do my “connecting”. If not for a colleague of mine who was living in Canberra about eight years ago, and whom I visited one weekend while I was working in Melbourne, I would never have visited places like the Australian War Memorial, Parliament House, Mt Stromlo Observatory, and the comms tower on Black Mountain. After all the visits I’ve made I’ve never seen the National Gallery, The Royal Australian Mint, and the dozens of other things worth visiting.

However, probably the number-one thing that’s getting to me:

* Having little (nothing) to do when I get there: I’m a technical person, and I feel most useful doing technical things. From that perspective, there is no need for me to be in Canberra to do what I’m doing. My presence there at the moment is merely a perception thing, building trust and a relationship; intangible things that probably are tremendously significant to the client and their perceptions, but don’t register with me at all.

I mentioned how well N is coping with me being away: he’s doing so well that I’m in awe. It seems like he’s keeping me sane. He got upset the first couple of times, but now he almost packs my bag for me! I took him to preschool the other day and this is the conversation that took place:

I said “now, Mummy’s going to come and get you this afternoon, mate.”

“Why?” said N.

Oh great, I thought, I almost made it without mentioning anything. “Well I’m going on the plane this afternoon,” I said.

“Are you going to Sydney, or Canberra?” he asked.  I was taken-aback: no tears, no don’t-go-daddy-I-need-you…

“Canberra,” I told him, “back to Canberra.”

“Oh, okay Daddy,” said N, “have a good trip. Bye!”, and off he ran to play with his friends.

Four. Years. Old.

So on Canberra… I think I’ve worked out which are the roads not to be on when I need to get from place to place (not having to spend much time in, near, or passing through Civic helps in that regard). I used to take a drive around after work at least once a trip (the closest I’ve been able to come to my “connecting”), but I’ve stopped doing that now as I think I’ve been to most of the districts. I have a bit of an idea what are the good food places, and which restaurants one can go to (as a male person dining alone) without ending up feeling like a total outcast.

Next update I might talk a bit more about the places I work and stay. Don’t expect anything too soon, though, as I have about 18 more months in which to do it…

[1] Okay, my little joke. The floors obviously aren’t a hundred years old. They do, however, sound like they belong on the sound-stage of a cinematic re-enactment of the voyage of the First Fleet.

It doesn’t take much to see just how rubbish I am at this blogging thing. The last few months have been full of interesting happenings, and the blog has been silent. I’m not going to embarrass myself by saying “I’ll do more frequent updates, really I will” because I know it won’t happen. Oh well, as appears in the gospel according to Dirty Harry, chapter Magnum Force, “A man’s got to know his limitations”.   

Okay, I’m sure it’s not really like that. I’m ashamed to admit this one bit me, though: ORDB, which 16 months ago called it quits but has still been online and more-or-less functional, decided last week to false-positive all queries (a mailing list thread on the issue can be seen here. Harsh, but fair–I do stuff-all email through this site but it still has been costing someone bandwidth to bounce my silly queries back to me. Multiply that up by all the part-time-sysadmins like me that don’t pay close-enough attention, follow old wiki articles, or get bad advice, and pretty soon you’re talking about a HEAP of bandwidth wasted to NOT provide a service any more. Shame it had to end like this though.

Just wended my way through another SLES 10 install on s390x. It’s b0rked though, and I’ll probably have to redo it. I had some kind of I/O error during the install which seems to have resulted in a couple of the filesystems being remounted read-only. Not too much trouble you’d think…

Some things aren’t starting because of missing binaries in /usr, frustrating but probably recoverable. The network startup is totally clagged though, and I can’t even begin to work out how what happened… happened.

During bootup, at the time it tries to configure the network interface, I get streams of error messages about problems running the “ip” command. The error text is full of garbage that the init script is trying to parse as text configuration–it looks like a corrupted filesystem or a binary file.

I manually configured the network (not a trivial task in s390x, it must be said), and started to poke around. I got this when I logged in as root:

Last login: Sat Mar 15 12:48:36 2008
/usr/X11R6/bin/xauth: error while loading shared libraries: libXau.so.6: cannot open shared object file: No such file or directory
-bash: read: read error: 0: Is a directory
lxs0za01:~ #

Okay, so I won’t get funky X-based YaST.  No problem, I’ve spent more time in the ncurses-mode YaST anyway…

lxs0za01:~ # yast
warning: the ncurses frontend is installed but does not work
You need to install yast2-ncurses to use the YaST2 text mode interface
lxs0za01:~ #

WHAT!!! What the @#!$ happened there?!?!?

Okay, so I’ve calmed down about that, so I go looking for the problem with the network initialisation…

lxs0za01:~ # cd /etc/sysconfig/network
lxs0za01:/etc/sysconfig/network # ls -go ifcfg*
-rw-r–r– 1   141 2006-06-17 07:30 ifcfg-lo
lrwxrwxrwx 1    16 2008-03-15 02:23 ifcfg-qeth-bus-ccw-0.0.0f00 -> /lib64/ld-2.4.so
-rw-r–r– 1 27470 2006-06-17 07:30 ifcfg.template
lxs0za01:/etc/sysconfig/network #

Priceless. You can’t make this stuff up. I cannot for the life of me work out how this could possibly have happened. I guess I just blame it on a whacked-out filesystem and move along.

Okay, so both of these issues probably have extenuating circumstances unrelated to SLES or YaST… but it’s nice to have a vent now and then. I’ll write up something a bit fairer once I fix this b0rkedness.

“Once upon a time, in a land far from here, there was a clothing factory. One part of this clothing factory made uniforms. The uniforms came in many shapes and sizes: some were sporting team outfits, some were “corporate wardrobe”, others were emergency services uniforms… but they all looked alike. Few of the people working in the factory spent any time actually making uniforms — more time was spent on things like adding or removing badges and insignia and players’ names from the uniforms, or completing reports to state that the holes in customers’ uniforms were appropriately mended (whether or not any such holes actually existed)…

“Sometimes the sleeves would fall off, and the pockets would be on the back instead of the front (or on the front instead of the back), and when this would happen and the customers complained the managers of the factory would make lots of noise and implement  more procedures and promise the customers it wouldn’t happen again. The factory workers would have to complete more reports, spending more time on the reports and still less time making uniforms.

“Many of the factory workers were unhappy, and dreamed of getting back to making uniforms instead of writing reports and sewing badges. Others dreamed of designing uniforms. Still others dreamed of moving to a different part of the factory, where suits or pyjamas or swimming togs or wedding dresses were made without the need for any of the strict rules that the managers said were required when making uniforms…”

Part Two coming soon…

Since I watch Planet KDE it was easy to get caught up in the excitement around the launch of the new version of KDE (the announcement is here). I was unable to resist giving it a try on the laptop! So this post is coming from Konqueror 4.0.0.

I tried an early Beta of the KDE 4.0 Live CD, but it was still using the KDE3 Kicker and was also a bit unstable. I wasn’t sure if it was the fact I was running in a virtual machine that made the graphics a bit flaky or whether it really was beta-quality code making things a bit funny. The KDE team put a lot of effort into bug-swatting in the weeks leading up to 4.0 being tagged, and it’s a lot better now!

This announcement from the Kubuntu folk shows how to get the KDE 4 packages installed on Gutsy. KDE 4 installs in a different path to KDE 3, so you can try out KDE 4 without affecting your existing environment.

I did have a bit of a heart-starter with this though, as apt-get wanted to remove a package called “kdebase-bin-kde3″, which looked risky! It’s okay though, as equivalent binaries are provided by “kdebase-bin-kde4″. In fact, if you follow Kubuntu’s instructions exactly, you should not see the issue: it happened to me because I did a system update after adding the Kubuntu PPA repository but before installing KDE 4. The system update brought a bunch of updated KDE 3 packages out of the PPA, one of which was to replace the standard “kdebase-bin” package with a “kdebase-bin-kde3″.

First impressions are that Oxygen (the new artwork for 4.0) looks great — it’s a very modern look. Some might think it borrows from Vista, but to me it’s got as much of Mac OS X’s appearance as that of Aero. Plasma (the desktop shell) does some interesting things, like turning desktop icons into widgets, but I’m yet to spend enough time with it to experience the other improvements it brings.
The biggest thing I’m looking to trying out is the compositing built into the window manager, KWin. Unfortunately the laptop is a bit old for this to work well (or at all in fact), so I’ll either have to find some magic Xorg setting or get the KDE 4 packages on the desktop machine. I’ve had trouble running Beryl and Compiz thanks to something about the terminal program Yakuake tickling a long-lived bug in X11 (I think part of the reason it’s long-lived is that the X11 folks don’t accept it as a bug but rather a fringe case that Yakuake shouldn’t be exercising, hence a stand-off) so it will be interesting to see if KWin has the same kind of issue.

As for bugs, well there look like plenty. As I’m keying this, Konqueror is chewing 100% CPU and the characters are delayed by a couple of seconds (and of course, now that I observe this, it stops doing it). Still with Konqueror, this is about the third time I’ve tried to post this thanks to Konqueror segfaulting for strange reasons. Also, the Alt-F2 program launcher reports that it was unable to launch whatever you told it to, even though it does so successfully.

There has been plenty written by the KDE folks about the “1.0.0 release of KDE 4″, and they’re copping a fair amount of stick from people who think they’ve done the wrong thing by releasing as 4.0.0. I’m on KDE’s side. Although many KDE folks have used their KDE 4 builds as their daily desktop for months, I haven’t seen anyone who wears a KDE hat recommending that others do so. The term “will eat your children” has been used to describe KDE 4 by folks from the KDE team, so there has never been any pretense that KDE 4.0.0 would be a daily desktop for the majority of users. I’ve never really participated in large-scale software development, but I can see their motivation for releasing what they had as 4.0.0 — I’m proof of it. As long as it was a beta I was not really all that fussed about trying it out; even after there were release candidates I wasn’t all that keen. As soon as you call it a release, however, your early-adopters rush in and kick the tyres and your real testing can start.

By being open about 4.0.0’s status (and I don’t think you can get more open than “will eat your children”), they can make sure that subsequent releases are a lot better than they would be if they dragged on in perpetual beta — the model that Google and the Web 2.0 fraternity seem to insist is better, plodding on for months hiding behind beta status and its implicit “get out of jail free” card.

Instead, KDE has shown the courage to take their code, along with its bugs, and hold it up as something they are proud to give to the world. It’s the foundation not only for future releases of KDE, but possibly the start of new ways that people work with their computers. By working with the community, instead of closeted away from it, I believe the KDE team will succeed.

Okay, so that finished a bit more ra-ra than I planned! Seriously, give KDE 4.0 a try… but if you aren’t happy to suffer a few bugs then by all means wait until 4.0.1 or even 4.1. Oh, and be free.

In part one I mentioned how I was considering using Google Talk as my main chat ID. As it turns out, I talked myself out of it pretty quickly after I delved into using Google Talk to connect to MSN and other services as I do now with my own Jabber server. While there are a lot of links around for using Jabber transports to hook your Google Talk ID to other services, there’s a tiny catch… well, actually, I think it’s a bloody great huge catch personally.

You see, it wasn’t until I read the how-tos that it became clear how it works. The trick is that Google doesn’t run Jabber transports on their own servers, so you therefore need to take advantage of various “open” Jabber servers that do (“open” in this context refers to a server that lets you use its transports without necessarily being a registered user there).

Seeing there didn’t seem to be any restrictions on the servers that could be used, I figured that I could use my own server. Sure enough, after the right incantations to expose the service on the ‘net, I could connect my Google Talk ID through the Jabber-MSN transport on my server to my MSN account. Yay, right? Well, not really — each little test message I sent in either direction incurred three trips over my Internet connection! Yes, three: one to go from my Google Talk client to Google, one back from Google to the transport on my Jabber server, then a third from the transport to MSN. Obviously the same happens in reverse as well (for incoming messages from MSN).

Seeing this as a less than optimum setup, and also being wary of getting listed as a Google Talk-friendly Jabber transport provider, I lopped the transport’s external visibility and went back to using my own JID for transport access. It’s a bit of a shame too; since fring (mentioned briefly in my last post) doesn’t let me connect to an arbitrary Jabber server, to keep connected to everything I’d need two mobile chat programs running.

It’s not like I do that much IM that I need to keep all this running, but it is at least a little bit interesting…

I reactivated an idle Google account the other day. A friend of mine from the Netherlands invited me ages ago but I never really did anything with it until I discovered that a Google Mail account can be used for other Google stuff as well, including Google Talk. I read that Google Talk is based on Jabber and works with any Jabber client, so I flicked over to Kopete and plugged in the details. Sure enough it worked… but then it got interesting.

I run a Jabber server for internal things. I wanted to have a secure, private chat facility to use over VPN with my nephews; I want to someday migrate my Nagios IRC bot to Jabber; and I use transports to link into MSN and Yahoo! to reach friends on those networks. The last point is great: I really like the fact that now, from whatever Jabber client I use (even the mobile ones I’ve played with) that I merely connect to my Jabber server and I’m online on MSN and Yahoo! as well.

Google Talk, though, has proven to be a bit of a challenge. It’s actually working like a tower, even though it’s based on (arguably) the most open of the IM platforms! You see I more-or-less took for granted that “transport” way of doing things, using my Jabber server to bridge to other networks. There’s no Jabber transport for Jabber though!

What I want to do kind-of flies in the face of how Jabber is designed. Ideally, you’re supposed to only have one Jabber ID (JID) — Jabber creates an open network with servers establishing connections when needed, very much like e-mail, and you only need an ID on one server to be able to chat with anyone on any other server. So what I wanted to do, which was connect to one Jabber server and have it “relay” messages to an ID on a different server is just not necessary with Jabber. Nor should it be necessary for Google Talk users to send messages to me using my Google Talk ID only — they can send straight to my JID on my Jabber server.

In the early days of Google Talk, Google had not enabled the “server-to-server” functionality that allowed this kind of communication to happen. Google Talk worked just like MSN, Yahoo! or AIM — you had to have a Google Talk account to chat with anyone on Google Talk. While this was the case, folks were looking making a Jabber-Jabber transport for connecting Jabber servers to Google Talk. At some point, though, Google opened the connectivity paths that allowed Google Talk to exist on the open Jabber network (I’ve tested this for myself). Once this happened, the need for a  ”Google Talk Transport” for Jabber evaporated in most people’s minds.

The solution nowadays is to use a client that supports multiple connections, and connect to your Jabber and Google Talk accounts at the same time. It works of course, but you don’t get the nice benefits that a transport provides — the main one being access to all your IM services and accounts from a single server connection.

So now, having resigned myself to not being able to bring my home JID and Google Talk ID together, the question arose: do I still need my own Jabber server? My current fave mobile IM client only connects to Google Talk… Could I get by just using the Google Talk service? Find out in Part two!