Status update, 16/02/2024

Some months you work very hard and there is little to show for any of it… so far this is one of those very draining months. I’m looking forward to spring, and spending less time online and at work.

Rather than ranting I want to share a couple of things from elsewhere in the software world to show what we should be aiming towards in the world of GNOME and free operating systems.

Firstly, from “Is something bugging you?”:

Before we even started writing the database, we first wrote a fully-deterministic event-based network simulation that our database could plug into. … if one particular simulation run found a bug in our application logic, we could run it over and over again with the same random seed, and the exact same series of events would happen in the exact same order. That meant that even for the weirdest and rarest bugs, we got infinity “tries” at figuring it out, and could add logging, or do whatever else we needed to do to track it down.

The text is hyperbolic, and for some reason they think it’s ok to work with Palantir, but its an inspiring read.

Secondly, a 15 minute video which you should watch in its entirety, and then consider how the game development world got so far ahead of everyone else. Here’s a link to the video.

This is the world that’s possible for operating systems, if we focus on developer experience and integration testing across the whole OS. And GNOME OS + openQA is where I see some of the most promising work happening right now.

Of course we’re a looooong way from this promised land, despite the progress we’ve made in the last 10+ years. Automated testing of the OS is great, but we don’t always have logs (bug), the image randomly fails to start (bug), the image takes hours to build, we can’t test merge requests (bug), testing takes 15+ minutes to run, etc. Some of these issues seem intractable when occasional volunteer effort is all we have.

Imagine a world where you can develop and live-deploy changes to your phone and your laptop OS, exhaustively test them in CI, step backwards with a debugger when problems arise – this is what we should be building in the tech industry. A few teams are chipping away at this vision – in the Linux world, GNOME Builder and the Fedora Atomic project spring to mind and I’m sure there are more .

Anyway, what happened last month?

Outreachy

This is the final month of the Outreachy internship that I’m running around end-to-end testing of GNOME. We already had some wins, there are now 5 separate testsuites running against GNOME OS, unfortunately rather useless at present due to random startup failures.

I spent a *lot* of time working with Tanju on a way to usefully test GNOME on Mobile. I haven’t been able to follow this effort closely beyond seeing a few demos and old blogs. This week was something of a crash course on what there is. Along the way I got pretty confused about scaling in GNOME Shell – it turns out there’s currently a hardcoded minimum screen size, and upstream Mutter will refuse to scale the display below a certain size. In fact upstream GNOME Shell doesn’t have any of the necessary adaptations for use in a mobile form factor. We really need a “GNOME OS Mobile” VM image – here’s an open issue – it’s unlikely to be done within the last 2 weeks of the current internship, though. The best we can do for now is test the apps on a regular desktop screen, but with the window resized to 360×720 pixels.

On the positive side, hopefully this has been a useful journey for Tanju and Dorothy into the inner workings of GNOME. On a separate note, we submitted a workshop on openQA testing to GUADEC in Denver, and if all goes well with travel sponsorship and US VISA applications, we hope to actually meet in person there in July.

FOSDEM

I went to FOSDEM 2024 and had a great time, it was one of my favourite FOSDEM trips. I managed to avoid the ‘flu – i think wearing a mask on the plane is the secret. From Codethink we were 41 people this year – probably a new record.

I went a day early to join in the end of the GTK hackfest, and did a little work on the Tiny SPARQL database, formerly known as Tracker SPARQL. Together with Carlos we fixed breakage in the CLI, improved HTTP support and prototyped a potential internship project to add a web based query editor.

My main goal at FOSDEM was to make contact with other openQA users and developers, and we had some success there. Since then i’ve hashed out a wishlist for openQA for GNOME’s use cases, and we’re aiming to set up an open, monthly call where different QA teams can get together and collaborate on a realistic roadmap.

I saw some great talks too, the “Outreachy 1000 Interns” talk and the Fairphone “Sustainable and Longlasting Phones” were particular highlights. I went to the Bytenight event for the first time and and found an incredible 1970’s Wurlitzer transistor organ in the “smoking area” of the HSBXL hackspace, also beating Javi, Bart and Adam at Super Tuxcart several times.

Cross-border rail in 2023

A new discovery of mine for 2023 was the #CrossBorderRail movement. It’s mainly a project of Jon Worth who is investigating all current and former border crossings in the EU by train and bike. His motivation, aside from the fact that I guess he likes to explore, is to highlight obvious issues and improvements which are needed to get more people onto trains and off of short-haul flights. I have some experiences to share in that area myself.

Back in June I made an unambitious commitment to take no more than 10 flights in 2023, which I didn’t actually meet having taken the 11th flight to get back to the UK for Christmas. (I think the climate crisis is built on people making unambitious targets and then failing to meet them anyway). Anyway, I decided to return by train to Santiago.

I have plenty of experience taking unadvisably long overland trips. The apex of this being driving to Mongolia in 2016 and taking the train back from Ulan Ude, Russia to Manchester. By comparison, travelling from North Wales to Galicia seems pretty straightforwards. However due to a combination of factors, this trip in late 2023 had more issues than that much longer journey of 2016. Here’s how it went.

Day 1

The 28th and 29th December are some of the worst days of the year to travel, as half the staff are on holiday, almost everyone in the world is travelling for holidays or family visits, and the weather is terrible.

Storm Gerrit (yes, really) blew a tree onto the line to London the night before I left. Impressively it was clear by 8AM. However, 2 previous trains to London were cancelled so three train loads of people crammed into one, barely fitting. A common experience in the UK in 2023. The staff created some order from the chaos and I guess everyone was just happy to make it to London only 30 minutes late.

I got to the Eurostar terminus with about 60 minutes to go through the border checks and no time to grab any food. This was only just enough time. There seemed to be only 1 guard checking every single UK passport and he wasn’t in a rush. I made it to the train, but I would be surprised if everyone who was further back in the queue managed to make it through.

The journey to Paris was pleasantly uneventful. In fact, nothing more went wrong until I was across the border into Spain.

The last time I did this overland trip, I went via Paris and Barcelona. The weakest point of that trip in terms of infrastructure is going between Barcelona and France border: there are 2 very expensive fast trains; or you can a take a 3 hour metro train to Latour de Carol and a night train. This route is vulnerable to delays - if you miss a connection you may need a last-minute hotel in Paris or Barcelona, which is an expensive problem.

So I took the route recommended by the Interrail app and by the Man in Seat 61, via Hendaye and Irún. Having left at 07.45 UK time, I got to Hendaye 22:45 French time and walked 20 minutes to a nice hotel in Irún. 50€ for a single room. Much cheaper than Paris!

Day 2

There’s one train a day direct from Irún to Madrid, leaving at 08.45. I was going to catch it to Segovia and then head to Santiago arriving around 17.00. Pretty good for a 1000km+ journey across the entire width of Spain.

At 08.47 I watched that train leaving Irún on time and with an empty seat where I should have been.

My mind doesn’t work right at 08:00. I left the hotel and walked away from the station, without checking – I got right back to the French border before realizing my mistake. Running, I got back to the station area and then got lost again trying to find the way in. The regular entrance is closed. The temporary entrance is pictured below… can you see it?

Yes… that tiny staircase at the end of the plaza. Perhaps a sign or two would help dumb travellers like me from getting lost and day 2 might have had a happier start.

Missing a high speed train can be an expensive mistake. Booking new tickets on the day can cost 200€ or more. If there are tickets available at all. Spanish trains require everyone to have a seat reservation, and if there are no more seats then the train is marked as “Full” and you can’t travel. There’s no possibility of standing room here.

I asked for help in the Irún ticket office and the lady was not optimistic about my chances. I had two things on my side though. The Interrail pass meant I didn’t need full tickets, I only paid around 15€ for new seat reservations. And as luck would have it, 1 place remained on the slow train to Galicia that day, which became mine.

So thanks to my early morning cockup I got home for 23:30. A long day – I expected to arrive at 17:00 – but a decent option. My other options would have been an 18 hour bus along the coast, or an expensive domestic flight.

The good and the bad

Why does the train journey get a 1500 word blog post, while the equivalent journey by plane does not?

Air travel can also be complicated – once, Vueling simply cancelled my flight on the day of travel, without notification and no viable alternative route, requiring an expensive new ticket to Porto and an overnight bus. It didn’t make an interesting blog post though. Events like this are covered in the mainstream press all the time, while cross-border train travel is still considered a minority pursuit.

I hope we get to a point where cross-border rail travel becomes mainstream and boring. We need alternatives to air travel. Every overland trip you do is worthwhile to show interest in these alternative routes. To look at it another way, you can’t complain about the trains being shit if you never try to use them. In a year where despots and colonizers are sending fighter jets and bombers to destroy the homes and the lives of innocent people in Ukraine and Palestine, by comparison the commercial aviation industry looks pretty innocent. But it causes somewhere between 4% and 10% of all greenhouse gas emissions and the figure is rising.

Besides funding less CO² emissions the train route has advantages: comfortable seats, better scenery, better baggage allowance with no worries about what might be classified as a liquid, and friendly staff along the way. I got some work done — almost all the trains had power sockets, and in France there’s even on-board wifi that works.

Organising a trip like this requires expertise. The Interrail app requires some practice. The Hendaye / Irún border doesn’t appear in most booking websites. The Seat 61 website is an invaluable resource when organising overland trips. There are also travel agents who do this if you have extra money to burn.

Cost

Talking of money, how much did it cost?

The Interrail pass was 258€. I used it to travel in the UK so it covered at least €125 of train fares there, let’s say 125€ went towards the trip home. The big problem is the Eurostar ticket – nearly 200€! Seat reservations were another 40€, and I paid 50€ for a hotel in Irún, so let’s say the total trip cost was 415€.

Compared to a flight – my family don’t live near any airport, so cheap early-morning flights aren’t an option. In this December period I expect to pay 200€ for a London-Santiago flight including baggage, plus 100€ to get to Gatwick by train, so around 300€ in total. Cheaper but comparable.

How do we make the train route practical?

The biggest blocker is time, of course – 30 to 40 hours is a whole weekend. I don’t get enough time off work to spend 4 days of each trip on travel. Working on the train isn’t realistic either; some of my work is commercially sensitive, and I can’t reasonably ask the person next to me to sign an NDA. Also, Spain there is no internet on the train, and in the UK there are no seats.

Night trains would be a huge help. The Trenhotel between Galicia and Barcelona needs to return. The proposed European Sleeper train between Barcelona and Brussels will be a game changer, if they manage to do it.

The Eurostar is in crisis, it is significantly worse since Brexit, and that one dude slowly stamping every passport is certainly not able to handle any more passengers per hour. Since capacity is limited through the station, prices for the Eurostar are prohibitively expensive. Airports have more than one passport gate – why not St. Pancras Station? This is a political issue which requires political will to solve.

France and Spain need to cooperate more on rail travel. Things have actually got worse in this regard. Imagine if SCNF ran a train to Irún, or Renfe had a train starting in Hendaye – the line is there, just – again – the political will is not.

Beyond that, making the city connections join up nicely would help hugely. The railways in Paris follow the famous “Legrand Star” design, which means you probably have to travel 30 minutes by metro to make any long distance train connection. This just isn’t practical if you have reduced mobility or small children in your party.

If you’re interested in this stuff – which you must be if you read down to here – I highly recommend following Jon Worth who seems to be the most rational and realistic voice in this area at the moment, and the #CrossBorderRail hashtag on the Fediverse.

Anyway, here’s to a 2024 filled with fun travel plans!

Quickball media server v2

As a Linux hacker-type I am often searching for some way to apply my rather specialized skillset to a real world problem. And I am always after some sovereignty over my music collection. So I came up with the idea to make some kind of music player using a Raspberry Pi in an old radio case.

What I wanted

What I got

In 2020 I got as far as setting up the Pi, in a cardboard box rather than the imagined retro radio case. It’s worked mostly ok since then, I always had some friction with the Raspbian Linux distribution though; I mean Debian is not the most up-to-date distro, and Raspbian is not even up to date with the latest Debian release. But Raspbian was the only OS with sufficient driver support to be useful for workloads involving graphics, audio, or Bluetooth.

Until now, it seems! After lots of great work from various folk, Fedora now has full Pi support, using a UEFI bootloader, with graphics acceleration, etc.

Recently I did that thing where you try to mix packages from Debian repos, thinking you might be able to do it safely this time, and the system winds up broken beyond repair, as usual. So I decided to throw it away and start again on top of the rpm-ostree based Fedora IoT distro.

The Pi is now back in action and mostly working, and the setup experience was tolerable. Here’s a summary of what I did.

The Base

It’s a Pi 4 connected to:

• a TV via HDMI
• an external HD via USB
• ethernet via a cable

The Fedora IoT install instructions are good, although somehow I didn’t get the right SSH key installed and instead had to set init=/bin/sh on first boot to gain access.

I created a regular user named “pi” and did the usual prep for a toy system: disable SELinux and disable firewalld, to avoid losing precious motivation on those.

I want to access the Pi by name; so I enabled Multicast DNS in the systemd-resolved config, and now I can refer to it with a name like pi.local on my home network.

All of the setup is automated in Ansible playbooks, when I break the machine I can rebuild it without having to go back to this blog post.

Audio

The Pi’s analogue audio output isn’t supported in Fedora IoT 38, but HDMI audio is. HDMI audio is handled by Broadcom VideoCore 4 hardware, driven by the vc4 driver.

As of Fedora IoT 38 this driver is blocklisted on the kernel commandline, and HDMI audio devices only appear after running modprobe vc4. Removing the block caused Bluetooth to stop working; I don’t really want to know what’s going on here, so instead I added a systemd unit to run modprobe vc4 after basic.target, and moved on.

I also had to add my pi user to the audio system group, done using systemd-sysusers in `/etc/sysusers.d`. Now I can see the audio devices, and (once installed) so can Pipewire and Wireplumber:

> aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: vc4hdmi0 [vc4-hdmi-0], device 0: MAI PCM i2s-hifi-0 [MAI PCM i2s-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: vc4hdmi1 [vc4-hdmi-1], device 0: MAI PCM i2s-hifi-0 [MAI PCM i2s-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Amazingly, using aplay I can now play audio from my TV.

Bluetooth

The Pi has a combined Bluetooth+Wifi adapter, the BCM4345C0. By default this needs manual probing with bcattach to make it appear. Following the overlays README, we can add this line to cat /boot/efi/config.txt for it to appear automatically on boot:

dtparam=krnbt=on

You can confirm it’s working by checking if /sys/class/bluetooth/ is populated with the hc0 controller.

From there I followed this guide from our friends at Collabora, “Using a Raspberry Pi as a Bluetooth speaker with PipeWire” and, rather amazingly, I managed to cast from my phone to the television a couple of times, after manually trusting and connecting the device from bluetoothctl on the commandline.

Ideally I want to be able to stream audio without having to get out my laptop and log in to the speaker via SSH each time I arrive home so that it connects to my phone. Another short Python script named bluetooth-autoconnect solves that by watching for trusted devices to appear and connecting with them automatically. So I only have to SSH into the machine once per device to set it as trusted.

I still get connection failures sometimes, restarting Wireplumber usually works around those.

External hard disk

The external HD full of music and media doesn’t get automatically mounted, so I installed UDisks2 and udiskie to automatically mount it. I added a storage group containing my ‘pi’ user, and a service running udiskie as ‘pi’ so that the external disk is mounted by this user.

The external disk I used was NTFS-formatted, which was a terrible idea. Every time the machine booted the filesystem would be marked dirty, no matter how cleanly I unmounted it before shutdown. I gave up on NTFS and copied the data to a second disk which is ext4-formatted.

NTFS: just say no.

Kodi

I figured getting the Kodi media server to work would be the biggest challenge. On Raspbian it’s as simple as apt install kodi and everything just works, but Fedora don’t even package Kodi (presumably due to patent issues?).

The right way to do this is to use Flatpak. The ARM64 build of Kodi is now ready. (Pro tip, you can push a draft PR to test your changes on the Flathub ARM builder rather than spending a week building it on an emulated ARM device on a small laptop.)

Flatpak needs a display server to work with, and my plan is to run a minimal Weston compositor following these instructions.

Other services

For completeness, here are the other services I’m running. These are all containers, in keeping with the design of Fedora IoT.

• Caddy web server (using official Alpine-based image)
• Jellyfin media server (using official container image)
• Samba (using ServerContainers/samba image)

One very nice thing of switching from Docker to Podman is the improved systemd integration. If you just deploy a container then it doesn’t persist across reboots, so you need a systemd service as well. Podman can generate these, and Ansible’s containers.podman.podman_container module makes it easy. The Ansible playbook to deploy Samba looks like this:

  - name: Samba container
    containers.podman.podman_container:
      name: samba
      ...
      # Let systemd manage service
      restart_policy: "no"
      generate_systemd:
        path: ~/.config/systemd/user
        restart_policy: on-failure

  - name: Start systemd unit
    systemd_service: daemon_reload=true name=container-samba state=restarted scope=user

No need to manually create a systemd unit for each container any more \o/

Calliope + Beets + Tracker Miner FS will be the next step, but first I need to bulid an ARM64 compatible container containing all the necessary pieces.

And perhaps that retro radio case – I do have some old cassette walkmans that might do the job.

Have you got any small home servers running? Show me!

Status update, 17/08/2023

Hello!
As often, these are some thoughts without any grand announcements to accompany them.

It’s just passed 5 years since I arrived in Santiago de Compostela, without much of a plan, and here I am. Summer a bit milder than last year which is great as I have got really back into cycling again. Compostela apparently has the highest ratio of tourists to locals at the moment in all Spain – 1.3 visitors for every resident, its very noticeable if you go to the old town. Meanwhile in the UK even Conservative news outlets now recommend leaving. I personally miss UK life but what I really miss is the UK of 2013, before the last decade of Conservative-led self destruction.

I already posted about GUADEC and not much has happened since. I have done some development on a Raspberry Pi-based home media server that, until recently, ran Raspbian and now runs Fedora IoT. I wanted to test an OSTree-based OS for a while and this seemed like a good opportunity as Fedora recently gained support for the Pi’s GPU.

The atomic updates with rpm-ostree work really nicely, it can be slow on ARM + SDcard, but that’s a good incentive to keep the base OS small and use containers for hosted services. This change in approach from “distro packages are the best option” to “containers are the best option” is what makes this migration non trivial.

Raspbian and Debian very much encompass the “traditional distro” model where all usecases should be satisfied with packages. Some folk still claim this to be the One True Way. In an ideal world I would agree. When it works well the results are great, the Kodi package on Raspbian is a great example which *just works*, thanks to hard work by many people. That said, Debian isn’t the newest collection of packages and Raspbian is not even based on the newest Debian release. It’s not even 64-bit by default. If I did have time to try and push these efforts forwards, where would I start?

Containers are not a panacea. Configuring services by adding snippets of config file into different environment variables can be maddening. But I am enjoying the “small distro” concept overall and finding suitable containers for different services hasn’t been difficult. So what about Kodi?

There is a Flatpak for Kodi and installing Flatpak on the Pi was not difficult. The Kodi Flatpak is only available for x86_64, though. How hard can it be to build for arm64?

Well, as always Hubert Figuiere already made a start. I updated his PR here. Building it locally on my laptop using flatpak-builder and qemu-system-arm took most of this week, and the resulting binary manages to load and display its UI, which is surprisingly good progress. The process doesn’t respond to keyboard input, so when I have a free minute I get to do some more debugging. Let’s see how it goes before I get bored.

Status update, 16/06/2023

This blog is about what’s on my mind, and recently I’ve been thinking about aeroplane flights, and limiting how many I take every year.

I’m not trying to make anyone feel impressed or ashamed here, or create rules that I insist everyone needs to live by. And I am no expert in climate science, but we need to start talking about this stuff more – as you probably spotted, the climate is uncomfortably hot now, caused mainly by all the “greenhouse gases” that we keep producing in our industrious human civilization.

Commercial aviation accounts for 3.5% of global warming today, according to the 3rd result in a web search I just did. So planes aren’t the biggest issue – the biggest issue is a massive methane leak from an oil well in Turkmenistan, and I guess the best contribution I could make to avoid further heating would be to plug up that hole somehow. From here though, it seems more realistic for me to look at reducing how much money I give to airlines and oil companies.

Plane spotting

In 2020 I didn’t go anywhere due to the COVID pandemic. so lets start with 2021.

• 2021: 3 flights, 6 hours total. (Including work stuff, family visits for Christmas).
• 2022: 10 flights, 25 hours total (Including work stuff, family visits, Linux App Summit, OSSEU, the Audio Developer Conference, and an incredible climbing trip in Canarias)
• 2023: So far, 6 flights, 6 hours total (the usual, plus FOSDEM and an incredible bike holiday in Wales)

My current goal is that 2023 will have no more flights than 2022. I’ve started tracking them in a spreadsheet to make me think harder about avoiding them. It’s easy to get excited about a holiday, conference, concert etc. and forget about the broader goal of not taking flights all the time. It’s also very hard to avoid them all – I’m sure I will take a flight to GUADEC in Latvia next month, which is basically as far away from me as you can get without leaving Europe. (I am hoping I can take a boat to the UK afterwards though, let’s see).

Everybody’s circumstances are different of course. I chose to live 2000km from my family, which implies extra travel. Some folk live 20,000km from their family, which implies even more. There’s no point judging people for how their life turned out. (Although it is good to judge billionaires and politicians flying everywhere in private planes – fuck those guys!!!).

My last trip from the UK supposedly avoided 4% of emissions of one passenger by using “sustainable fuel”, so that’s about 0.03% of the total emissions, and they emailed me this nice certificate.

The aviation industry claim that flying is actually great anyway, soon aeroplanes will be powered solely by wildflowers and insect wings, etc. etc. This is exactly what a business that sells plane tickets would say, so I am skeptical, and there is plenty of evidence of “greenwashing” such as this recent “Science Based Target Initiative“. Ultimately its hard for airline managers to turn down a fat targets-related bonus just to try and help the future of civilization … a sort of “prisoners dilemma”.

Rambling about trains

Many people assume you can take a boat between the UK and Spain, and you can do but the boat takes over 25 hours and leaves you in either Santander or Portsmouth, neither of which are very well-connected.

High speed trains are a better option, and they already exist for most of the trip: Santiago – Barcelona is less than 8 hours (not bad to travel 1000km); and Paris -> Manchester is also only 6 hrs door to door (to go 800km). The missing piece for me is Barcelona to Paris.

There’s no sleeper train option between Spain to France, since the Spanish government decided to remove it in 2012. During the day there about 2 trains, which take 6 hours each, unfortunately 8 hrs + 6 hrs + 6 hrs makes 20 hrs means you can’t do this trip in a single day. You need an expensive hotel either in Barcelona or Paris to make the trip in addition to that expensive express train to Paris. So I’ve only taken this route once so far, and it involved a few days in the French Pyrenees. (As an aside, that would be an option for every trip if it wasn’t for the restrictions of my job! In fact most of these issues would be more tractable if I could work less days a week or just take days offline sometimes.)

The French government have seen sense already and run excellent night trains from the border up to Paris, but the Spanish government have not seen any sense at all, and the connections from Barcelona are hopeless. There is some hope though, from the European Sleeper cooperative who are working on a Barcelona -> Brussels night train, due some time in 2024/2025. That could make the Santiago -> Manchester trip feasible overland in around 24 hours – which would be incredible!! They are running a share offer at the moment, if you find that sort of thing interesting.

So that’s what’s on my mind as the hot summer weather picks up across Europe.

As I said above – I’m not posting this in order to feel superior to folk who travel more by aeroplane, nor to show off about being part of the lucky 20% of people who are able to travel by aeroplane at all.

But I would like to see more people talking openly about the future and the climate crisis. When you have a problem, usually the hardest thing is to talk about it. It requires you to admit that there is a real problem which is not much fun. A lot of oil companies would prefer that we stay in denial. (They are happy making record profits!). Stay safe out in the heat.

Status update 17/03/2023

Hello from my parents place, sitting on the border of Wales & England, listening to this excellent Victor Rice album, thinking of this time last year when I actually got to watch him play at Freedom Sounds Festival, which was one of my first adventures of the post-lockdown 2020s.

I have many distractions at the moment, many being work/life admin but here are some of the more interesting ones:

• Playing in a new band in Santiago – Killo Karallo – recording some initial music which is to come out next week
• Preparing new Vladimir Chicken music, also cooked and ready for release in April
• Figuring out how we can grow the GNOME OpenQA tests while keeping them fun to work with. Here’s an experimental commandline tool which might help with that.
• Learning about marketing, analytics, and search engine optimization.
• Trying out the new LLaMA language model and generally trying to keep up with the ongoing revolution in content generation technology.

Also I got to see real snow for the first time in a few years! Thanks Buxton!

Status update, 15/12/2022

The last update of 2022, and the 14th since I started writing these.

Before we begin it’s vital to check my list of top albums from 2022, which I’m sharing to Mastodon:

• #5 – Fred Thomas – Another Song About Riding The Bus
• #4 – Klawo – Klawo
• #3 – The Slackers – Don’t Let The Sunlight Fool Ya

For #2 and #1 you will have to follow @vladimirchicken@mastodon.art I guess!

I’m commit full time at work to a project, as is normal, and a couple of spare hours a week lets me push forward a few things in GNOME.

(By the way, if you have a few hours to donate towards improving GNOME, Georges has some ideas for you).

I’ve been intermittently looking at OpenQA testing of GNOME since the summer, and I just posted a short progress report about that over on discourse.gnome.org.

I’ve also been looking at search in GNOME since about 2012, and the two things are hopefully about to converge.

When I first started volunteering my time to help maintain the Tracker search engine, CI was a luxury that GNOME couldn’t afford. Testing a patch for Tracker required a maintainer to do the following:

• download it from Bugzilla
• apply it to the local Git tree
• rebuild, which often involved 5+ minutes waiting for Autotools to check if we’d travelled back in time to the 1980’s since the last build
• run the test suite (those bits which worked)
• install and manually test the change

If anything went wrong due to the patch, the maintainer had to reply to the patch author what went wrong, and check back tomorrow to repeat the process. It’s really only due to the dedication of the maintainer team that anything ever got done.

Things have improved dramatically since the project moved to Gitlab, adopted Gitlab CI and resurrected the ‘functional-tests’ test suite which provides a model of the real desktop search process.

The ‘functional-tests’ model is useful but it is just a model, there are many more components involved when you actually start searching in GNOME Shell. The test surface includes: tracker-miner-fs and libtracker-sparql, GNOME Shell, the many search providers, Nautilus, the content apps (Music, Photos, etc), the GNOME platform libraries, the service manager (systemd), right down to the kernel itself. (If you see the background indexer taking more resources than the Shell, the problem lies in part with the kernel’s scheduler wrongly prioritizing background processes).

There are some plans in the air about changing the design and implementation of search in GNOME. This is really exciting to see – I have ideas myself – but the first step should be to better test what we already have.

And so, I recently ran a survey on what people search, and when I next get a little time to spend on GNOME, I hope to start adding some test content and search tests to OpenQA, so we can exercise the whole search stack, and confidently work to make it better.

Status update, 19/11/2022

Audio Developer Conference

I was at ADC 2022 last week – thanks to Codethink as always for covering the cost and allowing me 2 days time off to attend. It was my first time attending in person, and besides the amazing talks (which will appear online here around the end of this month), I had somehow never realized how many players in the music tech world are British. Perhaps because I always hang out in Manchester and further north while all the activity is happening in Cambridge and London.

Indeed the creator of the famous JUCE Framework is a Brit and was busy at the conference announcing a new(ish) language designed for DSP pipelines and plugins, cleverly named Cmajor.

ADC has no lightning talks but instead an “Open mic night”. I naively pictured a concert in a pub somewhere and signed up to play, but in practice it was closer to lightning talks and there were no instruments to play. For better or worse I had the session for Maharajah on a laptop and did an improvised 5 minute version with judicious (or not) or use of timestretching to speed through the slow parts. Even more fun than regular lightning talks.

I highly recommend the event! Despite the surfeit of Mac users!

Microblogging

Every article published during November 2022 must mention the Twitter meltdown and this is no exception! I’ve personally enjoyed using reading Twitter for years, always without Infinite Scroll and with “Latest tweets” instead of “Home” (i.e. content from people I follow, not entertaining clickbait for my monkey-brain). The sites only value was always its enourmous userbase and now there’s a real wave of migration to Mastodon, like, celebrities and all, I’m spending more time as @vladimirchicken@mastodon.art.

I made that account a while back, initially just for half-baked song ideas, and I will try to keep it more playful and positive than my old Twitter account. For work-related content I already have a very serious blog, after all. (That’s this blog!)

Nushell

At work I’m still spending every day deep in JSON and CSV data dumps and using Nushell to make sense of it all.

It’s really good for exploring data, and you can immediately make a guess at what this Nushell pipeline might do:

fetch http://date.jsontest.com/ | get date

I find the syntax a lot easier than jq, where even basic filters quickly start to look like K or Brainfuck.

I wrote my first Nushell scripts, which is is also very nice, shell functions (“custom commands”) can have type annotations for arguments, default values and each script can have a main function where you define the commandline arguments. 400% nicer than Bash scripts.

I don’t have time to get involved enough in the project right now to open useful bug reports, so I’m going to post a list of gripes here instead. Hopefully its useful for other folk evaluating Nushell so you can get an idea of what’s still missing as of version 0.70:

• Writing a long line (one that wraps to the next line) makes cursor movement increasingly slow… at least on WSL
• where can compare a column to a value, e.g. where date < 11-19-2020, but you can’t compare two columns, which is weird, SQL where can do that.
• The CTRL+C behaviour sucks. Run 100 slow python scripts in a loop, and you have to ctrl+c out of every single one.

Status update, 16/08/2022

Building Wheels

For the first time this year I got to spend a little paid time on open source work, in this case putting some icing on the delicious and nourishing cake that we call BuildStream 2.

If you’ve tried the 1.9x pre-releases you’ll have seen it depends on a set of C++ tools under the name BuildBox. Some hot codepaths that were part of the Python core are now outsourced to helper tools, specifically data storage (buildbox-casd, buildbox-fuse) and container creation (buildbox-run-bubblewrap). These tools implement remote-apis standards and are useful to other build tools, the only catch is that they are not yet widely available in distros, and neither are the BuildStream 2 prereleases.

Separately, BuildStream 2 has some other hot codepaths written with Cython. If you’ve ever tried to install a Python package from PyPI and wondered why a package manager is running GCC, the answer is usually that it’s installing a source package and has to build some Cython code with your system’s C compiler.

The way to avoid requiring GCC in this case is to ship prebuilt binary packages known as wheels. So that’s what we implemented for BuildStream 2 – and as a bonus, we can bundle prebuilt BuildBox binaries into these packages. The wheels have a platform compatibility tag of “manylinux_2_28.x86_64” so they should work on any x86_64 host with GLIBC 2.28 or later.

Connecting Gnomes

I didn’t participate in GUADEC this year for various reasons, I’m very glad to see it was a success. I was surprised to see six writeups of the Berlin satellite event, and only two of the main event in Mexico (plus one talk transcript and some excellent coverage in LWN) – are Europeans better at blogging? 🙂

I again saw folk mention that connections *between* the local, online and satellite events are lacking – I felt this at LAS this year – and I still think we should take inspiration from 2007 Twitter and put up a few TVs in the venue with chat and microblog windows open.

The story is, that Twitter launched in 2006 to nobody, and it became a hit only after putting up screens at a US music festival in 2007 displaying their site where folk could live-blog their activities.

Hey old people, remember this?! Via webdesignmuseum.org

I’d love to see something similar at conferences next year where online participants can “write on the walls” of the venue (avoiding the ethically dubious website that Twitter has become).

On that note, I just released a new version of my Twitter Without Infinite Scroll extension for Firefox.

Riding Trains

I made it from Galicia to the UK overland, actually for the first time. (I did do the reverse journey already by boat + van in 2018). It was about 27 hours of travel spread across 5 days, including a slow train from Barcelona into the Pyrenees, and a night train onwards to Paris, and I guess cost around 350€. The trip went fine, in comparison to the plane I had booked to return which was cancelled by the airline without notification, so the return flight+bus ended up costing a similar amount and taking nearly 15 hours. No further comment on that but I can recommend the train ride!

High speed main line through the French Pyrenees

Status update, 19/05/2022

I had some ambitious travel plans last month – ambitious by 2020’s standards, anyway – and somehow they came off without any major issues or contagions. In Cologne I was amazed to go to Freedom Sounds festival and witness the return of the Singers ATX alongside host of other ska legends. And then my first ever trip to Italy where I attended Linux App Summit. It was a treat to be around people who are as enthusiastic as I am about the rather niche topic of Linux desktop apps… but hopefully it’s a topic which is becoming less niche.

While safely back home I then managed to contract COVID-19, so this status update is going to be fairly short.

Things I spent time on that may be of interest to someone, include

• a minimal “GUI toolkit” for the monome grid, written partly as another Rust exercise, and partly to make experimental beats. Fun to write. A lot of this was developed offline while travelling, and I discovered Rust’s cargo doc feature which is an incredible tool for the disconnected hacker. Code dump here.
• related to the above, packaging the user space driver for the monome grid as a systemd portable service.
• updating Fedora on my partners laptop. This particular laptop was lacking a Wifi driver but that is now solved in Fedora 36. I can provide yet another datapoint that the GNOME 42 screenshot experience creates joy and happiness.
• watching the various new synthesizers unveiled the Superbooth 2022 event.
• a single experimental beat, plus some production on a new song due in a few months

Status update, 15/04/2022

As i mentioned last month, I bought one of these Norns audio-computers and a grey grid device to go with it. So now I have this lap-size electronic music apparatus.

Its very fun to develop for – the truth is I’ve never got on well with the “standard” tools of Max/MSP and Pure Data, I suspect flow-based programming just isn’t for me. As within an hour of opening the web-based Lua editor on the Norns device I already had a pretty cool prototype of something I hadn’t even intended to make. More on that when I can devote more time to it.

There is some work to make the open-source Norns software run on the Linux desktop in a container. It seems the status is “working but awkward.” I thought to myself – “How hard could it be to package this as a Flatpak?”

As soon as I saw use of Golang and Node.js I knew the answer – very hard.

In the world of Flatpak apps we are rigorous about doing reproducible builds from controlled inputs. The flatpak-builder tool requires that all the Go and JavaScript dependencies are specified ahead of time, and build tools are prevented from connecting to the internet at build time. This is a good thing, as the results are otherwise unpredictable. The packaging tools in the Go and Node.js ecosystems make this kind of attention to detail difficult to achieve.

There is a solution, which would be to build the components using BuildStream and develop plugins to integrate with the Node.js and Go worlds, so it can fetch deps ahead of time. There’s an open issue for Go plugin. For Node.js I am not sure if anything exists.

This led me to thinking – how come the open-source, community developed world of Flatpaks is so much better and doing reliable, reproducible builds than the corporate environments that I see in my day job?

One last cool thing – thanks to instructions here by @infinitedigits I was able to produce this amazing music video!

View this post on Instagram

A post shared by Vladimir Chicken (@vladimirchicken)

Status update, 18/03/2022

This month has had a big focus on music! I just released a 4 track EP called Rust In Peace, which you can listen to on various popular music platforms and (better still) download it from Bandcamp.

The COVID19 pandemic is not over (I can name 5 folk who have been COVID+ just this week), but its effects on society are becoming less, to the point I could even do a launch gig for the EP in the amazing Café Arume in Santiago – my first real “gig” since 2018.

A similar effect is happening at work, in that work-related travel is becoming possible. My current project, on a huge codebase that resembles a famous painting by Hieronymus Bosch, involves making changes that affects 100s or 1000s of developers, who we meet via Slack DMs and audio-only Zoom calls. For the first time in years, on-site visits are a possibility again and while I’m genuinely not keen to make a habit of intercontinental air travel, meeting the organisation face to face at least once would make a huge difference to my day to day work – the issues are much more about managing processes and people than any specific technologies, and this is super hard to do with strangers.

I am also hoping to travel to Italy next month to attend Linux App Summit 2022 in person, so, see you there? Talk submissions for LAS (online + face to face) are open until midnight tonight (18th March), so perhaps its not too late to submit a talk or lightning talk!

A few other interesting steps I’ve taken this month:

• Learn GNOME Shell keyboard shortcuts for moving and resizing windows. You can do a lot with Meta+F7/F8, it turns out.
• Implement the first “Special mix” playlist generator for Calliope; it takes a histogram of my Listenbrainz history, picks a year and finds 60 minutes of songs that I first discovered in that year. Like the old “Mystery years” radio show, but with better music.
• Buy a Norns audio-computer; basically a RPi 3 with a rich ecosystem of experimental audio effects, written Lua and Supercollider. I have been tempted recently by some of the amazing modern guitar pedals like Chase Bliss Mood and Hologram Microcosm, and this is my attempt to avoid buying any of those 🙂

Status update, 19/12/2021

Its a time to be thankful for what you can do, rather than be pissed off about things that you can’t do because we’re in the 3rd year of a global pandemic.

I made it home to Shropshire, in time to cast an important vote, and to spend Christmas with my folks… something I couldn’t do last year.

Work involves a client with software integration difficulties. Our goal is to enable a Python 3 migration in the company, which involves a tangle of dependencies in various languages. The interesting aspect is that we’re trialling BuildStream as the solution. We know BuildStream can control the mix of C/C++/Go/Java/etc. dependencies, in a way which Python-only tools like virtualenv cannot, and we hope it will be less friction compared to introducing a fullblown packaging system like DPKG. The project is challenging for a number of reasons and I am not enjoying working over VPN+SSH to another continent, but I’m sure we will learn a lot.

I was excited to see Your Year in Music available on Listenbrainz this year. I’d love to be able to export the generated playlists with Calliope, but I don’t have the time to implement it myself.

Besides that I am mostly concentrating on relaxing and finishing off some music. Here’s the weather in Wales today – cloud or snow?

Return to Codethink

2020 was a year full of surprises, so surprise that I finished it by returning to work in the same job that I left exactly 3 years ago.

There are a few reasons I did that! I will someday blog in more detail about working as a language teacher. It’s a fun job but to make the most of it you have to move around regularly, and I unexpectedly found a reason to settle in Santiago. Codethink kindly agreed that I could join the ongoing remote-work revolution and work from here.

Three years is a long time. What changed since I left? There’s a much bigger and nicer office in Manchester, with nobody in it due to the pandemic. The company is now grouped into 4 internal divisions. This is still an experiment and it adds some management overhead, also helps to maintain a feeling of autonomy in a company that’s now almost 100 people. (When I started there ten years ago, I think there were seventeen employees?!)

I also want to mention some research projects that my colleagues are working on. Codethink is a services company, but has always funded some non-customer work including in the past work on dconf, Baserock, Buildstream and the Freedesktop SDK. These are termed ‘internal investments’ but they are far from internal, the goal is always to contribute to open software and hardware projects. The process for deciding where to invest has improved somewhat in my absence; it still requires some business case for the investment (I’m still thinking how to propose that I get paid to work on music recommendations and desktop search tools all day), but there is now a process!

Here are two things that are being worked on now:

RISC-V

My contribution to Codethink’s RISC-V research was writing an article about it. The tl;dr is we are playing with some RISC-V boards, mainly in the context of Freedesktop SDK. Since writing that article the team tracked down a thorny bug in how qemu-user uses GLib that had been blocking progress, and got GNOME OS running in qemu-system-riscv. Expect to see a video soon. You can thank us when you get your first RISC-V laptop 🙂

Bloodlight

I never worked on a medical device but some of my colleagues have, and this led to the Bloodlight project. It’s an open hardware device for measuring your heart rate, aiming to avoid some pitfalls that existing devices fall into:

Existing technology used in smart watches suffers various shortcomings, such as reduced effectiveness on darker skin tones and tattoos.

There is a lot of technical info on the project on Github, including an interesting data processing pipeline. Or for a higher level overview, the team recently published an article at coruzant.com.

As is often the case, I can’t say exactly what I’m working on right now, other than it’s an interesting project and I am learning more than I ever wanted about Chromium.

Into the Pyramid

November 2019 wasn’t an easy month, for various reasons, and it also rained every single day of the month. But there were some highlights!

LibreTeo

At the bus stop one day I saw a poster for a local Free Software related event called LibreTeo. Of course I went, and saw some interesting talks related to technology and culture and also a useful workshop on improving your clown skills. Actually the clown workshop was a highlight. It was a small event but very friendly, I met several local Free Software heads, and we were even invited for lunch with the volunteers who organized it.

Purr Data on Flathub

I want to do my part for increasing the amount of apps that are easy to install Linux. I asked developers to Flatpak your app today last year, and this month I took the opportunity to package Purr Data on Flathub.

Here’s a quick demo video, showing one of the PD examples which generates an ‘audible illusion’ of a tone that descends forever, known as a Shepard Tone.

As always the motivation is a selfish one. I own an Organelle synth – it’s a hackable Linux-based device that generates sound using Pure Data, and I want to be able to edit the patches!

Pure Data is a very powerful open source tool for audio programming, but it’s never had much commercial interest (unlike its proprietary sibling Max/MSP) and that’s probably why the default UI is still implemented in TCL/TK in 2019. The Purr Data fork has made a lot of progress on an alternative HTML5/JavaScript UI, so I decided this would be more suitable for a Flathub package.

I was particularly motivated by the ongoing Pipewire project which is aiming to unify pro and consumer audio APIs on Linux in a Flatpak-friendly way. Christian Schaller mentioned this recently:

There is also a plan to have a core set of ProAudio applications available as Flatpaks for Fedora Workstation 32 tested and verified to work perfectly with Pipewire.

The Purr Data app will benefit a lot from this work. It currently has to use the OSS backend inside the sandbox and doesn’t seem to successfully communicate over MIDI either — so it’s rather a “tech preview” at this stage.

The developers of Purr Data are happy about the Flatpak packaging, although they aren’t interested in sharing the maintenance effort right now. If anyone reading this would like to help me with improving and maintaining the Purr Data Flatpak, please get in touch! I expect the effort required to be minimal, but I’d like to have a bus factor > 1.

Tracker bug fixes

This month we fixed a couple of issues in Tracker which were causing system lockups for some people. It was very encouraging to see people volunteering their time to help track down the issue, both in Gitlab issue 95 and in #tracker on IRC, and everyone involved in the discussion stayed really positive even though it’s obviously quite annoying when your computer keeps freezing.

In the end there were several things that come together to cause system lockups:

• Tracker has a ‘generic image extraction’ rule that tries to find metadata for any image/* MIME type that isn’t a .bmp, .jpg, .gif, or .png. This codepath uses the GstDiscoverer API, the same as for video and audio files, in the hope that a GStreamer plugin on the system can give us useful info about the image.
• The GstDiscoverer instance is created with a timeout of 5 seconds. (This seems quite high — the gst-typefind utility that ships with GStreamer uses a timeout of 1 second).
• GStreamer’s GstDiscoverer API feeds any file where the type is unknown into an MPEG decoder, which is effectively an unwanted fuzz test and can trigger periods of high CPU and memory usage.
• 5 seconds of processing non-MPEG data with an MPEG decoder is somehow enough to cause Linux’s scheduler to lock up the entire system.

We fixed this in the stable branches by blocking certain problematic MIME types. In the next major release of Tracker we will probably remove this codepath completely as the risks seem to outweigh the benefits.

Other bits

I also did some work on a pet project of mine called Calliope, related with music recommendations and playlist generation. More on this in a separate blog post.

And I finally installed Fedora on my partner’s laptop. It was nice to see that Gnome Shell works out-of-the-box on 12 year old consumer hardware. The fan, which was spinning 100% of the time under Windows 8, is virtually silent now – I had actually thought this problem was due to dust buildup or a hardware issue, but once again the cause was actually low-quality proprietary software.

What I did in October

October in Galicia has a weather surprise for every week. I like it because every time the sun appears you feel like you gotta enjoy it – there might be no more until March.

I didn’t do much work on Tracker this month, beside bug triage and a small amount of prep for the 2.3.1 stable release. The next step for Tracker 3.0 is still to fix a few regressions causing tests to fail in tracker-miners.git. Follow the Tracker 3.0 milestone for more information!

Planalyzer

In September I began teaching English classes again after the summer, and so I’ve been polishing the tool that I wrote to index old lesson plans.

It looks a little cooler than before:

I’m still quite happy with the hybrid GTK+/webapp approach that I’m taking. I began this way because the app really needs to be available in a browser: you can’t rely on running a custom desktop app on a classroom PC. However, for my own use running it as a webapp is inconvenient, so I added a simple GTK+/WebKit wrapper. It’s kind of experimental and a few weird things come out of it, like how clipboard selections contain some unwanted style info that WebKit injects, but it’s been pretty quick and fun to build the app this way.

I see some developers using Electron these days. In some ways it’s good: apps have strong portabilility to Linux, and are usually easy to hack on too due to being mostly JavaScript. But having multiple 150MB binary builds of Chromium dotted about my machine makes me sad. In the Planalyzer app I use WebKitGTK+, which is already part of GNOME and it works very well. It would be cool if Electron could make use of this in future 🙂

Hydra

I was always interested in making cool visuals, since I first learned about the PC demoscene back in the 1990s, but i was never very good at it. I once made a rather lame plasma demo using an algorithm i copied from somewhere else.

And then, while reading the Create Digital Music blog earlier this year, I discovered Hydra. I was immediately attracted by the simple, obvious interface: you chain JavaScript functions together and visuals appear right behind the code. You can try it here right away in your browser. I’ve been out of touch with the 3D graphics world forever, so I was impressed just to see that WebGL now exists and works.

I’ve been very much in touch with the world of audio synthesizers, so Hydra’s model of chaining together GL shaders as if it was a signal chain feels very natural to me. I still couldn’t write a fragment or a vertex shader myself, but now I don’t need to, I can skip to the creative part!

So far I’ve only made this rather basic webcam mashup but you can see a lot more Hydra examples in the @hydra_patterns Twitter account.

I also had a go at making online documentation, and added a few features that make it more suitable to non-live coding, such as loading prerecorded audio tracks and videos, and allowing you to record a .webm video of the output. I’m not sure this stuff will make it upstream, as the tool is intended for live coding use, but we’ll see. It’s been a lot of fun hacking on a project that’s so simple and yet so powerful, and hopefully you’ll see some cool music videos from me in the future!

2017 in review

I began this year in a hedge in Mexico City and immediately had to set off on a 2 day aeroplane trek back to Manchester to make a very tired return to work on the 3rd January. From there things calmed down somewhat and I was geared up for a fairly mundane year but in fact there have been many highlights!

The single biggest event was certainly bringing GUADEC 2017 to Manchester. I had various goals for this such as ensuring we got a GUADEC 2017, showing my colleages at Codethink that GNOME is a great community, and being in the top 10 page authors on wiki.gnome.org for the year. The run up to the event from about January to July took up many evenings and it was sometimes hard to trade it off with my work at Codethink; it was great working with Allan, Alberto, Lene and Javier though and once the conference actually arrived there was a mass positive force from all involved that made sure it went well. The strangest moment was definitely walking into Kro Bar slightly before the preregistration event was due to start to find half the GNOME community already crammed into the tiny bar area waiting for something to happen. Obviously my experience of organizing music events (where you can expect people to arrive about 2 hours after you want them somewhere) didn’t help here.

Codethink provides engineers with a travel budget a little bit of extra leave for attending conferences; obviously what with GUADEC being in Manchester I didn’t make a huge use of that this year, but I did make it to FOSDEM and also to PyConES which took place in the beautiful city of Cácares. My friend Pedro was part of the organizing team and it was great to watch him running round fighting fires all day while I relaxed and watched the talks (which were mostly all trying to explain machine learning in 30 minutes with varying degrees of success).

Work wise I spent most of my year looking at compilers and build tools, perhaps not my dream job but it’s an enjoyable area to work in because (at least in terms of build tools) the state of the art is comically bad. In 10 years we will look back at GNU Autotools in the way we look at a car that needs to be started with a hand crank, and perhaps the next generation of distro packagers will think back in wonder at how their forebears had to individually maintain dependency and configuration info in their different incompatible formats.

BuildStream is in a good state and is about to hit 1.0; it’s beginning to get battle tested in a couple of places (one of these being GNOME) which is no doubt going to be a rough ride — I already have a wide selection of performance bottlenecks to be looking at in the new year. But it’s looking already like a healthy community and I want to thanks to everyone who has already got behind the project.

It also seems to have been a great year for Meson; something that has been a long time coming but seems to be finally bringing Free Software build systems into the 21st century. Last year I ported Tracker to build with Meson, and have been doing various ongoing fixes to the new build system — we’re not yet able to fully switch to Autotools primary because of issue #2166, and also because of some Tracker test suite failures that seem to only show up with Meson that we haven’t yet dug into fully.

With GUADEC out of the way I managed to spend some time prototyping something I named Tagcloud. This is the next iteration of a concept that I’ve wanted since more or less forever, that of being able to apply arbitrary tags to different local and online resources in a nice way. On the web this is a widespread concept but for some reason the desktop world doesn’t seem to buy into it. Tracker is a key part of this puzzle, as it can deal with many types of content and can actually already handle tags if you don’t mind using the commandline so part of my work on Tagcloud has been making Tracker easy to embed as a subproject. This means I can try new stuff without messing up any session-wide Tracker setup, and it builds builds on some great work Carlos has been doing to modernize Tracker as well. I’ve been developing the app in Python, which has required me to fix issues in Tracker’s introspection bindings (and GLib’s, and GTK+’s … on the whole I find the PyGObject experience pretty good and it’s obviously been a massive effort to get this far, but at the same time these teething issues are quite demotivating.) Anyway I will post more about Tagcloud in the new year once some of the ideas are a bit further worked out; and of course it may end up going nowhere at all but it’s been nice to actually write a GTK+ app for the first time in ages, and to make use of Flatpak for the first time.

It’s also been a great year for the Flatpak project; and to be honest if it wasn’t for Flatpak I would probably have talked myself out of writing a new app before I’d even started. Previously the story for getting a new app to end users was that you must either be involved or know someone involved in a distro or two so that you can have 2+ year old versions of your app installable through a package manager; or your users have to know how to drive Git and a buildsystem from the commandline. Now I can build a flatpak bundle every time I push to master and link people straight to that. What a world! And did I mention GitLab? I don’t know how I ever lived without GitLab CI and I think that GNOME’s migration to GitLab is going to be *hugely* beneficial for the project.

Looking back it seems I’ve done more programming stuff than I thought I had; perhaps a good sign that you can achieve stuff without sacrificing too much of your spare time.

It’s also been a good year music wise, Manchester continues to have a fantastic music scene which has only got better with the addition of the Old Abbey Taphouse where I in fact spent the last 4 Saturdays in a row. Last Saturday we put on Babar Luck, I saw a great gig of his 10 years ago and have managed to keep missing him ever since but things finally worked out this time. Other highlights have been Paddy Steer, Baghdaddies and a very cold gig we did with the Rubber Duck Orchestra on the outdoor stage on a snowy December evening.

I caught a few gigs by Henge who only get better with time and who will hopefully break way out of Manchester next year. And in September I had the privilege of watching Jeffrey Lewis supported by The Burning Hell in a little wooden hut outside Lochcarron in Scotland, that was certainly a highlight despite being ill and wearing cold shoes.

I didn’t actually know much of Scotland until taking the van up there this year; I was amazed that such a beautiful place has been there the whole time just waiting there 400 miles north. This expedition was originally planned to be a bike trip but ended up being a road trip, and having now seen the roads that is probably for the best. However we did manage a great bike trip around the Netherlands and Belgium, the first time I’ve done a week long bike trip and hopefully the beginning of a new tradition ! Last year I did a lot of travel to crazily distant places, its a privilege to be able to do so but one that I prefer to use sparingly so it was nice to get around closer to home this year.

All in all a pretty successful year, not straightforward at times but one with several steps in the directions I wanted to head. Let’s see what next year holds 🙂

Manchester transport consulation

Manchester council is running a transport survey at the moment.

There’s nothing I like more than ranting into pointless online forms, and here’s an extract from my response:

This lane (from Mad Cycle Lanes of Manchester) is a favourite.

Manchester is not the worst city to cycle in due to having largely quite
wide roads, but cycle infrastructure is almost always an afterthought
with many issues. Most cycle lanes pop in and out of existence, forcing
cyclists into the road at dangerous points. For example London Road
going south just past the A57(m) overpass has a cycle lane which
suddenly ends forcing cyclists to pull out in front of fast-moving
traffic that has just come off a motorway. Cycle lanes sometimes
coexist with tram lines, which is very dangerous as mountain bike wheels
are just wide enough to get stuck in the tram lines. When wet, tram
lines are also very slippery which makes them dangerous to cross on a
bike. I have been injured twice as a result of cycle lanes that cross
tram lines. There are also cycle lanes which go onto narrow stretches of
pavement, for example the corner by London Road Fire Station and
Munroe’s Hotel which is painted to look like a cycle lane but is also
clearly a walk way and is too narrow to function as both. In some places
there are cycle lanes which are rendered useless by cars parking in them
or next to them.

Walking should be the main mode of transport to get around the city. At
the moment walking around takes longer than it should because so much
time is spent waiting at traffic lights.

I actually don’t mind cycling round the city too much, and the traffic jams are actually great for cycle safety because cars generally don’t hit you if they aren’t going anywhere. But the absurdity of our existing cycle infrastructure needs to be recognised.

There are more great examples from round the country here and here.

Manchester

Last night an suicide attack took place in Manchester killing at least 22 people. I don’t have much to comment on that apart from that everyone’s thoughts are with those who have been injured or lost friends and family to the attack, and to quote a friend of mine:

If you think you can sow disunity in Manchester with a bomb, you don’t know Manchester.

Ten years of Codethink

Spring is here and it is the 10th anniversary celebration of Codethink.  Nobody could have orchestrated it this way but we also have GUADEC happening here in Manchester in a few months and it’s the 20th anniversary of GNOME.  All roads lead to Manchester in 2017!

The company is celebrating its anniversary in various ways: cool new green-on-black T-shirts, a 10 years mug for everyone, and perhaps more significantly a big sophisticated party with a complicated cake.

The party was fun with a lot of old faces some who had travelled quite far to be there. The company was and still is a mix of very interesting and weird people and although we spend most of our time in the same room studiously not talking to eachother we do know how to celebrate things sometimes!

It was odd in a way being at a corporate party with fancy food and a function band and 150 guests in an enourmous monastery given that back when I joined the entire Manchester staff could go for lunch together and all sit at the same table. The first company party I went to was in Paul Sherwood’s conservatory, in fact the first few of them were there. It’s a good sign for sure that the company has quadrupled (or more) in size in the ensuing 6 years.

In hindsight I was quite lucky to have a world class open source software house more or less on my doorstep. I spent a long time trying to avoid working in software (and trying to avoiding working at all), but I did do a Summer of Code project back in 2009 or 2010 mentored by Allison Lortie, who then worked for Codethink and noted that I lived about 5000 miles closer to her office than she did.It was an obvious choice to apply to there when I graduated from University and luckily it was just at a time when they were hiring so I didn’t have to spend too long living on 50p a week and eating shoes for dinner. It was very surreal for the first few months of working there as a world which I’d previously only been involved via a computer turned into a world of real people (plus lots of computers), in fact the whole first year was pretty surreal what with also adapting to Manchester life and discovering the how much craziness there is underneath the surface of the technology industry.

I had no idea what the company did beforehand, and even now the Codethink website doesn’t give too much away. I saw contributions to Free Software projects such as Tracker and dconf (and various other things that were happening 7 years ago) but I didn’t know what kind of business model came out of that activity. It turned out that neither did anyone else at that point; the company grew out of consulting work from Nokia, but the Elopcalypse had just happened and so on starting I got involved in all sorts of different things as we looked for work in different areas: everything from boot speed optimizations and hardware control, to compiler testing and bugfixing, build tools, various automated testing setups, and more build tools, to Python and Ruby webapps, data visualisations, OpenStack, systems administration, report writing and more. Just before Christmas 2011 I got offered to go work in Korea, the catch being that I had to go in 2 days time, and the following year I spent another memorable month there (again with about 2 days notice). I also had month long stints in Bulgaria, and Berlin although these were actually planned in advance, plus all sorts of conferences as the company started to sponsor attendance and a couple of days off for such things. Most importantly of course I got involved in rock climbing which is now pretty much my favourite thing.

Since a long time now it’s felt like the company has a solid business model and while the work we do is still all over different sectors I think I can sum it up as bridging the gap between the worlds of corporate software projects and open-source software projects.  We have some great customers who engage us to do work upstream on Free Software projects which is ideal,  but far from everything we work on is Free Software, and we also work in various fields that I’m pretty unexcited about such as automotive and finance. It’s very hard to make money though if you  spend all your time working on something that you then give away so it’s a necessary compromise in my eyes.

And even in entirely closed source projects having knowledge of all the great Free Software that is available gives us an advantage. There are borderline-unusable proprietary tools still being sold by major vendors to do things like version control, there are unreliable proprietary hardware drivers being sold for hardware that has a functional and better open source driver, there are countless projects using medieval kernels, obsolete operating systems and all sorts of other craziness.Working for a company that trusts its employees is also pretty important, I meet operating systems engineers there are working on Linux-based devices whose corporate IT departments force them to use Windows, so right they trust them to maintain the operating system used in millions of cars but they don’t trust them to maintain the operating system on their laptop.

One thing Codethink lacks still is a model for providing engineer time to help with ongoing maintainance and development of different free software projects. There have been attempts at doing so within the company and I acknowledge it’s very difficult because the drop in, drop out nature of a consultant engineer’s time isn’t compatible with the ongoing time commitment required to be a reliable maintainer. Plus good maintenance skills require years to develop and either require someone experienced with a lot of free time to teach them to you, or they require you to maintain a real world project which you mess up continually and learn every lesson the hard way. Of course open source work that comes out of customer projects is highly regarded and if you’re lucky enough to have unallocated time it can sometimes be used to work through the backlog of bug fixes and feature additions for different tools you use that one inevitably develops as a full time software engineer. Again, it amazes me how many companies manage to actively prevent their developers from pushing things upstream.

We have been maintaining Baserock for years now (and many people have learned lots of lessons the hard way from it :-); BuildStream development is ongoing and I’m even still hopeful we can achieve the original goal of making it an order of magnitude easier to produce a high quality Free Software operating system. I should note that Codethink also contributes financially to conferences and projects in various ways.

I should also point out that we are still hiring. This wasn’t intended to be a marketing essay in which I talked up how great the company is, but it kinda has turned out out that way. I guess you take that as a good sign. My real underlying goal was to make it a bit clearer what it’s like to work here which I hope I’ve done a little.

I am quite proud of the company’s approach to hiring, we take in many graduates who show promise but never got involved in community-driven software projects or never really even got into programming except as a module in a science degree or whatever. Of course we also welcome people who do have relevant experience but they can be hard to find and focusing on them can also have an undesired effect of selecting based on certain privileges. I was debating with Tristan last week whether a consultancy is actually a good place for inexperienced developers to be, there is the problem that you don’t get to see the results of your work very often, you often move between projects fairly frequently and so you might not develop the intuition needed for being a good software maintainer, which is a complex topic but boils down to something like: “Is this going to cause problems in 5 years time?” There’s no real way around this, all we can do is give people a chance in an environment with a strong Free Software culture and that is pretty much what we do.

Ideally here I’d end with some photos from the party but I’m terrible at taking photos so it’s just all the back of people’s heads and lurid green lighting. Instead here’s a photo of a stranger taking a photo of me this afternoon while I was out biking round the river Mersey this afternoon.

Cake photo by Robert Marshall