Me and my dumb brain

Today those polite folk from up north have been talking about mental health and how it affects them and those around them. Mental health is one of those issues that we too rarely talk about. I think it’s partly because you can’t see the problem and unless you have experienced it yourself it can be very difficult to understand. There has always been a stigma attached as if for some reason having something wrong with your mind is much worse than having any other kind of medical issue.

Whether you know it or not you’ve known people with mental health problems. Maybe chronic conditions that they will have to learn to live with for the rest of their lives, maybe just short term problems that time and therapy can help with. I’ve always been very impressed at how brave some of my friends have been to come out, as it were, to me or their community about the difficulties they face. Today it felt like it should be my turn.

For most of my adult life I’ve suffered from social anxiety. It’s never been too much of a problem but being in public spaces and interacting with people I don’t know well has been hard. In particular this affects me with phone calls and video conferences which I try to avoid wherever possible. Mostly I can push myself past this, particularly if there are a few drinks to be had to calm my nerves, but it still affects the things I will consider doing meaning I just stay at home a lot.

Some years ago though things got a little harder. Since then I’ve been suffering from IBS, a simple name for a condition that isn’t really understood and is mostly the diagnosis you get if you have chronic stomach problems and no-one can identify why. As you might expect from that there isn’t really a cure or anything that will help symptoms for everyone. For some people one thing works, for others something else will work. For some it only lasts a while, for others it lasts a lifetime. One thing that is true for many people with IBS is that it causes anxiety and depression and that has been true for me for about four years now.

Going to places I’m not familiar with, long journeys, anything where I’m expecting to be stuck somewhere and unable to go to a bathroom at short notice all trigger my anxiety. Add in the social anxiety which makes me feel pressured to not act out of the ordinary when I’m around other people when this is going on and you’ve got a great combination for me never wanting to leave the house. Add in other normally manageable life stresses and you’ve got the perfect recipe for panic attacks when I do. Oh and did I mention that stress and anxiety make IBS worse?

Thankfully I’m in the position that no one of my problems are debilitating, it is just the combination of them and other normal sources of stress and how they feed each other that makes me go through periods of extreme anxiety often with short periods of depression. Alleviating one of them helps to alleviate the others automatically. Shortly before ChloŽ was born my IBS was largely gone. The anxiety remained but I went into therapy and for a time things were mostly normal. Unfortunately things have taken a downturn lately. One thing’s for sure, adding a baby to your family sure increases your stress levels and along with some medical issues with family members I’ve got caught back in the viscous cycle. And so now I’m taking anti-depressants to help stop the panic attacks and get my anxiety under control again. Medication is not something I take likely, but it is necessary right now while I get back into therapy.I’m very proud of the family I have who have supported me since opening up about this. It’s made the difference between feeling like someone who has to hide themselves and someone who can actually be themselves, which is very important for someone with social anxiety. If someone opens up to you about something like this, please accept and support them. You may not be able to understand exactly what they’re going through but taking the pressure off by letting them know that you’ll help in any way you can means a lot.

Improving the performance of the add-ons manager with asynchronous file I/O

The add-ons manager has a dirty secret. It uses an awful lot of synchronous file I/O. This is the kind of I/O that blocks the main thread and can cause Firefox to be janky. I’m told that that is a technical term. Asynchronous file I/O is much nicer, it means you can let the rest of the app continue to function while you wait for the I/O operation to complete. I rewrote much of the current code from scratch for Firefox 4.0 and even back then we were trying to switch to asynchronous file I/O wherever possible. But still I used mostly synchronous file I/O.

Here is the problem. For many moons we have allowed other applications to install add-ons into Firefox by dropping them into the filesystem or registry somewhere. We also have to do things like updating and installing non-restartless add-ons during startup when their files aren’t in use. And we have to know the full set of non-restartless add-ons that we are going to activate quite early in startup so the startup function for the add-ons manager has to do all those installs and a scan of the extension folders before returning back to the code startup up the browser, and that means being synchronous.

The other problem is that for the things that we could conceivably use async I/O, like installs and updates of restartless add-ons during runtime we need to use the same code for loading and parsing manifests, extracting zip files and others that we need to be synchronous during startup. So we can either write a second version that is asynchronous so we can have nice performance at runtime or use the synchronous version so we only have one version to test and maintain. Keeping things synchronous was where things fell in the end.

That’s always bugged me though. Runtime is the most important time to use asynchronous I/O. We shouldn’t be janking the browser when installing a large add-on particularly on mobile and so we have taken some steps since Firefox 4 to make parts of the code asynchronous. But there is still a bunch there.

The plan

The thing is that there isn’t actually a reason we can’t use the asynchronous I/O functions. All we have to do is make sure that startup is blocked until everything we need is complete. It’s pretty easy to spin an event loop from JavaScript to wait for asynchronous operations to complete so why not do that in the startup function and then start making things asynchronous?

Performances is pretty important for the add-ons manager startup code, the longer we spend in startup the more it hurts us. Would this switch slow things down? I assumed that there would be some losses due to other things happening during an event loop tick that otherwise wouldn’t have but that the file I/O operations should take around the same time. And here is the clever bit. Because it is asynchronous I could fire off operations to run in parallel. Why check the modification time of every file in a directory one file at a time when you can just request the times for every file and wait until they all complete?

There are really a tonne of things that could affect whether this would be faster or slower and no amount of theorising was convincing me either way and last night this had finally been bugging me for long enough that I grabbed a bottle of wine, fired up the music and threw together a prototype.

The results

It took me a few hours to switch most of the main methods to use Task.jsm, switch much of the likely hot code to use OS.File and to run in parallel where possible and generally cover all the main parts that run on every startup and when add-ons have changed.

The challenge was testing. Default talos runs don’t include any add-ons (or maybe one or two) and I needed a few different profiles to see how things behaved in different situations. It was possible that startups with no add-ons would be affected quite differently to startups with many add-ons. So I had to figure out how to add extensions to the default talos profiles for my try runs and fired off try runs for the cases where there were no add-ons, 200 unpacked add-ons with a bunch of files and 200 packed add-ons. I then ran all those a second time with deleting extensions.json between each run to force the database to be loaded and rebuilt. So six different talos runs for the code without my changes and then another six with my changes and I triggered ten runs per test and went to bed.

The first thing I did this morning was check the performance results. The first ready was with 200 packed add-ons in the profile, should be a good check of the file scanning. How did it do? Amazing! Incredible! A greater than 50% performance improvement across the board! That’s astonishing! No really that’s pretty astonishing. It would have to mean the add-ons manager takes up at least 50% of the browser startup time and I’m pretty sure it doesn’t. Oh right I’m accidentally comparing to the test run with 200 packed add-ons and a database reset with my async code. Well I’d expect that to be slower.

Ok, let’s get it right. How did it really do? Abysmally! Like incredibly badly. Across the board in every test run startup is significantly slower with the asynchronous I/O than without. With no add-ons in the profile the new code incurs a 20% performance hit. In the case with 200 unpacked add-ons? An almost 1000% hit!

What happened?

Ok so that wasn’t the best result but at least it will stop bugging me now. I figure there are two things going on here. The first is that OS.File might look like you can fire off I/O operations in parallel but in fact you can’t. Every call you make goes into a queue and the background worker thread doesn’t start on one operation until the previous has completed. So while the I/O operations themselves might take about the same time you have the added overhead of passing messages between the background thread and promises. I probably should have checked that before I started! Oh, and promises. Task.jsm and OS.File make heavy use of promises and I have to say I’m sold on using them for async code. But. Everytime you wait for a promise you have to wait at least one tick of the event loop longer than you would with a simple callback. That’s great if you want responsive UI but during startup every event loop tick costs time since other code might be running that you don’t care about.

I still wonder if we could get more threads for OS.File whether it would speed things up but that’s beyond where I want to play with things for now so I guess this is where this fun experiment ends. Although now I have a bunch of code converted I wonder if I can create some replacements for OS.File and Task.jsm that behave synchronously during startup and asynchronously at runtime, then we get the best of both worlds … where did that bottle of wine go?