It’s time to pay off some technical debt!

I have been running an email server since forever. Way back in 2004 I switched Linux distros to the newly-hatched Ubuntu. There were things I liked about it, particularly when they started offering long-term support (LTS) versions that would receive updates for years. I was running Postfix for the MTA and Dovecot for the user-facing side, and everything went really well. Every two years, a new LTS release of Ubuntu would come out, I would upgrade and spend most of a day cleaning up the inevitable rough edges, and Life Was Good™.

Fast-forward to April, 2018. My mail server had been chugging happily along on Ubuntu 16.04 (LTS) for two years. The 18.04 (LTS) version was out, and it was time to upgrade. My, life, though, had changed. I was CEO, CTO, and the medical director at my company and I was a single dad to two awesome boys. My email server had gone from supporting my personal mailbox and a couple of friends’, to being the email server for my company and the de facto mail clearinghouse for several not-for-profit organizations I was working with. I never actually counted, but I believe there were close to two thousand user mailboxes at the time. My company was heavily tied into email: faxing, texting, filing in the EMR, status reporting, and probably a few other things I’ve repressed all flowed through the mail server. I was getting very little sleep, and when I updated the mail server to 18.04 and Postfix refused to start up, I immediately reverted to 16.04. It had three more years of LTS life left. Plenty of time to update.

Suffice to say that my life didn’t get any simpler. Eventually, the release reached end-of-life. I had announced my retirement (a year in advance) and was busier than ever trying to get business wrapped up. But wait! Canonical (Ubuntu’s corporate sponsor) offered extended support for a small fee. That would buy me another couple of years of updates. Plenty of time…

Now it’s February, and extended support goes away at the end of March. Fortunately, I am now fully retired and have a little bit more time. Still, despite years of warnings that things were going to get dicier for the mail server as I worked less and traveled more, there are still a lot of people using it. So I have devised an upgrade plan that should avoid any long-term disruptions, at the expense of having a number of less-than-hour-long events.

So if you’re using email at any of the domains I host, expect to occasionally have outages of less than an hour. Some MTAs (Google, last I looked) have been cutting way down on the time before they generate error messages to senders, so some people sending mail might get error messages. The messages will say that the mail is delayed and they’ll keep trying, but of course senders will simply be screaming YOUR EMAIL IS BROKEN.

It can’t be helped.