2024-10-07 Outage - hardware failure

One of the machines on which the forum runs experienced a significant hardware failure, details are unknown to me as the hosting provider didn't provide clarity, but it resulted in them restoring a snapshot from just before the failure to a new piece of metal.

Unfortunately this resulted in changes in IP addresses, and that resulted in my iptables and ufw rules not allowing the machines to speak to each other, which then triggered the outage.

I was on mobile and out for a few hours so couldn't get back in and fix it even though I suspected firewalls were to blame.

It's back up now, should survive... I've made a couple of small changes to make it survive better next time, but I won't know if they work until the next time the machine reboots... I may trigger that later just to give myself peace of mind.

As always, thank you @velocio

Most productive few hours of my life!

Thanks for all your hard work you continue putting in looking after the site.

🙏

Best site on the www

Just spent 3hrs staring into space.

WTF happened?

Where am I?

Thank you @Velocio
This place is one of a kind so thank you for creating and taking care of it.

Many thanks.