🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

*throws another remediation onto the pile*

It has been long enough since the last time this happened that all the runbooks are wrong.

We are currently searching for the cron job that keeps this all working.

It turns out there was a dependency we weren’t aware of.

This is the cost of that cool feature rushed out last year.

*throws vendor you didn’t get to choose under the bus*

In this root cause analysis we’ll attempt to convince you we know what we are doing…

Oops! That isn’t how that supposed to go.

Hey my cousin’s friend got the vaccine and his incidents got shorter, tests finished faster, and code worked the first time. I’m not saying you shouldn’t get it, but do your research.

Shameless plug for learning from incidents instead of just #hugops. 🥰

Sorry for the interruption, our teams are gearing up to review that incident using the “Five Whines” technique and get to the bottom of things.

Sorry for the delay in that bug fix, company refreshed all the laptops for improved build times but then nobody could remember how to deploy.

Feature deploys have stabilized after deploying our internal deploy queue based on NFT derivatives. Risky deploy? Add your patch to the pile and let someone else get paid to risk the outage.

Turns out disks still fill up in 2022.

Things have seemed awfully bug free lately, and we haven’t received any notifications of changes to our existing tickets. So productive!

Sorry, we were slow bringing back the site because we were instead focusing on a fun and collaborative work environment.

The duration of that outage was definitely not affected by the distraction of the crumbling markets.

Yeah, that happened to that other company but it totally can’t happen to us. They are idiots.

Right, soooooo about that…

Disks still fill up in 2024.