We had managed to make it through the current incident unscathed until a few minutes ago but it appears to be worsening. Many (but not all!) of our ap

fly.io site is currently inaccessible..

submited by
Style Pass
2024-11-26 07:00:03

We had managed to make it through the current incident unscathed until a few minutes ago but it appears to be worsening. Many (but not all!) of our apps in iad are reporting no machines.

Not that I’m aware, and I don’t think we should expect it for a hot minute—lots of signal to indicate this has ballooned (see what I did there) into a widespread, likely even global, outage.

Things seem to be getting better, though I still have no CLI access at all which is making it really hard to restore our services

Yep, we’re in the same boat. Deploys are still 504-ing (not Depot), and all attempts to roll existing instances are also 504-ing. Not out of the woods yet.

Down for me as well as of 7:59 MST. 504’ing after waiting for the depot. Sucks as I had just gotten a solution ready to test apparently right as it went down. Always how it goes lol

@bobbyhiddn Has similar incidents happened before? In my view, an outage lasting several hours is a very serious incident. If such things happen frequently, we may need to seriously consider migrating away from Fly.io. I really like the convenience that Fly.io brings, but stability is always the highest priority as we operate in the financial payment industry.

Leave a Comment