Debug logs… can’t live without them, can’t live with them. “Debugging in production" is always tricky. Logging at debug level at all times is too verbose. The log noise is useless and it would take up too much space. Not logging them at all defeats the entire point of having them. So where’s the happy medium?
Let’s begin with the understanding that debug logs are for debugging — which means there’s some issue or an error. We can all agree that we want logs some times, but not all the time. So what can you do? Here are a few solutions I’ve come across. I’ll start with my favorite. Hopefully your company doesn’t make these common mistakes and will allow you to do this!
My favorite approach is a trigger-based log dump. Most modern languages and frameworks will allow you to have multiple log handlers. You can add a handler that keeps track of all (debug included) logs, but only saves N-recent ones in memory. You will need to catch an event, such as an Exception or an ERROR-level log, that will instruct your log handler to dump all N-recent logs.
✅ Upside: Automatically get debug logs for all failures the first time they happen. No need to repeat/wait for the bug. No overhead for your logging infrastructure. Adding “too many” debug logs is ok!