Smoke Alarm Vs. Car Alarm
Dan Slimmon has written a great post with the same title focussing on the tradeoffs between the sensitivity and specificity of alarms for your monitoring systems and how to avoid base rate fallacy. They also painted a perfect analogy between smoke alarms vs. car alarms to explain the base rate fallacy in the same post.
Tl;Dr is that when a smoke alarm goes off, we immediately get up and do some action, i.e., check for fire, call 9-1-1 or run out of the house. When the car alarm goes off, we don’t even bother getting up.
Translating that analogy to on-call means that every team should strive to have a minimally appropriate number of smoke alarms for their team-owned services and convert most smoke alarms to car alarms when possible.
I completely agree that this analogy encourages every team I have been part of to adopt it. The one aspect from Dan’s post that I would like to address in this post is what happens when teams fail to do so, i.e., having too many smoke alarms or ignoring them for too long.
