Errors can be thought of in two ways: bias and noise. Bias is when errors are in the same direction. Noise is variability in judgments that should be identical.
Noise can be good. Disagreements and contrarian thinking are essential ingredients to innovation—the market tests competing strategies. But there are many decisions where noise is a problem. Those of us who think analytically often believe that random errors cancel each other out. However, in performance ratings, judgment calls, and measurement, noise is highly detrimental to companies.
But even we when eliminate bias, there still exists systemic noise in the system that causes wrong decisions. Kahneman describes two types of noise: level noise and pattern noise. Level noise is variation across individuals. In a performance review, some reviewers are more generous than others. That's level noise. Judgment scales are ambiguous ("on a scale of 1 to 10"), and words may mean different things to different people ("exceeds expectations"). Pattern noise is the difference in the personal responses of people to the same things. It could be due to differences in principles or values that a person holds, consciously or unconsciously.
What's measured gets managed. Kahneman and the authors did a study on the level of noise in a company. The executives estimated differences ranged from 5% to 10% in the organization. The results were shocking. They showed that the "noise index" ranged from 34% to 62%.