Financial professionals who relish digging deeply into why we make mistakes and what we can do about it, and who can digest enough academic-driven jargon to choke a horse, will find choice pickings in “Noise: A Flaw in Human Judgment.’’ Written by the trio of Nobel prize-winning psychologist and economist Daniel Kahneman of Princeton University (author of “Thinking, Fast and Slow’’), HEC Paris professor Olivier Sibony (author of “You’re About to Make a Terrible Mistake!’’) and Harvard University legal scholar Cass R. Sunstein (co-author, with Nobelist Richard H. Thaler, of “Nudge’’), “Noise’’ is the authors’ term for “undesirable variability in judgment of the same problem.’’
First example: Using a shooting range session to illustrate their point, the authors say that the differing results show both “bias and noise — systematic deviation and random scatter — (which) are different components of error. The shooting range is a metaphor for what can go wrong in human judgment especially in the diverse decisions that people make on behalf of organizations.’’
Variable judgment leads to all sorts of prickly problems: doctors misdiagnosing illness; fingerprint experts erring in identifying a potential murderer; corporations hiring the wrong candidate; psychiatrists disagreeing on how to define patients’ ailments; and professional forecasters wrongly predicting volume of sales, growth, bankruptcy and unemployment.
Misjudgments threaten health and ruin lives, and, less profoundly, demoralize and confuse, Kahneman et al. say, and faulty judgments can cost big money.
Forecasting Problems
For financial advisors, chapter 21 of the book may be particularly interesting since it examines problems of judgment in forecasting. The authors note the answers to forecasting questions like these have major consequences, especially since private and public institutions often depend on them: “What is the unemployment likely to be in the next quarter? How many electric cars will be sold next year? What will the effects of climate change be in 2050? … What will be the annual earnings of a particular company? …”
As a whole, forecasters tend to be overly optimistic and overconfident, the book notes, and there is a lot of variability, or noise, in their projections.
How can noise be reduced in forecasts? The authors focus on two strategies: selecting better judges and aggregating multiple independent estimates.
The inspiration for “Noise’’ began some years ago, when two of the authors were affiliated with a consulting firm that conducted a “noise audit’’ of an insurance giant’s underwriters and claims adjusters’ decisions.
The audit convinced an executive that the company’s annual cost of noise in underwriting from excessive quotes and underpriced contracts “was in the hundreds of millions of dollars.’’
Kahneman et al. spend several chapters advising readers how to improve the task of judgment. There’s the artificial intelligence route — replacing judgment with algorithms — that they say “will eliminate noise entirely.’’ But they hedge their bets and say AI “is not and will not soon be a universal substitute for human judgment.’’ So, selecting the best judges is the sensible way to go, they say.
How to Conduct a Noise Audit
They also include an appendix on how to conduct a noise audit, and it’s not an easy assignment: The authors say audits require a lot of work and attention to detail because audit findings will be challenged if major flaws are revealed. The steps include creating a project team “that will be responsible for all phases of the study.” For legal cases, this includes preparing a questionnaire that explains the reasoning of each judge for each case and a list of the facts of the case. In the case of a corporation, this includes a meeting with executives to discuss the project team’s audit findings, followed by a discussion of possible outcomes, and for all audits, statistical analyses that measure the level of noise.
“Every detail of the cases and the procedure should therefore be considered with hostile scrutiny in mind,’’ they write.
“Noise” is worth the read for advisors who want to explore ideas on reducing bias and making sounder decisions that are likely to produce better outcomes.
“Noise’’ by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein. 454 pages. $32. Little, Brown Spark.
Eleanor O’Sullivan is an award-winning journalist.
