Experts' Predictions As Good As Random Chance

Nate Silver is an acclaimed and respected author, statistician, political blogger who wrote a book titled "The Signal and The Noise; Why So Many Predictions Fail - But Some Don't." I'm currently still working my way through it and have thus far rather enjoyed Silver's insights on the financial crisis and why it was missed by even the wisest economic advisers.

While discussing the fall of the Soviet Union in the late 1980s, Silver cites professor Philip Tetlock's statistical findings on experts' predictions of political and economic issues. Silver says that Tetlock found out "The experts in his survey - regardless of their occupation, experience, or subfield - had done barely any better than random chance..." and "They were grossly overconfident and terrible at calculating probabilities: about 15 percent of events that they claimed had no chance of occurring in fact happened, while about 25 percent of those that they said were absolutely sure things in fact failed to occur."

While I respect and appreciate Silver's thoughts, I worry these sentiments are slightly misconstrued. I understand that "had barely done better than random chance" means the experts correctly predicted events just over 50% of the time. Alas, the second quote is what bothers me. Naturally, coming from an "expert," it is disappointing to find that their "no chance" and "absolutely sure things" were incorrectly guessed even some of the time; however, I would not feel comfortable trusting a coin flip as much as a learned professional on generally straightforward choices. If the statistics showed that half of the time, the experts failed to predict the events they were sure would or would not happen, then I would be ok with Silver's thoughts. But, I feel that an educated guess is much more reliable than a coin flip over a large sample size.

Even though the experts picks on "sure things" were wrong up to 1/4 of the time, that still means their predictions were 75-25. Is that not a better chance than a coin flip? I do not like the idea of trusting random chance over an expert especially when an educated opinion on a "seemingly" obvious topic can provide a better than random chance prediction.

- Isaac M. Comelli (6/10/13)