What do you do when the industry’s most popular investment gurus show less market-timing accuracy than flipping a coin? You start flipping coins, or better yet, stop trying to time markets.
CXO Advisory Group has been collecting data from market forecasters since 1998. The firm has tracked and graded thousands of market forecasts made by dozens of popular gurus over the years. The overall results are not good. CXO has concluded that the market experts accurately predicted market direction only 48 percent of the time.
Although gurus tended to come and go during their collection period, CXO found forecasting ability tends to stay about the same. Figure 1 illustrates the cumulative accuracy grade since data collection began in 1998.
Figure 1: Guru Accuracy Compared to the Number of Forecasts
Source: CXO Advisory Group, LLC
It only took 2 years and about 200 predictions before the accuracy rating fell below 50 percent in early 2000. The cumulative accuracy has stayed below 50 percent ever since. By 2008, CXO had collected and graded more than 5,000 predictions and the rating stabilized at about 48 percent.
The consistent 48 percent accuracy grade has led CXO to make a command decision. They will stop collecting and grading new forecasts in 2013. The preliminary study on the CXO website graded 6,459 forecasts. Their final report will include an additional 126 grades for 2012, although they are not expected to materially affect the preliminary results.
Steve LeCompte of CXO explains the decision to end this popular study, “Ending the collection of forecasts is a matter of effort required versus incremental learning. The effort entails not only collecting and grading forecasts, but also continually finding new gurus to replace those that drop out of sight. The accuracy rate has been very stable for years, so I strongly doubt continuing to collect forecasts would materially affect findings.”
The simple average grade for the gurus has been 48 percent over the years, although some gurus performed better than others. Figure 2 illustrates the grade distribution around the average.
Figure 2: Individual Guru Accuracy
Source: CXO Advisory Group, LLC
The best performing guru had a 68.2 percent grade while the worst performing one had a 21.7 percent grade. Among gurus with more than 100 observations, the best grade was 65.0 percent while the worst was 28.9 percent. As a matter of interest, popular “Mad Money” television host Jim Cramer had a 46.8 percent accuracy rating based on 62 market predictions.
One question that entered my mind was how thousands of up-or-down independent predictions could consistently come out to less than 50 percent accuracy? According to the preliminary report, there could be many reasons. First, CXO may have had a grading bias derived from some unintentional animus toward gurus. Second, several gurus may have had motives other than accuracy in publishing forecasts. For example, gurus who make frequent public pronouncements may be those most prone to extreme forecasts to attract attention (and customers) by stimulating greed and fear. Third, there is likely trend-following behavior among the gurus, which leads to more wrong forecasts than right forecasts.
I am sad to see CXO end their guru grade tracking. It provides an excellent real time example of how sales skill triumphs over investment skill on Wall Street. Forecasting isn’t about predicting the market; it’s about marketing the prediction. As one newsletter guru told me years ago, “Given a choice between great marketing and great forecasting, I’d pick great marketing every time.”