Stats throw the Indicator for a loss
As if the stock market didn't already have several strikes against it, here's another: The New England Patriots are 12-point favorites to win today's Super Bowl.
Those odds add up to a bearish forecast because of the venerable Super Bowl Indicator. The New York Giants, who need to win if the Super Bowl Indicator is to be bullish for the rest of 2008, are major underdogs.
To be sure, many dismiss this Indicator as obviously silly. But some stubbornly believe in it, and still others — even if they don't know what to make of it — are impressed by its track record.
But how impressive is its track record, really? It's worth asking this question because, even if you believe the Super Bowl Indicator is nothing more than a Wall Street urban legend, no one is immune to making incorrect or unsupported inferences from the statistical evidence. So there's an object lesson here.
David Aronson, an adjunct professor of finance at Baruch College in New York, wrote "Evidence-Based Technical Analysis," a book in which he discusses how to use the "scientific method and statistical inference" when judging investment strategies.
Aronson focuses particularly on the pernicious role that such "data mining" as the SBI can play in making a track record look statistically significant when in fact it isn't. Data mining, of course, is the practice of searching and searching through a data set until one finds a pattern that, on the surface, looks to be meaningful.
To know for sure if the Super Bowl Indicator's track record is nothing more than a false positive, Aronson argues, you would need to know the total number of hypotheses that were tested before the Indicator was "discovered."
Unfortunately, according to Aronson, these failed hypotheses are "invisible to us. They're in the desk drawer of the person who tested them, found them not to work, and never reported his failure."
And though we can't know for sure how many failed hypotheses came before the Super Bowl Indicator was created, it would not be surprising if that number was quite large — large enough to transform the Indicator's apparently great track record into something to which we should not even give the time of day.
The first mention of the Super Bowl Indicator comes from was 1984. So the period since then constitutes a perfect out-of-sample test. And guess what: Its results since 1984 are not even remotely significant from a statistical point of view. It's only after the Super Bowl results from pre-1984 are included that the Indicator appears to be decisive.
Last year was a good illustration. The Indianapolis Colts, who won the Super Bowl last Feb. 4, could trace their roots to one of the original NFL teams — and, therefore, the stock market should have performed well since then. In fact, the Standard & Poor's 500 index is 6.5 percent lower today than then.
To be sure, some could claim that the Indicator got it right last year, since the S&P 500 at the end of 2007 was higher than where it stood during last year's Super Bowl (though only slightly). But even if the Indicator is judged a success or failure at the end of the year, as opposed to waiting until the subsequent Super Bowl, it's been no better than a coin flip this decade.
You can be just plain wrong, or you can be wrong in interesting ways. IThe Super Bowl Indicator falls into this latter category: Even though it is wrong, its errors can teach us how to be better investors.