When You Wish Upon a Star

Few accolades matter as much to a mutual fund as Morningstar's five-star designation. Earning that distinction leads to outsized flows, larger management fees and improved career prospects for the fund's managers.

For investors, though, buying a five-star fund is a much less certain proposition.

Whether an investor can expect higher returns from higher rated funds is an important question. Since Morningstar revised their methodology to calculate ratings in 2002, one major academic study has been published, by Pace University professors Aron Gottesman and Matthew Morey. Among the other studies that have been conducted are two that we have published at Advisor Perspectives.

The results of those studies, as well as data provided by Morningstar, suggest that ratings have limited value as predictors of future performance. This is not surprising. At its core, Morningstar's methodology is based on a fund's three-, five- and ten-year historical performance. Few subjects have been studied as carefully as whether historical fund performance can predict future performance and, on this count, the answer is clear. Virtually no scientific evidence exists that future performance, for periods longer than a year or two, can be predicted solely based on past results.

Morey and Gottesman evaluated the performance of actively managed U.S. equity funds rated by Morningstar for the three years beginning June 30, 2002. They found "widespread support for the notion that the new Morningstar rating system can predict future performance, at least within the first three years out-of-sample."

Their results were widely publicized and helped secure the value and status attributed to a five-star rating. They employed rigorous methodology, adjusting for risk and survivorship and concluded that the alpha (risk-adjusted return) for funds increased at each level of star rating, from one-star to five-star funds.

One problem remained, however.

The 2002-2005 time frame used by Morey and Gottesman was mostly a bull market. In a bull market, actively managed funds have less value and investors would be well served by index funds. A true evaluation of active managers and of the systems (like Morningstar's ratings) used to measure them requires a full market cycle, including bull and bear markets.

The last three years, for better or worse, provided the kind of turbulent conditions that are ideal for measuring managers. At Advisor Perspectives, we conducted two studies to measure the predictive power of Morningstar's ratings. The first, like Morey and Gottesman, covered a bull market, from 2004-2006. The second covered the three years ending September 30, 2009--a period including the bear market of 2008 and the bull market of 2009.

We used different methodology than Morey and Gottesman. For an advisor or an investor, the average return across a rating category is not very useful. The only way that return can be achieved is by investing in all funds with that rating--an obvious impracticality. Instead, we calculated the probability that a fund with a given rating would outperform one with a different rating. Our goal was to provide useful information for the typical decision an investor or advisor faces--whether to move up or down to a higher or lower rated fund.

Our results for the 2004-2006 period confirmed those of Morey and Gottesman; we found the ratings were reasonably predictive. We found, however, that the ratings lost virtually all of their predictive power when measured over the full market cycle ending 9/30/09. You might was well flip a coin when deciding whether to move to a fund one star higher or lower; the probabilities were very close to 50%.

Nonetheless, the larger issue remains unresolved. Morningstar ran a seven-year analysis, from mid-2002 to mid-2009, and found consistent predictive power in the ratings across all asset classes. Their study did not correct for survivorship bias, which introduces distortions that quickly grow as the time period of a study lengthens. A seven-year study is particularly vulnerable to those distortions.

It may also be that the results of the seven-year period chosen by Morningstar will turn out to be random luck. Results for other seven-year periods may be strikingly different. If Morningstar's results prove to be robust, though, it would be a breakthrough. No academic study has shown that past performance can be predictive of future results for periods as long as seven years.

When you wish upon a star, anything your heart desires may come true. Just don't expect it to come true too often.

Robert Huebscher is founder and CEO of Advisor Perspectives (www.advisorperspectives.com), a Web site and newsletter serving the financial advisory industry.