Learning Through Backtesting: You Might Be Unhappy 40-50% Of Time And Still Do Very Well

In this post, we will check back in on the 'Sample Portfolio' -- which was a portfolio we set up over 2 years ago and included as the example portfolio for new members. The idea behind it was to show one type of simple mix of ETFs that can serve as good complements to each other in terms of a rotation strategy.

As a review, you can see that this rotation portfolio has performed pretty well over the past 2 years.

(click to enlarge)

But the more specific point we wanted to make here was in the statistical profile that accompanies the report (marked with an arrow at the bottom) and glean what might be surprising to some.

In the last 18 monthly results (2011 & 2012), while this strategy is well ahead of the major indexes on a cumulative basis -- it has actually underperformed the S&P 500 in 10 out of 18 of those months. Yes, that is right -- it has handily beat the S&P 500 overall while still underperforming more times than not in terms of just counting the months. That seems counter-intuitive -- but this actually isn't that unusual in investing.

Many times you will have a strategy that shows good overall results --- make a habit of scanning down to that number in the table below and take a look at how often (as a %) it outperforms your inputted benchmark. If you do this a lot, you will start to understand some of the common pitfalls investors face --- which we would classify as a behavioral finance issue.

This is the important part: if you didn't know this kind of thing, imagine how your psychology might affect your investing process. The start to 2012 is a good example. Our allocations board portfolio fell behind the S&P 500 return on a Year-To-Date basis in March (we are up a healthy percentage -- but below on a relative basis). Was this underperformance meaningful? No, we don't think so. If we believe that our strategy is solid --- we only care about our equity curve (account balance) over time. Through backtesting, we are armed with the knowledge that good strategies might indeed underperform in 40% or more of the months and this does nothing to change the validity of the strategy.

Summary: We all want to outperform the indexes in every single period --- but if you obsess about this kind of thing --- chasing a benchmark around in a short-term time-frame and are then subject to changing your process -- it will ultimately be your undoing. The behavioral/mental side of investing is hugely underrated. The specific numbers in this one example are not the point. The point here is to use the portfolio backtesting process as a way to battle against the behavioral biases that keep you from doing well in investing over time. In this case, we can plainly see how the % of months outperforming vs underperforming is not a good indicator of intermediate and longer-term performance. In our view, backtesting your ideas can greatly help because if you are like us, you have to SEE it and imprint it on your brain to actually believe it. Only through a good, thorough research process will we ever actually learn what is consistent with long-term results and what is noise. Focus very intently on the investment process and not the short-term noise.

Disclosure: I have no positions in any stocks mentioned, and no plans to initiate any positions within the next 72 hours.