It seems that every argument supporting Joel Greenblatt’s magic formula examines some flavor of a backtest to prove its effectiveness. The problem with backtests is that the results don’t tell the complete picture. Backtests are fundamentally flawed, so we should find other ways to validate or disprove the magic formula strategy.
Backtests are dangerous indicators of success for two reasons.
- Time periods can be easily cherry picked
- Survivorship bias can skewer results significantly
When it comes to backtests… the shorter the time period examined, the more important timing is. This should be obvious, as results can fluctuate greatly even in short 1-3 year periods.
But results also fluctuate over the very long term. For example, examining the performance of buy and hold during 1999 would show it greatly outperforming market timing. That same study after the 2000 crash would show buy and hold as a sub-optimal strategy. The media loves to place significance to these performance numbers as they make for sensational headlines.
On a related note, out performance between value stocks, growth stocks, and momentum trading also fluctuates depending on the starting and ending time period. Small caps and large caps performance naturally fluctuates. Various asset classes are better investments at different times.
So you can see, performance numbers even on a CAGR basis greatly differ due to the natural cycling of investments and asset classes. And even though the impact reduces over a long enough time frame, there’s still a significant difference especially at market extremes.
Take this obvious example.
An examination over 20 years would see the S&P 500 returning roughly 6.6% a year from 1989 – 2009. A cherry picked time period just slightly shifted from 1991 – 2011 of the same index would be 14.8%.
The strong diversity of strategies by various fund managers shows their performance to fluctuate depending on the time period as well. Guys like Buffett had staunch under performance to the market average during the late 90’s bubble. Yet it only took a couple years to reverse that performance data.
So while Greenblatt achieved a CAGR of 40 – 50% over a 10 year period, this alone isn’t enough evidence to prove a strategy’s validity partly due to the weakness of backtests.
Of course, along with the inherent problems with backtests and past performance data are the many stock screeners which cherry pick stocks from the past and further skew the resulting data. The further back a strategy looks back, the harder it is to obtain financial data—especially before the internet.
As far as looking for SEC financial figures online, finding any data before 1992 – 1994 is impossible if you’re strictly using their own website, sec.gov. You’d likely have to go to a library to get easy and free access.
So the limited availability before 1992 either constrains the number of people able to publish relevant backtests or constrains the backtest to the last 25 year time period.
As we’ve seen previously, even a longer time period like 20 – 25 years can be greatly misinterpreted based on current market prices and how they fall in the bear or bull market cycle.
Don’t also discount the significant effect of companies that either fell off a major index or went completely bankrupt. Many backtests simply don’t include such companies. There’s no use in providing this data for many financial websites.
Stock screeners and backtests both tend to use stocks that are still currently trading. Screeners must sift through data already there, and many stocks that fell off aren’t included in past databases, especially the farther back you go. It makes a compounding effect in skewing the results.
Let’s be clear, I’m not completely discounting the value of a backtest.
It provides a good starting point but isn’t the end-all be-all for proving or disproving a strategy. I believe a strategy like the magic formula should be valid on a logical level.
If the reasons why the magic formula works does make sense on a fundamental business level, then coupling this evidence with past performance track records of Joel Greenblatt will suffice.
It’s no different than studying Graham or Buffett and understanding what the numbers mean, why they work like they do, and how such a mindset can be implemented by the average investor. As I’ve written before, blinding following a margin of safety formula can be both difficult and disastrous, as can blindly accepting the results of a backtest.
What is the Magic Formula exactly?
Joel Greenblatt wrote a bestselling book outlining his strategy called The Little Book that Beats the Market. The magic formula ranked stocks based on these 2 metrics.
- Earnings yield
- Return on capital
Following the magic formula investing strategy also comes with these conditions:
- A stock should have at least $50 million in market cap
- Financial, utility and international stocks are prohibited
- Buy the top ranked 5-7 stocks every 2 to 3 months
- Sell each stock after holding for a year (1 week beforehand if the stock is a loser for a short term capital loss deduction, 1 week afterwards for a winning stock for a long term capital gain)
With a few disclaimers:
- Your portfolio will be fully diversified in about 10 months
- Outperformance should take effect in about 3-10 years
Now, let’s simply look at the formulas for earnings yield and return on capital to deduce what they mean on simple terms and if they make sense for finding solid stocks that are likely to outperform. [click to continue…]