How Should We Measure Performance?
I asked a young Warren Buffett. His answers didn't disappoint.
Thomas Chua of Steady Compounding shared some notes from the Value Investor Conference in Omaha. The notes from Chuck Akre's presentation included this quote about Chuck's favorite question for management:
Meeting management is a key part of their due diligence, and Chuck often asks: How do you measure success?
Then let them talk about it, leaving it open-ended and broad.
This is done in order to better understand management’s temperament.
This got me thinking about how I should be defining my investing success.
There's a lot to think about. What should we measure performance against? Over how long of a timeframe? Should portfolio value be the only metric? Should volatility come into play? Risk-adjusted returns?
I had a lot of thoughts swimming around in my head. To help me clarify, I turned to someone who I respect. This person has also openly discussed his thoughts on evaluating performance, Warren Buffett. The Berkshire letters don't reflect how to judge an individual manager well. So I went to the older partnership letters. These are more useful when thinking about how to evaluate my own performance.
What to measure against
My gut feeling was that I needed a benchmark that's an alternative to active investment. It should be some kind of index or a broad based fund. Warren Buffett agreed with me, which is always a good thing. He chose the Dow Jones Industrial Average.
While the Dow is not perfect (nor is anything else) as a measure of performance, it has the advantage of being widely known, has a long period of continuity, and reflects with reasonable accuracy the experience of investors generally with the market. I have no objection to any other method of measurement of general market performance being used, such as other stock market averages, leading diversified mutual stock funds, bank common trust funds, etc.
However, most partners, as an alternative to their investment in the partnership, would probably have their funds invested in a media producing results comparable to the Dow, therefore, I feel it is a fair test of performance.
Why’d Buffett pick the Dow? Because it was the most widely followed index at the time. As he said, if people weren’t invested with him, they’d probably be in some kind of broad-based fund that would track the Dow. The S&P 500 started in 1957, the first year of partnership letters I found.
There are a lot more funds and indexes to choose from today than there were in 1957. I don’t think the Dow still tracks the alternative investment an average investor would make. Investing in index funds is so popular, it seems that using one of those would be appropriate. I’m going to go with the granddaddy of them all: VTSAX. I chose it for a few reasons:
It’s hugely popular.
It’s got a low expense ratio.
It matches my universe of possible investments.
It’s a very likely alternative to my active investments.
What’s the goal?
To evaluate performance, we have to set a goal. I thought about things like risk-adjusted returns and volatility. I also had a few hypothetical situations come into my head. As an example, what if I bought a company that showed great growth in free cash flow, but market price stayed flat and never reflected it? Should that be evaluated differently than a company whose share price appreciated at a rate faster than earnings or free cash growth?
Ultimately, I decided that was all too complicated. Outperformance of the benchmark is the goal. If the returns from my investments don’t outperform the benchmark over the long-term, there was no point in the work.
Here’s Buffett again with his thoughts on this:
My continual objective in managing partnership funds is to achieve a long-term performance record superior to that of the Industrial Average. I believe this Average, over a period of years, will more or less parallel the results of leading investment companies. Unless we do achieve this superior performance there is no reason for existence of the partnerships.
Over how long?
A long time. This part is hard for me. I want to evaluate my performance frequently. I want to know how I did this week, this month, and this year. But that’s not realistic. Buffett has set a bare minimum of three years, but he greatly prefers 5 years. Here he is, writing to his partners.
one year is far too short a period to form any kind of an opinion as to investment performance, and measurements based upon six months become even more unreliable. One factor that has caused some reluctance on my part to write semi-annual letters is the fear that partners may begin to think in terms of short-term performance which can be most misleading. My own thinking is much more geared to five year performance, preferably with tests of relative results in both strong and weak markets.
In another change since 1957, we’ve just come out of the longest bull market in history and have had historically low rates between the fall of 2007 and the fall of 2022. I’m not sure that 5 years is even long enough at this point. We’ve had a CAPE ratio of over 20 for the past 13 years. The long-term average is around 16. Even today with a potential default on the national debt, the longest and deepest yield curve inversion in history and a widely expected recession, the current CAPE ratio is 28.9. This is not the type of market that Buffett wanted his partners evaluating his performance in (emphasis mine):
However, I have pointed out that any superior record which we might accomplish should not be expected to be evidenced by a relatively constant advantage in performance compared to the Average. Rather it is likely that if such an advantage is achieved, it will be through better-than-average performance in stable or declining markets and average, or perhaps even poorer- than-average performance in rising markets.
I would not describe the market of the past decade as stable or declining. It’s a pretty strong uptrend from 2009 to today. There are a few dips, notably 2018 into 2019, the short, sharp COVID decline in March of 2020. There is also the recent bear market followed by the sideways market we’re in today. None of these lasted long enough to build up the advantages Buffett was talking about. In fact, I’d argue that based on this quote, he would have expected his performance to match or underperform the market for most of the last 20 years.
Using monthly close prices for VTSAX, we can cherry-pick a 3-year period from March of 2017 to March of 2020 and get a total return of only 12%. Even that isn’t terribly sideways. It’s a 4% yearly return during a period where bonds rates were essentially zero. Going one month shorter to February gets us a 30% return. Going one month longer to April of 2020 gets us a 27.8% return. Like I said, it’s cherry-picked and not reflective of the larger trend.
Looking at the monthly closing prices for VTSAX for the past 13 years shows a pretty stable rise.
What else should come into play?
I don’t think much. Ben Graham taught us that volatility in prices is not risk. I thought about looking at max drawdowns, but if we’re evaluating over a period of years, they shouldn’t matter. Buffett had some wisdom about this for his partners too:
Our job is to pile up yearly advantages over the performance of the Dow without worrying too much about whether the absolute results in a given year are a plus or a minus. I would consider a year in which we were down 15% and the Dow declined 25% to be much superior to a year when both the partnership and the Dow advanced 20%.
To further illustrate the point that even great investors have down periods, I’ll leave close with another quote from Thomas Chua’s notes on Chuck Akre’s presentation:
Chris went on to share a research study, that it’s inevitable to periodically underperform the market. Based on backtesting, the study created what a God portfolio would look like and created these portfolios knowing what had already happened.
Even though these portfolios maximized returns over time, they occasionally underperformed the market over time.
With the short termism of most investors, even God would have been fired, even if the returns were amazing in the longer term.
Even God would have been fired, and even Buffett has been accused of losing his touch. I’ll let you know how my portfolio is doing in 5 years. If we’re in another historic bull run, or if I’ve had a short-term drawdown, don’t judge me too harshly though.