The Single Greatest Predictor of Future Stock Market Returns

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Topic Author
ArthurDent
Posts: 35
Joined: Thu Jan 06, 2011 2:54 am

The Single Greatest Predictor of Future Stock Market Returns

Post by ArthurDent »

Saw this interesting blog post today on a supply-demand framework for stock prices rather than a valuations framework.
http://philosophicaleconomics.wordpress ... t-returns/
From the above link:
In this piece, I’m going to do five things. First, I’m going to explain, in very simple terms, the accounting principles behind the metric. The explanation will include instructions (with ready-made links) for how to graph the metric in FRED. Second, I’m going to discuss the dynamics of asset supply, with a special focus on equities. Third, I’m going to challenge the conventional framework for understanding the relationship between valuation and stock market returns. Fourth, I’m going to introduce a new framework, one that relates stock market returns to equity asset supply. Fifth, I’m going to present a scatterplot of the predictive performance of the metric alongside other metrics, and discuss what the metric is currently forecasting for U.S. equity returns. I’m going to conclude by briefly touching on the question of whether or not the current U.S. stock market is “overvalued.”
Browser
Posts: 4857
Joined: Wed Sep 05, 2012 4:54 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by Browser »

That's an interesting post, but I'm totally confused by the circular logic. He shows that the percentage of investor allocation to stocks is inversely correlated over 90% to subsequent 10-year returns from stocks. Here's the chart:

Image

But the percentage allocation to stocks is nothing more than the total capital invested in stocks, which is a function of the price of stocks. So, investors have a higher allocation to stocks when the price of stocks rises and the relative value of their stock holdings increases, and they have a lower allocation to stocks when the price of stocks fall and the relative value of their stock holdings decreases. An, lo and behold, high stock prices tend to be followed by lower stock prices a decade later -- let's coin a term for this phenomenon: "mean reversion." It turns out that it is better to buy when stock prices are low, and sell when they are high. This is astounding news. Now if we can just get a handle on the operational definitions of "high" and "low" we'll be in business!
We don't know where we are, or where we're going -- but we're making good time.
stlutz
Posts: 5585
Joined: Fri Jan 02, 2009 12:08 am

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by stlutz »

But the percentage allocation to stocks is nothing more than the total capital invested in stocks, which is a function of the price of stocks. So, investors have a higher allocation to stocks when the price of stocks rises and the relative value of their stock holdings increases, and they have a lower allocation to stocks when the price of stocks fall and the relative value of their stock holdings decreases. An, lo and behold, high stock prices tend to be followed by lower stock prices a decade later -- let's coin a term for this phenomenon: "mean reversion." It turns out that it is better to buy when stock prices are low, and sell when they are high. This is astounding news. Now if we can just get a handle on the operational definitions of "high" and "low" we'll be in business!
Read some of the other posts on his blog and it becomes a little more clear. Most helpful is his discussion of prices before the '87 crash (http://philosophicaleconomics.wordpress ... ey-happen/). Using PE ratios (or CAPE), one would have said prices were "high". Yet, stocks returned over 14% per year over the next 10 years, despite the crash. His proposed framework shows that stocks were a relatively low percentage of investor portfolios in 1987. In short, prices were low, not high.

The basic logic is that:

1) Most new securities issuance is in the form of fixed income. The size of the fixed income market grows about 8% per year.
2) Corporations don't like to issue new equity--buybacks and bankruptcies balance out IPOs and new share issuance.
3) The relative value of stocks and fixed income will reflect aggregate investor preferences. If investors in the aggregate want to hold a 60/40 portfolio, prices will adjust to achieve this result.
3) Therefore, if investors are going to maintain a fixed allocation of bonds and equities, then the price of equities has to go up by 8% per year.
4) Moreover, if investors currently hold a very low allocation to equities, then there is more potential demand and hence price appreciation out there. Conversely, if investors hold a high equity allocation, there is more potential supply and hence price depreciation.

Very simple, yes, but I found it very interesting as the approach offers no opinion as to what the price of stocks "should" be.
User avatar
Taylor Larimore
Posts: 32842
Joined: Tue Feb 27, 2007 7:09 pm
Location: Miami FL

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by Taylor Larimore »

Arthur:

Many years ago I purchased Jeremy Siegel's great book, Stocks for the Long Run. It contained a chart that I have never forgotten (especially in Bear Markets). Jack Bogle copied Professor Siegel's chart on page 7 in Jack's Common Sense on Mutual Funds.

I have never seen another chart that gives me more confidence about the wisdom of "Stay the course" and the long-term direction of the U.S. stock market. The chart below is from a later edition of Stocks for the Long Run:

Image

Past performance does not guarantee future performance.

Michael: Thank you for teaching your dad to post images.

Best wishes.
Taylor
"Simplicity is the master key to financial success." -- Jack Bogle
User avatar
camontgo
Posts: 329
Joined: Tue Oct 27, 2009 1:46 pm
Location: Chicago, IL
Contact:

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by camontgo »

The author has now posted a new post which includes a critique of his earlier post...another interesting read :happy

http://philosophicaleconomics.wordpress ... e-fitting/
"Essentially, all models are wrong, but some are useful." - George E. P Box
Leeraar
Posts: 4109
Joined: Tue Dec 10, 2013 7:41 pm
Location: Nowhere

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by Leeraar »

Taylor Larimore wrote:
Image

Michael: Thank you for teaching your dad to post images.

Best wishes.
Taylor
Taylor,

A most wonderful chart, but also a candidate for the most hideous design!

L.
You can get what you want, or you can just get old. (Billy Joel, "Vienna")
User avatar
JoMoney
Posts: 16260
Joined: Tue Jul 23, 2013 5:31 am

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by JoMoney »

The chart's representation of the real (inflation adjusted) return on bonds for the 40 year period between 1941 and 1981 concerns me as someone with a long-term investing horizon. Granted we don't know what the future brings, so vigilant diversification is best... but it also suggests bonds aren't a long-term safe haven either.
"To achieve satisfactory investment results is easier than most people realize; to achieve superior results is harder than it looks." - Benjamin Graham
billyt
Posts: 1111
Joined: Wed Aug 06, 2008 9:57 am

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by billyt »

Yes, inflation can be a big problem for bonds. Thank goodness someone invented TIPS.
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by longinvest »

Such charts are very misleading. As the chart starts on a specific date, it hides future differences. The same chart started in 1929 would show a different picture of stocks and bonds, for example.

Yep. I've been fooled by such charts in the past.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Browser
Posts: 4857
Joined: Wed Sep 05, 2012 4:54 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by Browser »

Does anyone have a chart of asset returns from 2014 to 2114? I've misplaced mine.
We don't know where we are, or where we're going -- but we're making good time.
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by longinvest »

Browser wrote:Does anyone have a chart of asset returns from 2014 to 2114? I've misplaced mine.
+1
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
ourbrooks
Posts: 1575
Joined: Fri Nov 13, 2009 3:56 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by ourbrooks »

Yeah, I know, the past doesn't predict the future. Then, the only safe retirement savings rate is 100% percent, not because it achieves any particular amount, but because if you can live now without spending any of your income, then you can do the same thing in retirement without worrying about retirement savings. If you don't ever want to predict the investment future, then consider committing a crime which will get you a lifetime prison sentence.

Like it or not, we must predict the future in order to make plans. Even if our predictions have a low probability of correctness, they're likely to be better than doing nothing, which is a form of prediction in its own right. For prediction purposes, the chart posted by Taylor has very little information. Yup, over 200 years, stocks have gone up and up, on average, but I'm not going to live for 200 years. In fact, some of the little downward bumps and wiggles in the chart have lasted longer than most people have to save for retirement.

In contrast, the article on predicting market returns does provide some real help. If you're 10 years from retirement, there's a high likelihood that stocks will return in the neighborhood of 6%. Trying to decide whether it pays to move closer to your work before you retire? If you use the historical average return from stocks to determine when you'll have enough money to retire, you're more likely to be wrong than if you use the figure from the article.

Let me also point out that the kind of predictions made in the article don't lend themselves to that great Boglehead sin, market timing. If stocks are going to return around 6% over the next decade, should an investor get in or out of the market now? That, of course, depends on predicting the performance of inflation and other asset classes. In the absence of accurate predictions of these other assets, most investors would be better off "staying the course" and getting the predicted 6% than hopping in and out of the market based on mostly wrong short term indicators.
Levett
Posts: 4177
Joined: Fri Feb 23, 2007 1:10 pm
Location: upper Midwest

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by Levett »

Browser,

See if you can find your chart for Jan. 1, 2014 - Dec. 31, 2014.

That will do!

I'm a stocks-in-the-short-run kinda guy. :wink:

Lev
rkhusky
Posts: 17764
Joined: Thu Aug 18, 2011 8:09 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by rkhusky »

It's funny when people mock looking at past performance, but then go ahead and base their asset allocation on past performance.
Browser
Posts: 4857
Joined: Wed Sep 05, 2012 4:54 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by Browser »

rkhusky wrote:It's funny when people mock looking at past performance, but then go ahead and base their asset allocation on past performance.
What's even sillier is that we never keep track of our predictions or expectations and evaluate how things actually turned out later. For the New Year, make 3 predictions about what you think will happen in some important areas of your life over the next 5 years. Write them down and check what happened when the 5 years is up. Of course, those predictions will be based on what has happened already in your life, and I'm willing to bet a large sum of money that all of them will be dramatically incorrect. Yet we continue to muck along, acting as if we really can predict the future based on the past. As the sage once said, "Life is what happens to you while you're busy making other plans."
We don't know where we are, or where we're going -- but we're making good time.
User avatar
Taylor Larimore
Posts: 32842
Joined: Tue Feb 27, 2007 7:09 pm
Location: Miami FL

Learning from the past

Post by Taylor Larimore »

For prediction purposes, the chart posted by Taylor has very little information. Yup, over 200 years, stocks have gone up and up, on average, but I'm not going to live for 200 years. In fact, some of the little downward bumps and wiggles in the chart have lasted longer than most people have to save for retirement.
I started investing in 1950 at the age of 26. At that time the S&P 500 was under 20. Today it is over 1,800 (not including dividends).

If I had seen a similar chart in 1950 -- and if it convinced me to invest in the market and stay-the-course -- I would be very wealthy today.

Happy Holiday!
Taylor
"Simplicity is the master key to financial success." -- Jack Bogle
ourbrooks
Posts: 1575
Joined: Fri Nov 13, 2009 3:56 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by ourbrooks »

Browser wrote:
rkhusky wrote:It's funny when people mock looking at past performance, but then go ahead and base their asset allocation on past performance.
What's even sillier is that we never keep track of our predictions or expectations and evaluate how things actually turned out later. For the New Year, make 3 predictions about what you think will happen in some important areas of your life over the next 5 years. Write them down and check what happened when the 5 years is up. Of course, those predictions will be based on what has happened already in your life, and I'm willing to bet a large sum of money that all of them will be dramatically incorrect. Yet we continue to muck along, acting as if we really can predict the future based on the past. As the sage once said, "Life is what happens to you while you're busy making other plans."
So I should never buy furniture, because I can't predict where I'll be living in five years and I might be living in a furnished apartment? I also should never save for retirement because I can't predict investment returns and every class of asset might have negative returns? We muck along, acting as if we can really predict the future based on the past, because the only other choice is to make decisions without prediction, in other words, to make them completely randomly.

Predictions don't have to be correct all of the time to still be bases for useful action. Suppose that I had the ability to predict the flip of a coin 51% of the time. I'd still be wrong nearly half the time but if I could find a casino to bet against me, I'd still break the casino.

Suppose I need to make some other life decisions based on whether or not I will have enough money in 10 years to retire in 10 years. It doesn't help to say that you can't predict investment returns; without some prediction, the only other way to make the decisions is by flipping a coin. Instead, I could presume historical market returns or I could use prediction methodologies based on CAPE or I could use the methodology proposed in the paper linked to at the beginning of the thread. There's no guarantee that any one of them is correct, but the odds of making the correct decisions are probably higher with any one of them than with a coin flip.
Browser
Posts: 4857
Joined: Wed Sep 05, 2012 4:54 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by Browser »

Of course we make predictions: I predict that when I'm crossing the street I won't get hit by a bus. We can't avoid making predictions or we'd be paralyzed. But that doesn't mean that the predictions have any validity in terms of what actually happens. We predict investment returns way out there, many years from now, based on what has happened in the past. I can guarantee that the returns you get will not even be close to what you expect. Nothing you predict several years in the future is going to turn out as you expect. Kinda perplexing isn't it? We muck along as if our expectations are accurate, we never actually check to see if they were after the fact, and if we do check up we usually change the story in our own minds so we think our expectations were accurate. Hey, I predicted the subprime mortgage financial crisis -- didn't you? I made millions. Happy sailing...
We don't know where we are, or where we're going -- but we're making good time.
ourbrooks
Posts: 1575
Joined: Fri Nov 13, 2009 3:56 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by ourbrooks »

Taylor Larimore wrote:
For prediction purposes, the chart posted by Taylor has very little information. Yup, over 200 years, stocks have gone up and up, on average, but I'm not going to live for 200 years. In fact, some of the little downward bumps and wiggles in the chart have lasted longer than most people have to save for retirement.
I started investing in 1950 at the age of 26. At that time the S&P 500 was under 20. Today it is over 1,800 (not including dividends).

If I had seen a similar chart in 1950 -- and if it convinced me to invest in the market and stay-the-course -- I would be very wealthy today.

Happy Holiday!
Taylor
So Taylor is predicting that over just the next ten years I will get the same average stock returns he's gotten since 1950? I'm impressed and I'll note this down; it will be interesting to see how accurate Taylor's prediction is a decade from now.

I do note that there's nothing in the results of either the CAPE based predictions or the investor sentiment based predictions that would suggest that staying the course is a bad idea. A 6% return might not be as large as in the past but it still is likely to be better than what you would get by taking the money out. The real value of these predictions is to allow investors to make more realistic financial plans.
alexost
Posts: 116
Joined: Wed Sep 05, 2012 1:59 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by alexost »

JoMoney wrote:The chart's representation of the real (inflation adjusted) return on bonds for the 40 year period between 1941 and 1981 concerns me as someone with a long-term investing horizon. Granted we don't know what the future brings, so vigilant diversification is best... but it also suggests bonds aren't a long-term safe haven either.
My thoughts exactly. I am 39, about 94% stocks and considering adjusting my risk to 80/20 in January instead of at 40 like I previously planned. But am I going to get negative returns from my bond allocation for the next 20-25 years? What would that do to my total real return? Seems there's an above average possibility bonds could subtract from my real total return.
rkhusky
Posts: 17764
Joined: Thu Aug 18, 2011 8:09 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by rkhusky »

We invest in stocks because we expect stocks to provide the best return over other types of investment. Where does that expectation come from? Looking at past returns. We temper our stock investments by adding some bonds to the mix, based on our time horizon, due to the expected volatility of stocks. Where do our expectations for stock volatility come from? Looking at past returns.
And, based on past returns, we consider a 20% stock market drop likely, a 50% drop possible, but a 90% drop very unlikely.
pkcrafter
Posts: 15461
Joined: Sun Mar 04, 2007 11:19 am
Location: CA
Contact:

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by pkcrafter »

JoMoney wrote:
so vigilant diversification is best..
+1...Always, in all cases.

Ourbrooks wrote:
Yup, over 200 years, stocks have gone up and up, on average, but I'm not going to live for 200 years. In fact, some of the little downward bumps and wiggles in the chart have lasted longer than most people have to save for retirement.


True. Do investors really have a long term, or they simply faced with a series of abnormal periods lasting from 10 to 20 years?

On the subject of predicting the market, GMO makes 7 year forecasts every quarter. Most others make 10 year forecasts every year. Think about it!

Paul
When times are good, investors tend to forget about risk and focus on opportunity. When times are bad, investors tend to forget about opportunity and focus on risk.
dbr
Posts: 46181
Joined: Sun Mar 04, 2007 8:50 am

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by dbr »

pkcrafter wrote:
True. Do investors really have a long term, or they simply faced with a series of abnormal periods lasting from 10 to 20 years?
It is a fact that safe withdrawal rates have been strongly dependent on year of retirement. That dependence must come about because stock returns, bond returns, and inflation come as various series of abnormal periods lasting long enough to dominate whole retirements but being short enough to be unpredictable ex ante. 10-20 years would be at about the same period as retirements, maybe a little short, and therefore the worst possible time scale for such things if one is attempting to predict retirement outcomes.
Levett
Posts: 4177
Joined: Fri Feb 23, 2007 1:10 pm
Location: upper Midwest

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by Levett »

Moderation in all things (including asset classes) has served me quite well for 45 years of investing.

If you want to swing for the fences, makes sure you can handle strikeouts--especially make sure you can hit a curve!

If any asset class return is a sure thing, be sure to get your advisor to guarantee it.

If you don't get a guarantee, it means the only skin your advisor has in the game is your skin.

Think about it.

Lev
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by willthrill81 »

I think it's worth revisiting this topic. In the four years since the inception of this metric, it has continued to serve as a good predictor of subsequent ten year returns.

Image
Why haven’t you heard of this before?

My guess is that 10 years is too long a period to interest the typical investor, who is far more drawn to what the market will do this month or year. But some have paid attention to the indicator. Ned Davis Research, the quantitative research firm, for one, has been following it; Ned Davis calls its track record “remarkable.”
https://www.marketwatch.com/story/what- ... 2016-06-21
The Sensible Steward
michaelsieg
Posts: 625
Joined: Mon Jan 07, 2013 10:02 am

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by michaelsieg »

Interesting concept, thanks for reviving this thread. I had not heard about this metric before. Unfortunately the link to the original article does not seem work anymore, it would be interesting to read it.
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by willthrill81 »

michaelsieg wrote: Mon Mar 19, 2018 10:03 pm Interesting concept, thanks for reviving this thread. I had not heard about this metric before. Unfortunately the link to the original article does not seem work anymore, it would be interesting to read it.
It's below.

http://www.philosophicaleconomics.com/2 ... t-returns/
The Sensible Steward
AlohaJoe
Posts: 6609
Joined: Mon Nov 26, 2007 1:00 pm
Location: Saigon, Vietnam

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by AlohaJoe »

willthrill81 wrote: Mon Mar 19, 2018 9:25 pm I think it's worth revisiting this topic. In the four years since the inception of this metric, it has continued to serve as a good predictor of subsequent ten year returns.
I think the author intended his article to be a thought experiment rather than a serious prediction tool. In a later post he clarified:
The title, “The Single Greatest Predictor of Future Stock Market Returns”, was something of an intentional exaggeration, chosen not only to draw attention to an out-of-the-box (and, in my opinion, useful) way of thinking about equity returns, but also to take a subtle jab at commonly-cited valuation metrics. The title was not meant to be taken literally.

[...]

The right way to model returns–and to debate the valuation issue–is not to put together curve-fits (as I admittedly did in the prior piece)
nalor511
Posts: 5058
Joined: Mon Jul 27, 2015 1:00 am

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by nalor511 »

Is there a link/ticker to easily track this metric? (Average stock allocation)
User avatar
wintermute
Posts: 229
Joined: Mon Mar 15, 2010 10:36 pm

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by wintermute »

What has always been funny to me is that portfolios are always valued at marginal or nearly-marginal pricing. Prices aren't volume or supply/demand adjusted before multiplying by shares held.
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by willthrill81 »

AlohaJoe wrote: Mon Mar 19, 2018 10:30 pm
willthrill81 wrote: Mon Mar 19, 2018 9:25 pm I think it's worth revisiting this topic. In the four years since the inception of this metric, it has continued to serve as a good predictor of subsequent ten year returns.
I think the author intended his article to be a thought experiment rather than a serious prediction tool. In a later post he clarified:
The title, “The Single Greatest Predictor of Future Stock Market Returns”, was something of an intentional exaggeration, chosen not only to draw attention to an out-of-the-box (and, in my opinion, useful) way of thinking about equity returns, but also to take a subtle jab at commonly-cited valuation metrics. The title was not meant to be taken literally.

[...]

The right way to model returns–and to debate the valuation issue–is not to put together curve-fits (as I admittedly did in the prior piece)
An exaggeration, perhaps, but he may have stumbled onto something useful. Ned Davis seems to think so.
The Sensible Steward
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by willthrill81 »

nalor511 wrote: Mon Mar 19, 2018 11:00 pm Is there a link/ticker to easily track this metric? (Average stock allocation)
https://fred.stlouisfed.org/graph/?g=qis
The Sensible Steward
User avatar
nisiprius
Advisory Board
Posts: 52215
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by nisiprius »

longinvest wrote: Mon Dec 23, 2013 8:01 am Such charts are very misleading. As the chart starts on a specific date, it hides future differences. The same chart started in 1929 would show a different picture of stocks and bonds, for example.

Yep. I've been fooled by such charts in the past.
And they are also misleading because of lower and lower data quality as you go farther back, and because of the impossibility of comparing "the stock market" of 1850 with that of today. That's a version of Jeremy Siegel's chart, and Jason Zweig has attacked it in an article, Does Stock-Market Data Really Go Back 200 Years?. He is particularly skeptical about pre-1870 data, for good reason. One issue is dividend data, which is much harder to find than price data, and according to Zweig it is basically an estimate or guess, and in one book Siegel estimated it at 5.0% and in a later edition he'd changed that to 6.4%. Also according to Zweig--well, read the article.

The chapter on "Stock Market Returns from 1815-2014" in the 2015 Ibbotson SBBI Classic Yearbook says:
While we firmly believe that a 1926 starting data was approximately when quality financial data came into existence, our hope is that the continuing development of these [older] data sets will allow modern researchers ... to test a broad range of hypothesis... as well as open up new areas for more accurate analysis.
The earliest data in their set comes from a publication entitled The New York Shipping List, which in itself is a strong indication that the stock market wasn't very important and wasn't getting the careful attention it does today!

From 1870 to 1937 the usual source of data is the Cowles Commission report, "Common-Stock Indexes, 1871-1937." This is a full-sized book and a major piece of work, very obviously as serious and comprehensive as they could make it, with a great deal of detail on how they obtained their data. I started a thread on Reliability of stock data prior to 1939? about it. It seems like pretty data but it has some interesting issues. One is that regional exchanges were important in that era. The Cowles report estimates that they included about 90% of the stocks on the NYSE, but that the NYSE only accounted for about 67% of the total volume on "the 24 most important exchanges in the country." Thus, less than 2/3rds of "the US stock market." Even more important, about 1/6th of the stock market was "the curb exchange," which eventually became the AMEX. Cowles omitted it because the newspaper that published that data literally put a warning label on it:
he reason... is indicated by the following statement regularly carried by the Commercial and Financial Chronicle at the head of the Curb quotations prior to December 1920: "It should be understood that no such reliability attaches to transactions on the "Curb" as to those on the regularly organized stock exchanges... it is out of the question for anyone to vouch for the absolute trustworthiness of the record of "Curb" transactions, and we give it for what it may be worth." Frauds practiced on the New York Curb were so severely criticized in the report of the Hughes Commission in 1909 that organization was begun for the purpose of reform.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
lazyday
Posts: 3849
Joined: Wed Mar 14, 2007 10:27 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by lazyday »

camontgo wrote: Sun Dec 22, 2013 8:36 pm The author has now posted a new post which includes a critique of his earlier post...another interesting read :happy

http://philosophicaleconomics.wordpress ... e-fitting/
new link:

http://www.philosophicaleconomics.com/2 ... e-fitting/
Valuation and Stock Market Returns: Adventures in Curve Fitting
Posted on December 23, 2013 by philosophicalecon@gmail.com

My prior piece on asset supply has received significant interest, and so I feel an obligation to clarify.
motorcyclesarecool
Posts: 956
Joined: Sun Dec 14, 2014 6:39 am

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by motorcyclesarecool »

Browser wrote: Sun Dec 22, 2013 6:08 pm...So, investors have a higher allocation to stocks when the price of stocks rises and the relative value of their stock holdings increases, and they have a lower allocation to stocks when the price of stocks fall and the relative value of their stock holdings decreases. An, lo and behold, high stock prices tend to be followed by lower stock prices a decade later -- let's coin a term for this phenomenon: "mean reversion." It turns out that it is better to buy when stock prices are low, and sell when they are high. This is astounding news. Now if we can just get a handle on the operational definitions of "high" and "low" we'll be in business!
Gauging anything is of necessity going to be a question of which analogy to use. The best example I can think of is that of a fuel gauge. Car fuel gauges measure the height of fluid at a given point in a tank. It is a rough analog for the volume of fuel, which is an analog for the mass of fuel, from which, thanks to Avogadro, we can calculate the number of fuel molecules. Is it circular, or merely another analogy that mostly fits?

What analogy for valuation works best? I haven’t the foggiest. I believe on the one hand, like the weather, there are far too many variables in the equation to be able to measure. On the other hand, there sure are a lot of people going 100% equities.

My questions:
Why 10 year equity returns? Why not 5, or 20?
Why are the scales so radically different for the two lines? And why not show numbers back to 1927?
What about allocation to international equities and/or international debt instruments?
Understand that choosing an HDHP is very much a "red pill" approach. Most would rather pay higher premiums for a $20 copay per visit. They will think you weird for choosing an HSA.
2015
Posts: 2906
Joined: Mon Feb 10, 2014 1:32 pm

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by 2015 »

Browser wrote: Mon Dec 23, 2013 12:13 pm Of course we make predictions: I predict that when I'm crossing the street I won't get hit by a bus. We can't avoid making predictions or we'd be paralyzed. But that doesn't mean that the predictions have any validity in terms of what actually happens. We predict investment returns way out there, many years from now, based on what has happened in the past. I can guarantee that the returns you get will not even be close to what you expect. Nothing you predict several years in the future is going to turn out as you expect. Kinda perplexing isn't it? We muck along as if our expectations are accurate, we never actually check to see if they were after the fact, and if we do check up we usually change the story in our own minds so we think our expectations were accurate. Hey, I predicted the subprime mortgage financial crisis -- didn't you? I made millions. Happy sailing...
+1

Why We Listen To Bad Forecasts:

http://www.collaborativefund.com/blog/w ... forecasts/
The history of financial forecasts is long. And it is pathetic.

Twenty-two chief market strategists at major banks predict the S&P 500’s annual performance at the start of every year. The average difference between their forecast and what the market actually does is 14.7 percentage points per year. If, instead, you blindly assume the S&P 500 goes up by its historic average every year, the error is lower – 14.1 percentage points.

The history of predicting recessions and interest rates and economic growth is just as bad.

So why is there still rabid demand for forecasts?

It’s not as simple as “hope” or “ignorance” or “bias” – although there’s truth to each.

People listen to financial forecasts because:

The stakes are high. An accurate investment forecast can earn or cost you a fortune. Your future goals could depend on it.

Markets and economies are confusing. Why things happen is often unknown, the result of billions of people responding to their own unique incentives.

If economies confuse you – and if they don’t you’re not paying attention – and someone says “I can earn you a fortune if you listen to me and if you don’t you will lose a fortune,” the temptation to listen is huge. The stakes are so high that it can be dangerous to not listen to forecasts, despite their track record – just in case.
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by willthrill81 »

motorcyclesarecool wrote: Sun Mar 25, 2018 5:48 amWhy 10 year equity returns? Why not 5, or 20?
Because it works best for 10 years. This is largely the same conclusion that Nobel prize winner Shiller came to with his CAPE: certain time periods just work better than do others.

Some throw a lot of criticism on this data-driven approach, but as physicist Olivier Heaviside, who discovered the operator method for solving differential equations, said to his detractors for his inability to articulate the theoretical underpinnings of his discovery, "I do not refuse my dinner because I do not understand the process of digestion."
motorcyclesarecool wrote: Sun Mar 25, 2018 5:48 amWhy are the scales so radically different for the two lines? And why not show numbers back to 1927?
The two variables are measuring very different things, hence, the scales must be very different. The scale isn't important; the relationship between the variables is.

I believe that he didn't go back further due to the lack of available data on the average allocation to equity.
motorcyclesarecool wrote: Sun Mar 25, 2018 5:48 amWhat about allocation to international equities and/or international debt instruments?
Again, data availability is probably the driver here.
The Sensible Steward
User avatar
cheese_breath
Posts: 11786
Joined: Wed Sep 14, 2011 7:08 pm

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by cheese_breath »

You want to know the single greatest predictor of future stock market returns?

See my signature.
The surest way to know the future is when it becomes the past.
motorcyclesarecool
Posts: 956
Joined: Sun Dec 14, 2014 6:39 am

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by motorcyclesarecool »

willthrill81 wrote: Sun Mar 25, 2018 9:58 am
motorcyclesarecool wrote: Sun Mar 25, 2018 5:48 amWhat about allocation to international equities and/or international debt instruments?
Again, data availability is probably the driver here.
What I mean is that this model appears to be flawed in that it would appear to distort the average allocation to equities by lumping in international bonds and international equities with domestic equities. On its face it strikes me as a nonsense number.
Understand that choosing an HDHP is very much a "red pill" approach. Most would rather pay higher premiums for a $20 copay per visit. They will think you weird for choosing an HSA.
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: The Single Greatest Predictor of Future Stock Market Ret

Post by willthrill81 »

motorcyclesarecool wrote: Sun Mar 25, 2018 4:22 pm
willthrill81 wrote: Sun Mar 25, 2018 9:58 am
motorcyclesarecool wrote: Sun Mar 25, 2018 5:48 amWhat about allocation to international equities and/or international debt instruments?
Again, data availability is probably the driver here.
What I mean is that this model appears to be flawed in that it would appear to distort the average allocation to equities by lumping in international bonds and international equities with domestic equities. On its face it strikes me as a nonsense number.
From what I'm looking at, the Fed doesn't lump international equities or debt instruments into the numerator of this measure, and I don't think they include those in the denominator either. Regardless, as long as the method being used is consistent, it shouldn't really make a substantial difference.

If it is a 'nonsense number', it's predictive ability since 'discovery' has surely continued to be robust.

If you haven't already, I would recommend that you his accounting explanation as to why it works. It actually makes a lot of sense. The basic idea is that when investors' allocation to equities is high, the volume of capital is too great for returns to be high, and vice versa. This has had far greater predictive ability than any valuation based measure I've seen. Their problem is that valuations are not reliably mean-reverting; for instance, CAPE has been above its historic average almost continuously since 1992, and real returns from then until now were nearly identical to their long-term average.
The Sensible Steward
dknightd
Posts: 3727
Joined: Wed Mar 07, 2018 10:57 am

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by dknightd »

willthrill81 wrote: Mon Mar 19, 2018 10:19 pm It's below.

http://www.philosophicaleconomics.com/2 ... t-returns/
Thanks for bumping this, and updating the chart. Was an interesting read.

His follow up was also an interesting read
http://www.philosophicaleconomics.com/2 ... e-fitting/
If the supply of cash and bonds rises faster than normal, then stock prices should rise faster than normal, assuming the equity allocation preference doesn’t change over the period. Therefore, the future return should be higher. How does the metric model this fact? It doesn’t. For whatever reason, it gets lucky, and any effect of higher cash and bond supply growth gets offset by other coincidences, or just dismissed as visual noise.

Now, this doesn’t mean that allocation dynamics don’t affect stock prices, or that they aren’t an important driver of returns–they are, without question. We should pay attention to them, factor them into our market analyses. But, admittedly, the metric doesn’t deserve the reputation for predictive precision that the chart, by chance, affords it.
Retired 2019. So far, so good. I want to wake up every morning. But I want to die in my sleep. Just another conundrum. I think the solution might be afternoon naps ;)
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by willthrill81 »

Here's the most recent version of the chart showing the relationship between stock returns and the average investor allocation to stocks.

Image
https://financial-charts.effingapp.com/

I must say that I'm astounded at how close the lines are from 2004 (i.e. 10 years back from when this relationship was initially proposed for predicting 10 year subsequent returns) going forward.
The Sensible Steward
User avatar
dogagility
Posts: 3237
Joined: Fri Feb 24, 2017 5:41 am

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by dogagility »

Very interesting relationship. Trying to wrap my brain around why this relationship exists (doesn't look like randomness to me).
Some thoughts...
During large market declines (e.g. 2008/9), people reduce their equity exposure. Because equities are favorably valued after a large decline, the price would tend to increase during the next ten years... creating a larger 10 year average return.
The converse would be true during a bull run.
Is that it? Am I off base? Is it that simple?
Make sure you check out my list of certifications. The list is short, and there aren't any. - Eric 0. from SMA
Valuethinker
Posts: 49035
Joined: Fri May 11, 2007 11:07 am

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by Valuethinker »

ArthurDent wrote: Sun Dec 22, 2013 5:19 pm Saw this interesting blog post today on a supply-demand framework for stock prices rather than a valuations framework.
http://philosophicaleconomics.wordpress ... t-returns/
From the above link:
In this piece, I’m going to do five things. First, I’m going to explain, in very simple terms, the accounting principles behind the metric. The explanation will include instructions (with ready-made links) for how to graph the metric in FRED. Second, I’m going to discuss the dynamics of asset supply, with a special focus on equities. Third, I’m going to challenge the conventional framework for understanding the relationship between valuation and stock market returns. Fourth, I’m going to introduce a new framework, one that relates stock market returns to equity asset supply. Fifth, I’m going to present a scatterplot of the predictive performance of the metric alongside other metrics, and discuss what the metric is currently forecasting for U.S. equity returns. I’m going to conclude by briefly touching on the question of whether or not the current U.S. stock market is “overvalued.”
Are we talking nominal or real returns here?

Siegel found stock returns USA were 6% long run. Real.

Dimson & Marsh found international returns, long run, were 5% real in USD.

So when people say 6% here, do they mean real or nominal?

Hard to believe you can value the US market without considering international portfolio investors. US exceptionalism strikes again?
User avatar
snackdog
Posts: 3102
Joined: Wed Nov 12, 2014 3:57 am
Location: PNW

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by snackdog »

As valuations rise, people want to own more stocks. As they fall, they want to own less (and do own less due to their reduced value). Makes perfect sense and hard to imagine it working any other way.
BH Consumer FAQ: | Car? Used Toyota, Lexus or Miata. | House? 20% down and 3x salary. | Vacation house? No. | Umbrella? $1 million. | Goods? Costco.
dbr
Posts: 46181
Joined: Sun Mar 04, 2007 8:50 am

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by dbr »

Valuethinker wrote: Mon Jul 22, 2019 6:05 am
So when people say 6% here, do they mean real or nominal?
I would say that the usual convention is that it is nominal unless someone adds the word "real." It is conceivable that in any given conversation the context is real returns, but hopefully that would be clear somewhere. Sometimes it takes some close reading to be sure. Certainly published financial data is almost always just actual prices, meaning nominal, and if it is indexed for inflation it says so somewhere. I think it would be more common on this forum to assume nominal and not even think much about real terms compared to anyone just naturally assuming real terms first.
User avatar
unclescrooge
Posts: 6265
Joined: Thu Jun 07, 2012 7:00 pm

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by unclescrooge »

willthrill81 wrote: Sun Jul 21, 2019 11:48 pm Here's the most recent version of the chart showing the relationship between stock returns and the average investor allocation to stocks.

Image
https://financial-charts.effingapp.com/

I must say that I'm astounded at how close the lines are from 2004 (i.e. 10 years back from when this relationship was initially proposed for predicting 10 year subsequent returns) going forward.
What's the current stock allocation? :mrgreen:
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by willthrill81 »

unclescrooge wrote: Mon Jul 22, 2019 9:58 am What's the current stock allocation? :mrgreen:
According to FRED, 43.6%. That implies forward 10 year nominal returns of 3.58%. :(
The Sensible Steward
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by willthrill81 »

dbr wrote: Mon Jul 22, 2019 8:51 am
Valuethinker wrote: Mon Jul 22, 2019 6:05 am
So when people say 6% here, do they mean real or nominal?
I would say that the usual convention is that it is nominal unless someone adds the word "real." It is conceivable that in any given conversation the context is real returns, but hopefully that would be clear somewhere. Sometimes it takes some close reading to be sure. Certainly published financial data is almost always just actual prices, meaning nominal, and if it is indexed for inflation it says so somewhere. I think it would be more common on this forum to assume nominal and not even think much about real terms compared to anyone just naturally assuming real terms first.
Yes, it is nominal.
The Sensible Steward
Nowizard
Posts: 4842
Joined: Tue Oct 23, 2007 5:33 pm

Re: The Single Greatest Predictor of Future Stock Market Returns

Post by Nowizard »

Interesting that the "Past cannot predict future stock returns," but threads that discuss the topic always get traction. Our approach is based on a simple belief that if the market does not go up in the long-term future, those who have a thoughtful assessment of their portfolio and need for risk will "Lose less" than others, at least in the context of their needs. Sometimes, you make the most by losing the least.
Tim
Post Reply