A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

Let's imagine we have 2 stocks (Stock A and stock B) that have a correlation of 50%.
If we make portfolio A of 50% stock A and 50% stock B, then portfolio A is not 100% correlated with stock A or stock B. Does this mean we can diversify more by adding either stock A or stock B to portfolio A, because they are not 100% correlated?

Another example:
MSCI ACWI IMI has 9.189 holdings. All (or most) of these stocks are not 100% correlated with MSCI ACWI IMI. Does this mean we can diversify more by adding any of the stocks it consists of, because they are not 100% correlated with MSCI ACWI IMI?

More generally:
In almost all cases the stocks a portfolio consists of are not 100% correlated with the portfolio itself. Can we then get a diversification benefit by adding more of any of those that are not 100% correlated with the portfolio?
User avatar
burritoLover
Posts: 4097
Joined: Sun Jul 05, 2020 12:13 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by burritoLover »

I'll pose one to you. If you have 100 stocks and 99 of them are drug manufacturers and 1 stock is a bank and that 1 stock makes up 0.1% of the portfolio, would I increase diversification if I added more of that 1 stock? Assume that each stock is not 100% correlated to the entire portfolio.
sureshoe
Posts: 2165
Joined: Tue Jan 15, 2019 3:26 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by sureshoe »

larsnyborgpedersen wrote: Tue Dec 07, 2021 11:26 am Let's imagine we have 2 stocks (Stock A and stock B) that have a correlation of 50%.
If we make portfolio A of 50% stock A and 50% stock B, then portfolio A is not 100% correlated with stock A or stock B. Does this mean we can diversify more by adding either stock A or stock B to portfolio A, because they are not 100% correlated?

Another example:
MSCI ACWI IMI has 9.189 holdings. All (or most) of these stocks are not 100% correlated with MSCI ACWI IMI. Does this mean we can diversify more by adding any of the stocks it consists of, because they are not 100% correlated with MSCI ACWI IMI?

More generally:
In almost all cases the stocks a portfolio consists of are not 100% correlated with the portfolio itself. Can we then get a diversification benefit by adding more of any of those that are not 100% correlated with the portfolio?
I'll bite. I think "diversify" is a complex word. What you're really doing in your specific example is overweighting a particular asset. If you own Apple and MSCI, you're overweighting Apple, not diversifying. Concentrating assets generally does not make you more diverse. If I put $10k in MSCI and another $10k in some stock that is NOT in the index, I have added to my total stocks. However, with MSCI being lots of stocks, I really have un-diversified my portfolio some because I've created a large concentration in a single holding. I'm a bit sketchy on the exact research, but I remember from business school that you are theoretically diversified after ~7 stocks. Buying a single large index gets you this. Now you get into questions of sectors and asset classes. But either way, if you overweight, you're decreasing (not increasing) diversification.
Last edited by sureshoe on Tue Dec 07, 2021 12:10 pm, edited 2 times in total.
HootingSloth
Posts: 1050
Joined: Mon Jan 28, 2019 2:38 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by HootingSloth »

You might begin by defining what you mean by "more diversified." One place to start is Parmentier, Measures of Portfolio Diversification. It opens:
From a qualitative point of view, the concept of diversification is quite clear: a portfolio is well-diversified if shocks in the individual components do not heavily impact on the overall portfolio. Relatively simple to understand then but profoundly difficult to define. Indeed, there is no broadly accepted precise and quantitative definition of diversification.

The first who proposed a mathematical formalization of diversification in a portfolio selection context was Markowitz in 1952 with his Modern Portfolio Theory. Even if it was path-breaking for that time, it doesn’t clearly provide any proper definition of the term diversification or a specific measure of portfolio diversification.

Over the years, many different measures of diversification have been developed in the literature, each with its pros and cons. In the framework of this thesis, we have chosen to analyze six of them.
Global Market Portfolio + modest tilt towards volatility (80/20->60/40 as approach FI) + modest tilt away from exchange rate risk (80% global+20% U.S. stocks; currency-hedge bonds) + tax optimization
User avatar
nisiprius
Advisory Board
Posts: 52211
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by nisiprius »

In almost all cases the stocks a portfolio consists of are not 100% correlated with the portfolio itself. Can we then get a diversification benefit by adding more of any of those that are not 100% correlated with the portfolio?
No, not in general, because lack of perfect correlation does not guarantee a diversification benefit.

Here is a useful mathematical relationship that is valid if
  • we define "diversification benefit" to mean that the Sharpe ratio of the combined portfolio of A and B is higher than the Sharpe ratio of A alone; that is, adding B has improved the Sharpe ratio, and
  • we are either looking at a known past period of time over which we know exactly what the returns, standard deviations, and correlations were, or we somehow have accurate predictions of what they will be in the future.
Since we can label the two assets either way we like, let use us A for whichever one had the higher Sharpe ratio and B for the other.

Multiply the Sharpe ratio of A by the correlation ρ between A and B.
The portfolio can be improved only if the Sharpe ratio of B is higher than that product.

Consider the following example.

Let A be the Vanguard Total Stock Market Index Fund, VTSMX.
Let B be the Vanguard Emerging Markets Stock Index Fund, VEIEX.

Since inception, they have had a correlation of ρ = 0.76.

It is suggested that perhaps VEIEX would have provided a diversification benefit for VTSMX.

We apply the rule: the Sharpe ratio of VTSMX was 0.60.

Therefore, we need for VEIEX to have had a Sharpe ratio of 0.60 x ρ = 0.60 x 0.76 = 0.456 to obtain a diversification benefit.

But the Sharpe ratio of VEIEX was only 0.28. So it wasn't high enough.

The low correlation was not low enough to overcome the drag of B having a lower Sharpe ratio.

We can test this by comparing a portfolio of 100% VTSMX to, say, 80% VTSMX, 20% VEIEX. The mathematical rule says that adding VEIEX to VTSMX will lower the Sharpe ratio. In fact, we see that

Image

despite imperfect correlation, adding VEIEX did not create a diversification benefit. It managed to lower return (CAGR), while at the same time raising two measures of risk (St Dev and Max Drawdown), with the overall result being lower measures of risk-adjusted return (Sharpe and Sortino ratios).

Imperfect, or low, or even zero, or even negative correlation will not automatically or necessarily create a diversification benefit. The diversifying asset cannot improve "the portfolio as a whole" unless it is a reasonably good investment on its own.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

sureshoe wrote: Tue Dec 07, 2021 12:08 pm I'll bite. I think "diversify" is a complex word. What you're really doing in your specific example is overweighting a particular asset. If you own Apple and MSCI, you're overweighting Apple, not diversifying. Concentrating assets generally does not make you more diverse. If I put $10k in MSCI and another $10k in some stock that is NOT in the index, I have added to my total stocks. However, with MSCI being lots of stocks, I really have un-diversified my portfolio some because I've created a large concentration in a single holding. I'm a bit sketchy on the exact research, but I remember from business school that you are theoretically diversified after ~7 stocks. Buying a single large index gets you this. Now you get into questions of sectors and asset classes. But either way, if you overweight, you're decreasing (not increasing) diversification.
You talk a lot about overweighting, but overweighting compared to what? What is neutral? Do you consider equal weighting stocks as the most diversified or market-cap as the most diversified? Or risk parity?
I'm not necessarily overweighting something, just because I'm putting more money into it.
User avatar
arcticpineapplecorp.
Posts: 15080
Joined: Tue Mar 06, 2012 8:22 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by arcticpineapplecorp. »

larsnyborgpedersen wrote: Tue Dec 07, 2021 3:25 pm You talk a lot about overweighting, but overweighting compared to what? What is neutral? Do you consider equal weighting stocks as the most diversified or market-cap as the most diversified? Or risk parity?
I'm not necessarily overweighting something, just because I'm putting more money into it.
compared to the market. market cap weighting is a mirror of how money is invested/allocated. At one point I think Apple represented 6% of the total US market. That means $6 of every $100 invested in US stocks was invested in Apple at that time (could be different now).

If you put more or less money in one thing or another, or if you equal weight, you are saying you know more than the market has already figured out. Why do you believe you know more than the market knows? You might, you might not. Are you feeling lucky?
It's hard to accept the truth when the lies were exactly what you wanted to hear. Investing is simple, but not easy. Buy, hold & rebalance low cost index funds & manage taxable events. Asking Portfolio Questions | Wiki
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

arcticpineapplecorp. wrote: Tue Dec 07, 2021 3:50 pm
larsnyborgpedersen wrote: Tue Dec 07, 2021 3:25 pm You talk a lot about overweighting, but overweighting compared to what? What is neutral? Do you consider equal weighting stocks as the most diversified or market-cap as the most diversified? Or risk parity?
I'm not necessarily overweighting something, just because I'm putting more money into it.
compared to the market. market cap weighting is a mirror of how money is invested/allocated. At one point I think Apple represented 6% of the total US market. That means $6 of every $100 invested in US stocks was invested in Apple at that time (could be different now).

If you put more or less money in one thing or another, or if you equal weight, you are saying you know more than the market has already figured out. Why do you believe you know more than the market knows? You might, you might not. Are you feeling lucky?
I'm just thinking that the market-cap might not be the most diversified/optimal portfolio.
Investors are not homogeneous and have very different risk preferences, time horizons and goals. Some people are looking for lottery tickets, others want to preserve their wealth.
The market-cap is the aggregate investments of VERY different people, so it might not be the most optimal.
User avatar
arcticpineapplecorp.
Posts: 15080
Joined: Tue Mar 06, 2012 8:22 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by arcticpineapplecorp. »

larsnyborgpedersen wrote: Tue Dec 07, 2021 4:16 pm
arcticpineapplecorp. wrote: Tue Dec 07, 2021 3:50 pm
larsnyborgpedersen wrote: Tue Dec 07, 2021 3:25 pm You talk a lot about overweighting, but overweighting compared to what? What is neutral? Do you consider equal weighting stocks as the most diversified or market-cap as the most diversified? Or risk parity?
I'm not necessarily overweighting something, just because I'm putting more money into it.
compared to the market. market cap weighting is a mirror of how money is invested/allocated. At one point I think Apple represented 6% of the total US market. That means $6 of every $100 invested in US stocks was invested in Apple at that time (could be different now).

If you put more or less money in one thing or another, or if you equal weight, you are saying you know more than the market has already figured out. Why do you believe you know more than the market knows? You might, you might not. Are you feeling lucky?
I'm just thinking that the market-cap might not be the most diversified/optimal portfolio.
Investors are not homogeneous and have very different risk preferences, time horizons and goals. Some people are looking for lottery tickets, others want to preserve their wealth.
The market-cap is the aggregate investments of VERY different people, so it might not be the most optimal.
you can't know what will be optimal in advance. if you could, why wouldn't everyone pick the most optimal and then wouldn't there only be just one optimal portfolio?

the market portfolio is I believe the most "efficient" rather than optimal. You diversify away idiosyncratic risk by diversifying according to market cap. You don't take the sector, stock, size, style, country or manager risks that others take who don't own the market. These risks are often not compensated. In investing you should only take a risk for which you're likely to be compensated for.

When you own the market since you diversify away those risks, the only risk you are left taking is market risk. That's the systemic risk that the market in aggregate declines. That's the risk that you're getting compensated for. You take the market risk, to get the market's return. You get what you paid for.

Having different risk profiles, horizons and goals can be accomplished by holding bonds in addition to stocks. If you want less risk than 100% stock portfolio you hold bonds which lessens your risk and likely lessens your returns. This may be due to a shortened time horizon (no ability to take risk) or you've met your goal (no need to take risk) or less risk preferences (less willingness to take risk). These three factors were mentioned by you (in red above), but they go nicely with an investor's main task which is determining their asset allocation based on need, ability and willingness to take risk:

https://www.cbsnews.com/news/asset-allo ... -you-take/

https://www.cbsnews.com/news/asset-allo ... tolerance/

https://www.cbsnews.com/news/asset-allo ... -you-need/

https://www.cbsnews.com/news/asset-allo ... ing-goals/

you can tilt to small cap (size risk) value (style risk) or hold more tech (sector risk) etc if you want, but you won't know if that was optimal until after the fact. Many people believed in small cap and value which underperformed for a decade or more. Do you have the courage of your convictions? Will you give up in the face of adversity? These are the questions only you can answer.

Most experts (Bernstein, et. al) have said that the optimal portfolio is the one that you can stick with. Because regardless of how you slice and dice, if you don't hold on for dear life, then you probably won't get the returns you seek. Chasing performance rarely wins and investing requires extraordinary patience and intestinal fortitude. Look at the chart by Paul Merriman below. He's a believer and promoter of small cap value premium, but even he knows you've got to hang on for very long periods of underperformance relative to the market in order to get those short bursts of outperformance that make it worth it.

Image

will you stay the course or not?

you can always find a better portfolio than the one you have. there are at least 150 portfolios "better than yours":
https://www.whitecoatinvestor.com/150-p ... han-yours/

which of those 150 will be the best (optimal, right?). there's no way to know that.

People want to do better than the market, yet they can't even get the returns of the market because they market time, chase performance, capitulate, etc.

all the while the return of the market was right there for the taking.

should be easy right? If it's so easy, why do most investors get lower returns than the annual returns of the mutual funds in which they invest? It's not all due to DCA.
It's hard to accept the truth when the lies were exactly what you wanted to hear. Investing is simple, but not easy. Buy, hold & rebalance low cost index funds & manage taxable events. Asking Portfolio Questions | Wiki
luckyducky99
Posts: 415
Joined: Sun Dec 15, 2019 6:47 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by luckyducky99 »

larsnyborgpedersen wrote: Tue Dec 07, 2021 4:16 pm
arcticpineapplecorp. wrote: Tue Dec 07, 2021 3:50 pm
larsnyborgpedersen wrote: Tue Dec 07, 2021 3:25 pm You talk a lot about overweighting, but overweighting compared to what? What is neutral? Do you consider equal weighting stocks as the most diversified or market-cap as the most diversified? Or risk parity?
I'm not necessarily overweighting something, just because I'm putting more money into it.
compared to the market. market cap weighting is a mirror of how money is invested/allocated. At one point I think Apple represented 6% of the total US market. That means $6 of every $100 invested in US stocks was invested in Apple at that time (could be different now).

If you put more or less money in one thing or another, or if you equal weight, you are saying you know more than the market has already figured out. Why do you believe you know more than the market knows? You might, you might not. Are you feeling lucky?
I'm just thinking that the market-cap might not be the most diversified/optimal portfolio.
Investors are not homogeneous and have very different risk preferences, time horizons and goals. Some people are looking for lottery tickets, others want to preserve their wealth.
The market-cap is the aggregate investments of VERY different people, so it might not be the most optimal.
As someone pointed out, there are lots of ways to define diversification. Market-cap weighting is probably pretty good according to most reasonable definitions. So it's a good starting point. It will get you a good risk-adjusted return because it diversifies away much of the risk that can be diversified away. That is a kind of abstract way of looking at things.

Like you said, looking at things from the perspective of an actual investor is something else. Some people want to take on more risk for more potential reward. That is tangentially related to but not closely related to diversification. You still want your high-risk/high-reward portfolio to be diversified as much as you can because otherwise your risk-adjusted return goes down, and the potential payoff for taking on more risk decreases.

The common way of attacking that problem is with leverage. Apply leverage to a diversified portfolio, and you can keep the diversification while increasing the dispersion of possible outcomes. More risk, more potential reward. There are practical limits to how much leverage you can take on though. People who really want to gamble are forced to concentrate their bets, sacrificing diversification. They get more potential reward, but take on much more risk, the dispersion of outcomes increases even more, but the outcomes skew toward being negative. This is the lottery ticket type stuff you describe, though really taking on any undiversified position (overweighting an individual stock, or a sector, or a country, or whatever) has this effect.

There are grey areas. Factors are one. More risk, more potential reward. What about the risk-adjusted return? Does it or doesn't it diversify a market cap weighted portfolio? Lots of arguments about that. I won't go there here.

If the risk adjusted return of the market cap weighted portfolio is too much risk for someone, "people who want to preserve their wealth" they can deleverage with bonds or cash or fixed income. This is the standard advice especially around here, since for most people the amount of risk you get with a 100% equity portfolio, even if it's diversified, is still too much to handle behaviorally.

So when you say "optimal" when you refer to diversification, and "optimal" when you refer to someone who wants to take on more or less risk, you're describing optimizing two different things.
marcopolo
Posts: 8445
Joined: Sat Dec 03, 2016 9:22 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by marcopolo »

This just sounds like another factor investing thread.

camp1: Owning TSM gives us diversification

camp2: Adding SCV to TSM will provide more diversification

camp1: But, SCV is already in TSM

camp2: Yes, but SCV provides independent source of risk/return (i.e., somewhat uncorrelated) compared to TSM

repeat ad nauseum....
Once in a while you get shown the light, in the strangest of places if you look at it right.
psteinx
Posts: 5801
Joined: Tue Mar 13, 2007 2:24 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by psteinx »

To simplify a bit (hopefully):

Diversification is not an end in and of itself, but rather a means, to get lower risk for the same expected return.

i.e. A fund with 30 stock in it will have lower SD than the average component stock, but will have about the same expected return as the average component stock.

You have $30,000 in that fund, which is equal weighted (so ~$1K in each stock). Fine.

Now, you put $1,000 directly into one of the stocks, which of course has a correlation < 1.00 with the fund. BUT, that stock has a higher SD than the fund itself. So the < 1.00 correlation is offset by the > fund_average SD.
User avatar
nisiprius
Advisory Board
Posts: 52211
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by nisiprius »

HootingSloth wrote: Tue Dec 07, 2021 12:08 pm You might begin by defining what you mean by "more diversified." One place to start is Parmentier, Measures of Portfolio Diversification.
This is a major find, thank you. But it's going to take a lot of nibbling away on it before I can absorb much.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
User avatar
Beensabu
Posts: 5657
Joined: Sun Aug 14, 2016 3:22 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by Beensabu »

larsnyborgpedersen wrote: Tue Dec 07, 2021 11:26 am Let's imagine we have 2 stocks (Stock A and stock B) that have a correlation of 50%.
If we make portfolio A of 50% stock A and 50% stock B, then portfolio A is not 100% correlated with stock A or stock B. Does this mean we can diversify more by adding either stock A or stock B to portfolio A, because they are not 100% correlated?
In the stated example, you can diversify more by adding stock C to portfolio A, regardless of correlation. Get it?

Diversification: the process of allocating capital in a way that reduces the exposure to any one particular asset or risk

Diversification available across:

- asset classes (stocks / bonds)
- capitalizations (large / small)
- equity markets (US / exUS)
- equity styles (growth / value)

Fixed income diversification available across:

- bond markets
- credit quality
- currencies
- duration
- nominal vs. inflation-linked

You choose your own adventure. How diversified do you want to be?

Don't look back -- it won't help you.

Except to figure out this: When does each thing tend to shine? When does it tend to plunge into the abyss?

You weigh the potential benefits and disadvantages for yourself, and you find the right balance for you.

A balance that can be maintained through all market / economic conditions.

Screw risk adjusted returns and optimization. What can you hold? What can you actually hold? Because it's not going to work if you can't hold it.

They say you can change things around if life circumstances warrant. Forget that, when you're finding the balance.

What would you pick, if you weren't ever allowed to change it?

If you got one choice, and once you made it, that was it? What would it be?
"The only thing that makes life possible is permanent, intolerable uncertainty; not knowing what comes next." ~Ursula LeGuin
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

arcticpineapplecorp. wrote: Tue Dec 07, 2021 4:34 pm
larsnyborgpedersen wrote: Tue Dec 07, 2021 4:16 pm
arcticpineapplecorp. wrote: Tue Dec 07, 2021 3:50 pm
larsnyborgpedersen wrote: Tue Dec 07, 2021 3:25 pm You talk a lot about overweighting, but overweighting compared to what? What is neutral? Do you consider equal weighting stocks as the most diversified or market-cap as the most diversified? Or risk parity?
I'm not necessarily overweighting something, just because I'm putting more money into it.
compared to the market. market cap weighting is a mirror of how money is invested/allocated. At one point I think Apple represented 6% of the total US market. That means $6 of every $100 invested in US stocks was invested in Apple at that time (could be different now).

If you put more or less money in one thing or another, or if you equal weight, you are saying you know more than the market has already figured out. Why do you believe you know more than the market knows? You might, you might not. Are you feeling lucky?
I'm just thinking that the market-cap might not be the most diversified/optimal portfolio.
Investors are not homogeneous and have very different risk preferences, time horizons and goals. Some people are looking for lottery tickets, others want to preserve their wealth.
The market-cap is the aggregate investments of VERY different people, so it might not be the most optimal.
you can't know what will be optimal in advance. if you could, why wouldn't everyone pick the most optimal and then wouldn't there only be just one optimal portfolio?

the market portfolio is I believe the most "efficient" rather than optimal. You diversify away idiosyncratic risk by diversifying according to market cap. You don't take the sector, stock, size, style, country or manager risks that others take who don't own the market. These risks are often not compensated. In investing you should only take a risk for which you're likely to be compensated for.

When you own the market since you diversify away those risks, the only risk you are left taking is market risk. That's the systemic risk that the market in aggregate declines. That's the risk that you're getting compensated for. You take the market risk, to get the market's return. You get what you paid for.

Having different risk profiles, horizons and goals can be accomplished by holding bonds in addition to stocks. If you want less risk than 100% stock portfolio you hold bonds which lessens your risk and likely lessens your returns. This may be due to a shortened time horizon (no ability to take risk) or you've met your goal (no need to take risk) or less risk preferences (less willingness to take risk). These three factors were mentioned by you (in red above), but they go nicely with an investor's main task which is determining their asset allocation based on need, ability and willingness to take risk:

https://www.cbsnews.com/news/asset-allo ... -you-take/

https://www.cbsnews.com/news/asset-allo ... tolerance/

https://www.cbsnews.com/news/asset-allo ... -you-need/

https://www.cbsnews.com/news/asset-allo ... ing-goals/

you can tilt to small cap (size risk) value (style risk) or hold more tech (sector risk) etc if you want, but you won't know if that was optimal until after the fact. Many people believed in small cap and value which underperformed for a decade or more. Do you have the courage of your convictions? Will you give up in the face of adversity? These are the questions only you can answer.

Most experts (Bernstein, et. al) have said that the optimal portfolio is the one that you can stick with. Because regardless of how you slice and dice, if you don't hold on for dear life, then you probably won't get the returns you seek. Chasing performance rarely wins and investing requires extraordinary patience and intestinal fortitude. Look at the chart by Paul Merriman below. He's a believer and promoter of small cap value premium, but even he knows you've got to hang on for very long periods of underperformance relative to the market in order to get those short bursts of outperformance that make it worth it.

Image

will you stay the course or not?

you can always find a better portfolio than the one you have. there are at least 150 portfolios "better than yours":
https://www.whitecoatinvestor.com/150-p ... han-yours/

which of those 150 will be the best (optimal, right?). there's no way to know that.

People want to do better than the market, yet they can't even get the returns of the market because they market time, chase performance, capitulate, etc.

all the while the return of the market was right there for the taking.

should be easy right? If it's so easy, why do most investors get lower returns than the annual returns of the mutual funds in which they invest? It's not all due to DCA.

How can we know for sure that market-cap is the ONLY allocation that diversifies away idiosyncratic risk? Why does being a few percentage different allocation than market-cap introduce idiosyncratic risk?
Booglie
Posts: 152
Joined: Sun Aug 15, 2021 2:17 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by Booglie »

larsnyborgpedersen wrote: Tue Dec 07, 2021 11:26 am Let's imagine we have 2 stocks (Stock A and stock B) that have a correlation of 50%.
If we make portfolio A of 50% stock A and 50% stock B, then portfolio A is not 100% correlated with stock A or stock B. Does this mean we can diversify more by adding either stock A or stock B to portfolio A, because they are not 100% correlated?

Another example:
MSCI ACWI IMI has 9.189 holdings. All (or most) of these stocks are not 100% correlated with MSCI ACWI IMI. Does this mean we can diversify more by adding any of the stocks it consists of, because they are not 100% correlated with MSCI ACWI IMI?

More generally:
In almost all cases the stocks a portfolio consists of are not 100% correlated with the portfolio itself. Can we then get a diversification benefit by adding more of any of those that are not 100% correlated with the portfolio?
You would need to be more specific. What do you want to do? Reduce risk?

You're already as diversified as you can be with stock A and B on an equal weight approach.

But if you want to have the least risk possible with the most return, then you need to do an efficient frontier simulation, which will calculate historical volatility and returns for stock A and B, and will give you very different ratios. For example, if you find stock A is much safer historically than stock B, the algo may suggest, say, 90% stock A and 10% stock B.

Of course, you would be exposed to a different type of risk, i.e, your portfolio would be vulnerable to stock A going bankrupt.

Assuming they're both from the same sector, buying A and B does nothing at all to protect you from systemic risk, i.e, the market as a whole crashing.

You also have to check historical correlation. If stock A moves exactly the same as stock B, then you're not protecting yourself at all if you're purchasing both. You might as well go all in in just one.

The most effective way to protect yourself while staying fully invested would be to find stocks that move opposite from each other, i.e, one goes up when the other goes down. But good luck finding those assets in this market. Save for assets that are specifically engineered to move opposite to the market (i.e, an inverse ETF), finding assets with inverse correlation is almost impossible at this point.
User avatar
burritoLover
Posts: 4097
Joined: Sun Jul 05, 2020 12:13 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by burritoLover »

arcticpineapplecorp. wrote: Tue Dec 07, 2021 4:34 pm People want to do better than the market, yet they can't even get the returns of the market because they market time, chase performance, capitulate, etc.

all the while the return of the market was right there for the taking.

should be easy right? If it's so easy, why do most investors get lower returns than the annual returns of the mutual funds in which they invest? It's not all due to DCA.
I see you post this info typically as it relates to SCV but I don't believe I've seen you post this in the international threads. Wouldn't a 0-20% of international be "slicing and dicing" and trying to do "better than the market"? Or does the global market have a special exemption in your eyes?
User avatar
arcticpineapplecorp.
Posts: 15080
Joined: Tue Mar 06, 2012 8:22 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by arcticpineapplecorp. »

larsnyborgpedersen wrote: Wed Dec 08, 2021 4:00 am How can we know for sure that market-cap is the ONLY allocation that diversifies away idiosyncratic risk? Why does being a few percentage different allocation than market-cap introduce idiosyncratic risk?
i'm sure someone can explain this much better than i can, but

1. how else do you eliminate stock, size, style, sector, country, manager risks than owning the market? Look at the following chart:

Image

Can you find the optimal portfolio?

Wouldn't the optimal portfolio change from year to year?

How would you determine what 2022's optimal portfolio is based on 2021's optimal portfolio??

2. if it's only a few percentage points, how much is the difference really going to be? (some here say a tilt that isn't at least 10% or more isn't worth it because it's not enough to move the needle).
It's hard to accept the truth when the lies were exactly what you wanted to hear. Investing is simple, but not easy. Buy, hold & rebalance low cost index funds & manage taxable events. Asking Portfolio Questions | Wiki
sureshoe
Posts: 2165
Joined: Tue Jan 15, 2019 3:26 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by sureshoe »

larsnyborgpedersen wrote: Tue Dec 07, 2021 3:25 pm
sureshoe wrote: Tue Dec 07, 2021 12:08 pm I'll bite. I think "diversify" is a complex word. What you're really doing in your specific example is overweighting a particular asset. If you own Apple and MSCI, you're overweighting Apple, not diversifying. Concentrating assets generally does not make you more diverse. If I put $10k in MSCI and another $10k in some stock that is NOT in the index, I have added to my total stocks. However, with MSCI being lots of stocks, I really have un-diversified my portfolio some because I've created a large concentration in a single holding. I'm a bit sketchy on the exact research, but I remember from business school that you are theoretically diversified after ~7 stocks. Buying a single large index gets you this. Now you get into questions of sectors and asset classes. But either way, if you overweight, you're decreasing (not increasing) diversification.
You talk a lot about overweighting, but overweighting compared to what? What is neutral? Do you consider equal weighting stocks as the most diversified or market-cap as the most diversified? Or risk parity?
I'm not necessarily overweighting something, just because I'm putting more money into it.
https://www.investopedia.com/terms/o/ov ... g%20months.
I don't want to argue semantics, but "putting more money into something" is literally the definition of overweighting.

"Neutral" is also an opinion, but generally, market cap is used for weighting purposes.

As for diversification - It depends (as does weighting) at how you're looking at segmenting. There is no uniform agreement on what "diversification is". If I told people I own 100 stocks, am I diversified? Some people would say yes, but most people would say "not if they're all in the same sector". If I said, "oh, they are all different sectors", many people would say I'm diversified. But some people would say, "but what about other asset classes? Bonds, Alternatives, etc." If I said - "Ok, I have added 20% bonds and 5% alternatives", lots of people would consider me well diversified, but there would still be people asking, "How much of that is foreign vs. domestic?"

Simplifying: diversification is an opinion. There are lots of theories on it, on risk, the efficiency frontier, etc. I won't claim to be an expert. However, if you own a disproportionate amount of any asset or asset class, you are overweighting, and generally decreasing your diversification. The only logical reason to overweight is that you believe a particular stock or asset class or whatever is going to outperform the broader market, with the "broader market" being defined as the performance of some whole market index. (others might argue the S&P)
User avatar
burritoLover
Posts: 4097
Joined: Sun Jul 05, 2020 12:13 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by burritoLover »

There's virtually no difference between the S&P 500 and the US total stock market as far as return, standard deviation, max drawdown, sharpe ratio, and correlation over 50 years. We are talking differences of hundredths to a few tenths of a percent. That means historically you are seeing virtually no diversification benefit from adding 3500 small to mid US stocks to a market cap weighted fund.
dbr
Posts: 46181
Joined: Sun Mar 04, 2007 8:50 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by dbr »

You have two assets. You can calculate the expected return and the expected standard deviation of return from the weightings of the two assets, their expected returns, their expected standard deviation of returns, and the expected correlation of returns. Make a chart of those as a function of the weightings and see what it looks like. You can calculate derived statistics such a Sharpe ratio if you want. I would avoid even referring to the words diversify or diversification. If one wants to quibble about the statistical model involved then one can gather real world data and see what that looks like as a backtest or use historical data in a Monte Carlo simulation.
User avatar
arcticpineapplecorp.
Posts: 15080
Joined: Tue Mar 06, 2012 8:22 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by arcticpineapplecorp. »

burritoLover wrote: Wed Dec 08, 2021 7:55 am There's virtually no difference between the S&P 500 and the US total stock market as far as return, standard deviation, max drawdown, sharpe ratio, and correlation over 50 years. We are talking differences of hundredths to a few tenths of a percent. That means historically you are seeing virtually no diversification benefit from adding 3500 small to mid US stocks to a market cap weighted fund.
true, but people are very bad at thinking long term and instead focus on recency.

last year there were posts from people showing the underperformance of s&p500 relative to TSM, mostly I think because Tesla wasn't added to the S&P 500 until later in the year, but was in TSM prior, during it's period of going "to the moon!". So last year that additional diversification helped.

It can go the other way of course when large cap outperforms small/mid.

i say since there's no additional cost to own those extra 3500 small to mid US stocks, why not own them?
It's hard to accept the truth when the lies were exactly what you wanted to hear. Investing is simple, but not easy. Buy, hold & rebalance low cost index funds & manage taxable events. Asking Portfolio Questions | Wiki
User avatar
burritoLover
Posts: 4097
Joined: Sun Jul 05, 2020 12:13 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by burritoLover »

arcticpineapplecorp. wrote: Wed Dec 08, 2021 9:29 am
burritoLover wrote: Wed Dec 08, 2021 7:55 am There's virtually no difference between the S&P 500 and the US total stock market as far as return, standard deviation, max drawdown, sharpe ratio, and correlation over 50 years. We are talking differences of hundredths to a few tenths of a percent. That means historically you are seeing virtually no diversification benefit from adding 3500 small to mid US stocks to a market cap weighted fund.
true, but people are very bad at thinking long term and instead focus on recency.

last year there were posts from people showing the underperformance of s&p500 relative to TSM, mostly I think because Tesla wasn't added to the S&P 500 until later in the year, but was in TSM prior, during it's period of going "to the moon!". So last year that additional diversification helped.

It can go the other way of course when large cap outperforms small/mid.

i say since there's no additional cost to own those extra 3500 small to mid US stocks, why not own them?
The opposite can also happen where you have some of the Big Boys faceplant themselves and you get to ride that loss train all the way to the bottom in TSM. I would agree you might as well hold TSM but smids aren't doing a whole lot for you over time - I would argue considering overweighting small cap value as a viable option to increase diversification - the antithesis of large cap growth that dominates the TSM and a category which tends to do extremely well in recoveries relative to the market.
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

Booglie wrote: Wed Dec 08, 2021 7:01 am You're already as diversified as you can be with stock A and B on an equal weight approach.
How do you know equal weighting is the most diversified? What about market-cap, risk-parity, fundamental weighting or some other approach?
dbr
Posts: 46181
Joined: Sun Mar 04, 2007 8:50 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by dbr »

larsnyborgpedersen wrote: Wed Dec 08, 2021 10:06 am
Booglie wrote: Wed Dec 08, 2021 7:01 am You're already as diversified as you can be with stock A and B on an equal weight approach.
How do you know equal weighting is the most diversified? What about market-cap, risk-parity, fundamental weighting or some other approach?
If you want "diversification" to be a meaningful property of an investment portfolio you have to propose a measurable definition of it. You could propose Sharpe ratio. In the case of diversifiable risk a portfolio is fully diversified when there is no extra risk for the return expected due to single entity risk. I am not an investment professional so someone else could comment on how to measure "diversification."
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

sureshoe wrote: Wed Dec 08, 2021 7:46 am I don't want to argue semantics, but "putting more money into something" is literally the definition of overweighting.

"Neutral" is also an opinion, but generally, market cap is used for weighting purposes.
"putting more money into something" still requires some allocation to be chosen as neutral, and the allocation that is chosen as neutral has big implications for what is considered overweight.

You say that generally market-cap is used for weighting purposes, but why? That's what I'm trying to figure out.
I don't want to consider market-cap as neutral if there isn't any compelling arguments for it. Otherwise I might as well consider risk-parity, equal-weighting, fundamental weighting or something else as neutral.


sureshoe wrote: Wed Dec 08, 2021 7:46 am I won't claim to be an expert. However, if you own a disproportionate amount of any asset or asset class, you are overweighting, and generally decreasing your diversification.
How much is a disproportionate amount?
USA is currently 60.43% in market-cap. That's A LOT if you consider equal-weighting as neutral, but it's perfect if you consider market-cap as the neutral.

But which one is neutral? Market-cap, equal-weighting, risk-parity, fundamentals-weighting or something else? and WHY?
User avatar
burritoLover
Posts: 4097
Joined: Sun Jul 05, 2020 12:13 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by burritoLover »

larsnyborgpedersen wrote: Wed Dec 08, 2021 10:06 am
Booglie wrote: Wed Dec 08, 2021 7:01 am You're already as diversified as you can be with stock A and B on an equal weight approach.
How do you know equal weighting is the most diversified? What about market-cap, risk-parity, fundamental weighting or some other approach?
No one knows what is going to give you the most diversification benefit going forward - period. Correlations, volatility measurements, sharpe ratios, etc. change over time - they can be notably different depending on what time period you look at in the past. What you try to do is diversify across as many different risk assets that have an expected return as you can while considering costs. There isn't a pick X method which is the go-to for the best diversification although market cap is going to be the cheapest from ER and tax considerations, so it is as good default choice.
dbr
Posts: 46181
Joined: Sun Mar 04, 2007 8:50 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by dbr »

Overweighting is always with respect to an arbitrary definition of evenly weighted, whatever one wants to choose. The cap weighted distribution is often chosen, but the implications of any given choice have to be explored in detail.

An example of this kind of scheme would be a Fama-French factor loading on small and value using the definition that the factor loading is defined relative to the cap weighted total market. But it is all about using self-consistent arbitrary definitions. As with any scheme of understanding the issue is whether or not the picture is self-consistent, simple, and useful. That is all there is.
dcabler
Posts: 4541
Joined: Wed Feb 19, 2014 10:30 am
Location: TX

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by dcabler »

I always appreciated this thread, started by vineviz

viewtopic.php?f=10&t=285269

Cheers.
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by willthrill81 »

arcticpineapplecorp. wrote: Wed Dec 08, 2021 7:25 am
larsnyborgpedersen wrote: Wed Dec 08, 2021 4:00 am How can we know for sure that market-cap is the ONLY allocation that diversifies away idiosyncratic risk? Why does being a few percentage different allocation than market-cap introduce idiosyncratic risk?
i'm sure someone can explain this much better than i can, but

1. how else do you eliminate stock, size, style, sector, country, manager risks than owning the market? Look at the following chart:

Image

Can you find the optimal portfolio?

Wouldn't the optimal portfolio change from year to year?

How would you determine what 2022's optimal portfolio is based on 2021's optimal portfolio??

2. if it's only a few percentage points, how much is the difference really going to be? (some here say a tilt that isn't at least 10% or more isn't worth it because it's not enough to move the needle).
It can be easily demonstrated to not be at all necessary to own the entire stock market to essentially eliminate idiosyncratic risk. That has already been pointed out in the thread. For instance, the S&P 500 has tracked TSM and with nearly identical returns over the long-term.

You are correct that there are other risks beyond idiosyncratic risk, but market-cap weighting can only reduce some of those risks. For instance, investing in global stocks still leaves one greatly exposed to single country risk as about 60% of the world's stock value is in a single country (i.e., the U.S). Owning U.S. TSM means that a substantial proportion of one's holding are in big tech stocks. And so on.

The Callan table is arguably one of the very worst ways to present returns of various asset classes, as I described here.

If one accepts that stock returns are not explained only by market beta but also by other factors, then it stands to reason that gaining exposure to multiple factors, not just market beta, increases diversification, and the historic data we have are pretty clear about this, though some have just concerns about the validity of some of those data (e.g., nothing more than a data mined artifact).

If one wants to examine the long-term diversifying impact of various asset classes, it's far better to look at a portfolio's historic start date sensitivity or safe withdrawal rate.

The point you make which is very valid is that deviations from buy-and-hold of TSM often result in a big increase in behavioral risk. But let's not assume that buy-and-hold of TSM has no behavioral risk either; look at how many posts here back in 2008 and 2020, for instance, were people considering 'panic selling'.
The Sensible Steward
User avatar
arcticpineapplecorp.
Posts: 15080
Joined: Tue Mar 06, 2012 8:22 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by arcticpineapplecorp. »

willthrill81 wrote: Wed Dec 08, 2021 10:40 am The point you make which is very valid is that deviations from buy-and-hold of TSM often result in a big increase in behavioral risk. But let's not assume that buy-and-hold of TSM has no behavioral risk either; look at how many posts here back in 2008 and 2020, for instance, were people considering 'panic selling'.
right, but behavioral risk associated with tracking error (as a result of being different from the market and bailing when they don't get market beating returns) is different from behavioral risk of panic selling because their asset allocation with their portfolio of TSM/TBM was simply too aggressive.
It's hard to accept the truth when the lies were exactly what you wanted to hear. Investing is simple, but not easy. Buy, hold & rebalance low cost index funds & manage taxable events. Asking Portfolio Questions | Wiki
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by willthrill81 »

arcticpineapplecorp. wrote: Wed Dec 08, 2021 10:54 am
willthrill81 wrote: Wed Dec 08, 2021 10:40 am The point you make which is very valid is that deviations from buy-and-hold of TSM often result in a big increase in behavioral risk. But let's not assume that buy-and-hold of TSM has no behavioral risk either; look at how many posts here back in 2008 and 2020, for instance, were people considering 'panic selling'.
right, but behavioral risk associated with tracking error (as a result of being different from the market and bailing when they don't get market beating returns) is different from behavioral risk of panic selling because their asset allocation with their portfolio of TSM/TBM was simply too aggressive.
True, those situations are certainly not identical, but the point is that behavioral risk is a problem with any investment strategy.

Further, behavioral risks are not limited to deviations from owning 'the market'. Which poses a greater behavioral risk, bailing on something like SCV when it's underperforming TSM but still providing great returns (i.e., the last decade or so) or bailing on TSM when it has negative real returns for a decade while SCV's returns are still good (i.e., 2000-2009)?
The Sensible Steward
sureshoe
Posts: 2165
Joined: Tue Jan 15, 2019 3:26 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by sureshoe »

larsnyborgpedersen wrote: Wed Dec 08, 2021 10:18 am
sureshoe wrote: Wed Dec 08, 2021 7:46 am I don't want to argue semantics, but "putting more money into something" is literally the definition of overweighting.

"Neutral" is also an opinion, but generally, market cap is used for weighting purposes.
"putting more money into something" still requires some allocation to be chosen as neutral, and the allocation that is chosen as neutral has big implications for what is considered overweight.

You say that generally market-cap is used for weighting purposes, but why? That's what I'm trying to figure out.
I don't want to consider market-cap as neutral if there isn't any compelling arguments for it. Otherwise I might as well consider risk-parity, equal-weighting, fundamental weighting or something else as neutral.
To what end are you asking these questions? There will never be a universal consensus. I quite literally put the generally accepted "equal weighting", which is market cap across lots of indexes. Even there, some indexes and funds tinker with what is meant by market cap. Other indexes might choose earnings or some weird hybrid like the Dow.

I think you need to understand why you care about this specifically beyond just academic philosophizing. If the answer is you want a perfectly diverse portfolio, fine - but to what point? Are you trying to maximize risk adjusted return? There are hoards and hoards of research on all these questions, but you need to figure out the question you're actually trying to answer.
sureshoe wrote: Wed Dec 08, 2021 7:46 am I won't claim to be an expert. However, if you own a disproportionate amount of any asset or asset class, you are overweighting, and generally decreasing your diversification.
larsnyborgpedersen wrote: Wed Dec 08, 2021 10:18 am How much is a disproportionate amount?
USA is currently 60.43% in market-cap. That's A LOT if you consider equal-weighting as neutral, but it's perfect if you consider market-cap as the neutral.

But which one is neutral? Market-cap, equal-weighting, risk-parity, fundamentals-weighting or something else? and WHY?
Again, all you're really doing is questioning statements. The goal here isn't to win some arcane academic debate. What are you trying to achieve?

To answer your question > a disproportionate amount is one where you have more of a stock beyond what you have defined as neutral. In general, that's frequently market cap. In the USA example, are you suggesting you should own 50/50, US and international? If that, then why not own equal parts of each country? US is just 1 of 50 countries, or whatever. That's the problem - you can keep taking it to X degree.

If all you want to know is "why do people use Market Cap", it's generally because it reflects the true state and value of the market while also having a greater proportional impact to market returns/losses. We can argue all day whether Tesla SHOULD have its current value, but the fact is - it DOES have its current value, and as it changes - so does the market. GGB, TISI, and RGS could all quadruple in value (or go bankrupt) and have nowhere near impact to total market returns as a 20% change in TSLA.
sureshoe
Posts: 2165
Joined: Tue Jan 15, 2019 3:26 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by sureshoe »

dbr wrote: Wed Dec 08, 2021 10:24 am Overweighting is always with respect to an arbitrary definition of evenly weighted, whatever one wants to choose. The cap weighted distribution is often chosen, but the implications of any given choice have to be explored in detail.

An example of this kind of scheme would be a Fama-French factor loading on small and value using the definition that the factor loading is defined relative to the cap weighted total market. But it is all about using self-consistent arbitrary definitions. As with any scheme of understanding the issue is whether or not the picture is self-consistent, simple, and useful. That is all there is.
This is a good example - someone is always going to tinker with the definition. They're always going to try to get a "more perfect measure".

I think people struggle (maybe math-people more specifically) with the "there ain't no perfect answer" answer.
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by willthrill81 »

sureshoe wrote: Wed Dec 08, 2021 11:12 am I think people struggle (maybe math-people more specifically) with the "there ain't no perfect answer" answer.
"Perfect is the enemy of good."

The above can be interpreted in multiple ways. Some argue that TSM is 'good enough', and they may be right with regard to themselves at least. Others may choose to invest in factors knowing that their specific AA will not be 'perfect', but they may believe that they will at least have 'better' performance in some regard than owning TSM.
The Sensible Steward
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

willthrill81 wrote: Wed Dec 08, 2021 10:40 am If one wants to examine the long-term diversifying impact of various asset classes, it's far better to look at a portfolio's historic start date sensitivity or safe withdrawal rate.
Can I do this for myself?
User avatar
willthrill81
Posts: 32250
Joined: Thu Jan 26, 2017 2:17 pm
Location: USA
Contact:

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by willthrill81 »

larsnyborgpedersen wrote: Wed Dec 08, 2021 11:22 am
willthrill81 wrote: Wed Dec 08, 2021 10:40 am If one wants to examine the long-term diversifying impact of various asset classes, it's far better to look at a portfolio's historic start date sensitivity or safe withdrawal rate.
Can I do this for myself?
Do you mean 'how'? If so, Portfolio Charts has great tools that allow the measurement of both start date sensitivity and safe withdrawals rates for a wide variety of asset classes going back to 1970.
The Sensible Steward
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

sureshoe wrote: Wed Dec 08, 2021 11:10 am
larsnyborgpedersen wrote: Wed Dec 08, 2021 10:18 am
sureshoe wrote: Wed Dec 08, 2021 7:46 am I don't want to argue semantics, but "putting more money into something" is literally the definition of overweighting.

"Neutral" is also an opinion, but generally, market cap is used for weighting purposes.
"putting more money into something" still requires some allocation to be chosen as neutral, and the allocation that is chosen as neutral has big implications for what is considered overweight.

You say that generally market-cap is used for weighting purposes, but why? That's what I'm trying to figure out.
I don't want to consider market-cap as neutral if there isn't any compelling arguments for it. Otherwise I might as well consider risk-parity, equal-weighting, fundamental weighting or something else as neutral.
To what end are you asking these questions? There will never be a universal consensus. I quite literally put the generally accepted "equal weighting", which is market cap across lots of indexes. Even there, some indexes and funds tinker with what is meant by market cap. Other indexes might choose earnings or some weird hybrid like the Dow.

I think you need to understand why you care about this specifically beyond just academic philosophizing. If the answer is you want a perfectly diverse portfolio, fine - but to what point? Are you trying to maximize risk adjusted return? There are hoards and hoards of research on all these questions, but you need to figure out the question you're actually trying to answer.
I'm asking about this because I've been considering overweighting (where market-cap is chosen as neutral) emerging markets and maybe some specific sectors, because US and the technology sector is REALLY big in market-cap. I'm doing this to hopefully achieve a more reliable outcome. My goal is to have as big a expected return as possible, but still with a reliable outcome. I'm hoping to achieve this with diversification.
But whether I should underweight US and the technology sector depends upon whether market-cap will give me the most reliable outcome and the highest expected return?

Maybe US SHOULD be 60.64% of MSCI ACWI IMI, and maybe information technology SHOULD be 22.63% of MSCI ACWI?
I'm basically asking if there are OTHER benefits to market-cap than low cost and tax efficiency?
Does market-cap have the BEST diversification benefits or come something else give me a more reliable outcome with the same or higher expected return ?
Booglie
Posts: 152
Joined: Sun Aug 15, 2021 2:17 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by Booglie »

larsnyborgpedersen wrote: Wed Dec 08, 2021 10:06 am
Booglie wrote: Wed Dec 08, 2021 7:01 am You're already as diversified as you can be with stock A and B on an equal weight approach.
How do you know equal weighting is the most diversified? What about market-cap, risk-parity, fundamental weighting or some other approach?
I meant to say that if your sole and exclusive criterion is equal weight (i.e, ignoring everything else), then 50% A and 50% B is the most diversified you can be. Please read my explanation below on that post.
User avatar
burritoLover
Posts: 4097
Joined: Sun Jul 05, 2020 12:13 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by burritoLover »

larsnyborgpedersen wrote: Wed Dec 08, 2021 11:35 am
sureshoe wrote: Wed Dec 08, 2021 11:10 am
larsnyborgpedersen wrote: Wed Dec 08, 2021 10:18 am
sureshoe wrote: Wed Dec 08, 2021 7:46 am I don't want to argue semantics, but "putting more money into something" is literally the definition of overweighting.

"Neutral" is also an opinion, but generally, market cap is used for weighting purposes.
"putting more money into something" still requires some allocation to be chosen as neutral, and the allocation that is chosen as neutral has big implications for what is considered overweight.

You say that generally market-cap is used for weighting purposes, but why? That's what I'm trying to figure out.
I don't want to consider market-cap as neutral if there isn't any compelling arguments for it. Otherwise I might as well consider risk-parity, equal-weighting, fundamental weighting or something else as neutral.
To what end are you asking these questions? There will never be a universal consensus. I quite literally put the generally accepted "equal weighting", which is market cap across lots of indexes. Even there, some indexes and funds tinker with what is meant by market cap. Other indexes might choose earnings or some weird hybrid like the Dow.

I think you need to understand why you care about this specifically beyond just academic philosophizing. If the answer is you want a perfectly diverse portfolio, fine - but to what point? Are you trying to maximize risk adjusted return? There are hoards and hoards of research on all these questions, but you need to figure out the question you're actually trying to answer.
I'm asking about this because I've been considering overweighting (where market-cap is chosen as neutral) emerging markets and maybe some specific sectors, because US and the technology sector is REALLY big in market-cap. I'm doing this to hopefully achieve a more reliable outcome. My goal is to have as big a expected return as possible, but still with a reliable outcome. I'm hoping to achieve this with diversification.
But whether I should underweight US and the technology sector depends upon whether market-cap will give me the most reliable outcome and the highest expected return?

Maybe US SHOULD be 60.64% of MSCI ACWI IMI, and maybe information technology SHOULD be 22.63% of MSCI ACWI?
I'm basically asking if there are OTHER benefits to market-cap than low cost and tax efficiency?
Does market-cap have the BEST diversification benefits or come something else give me a more reliable outcome with the same or higher expected return ?
That isn't diversification - that is market timing. Diversification changes should be agnostic to current valuations. Tech wasn't always this concentrated in a market cap portfolio, neither were EM valuations so low. You can't time valuations to get a more reliable outcome.
afan
Posts: 8191
Joined: Sun Jul 25, 2010 4:01 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by afan »

The problem with the Sharpe ratio approach is that you need the future returns on the stocks. Which no one knows.
The variances of the stocks, and even more of a basket of stocks, is more predictable. I look on the value of adding stocks to the portfolio as reducing the idiosyncratic risk of the individual holdings. The more stocks you have, the less impact the wild swings of any one security has on your portfolio. How much reduction you get depends on the volatility of the individual stocks and how much of each you have in the portfolio.
That then leads to ask what should be the proportion of each stock in the portfolio. One could say that the market best estimate of the expected risk adjusted return would be the market portfolio, but that is not necessarily the highest Sharpe ratio.

If you have stock A and you propose to add some stock B, which is not perfectly correlated with A, then, ON AVERAGE you would anticipate a reduction in volatility of the portfolio. However, if stock A is a low volatility utility and stock B is a high volatility penny stock, adding B to A, in any amount, may well increase your volatility. A correlation coefficient of 0.5 means that only 25% of the variance in one is predicted by the variance in the other. That is a low figure, but it is not zero. The stocks are not negatively correlated.

Adding B to A would not necessarily increase the expected return. You would not necessarily expect an increase in risk adjusted return, those would depend on the volatilities, higher moments of the return distribution, returns on each stock and proportions of each stock.

I view "diversified" to mean "matches the market portfolio" since that portfolio has the unique characteristic of being the consensus forecast of optimal risk adjusted return. To deviate from the market portfolio then requires at least one of:
1. special, non-public, knowledge that the deviation will improve performance
OR
2. a utility function that differs from the consensus function of market participants. This requires that you know your own utility function, or at least enough about it to know how and by how much it differs from the market.

Most of us never have any special inside, non-public, knowledge on which is it legal to trade. Very few of us know their utility functions. Therefore, for most of us, the market portfolio is the best we can do ex ante.
The ex post optimal portfolio will almost never be the market portfolio.
We don't know how to beat the market on a risk-adjusted basis, and we don't know anyone that does know either | --Swedroe | We assume that markets are efficient, that prices are right | --Fama
Booglie
Posts: 152
Joined: Sun Aug 15, 2021 2:17 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by Booglie »

burritoLover wrote: Wed Dec 08, 2021 11:53 am
larsnyborgpedersen wrote: Wed Dec 08, 2021 11:35 am
sureshoe wrote: Wed Dec 08, 2021 11:10 am
larsnyborgpedersen wrote: Wed Dec 08, 2021 10:18 am
sureshoe wrote: Wed Dec 08, 2021 7:46 am I don't want to argue semantics, but "putting more money into something" is literally the definition of overweighting.

"Neutral" is also an opinion, but generally, market cap is used for weighting purposes.
"putting more money into something" still requires some allocation to be chosen as neutral, and the allocation that is chosen as neutral has big implications for what is considered overweight.

You say that generally market-cap is used for weighting purposes, but why? That's what I'm trying to figure out.
I don't want to consider market-cap as neutral if there isn't any compelling arguments for it. Otherwise I might as well consider risk-parity, equal-weighting, fundamental weighting or something else as neutral.
To what end are you asking these questions? There will never be a universal consensus. I quite literally put the generally accepted "equal weighting", which is market cap across lots of indexes. Even there, some indexes and funds tinker with what is meant by market cap. Other indexes might choose earnings or some weird hybrid like the Dow.

I think you need to understand why you care about this specifically beyond just academic philosophizing. If the answer is you want a perfectly diverse portfolio, fine - but to what point? Are you trying to maximize risk adjusted return? There are hoards and hoards of research on all these questions, but you need to figure out the question you're actually trying to answer.
I'm asking about this because I've been considering overweighting (where market-cap is chosen as neutral) emerging markets and maybe some specific sectors, because US and the technology sector is REALLY big in market-cap. I'm doing this to hopefully achieve a more reliable outcome. My goal is to have as big a expected return as possible, but still with a reliable outcome. I'm hoping to achieve this with diversification.
But whether I should underweight US and the technology sector depends upon whether market-cap will give me the most reliable outcome and the highest expected return?

Maybe US SHOULD be 60.64% of MSCI ACWI IMI, and maybe information technology SHOULD be 22.63% of MSCI ACWI?
I'm basically asking if there are OTHER benefits to market-cap than low cost and tax efficiency?
Does market-cap have the BEST diversification benefits or come something else give me a more reliable outcome with the same or higher expected return ?
That isn't diversification - that is market timing. Diversification changes should be agnostic to current valuations. Tech wasn't always this concentrated in a market cap portfolio, neither were EM valuations so low. You can't time valuations to get a more reliable outcome.
Sureshoe described this pretty well. No diversification system is perfect, and each has some sort of tradeoff.
For example, you can cheaply diversify with an S&P ETF, as they are composed of the 500 largest companies. But because lots of people do the same and due to the sheer amount of leverage on SPY derivatives, SPY ends up behaving as a single asset. So, while in theory you are diversified, in practice you aren't so much as you would think.

There are many other ways you could diversify, such as: inverse correlations (e.g, asset hedges); precious metals; treasuries, and so on. But inverse ETFs will penalize your performance; precious metals will increase volatility while not protecting you at all; treasuries will expose you to credit risk (no matter how small); and so on.

Ultimately, you can reduce risk significantly, but you can't completely eliminate it. No matter how small, there is always a chance you can lose money somewhere, somehow (systemic risk). Also, some diversification strategies might even hurt your performance instead of help, all while increasing costs. So you have to pick a method that you think is "good enough", depending on what exactly you want.
sureshoe
Posts: 2165
Joined: Tue Jan 15, 2019 3:26 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by sureshoe »

larsnyborgpedersen wrote: Wed Dec 08, 2021 11:35 am
I'm asking about this because I've been considering overweighting (where market-cap is chosen as neutral) emerging markets and maybe some specific sectors, because US and the technology sector is REALLY big in market-cap. I'm doing this to hopefully achieve a more reliable outcome. My goal is to have as big a expected return as possible, but still with a reliable outcome. I'm hoping to achieve this with diversification.
But whether I should underweight US and the technology sector depends upon whether market-cap will give me the most reliable outcome and the highest expected return?

Maybe US SHOULD be 60.64% of MSCI ACWI IMI, and maybe information technology SHOULD be 22.63% of MSCI ACWI?
I'm basically asking if there are OTHER benefits to market-cap than low cost and tax efficiency?
Does market-cap have the BEST diversification benefits or come something else give me a more reliable outcome with the same or higher expected return ?
So that becomes the multi-trillion dollar question. You're effectively looking to outperform a broad market index in terms of risk adjust returns. There is not going to be an answer. There are people on these boards who think you should always be 70/30. There are people who think you should be 100% US. The problem is always going to be that there is no definitive, empirical evidence to say who is correct. It's all after-the-fact and for a selected time period.

The main benefit to market cap is that it reflects the actual value of things today. It builds in all the future-looking variables. That is the closest to neutral you're going to get. And I assure you, there are people who disagree with me.
Thesaints
Posts: 5108
Joined: Tue Jun 20, 2017 12:25 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by Thesaints »

If stock A and B are not 50% correlated (i.e. +0.5), but are perfectly inversely correlated (i.e. -1), the 50/50 portfolio would have zero volatility, which we can fairly equate to it being perfectly diversified.
If I add more stock A, or B, to that portfolio does the diversification go up, or down ?
Scrooge McDuck
Posts: 36
Joined: Mon Apr 10, 2017 11:49 am
Contact:

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by Scrooge McDuck »

larsnyborgpedersen wrote: Wed Dec 08, 2021 11:35 am I'm basically asking if there are OTHER benefits to market-cap than low cost and tax efficiency?
Does market-cap have the BEST diversification benefits or come something else give me a more reliable outcome with the same or higher expected return ?
As I understand it, market-cap weighting is optimal under the Capital Asset Pricing Model (CAPM). But CAPM is highly idealized and depends on various assumptions that do not hold in the real world.
FactualFran
Posts: 2776
Joined: Sat Feb 21, 2015 1:29 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by FactualFran »

larsnyborgpedersen wrote: Wed Dec 08, 2021 10:06 am How do you know equal weighting is the most diversified? What about market-cap, risk-parity, fundamental weighting or some other approach?
The weighting that is most diversified depends on how portfolio diversification is measured. The Measures of Portfolio Diversification thesis that HootingSloth provided a link to examines a number of measures of portfolio diversification. With one of them, Shannon Entropy, the equal weighted portfolio is the most diversified.
afan
Posts: 8191
Joined: Sun Jul 25, 2010 4:01 pm

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by afan »

Scrooge McDuck wrote: Wed Dec 08, 2021 1:34 pm
larsnyborgpedersen wrote: Wed Dec 08, 2021 11:35 am I'm basically asking if there are OTHER benefits to market-cap than low cost and tax efficiency?
Does market-cap have the BEST diversification benefits or come something else give me a more reliable outcome with the same or higher expected return ?
As I understand it, market-cap weighting is optimal under the Capital Asset Pricing Model (CAPM). But CAPM is highly idealized and depends on various assumptions that do not hold in the real world.
The argument for market cap weighting does not depend on CAPM.
We don't know how to beat the market on a risk-adjusted basis, and we don't know anyone that does know either | --Swedroe | We assume that markets are efficient, that prices are right | --Fama
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

burritoLover wrote: Wed Dec 08, 2021 11:53 am
larsnyborgpedersen wrote: Wed Dec 08, 2021 11:35 am
sureshoe wrote: Wed Dec 08, 2021 11:10 am
larsnyborgpedersen wrote: Wed Dec 08, 2021 10:18 am
sureshoe wrote: Wed Dec 08, 2021 7:46 am I don't want to argue semantics, but "putting more money into something" is literally the definition of overweighting.

"Neutral" is also an opinion, but generally, market cap is used for weighting purposes.
"putting more money into something" still requires some allocation to be chosen as neutral, and the allocation that is chosen as neutral has big implications for what is considered overweight.

You say that generally market-cap is used for weighting purposes, but why? That's what I'm trying to figure out.
I don't want to consider market-cap as neutral if there isn't any compelling arguments for it. Otherwise I might as well consider risk-parity, equal-weighting, fundamental weighting or something else as neutral.
To what end are you asking these questions? There will never be a universal consensus. I quite literally put the generally accepted "equal weighting", which is market cap across lots of indexes. Even there, some indexes and funds tinker with what is meant by market cap. Other indexes might choose earnings or some weird hybrid like the Dow.

I think you need to understand why you care about this specifically beyond just academic philosophizing. If the answer is you want a perfectly diverse portfolio, fine - but to what point? Are you trying to maximize risk adjusted return? There are hoards and hoards of research on all these questions, but you need to figure out the question you're actually trying to answer.
I'm asking about this because I've been considering overweighting (where market-cap is chosen as neutral) emerging markets and maybe some specific sectors, because US and the technology sector is REALLY big in market-cap. I'm doing this to hopefully achieve a more reliable outcome. My goal is to have as big a expected return as possible, but still with a reliable outcome. I'm hoping to achieve this with diversification.
But whether I should underweight US and the technology sector depends upon whether market-cap will give me the most reliable outcome and the highest expected return?

Maybe US SHOULD be 60.64% of MSCI ACWI IMI, and maybe information technology SHOULD be 22.63% of MSCI ACWI?
I'm basically asking if there are OTHER benefits to market-cap than low cost and tax efficiency?
Does market-cap have the BEST diversification benefits or come something else give me a more reliable outcome with the same or higher expected return ?
That isn't diversification - that is market timing. Diversification changes should be agnostic to current valuations. Tech wasn't always this concentrated in a market cap portfolio, neither were EM valuations so low. You can't time valuations to get a more reliable outcome.
I'm not doing this because of valuations, but just that I think 60% for US is too much for one country and 22% for technology is too much for one sector.
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

afan wrote: Wed Dec 08, 2021 11:57 am The problem with the Sharpe ratio approach is that you need the future returns on the stocks. Which no one knows.
The variances of the stocks, and even more of a basket of stocks, is more predictable. I look on the value of adding stocks to the portfolio as reducing the idiosyncratic risk of the individual holdings. The more stocks you have, the less impact the wild swings of any one security has on your portfolio. How much reduction you get depends on the volatility of the individual stocks and how much of each you have in the portfolio.
That then leads to ask what should be the proportion of each stock in the portfolio. One could say that the market best estimate of the expected risk adjusted return would be the market portfolio, but that is not necessarily the highest Sharpe ratio.

If you have stock A and you propose to add some stock B, which is not perfectly correlated with A, then, ON AVERAGE you would anticipate a reduction in volatility of the portfolio. However, if stock A is a low volatility utility and stock B is a high volatility penny stock, adding B to A, in any amount, may well increase your volatility. A correlation coefficient of 0.5 means that only 25% of the variance in one is predicted by the variance in the other. That is a low figure, but it is not zero. The stocks are not negatively correlated.

Adding B to A would not necessarily increase the expected return. You would not necessarily expect an increase in risk adjusted return, those would depend on the volatilities, higher moments of the return distribution, returns on each stock and proportions of each stock.

I view "diversified" to mean "matches the market portfolio" since that portfolio has the unique characteristic of being the consensus forecast of optimal risk adjusted return. To deviate from the market portfolio then requires at least one of:
1. special, non-public, knowledge that the deviation will improve performance
OR
2. a utility function that differs from the consensus function of market participants. This requires that you know your own utility function, or at least enough about it to know how and by how much it differs from the market.

Most of us never have any special inside, non-public, knowledge on which is it legal to trade. Very few of us know their utility functions. Therefore, for most of us, the market portfolio is the best we can do ex ante.
The ex post optimal portfolio will almost never be the market portfolio.
But how can the market portfolio be the optimal risk-adjusted return, when the people trying to win have different risk preferences, goals and so on?
If all people were the same, then the market portfolio would produce something optimal for everyone, but if 90% of people want lottery tickets, then those lottery tickets become overpriced, and thus the market portfolio is not optimal for the 10% who don't want lottery tickets. They'll get much better risk-adjusted returns by buying the stocks that are NOT lottery tickets.
Topic Author
lnp
Posts: 166
Joined: Tue Aug 17, 2021 2:33 am

Re: A logic puzzle about diversification and non-correlated portfolio's. Can you solve it?

Post by lnp »

sureshoe wrote: Wed Dec 08, 2021 12:59 pm
larsnyborgpedersen wrote: Wed Dec 08, 2021 11:35 am
I'm asking about this because I've been considering overweighting (where market-cap is chosen as neutral) emerging markets and maybe some specific sectors, because US and the technology sector is REALLY big in market-cap. I'm doing this to hopefully achieve a more reliable outcome. My goal is to have as big a expected return as possible, but still with a reliable outcome. I'm hoping to achieve this with diversification.
But whether I should underweight US and the technology sector depends upon whether market-cap will give me the most reliable outcome and the highest expected return?

Maybe US SHOULD be 60.64% of MSCI ACWI IMI, and maybe information technology SHOULD be 22.63% of MSCI ACWI?
I'm basically asking if there are OTHER benefits to market-cap than low cost and tax efficiency?
Does market-cap have the BEST diversification benefits or come something else give me a more reliable outcome with the same or higher expected return ?
So that becomes the multi-trillion dollar question. You're effectively looking to outperform a broad market index in terms of risk adjust returns. There is not going to be an answer. There are people on these boards who think you should always be 70/30. There are people who think you should be 100% US. The problem is always going to be that there is no definitive, empirical evidence to say who is correct. It's all after-the-fact and for a selected time period.

The main benefit to market cap is that it reflects the actual value of things today. It builds in all the future-looking variables. That is the closest to neutral you're going to get. And I assure you, there are people who disagree with me.
But doesn't the competition among investors just mean that all assets have the same risk-adjusted returns? Because if that's true, than nothing truly days market-cap should provide the optimal risk-adjusted returns?
Also people have different risk preferences, time horizons and goals, so why should the market cap provide the optimal portfolio for anyone?
Post Reply