Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

About the 1871-1919 period, after reading Homer's chapter on the 19th century to complement the 20th century's chapter, this was really a different time... A very hectic time with severe ups and downs and a fledgling banking system overly influenced by political battles and immature markets.

By then government bonds were nascent and NOT viewed as a solid yardstick. Corresponding data is sketchy, larges government purchases distorted yields and premiums, therefore Shiller's mapping of Long-Term bonds to municipal bonds (much better documented and respected) appears to be the right choice. Homer also indicates (without elaborating) that tax exemption didn't distort the municipal bonds yields as they do today. Note that municipals historical returns came from Frederick Macaulay's work (yes, the Macaulay of the corresponding duration). I just can't find how to get access to a copy of Macaulay's book ("Average of New England Municipals"), but we have the corresponding (monthly) numbers in FRED.

As to short-term rates, they were varying widely, partly driven by seasonality (e.g. high rates during fall/xmas time), in a way totally disconnected from (more stable) long-term interest rates. Commercial paper was the yardstick of the day, but we may want to use annual averages instead of December rates to avoid distortions.

Back to the first half of the 20th century... I am still waiting for my copy of the Ibbotson's book about the Yield Curve. I saw an interesting comment from Homer though (validated by a couple of charts) indicating that the yield curve varied much more between bills and 10y maturities than between 10y, 20y and 30y maturities. This means that the current coarse assumption we currently have in the bond fund simulator of equating 10y to 30y yields to the 'Long-Term' number of the day (well, the month!) might be fairly reasonable, even if the average maturity associated with such LT bonds is rather uncertain and varying in time. And then we should apply the same logic for the end of the 19th century.
=> maybe this is where Prof. Shiller was coming from by associating such LT rates to a 10y maturity for his 'RLONG' series.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

Fascinating.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

Finally received "The Bible". The 1993 edition of "The Historical US Treasury Yield Curve" (Coleman, Fisher, Ibbotson). Which is probably more or less the same data as what CRSP makes available to academics and government agencies...

We can notably find Table 7-2, a nice synthetic table with end of December Par Bond Yields for fully taxable treasury bonds, between 1941 and 1992, for 1y, 2y, 5y, 10y, 20y maturities. Just that would be great progress as input to our simulations to complement FRED CMT data.

There are actually more detailed tables:
a) similar numbers (End of December) for 1925 to 1940, but then there are various considerations between fully tax-exempt and partially tax-exempt (PTE) bonds, as fully taxable bonds didn't exist by then (as Homer's book did describe). Shiller was quoting PTE rates from Homer for this time period, by the way.
b) full end-of-month series (with a few holes) for 1y, 2y, 3y, 4y, 5y, 7y, 10y, 15y, 20y and 'LONG' for both estimated Spot rates and Par Bond Yields. 'LONG' means the longest maturity available for a given date, which usually ranges between 20y to 30y, but is sometimes shorter, sometimes longer.

And then there are plenty of explanations about the math of bonds, how to derive yields, spot rates, etc. Will take me a while to unpack!

A quick check comparing Table 7/2 and the FRED CMT daily series showed that there is good alignment, but very often with a difference of a few points (e.g. up to 0.05 deltas). Now I have a dumb question... Is there any meaningful difference between a "Par Bond Yield" and a "Constant Maturity Treasury" rate?
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Thu Sep 03, 2020 4:25 pm Is there any meaningful difference between a "Par Bond Yield" and a "Constant Maturity Treasury" rate?
"Constant Maturity" means that the yield is estimated at specific "constant" maturities, like 10 years, even if there's no bond maturing in exactly 10 years. "Par bond" means that the bond's current price is equal to par (face value).

The 10-Year Treasury Constant Maturity Rate (DGS10) | FRED | St. Louis Fed page has a link to the Treasury Yield Curve Methodology:
The Treasury's yield curve is derived using a quasi-cubic hermite spline function. Our inputs are indicative, bid-side market quotations (not actual transactions) for the on-the-run securities obtained by the Federal Reserve Bank of New York at or near 3:30 PM each trading day. Because the on-the-run securities typically trade close to par, those securities are designated as the knot points in the quasi-cubic hermite spline algorithm and the resulting yield curve is considered a par curve. [...]
Wikipedia tells us that "an on the run security or contract is the most recently issued, and hence most liquid, of a periodically issued security."
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
Kevin M
Posts: 15789
Joined: Mon Jun 29, 2009 3:24 pm
Contact:

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by Kevin M »

siamond wrote: Thu Sep 03, 2020 4:25 pm Now I have a dumb question... Is there any meaningful difference between a "Par Bond Yield" and a "Constant Maturity Treasury" rate?
It's not a dumb question.

As longinvest notes, the Treasury says that the CMT yield curve is considered a par curve. However, on-the-run Treasuries do not trade exactly at par--they aren't even issued at par! For example, the most recently auctioned 30-year bond was priced at 99.243433, with a yield of 1.406% and coupon rate of 1.375%.

I wish that Treasury told us more about their "quasi-cubic hermite spline algorithm". Does their algorithm explicitly attempt to generate par yields, or just a yield that will be close to what the actual Treasury would be trading at if it had a maturity of exactly that that is shown in the table?

There are other sources of explicit par (and spot) yields, such as the FRB, and they provide yields for each year of maturity, not just selected years. When I've checked in the past, the CMT yields are close but not exactly the same as the FRB yields, but FRB uses off-the-run Treasuries as inputs to its algorithm, so that would introduce some differences.

At any rate, I follow the Treasury's guidance, and consider the CMT yields to be par yields--at least good enough for the kinds of things we do with them.

Kevin
If I make a calculation error, #Cruncher probably will let me know.
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

Kevin M wrote: Thu Sep 03, 2020 6:01 pm[...] At any rate, I follow the Treasury's guidance, and consider the CMT yields to be par yields--at least good enough for the kinds of things we do with them.
Just to clarify, the 'Yield Curves' book I just received is a mathematical model, reconstructing yields for each month and each (fixed) maturity in a systematic manner from a number of irregular data points (real life bonds issued in the past). The authors call it 'Estimated Par Bond Yields'. This seems pretty close in nature to what the Fed now calls CMT rates, isn't it? The book I have was issued in 1993, its first edition in 1955, maybe the CMT terminology didn't exist by then...

Anyhoo, this is what we have and this is undoubtedly a much better input than Shiller/Homer aggregate numbers for the 1941-1953 time period (and probably 1925-1940 too). I'll give it a go tomorrow, entering December yields from 1940 to 1992 from Table 7-2, and see if simulated bond fund returns in the 70s and 80s look reasonably similar to what we have with the FRED CMT rates. I suspect they will. Then we'll be on more solid ground to possibly decide to splice the early decades with the FRED CMT numbers...

EDIT: I ran my test, it worked well. It actually incrementally improved the Telltale charts against 30-10 and 5-1 Barclays indices (which start in the mid-70s). Next step: track down the 'long maturity' yields in the monthly tables and use them as an improved proxy for 30y treasuries.
User avatar
Kevin M
Posts: 15789
Joined: Mon Jun 29, 2009 3:24 pm
Contact:

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by Kevin M »

siamond wrote: Thu Sep 03, 2020 11:19 pm
Kevin M wrote: Thu Sep 03, 2020 6:01 pm[...] At any rate, I follow the Treasury's guidance, and consider the CMT yields to be par yields--at least good enough for the kinds of things we do with them.
Just to clarify, the 'Yield Curves' book I just received is a mathematical model, reconstructing yields for each month and each (fixed) maturity in a systematic manner from a number of irregular data points (real life bonds issued in the past). The authors call it 'Estimated Par Bond Yields'. This seems pretty close in nature to what the Fed now calls CMT rates, isn't it? The book I have was issued in 1993, its first edition in 1955, maybe the CMT terminology didn't exist by then...
<snip>
All such yield curves are models. All the ones I've looked at use actual yields as inputs to an algorithm that generates the modeled par or spot yields for specific maturities.

I think they all could be called constant maturity yield curves, which for Treasuries would be CMT. All of the ones I've seen generate yields for fixed maturities, like 1, 2, 3, etc. years, even though those exact maturities usually are not available. Since the same maturities are always used, they are constant maturity yields.

My only question is whether or not the Treasury CMT algorithm is designed to generate exactly par yields.

For example, the on-the-run 20-year Treasury is CUSIP 912810SQ2, with maturity 8/15/2040 and coupon rate 1.125%, but at Fidelity I see a bid price of 97.546 with bid yield of 1.265%. So does the CMT algorithm just adjust for the maturity being slightly less than 20 years, or also for the price being less than 100 (par)? For reference, the 20y CMT yield for today is 1.25%.

Par yield curves can be generated with off-the-run Treasuries, which is what the FRB par yield algorithm does, so the fact that the Treasury CMT algorithm uses on-the-run Treasuries (trading close to, but not at par) as inputs does not in itself define it as a par yield curve. You could use the same inputs to model a spot yield curve.

Again, I don't think it matters much, as the yields will be close enough to par yields for our purposes.

Kevin
If I make a calculation error, #Cruncher probably will let me know.
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

After reading more carefully the 1925 to 1940 commentary in the "Yield Curves" book, there is unfortunately not much we can use here, as the (estimated) par bond yield data is very discontinuous:
- 1925 to 1932: partially-tax-exempt
- 1930 to 1942: fully tax-exempt
- 1940 to 1992: fully taxable (1940 data is incomplete)

Not only are the series of a very different nature, but checking the numbers for overlapping years, the numerical discontinuity is way too large for comfort, so I don't think we have anything usable here up to 1940, which is unfortunate. Starting from 1941, we have full End-of-December yields for taxable treasuries, which is very useful.

Preliminary conclusions:

- the 1941 to 1961 yields (Table 7-2: 1y, 2y, 5y, 10y, 20y; end of December) would be great to complement the FRED CMT daily data which starts in 1962

- the 1941 to 1977 'long maturity' yields (monthly tables; end of December) could be a half-decent proxy for missing 30y FRED CMT data (which starts in 1978) --- better than 20y yields
=> here 'long maturity' means 20y to 30y (sometimes 40y), depending on real life bond availability

- before that, we have to fall back to the data previously discussed (e.g. commercial paper and aggregate 'long-term' bonds from FRED/Homer)
=> here 'long term' means an aggregate of bonds over 8y or 12y of remaining maturity, depending on the exact year, which I'd suggest to use as a flat proxy for 10y to 30y of maturity

- such approach would eliminate other discontinuities (e.g. no need for FRED CMT monthly series from 1953 to 1961; nor FRED 3-5y issues from 1940 to 1952)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

Time to reach closure on those new historical data sources. Here is my complete proposal (year XXXX means beginning of year in the following, all series are monthly, either monthly average or end-of-month values):

1871 to 1940

* 1 year rate from FRED Commercial Paper 4-6m rates
=> see the APY/BEY discussion in following posts

* 10+ years rate from Prof. Shiller's FRED sources for his 'RLONG' series (i.e. New England municipals until 1899; High-Grade municipals until 1918; Long-Term United States Bonds 1919+ series afterwards)

1941-1961

* 1y, 2y, 3y, 4y, 5y, 7y, 10y, 15y, 20y yields from 'Yield Curves' book (end-of-month tables, estimated PAR bond yields, fully taxable)

* 30y proxied to 'Long Maturity' yields from 'Yield Curves' book (end-of-month tables, estimated PAR bond yields, fully taxable)

1962-Now

* FRED CMT Daily series when possible (Dec 31st rates)

* 30y proxied to 'Long Maturity' yields from 'Yield Curves' book to fill the 1962 to 1977 gap

I updated my NBER/FRED spreadsheet, adding a 'Yields for Simulator' tab where I tallied the FRED series. Note that the 'FRED CMT Daily' numbers were copied by value from the 1.25 simulator.

I also created a Yields Curve spreadsheet, where I tallied the 1941+ monthly series from the 'Yield Curves' book and added some simple logic to fill a few gaps.

Longinvest, if you agree with my proposal, the next step is up to you... Feel free to recompute directly from FRED sources if you prefer.
Last edited by siamond on Thu Sep 17, 2020 10:10 pm, edited 2 times in total.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Thu Sep 10, 2020 5:18 pm 1871 to 1940
* 1 year rate from FRED Commercial Paper 4-6m rates (12 month average until 1919; December N-1 average afterwards)
=> I computed the 12m average for year N as the monthly average of 2nd semester year N-1 and 1st semester year N (as we seek an end-of-year proxy)
I think it's important to justify our choices.

The 1-year yield can serve two distinct purposes:
  1. Establish the price of a bond maturing in 1 year given its coupon and face value.
  2. Establish the internal rate of return of a one-year investment into a 1-year (or shorter) bond.
Unfortunately, in early periods (1871-...), we only have commercial paper 4-6m yields. $1,000 invested into commercial paper in December would have matured in (April to) June, 6-month later, and would have needed to be reinvested for an additional 6 months in June to become a 1-year investment. So, it would make sense to create a synthetic 1-year yield using the compound return of two half-year investments using December (D) and June (J) yields: (((1 + D) X (1 + J)) - 1). Note that D and J must be recovered from the annualized yields: D = (((1 + annualized December yield)^(1 / 2)) - 1).

There's a small annoying aspect to this, because it uses information from the future (it uses a June 1871 yield to establish a 1-year yield in December 1870), but I think it's good enough for periods where proper 1-year yields are missing.

Using this synthetic 1-year yield to establish the price of a bond maturing in 1 year given its coupon and face value isn't perfect, either, but I think it's good enough. (A bond holder, seeing the bond undervalued in December, could have decided to hold onto a few more months, it instead of selling it at a low price).

What do you think?
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Sat Sep 12, 2020 8:28 amSo, it would make sense to create a synthetic 1-year yield using the compound return of two half-year investments using December (D) and June (J) yields: (((1 + D) X (1 + J)) - 1). Note that D and J must be recovered from the annualized yields: D = (((1 + annualized December yield)^(1 / 2)) - 1). [...]
Yes, good point about compounding two 6m bills in a row, I actually did exactly that for Simba with the 3m T-bills.

Just a detail, shouldn't the "D" derivation be arithmetic (e.g. D/2 instead of (1+D)^(1/2)-1) instead of geometric? This always confuses me, but I thought this was the way bond rates worked. Because the annualized value is derived by the actual rate (the one matching the distribution frequency) by arithmetic means... So we need to reverse it the same way... Did I get that right?
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Sat Sep 12, 2020 12:05 pm
longinvest wrote: Sat Sep 12, 2020 8:28 amSo, it would make sense to create a synthetic 1-year yield using the compound return of two half-year investments using December (D) and June (J) yields: (((1 + D) X (1 + J)) - 1). Note that D and J must be recovered from the annualized yields: D = (((1 + annualized December yield)^(1 / 2)) - 1). [...]
Yes, good point about compounding two 6m bills in a row, I actually did exactly that for Simba with the 3m T-bills.

Just a detail, shouldn't the "D" derivation be arithmetic (e.g. D/2 instead of (1+D)^(1/2)-1) instead of geometric? This always confuses me, but I thought this was the way bond rates worked. Because the annualized value is derived by the actual rate (the one matching the distribution frequency) by arithmetic means... So we need to reverse it the same way... Did I get that right?
A $1,000 (par) bond maturing in 10 years with a $1,000 face value pays a $10 coupon every 6 months. It's probably for simplicity that the coupon rate is expressed as a ((2 X coupon) / face value) = 2% rate. But, the effective annualized yield is (((1 + 1%)^2) - 1) = 2.01%.

I was assuming that FRED published yields... It might be a good idea to check my assumptions.

If I pick FRED's 10-Year Treasury Constant Maturity Rate (I'm too lazy to track commercial paper detailed information), the title contains the word "rate", but later, the page says "For further information regarding treasury constant maturity data, please refer to the H.15 Statistical Release notes and Treasury Yield Curve Methodology". Footnote 3 of the H.15 document says that yields are "[a]nnualized using a 360-day year or bank interest".

The average year has 365.25 days. Using my example above, the "360-day" yield would be ((1 + 1%)^(360 / (365.25 / 2)) - 1) = 1.98%. Yet, the effective annualized yield (for a full year) is 2.01%.

The Bond fund simulator spreadsheet assumes that yields are annualized on a complete year basis (365 days or, for leap years, 366 days).

So, we're discussing about a difference of a few basis points between assumptions (coupon rate, annualized (365 or 366 days) yield, annualized (360 days) yield). You can pick whatever you prefer.
Last edited by longinvest on Sat Sep 12, 2020 1:35 pm, edited 10 times in total.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

Ok, I went for ((1 + D/2) x (1 + J/2) - 1) for now. Seemed more logical. I updated my historical spreadsheet accordingly.

PS. this is an NBER series, it's just that FRED made it available on its Web site, so we probably shouldn't assume this is the same logic as FRED CMT series. The best description I have (which isn't much) can be found here. I'll see if I can find a more detailed paper somewhere...

EDIT: fixed link.
Last edited by siamond on Mon Sep 14, 2020 8:55 am, edited 1 time in total.
User avatar
Kevin M
Posts: 15789
Joined: Mon Jun 29, 2009 3:24 pm
Contact:

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by Kevin M »

siamond wrote: Sat Sep 12, 2020 12:05 pm
longinvest wrote: Sat Sep 12, 2020 8:28 amSo, it would make sense to create a synthetic 1-year yield using the compound return of two half-year investments using December (D) and June (J) yields: (((1 + D) X (1 + J)) - 1). Note that D and J must be recovered from the annualized yields: D = (((1 + annualized December yield)^(1 / 2)) - 1). [...]
<snip>
Just a detail, shouldn't the "D" derivation be arithmetic (e.g. D/2 instead of (1+D)^(1/2)-1) instead of geometric? This always confuses me, but I thought this was the way bond rates worked. Because the annualized value is derived by the actual rate (the one matching the distribution frequency) by arithmetic means... So we need to reverse it the same way... Did I get that right?
I believe you are correct. Here is what Treasury says (underlines mine for emphasis):
Treasury wrote:Are the CMT yields annual yields?

CMT yields are read directly from the Treasury's daily yield curve and represent "bond equivalent yields" for securities that pay semiannual interest, which are expressed on a simple annualized basis. This is consistent with market practices for quoting bond yields in the market and makes the CMT yields directly comparable to quotations on other bond market yields. As such, these yields are not effective annualized yields or Annualized Percentage Yields (APY), which include the effect of compounding. To convert a CMT yield to an APY you need to apply the standard financial formula:

APY = (1 + I/2)2-1

Where "i" is the CMT rate expressed in decimals. For example, if the 5-year CMT rate was 8.00%, then the annualized effective yield, or APY, would be:

APY = (1 + .0800/2)2-1
APY = 1.081600 -1
APY = 0.081600

And, expressed as a percent:

APY = 8.16%
Source: Interest Rates - Frequently Asked Questions

Kevin
If I make a calculation error, #Cruncher probably will let me know.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

Kevin M wrote: Sat Sep 12, 2020 4:24 pm Here is what Treasury says (underlines mine for emphasis):
Treasury wrote:Are the CMT yields annual yields?

CMT yields are read directly from the Treasury's daily yield curve and represent "bond equivalent yields" for securities that pay semiannual interest, which are expressed on a simple annualized basis. This is consistent with market practices for quoting bond yields in the market and makes the CMT yields directly comparable to quotations on other bond market yields. As such, these yields are not effective annualized yields or Annualized Percentage Yields (APY), which include the effect of compounding. To convert a CMT yield to an APY you need to apply the standard financial formula:

APY = (1 + I/2)2-1

Where "i" is the CMT rate expressed in decimals. For example, if the 5-year CMT rate was 8.00%, then the annualized effective yield, or APY, would be:

APY = (1 + .0800/2)2-1
APY = 1.081600 -1
APY = 0.081600

And, expressed as a percent:

APY = 8.16%
Source: Interest Rates - Frequently Asked Questions
Thanks for the link.

I think that the bond simulator should use APYs.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Sat Sep 12, 2020 12:57 pm Ok, I went for ((1 + D/2) x (1 + J/2) - 1) for now.
This calculates an Annualized Percentage Yield (APY) instead of a Bond Equivalent Yield (BEY). I agree that APYs are more logical for the bond simulator's calculations, but, for consistency with other yield sources, I suggest to calculate a BEY ((((1 + D/2) X (1 + J/2))^(1/2) - 1) X 2), instead.

For its calculations, the bond fund simulator can recreate an APY yield curve from the historical BEY yield curve.
Last edited by longinvest on Sat Sep 12, 2020 10:48 pm, edited 1 time in total.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Sat Sep 12, 2020 4:28 pmI think that the bond simulator should use APYs.
I was actually going to raise a similar issue after finishing to sort out the updated historical data sources, but here we are, the two themes inter-twined earlier than I thought... Where I was coming from is that the simulator today doesn't take in account the fact that bond distributions are typically biannual, and the first semester distribution should be reinvested for the next 6 months, while the current model kind of assumes it is sitting idle until the end of the year. Kevin formalized the issue with BEY/APY terminology, which I didn't know...

So either we reformat the bond simulator to do math one semester at a time (using CMT/2 as the per-semester rate, using twice as many rungs) or we stay with an annual model, and then yes, we need to adjust the rates (and then the APY formula makes sense, agreed). I have a preference for the former approach (which I actually experimented with with my own cash flow spreadsheet) as it would make a better use of the historical monthly data we have and better illustrate the typical dynamic of bonds (with no need for BEY/APY confusing contortions), but the latter might be reasonable too.
longinvest wrote: Sat Sep 12, 2020 4:41 pm
siamond wrote: Sat Sep 12, 2020 12:57 pm Ok, I went for ((1 + D/2) x (1 + J/2) - 1) for now.
This calculates an Annualized Percentage Yield (APY) instead of a Bond Equivalent Yield (BEY). I agree that APYs are more logical for the bond simulator's calculations, but, for consistency with other yield sources, I suggest to calculate a BEY (((1 + D/2) X (1 + J/2))^(1/2) - 1), instead.

For its calculations, the bond fund simulator can recreate an APY yield curve from the historical BEY yield curve.
My formula is the annual effective return of a 6m bill, reinvested for 6 more months, so yes, it's an APY. If you're going to add a BEY to APY derivation in the model, then agreed, we need to adjust the inputs accordingly. I think your BEY formula is a per-semester rate though, it should be multiplied by 2 (arithmetically) to get to a true CMT-like rate.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Sat Sep 12, 2020 10:38 pm So either we reformat the bond simulator to do math one semester at a time (using CMT/2 as the per-semester rate, using twice as many rungs) or we stay with an annual model, and then yes, we need to adjust the rates (and then the APY formula makes sense, agreed).
The bond fund simulator spreadsheet uses an annual model where all (annual) coupons and the maturing rung are reinvested into a new bond every year. I'm aware that using an APY yield is cheating as it implies reinvesting the coupon in the same rung and at the same yield at mid-year, but I think that it's good enough for our purpose. I'd like to keep it on an annual model because I view the spreadsheet as an educational tool in addition to being useful to derive synthetic bond returns from historical rates.

More complex models (monthly, etc.) could be implemented using software. If you really want to get fancy, you'd probably want to derive (non-par) yields for aging (non-par) bonds. It's easy to add as much complexity as you want to a model. But, as for the accuracy of the outcome, I'm not sure that it adds much, as the constant maturity rates we're starting from are already approximate.
longinvest wrote: Sat Sep 12, 2020 4:41 pm I think your BEY formula is a per-semester rate though, it should be multiplied by 2 (arithmetically) to get to a true CMT-like rate.
Yes, of course. I've fixed the formula in my post.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Sat Sep 12, 2020 11:11 pmI'd like to keep it on an annual model because I view the spreadsheet as an educational tool in addition to being useful to derive synthetic bond returns from historical rates.
Well, given that US bonds are typically biannual (and that BEY/APY math is rather confusing), it seems to me that an educational goal would be better fulfilled by a biannual (per semester) model, which is closer to reality. Twice as many columns and rows in Excel isn't a big deal in this case. But hey, you maintain this spreadsheet, so... up to you. I'll give it a try to compare accuracy between an annual APY model and a biannual BEY/CMT model, but I agree I don't expect a big difference. Either way should be a solid improvement over the current model.
longinvest wrote: Sat Sep 12, 2020 11:11 pmMore complex models (monthly, etc.) could be implemented using software. If you really want to get fancy, you'd probably want to derive (non-par) yields for aging (non-par) bonds. It's easy to add as much complexity as you want to a model. But, as for the accuracy of the outcome, I'm not sure that it adds much, as the constant maturity rates we're starting from are already approximate.
I actually assembled a monthly model too in my own spreadsheet, but yes, it does become a little clunky.
longinvest wrote: Sat Sep 12, 2020 4:41 pm
siamond wrote: Sat Sep 12, 2020 10:38 pm I think your BEY formula is a per-semester rate though, it should be multiplied by 2 (arithmetically) to get to a true CMT-like rate.
Yes, of course. I've fixed the formula in my post.
Cool. I kept the APY math in my NBER spreadsheet, added the BEY derivative math, and referred to the latter in the 'Yields for Simulator' tab. It's now updated on Google Drive.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Sun Sep 13, 2020 11:08 am Well, given that US bonds are typically biannual (and that BEY/APY math is rather confusing), it seems to me that an educational goal would be better fulfilled by a biannual (per semester) model, which is closer to reality. Twice as many columns and rows in Excel isn't a big deal in this case. But hey, you maintain this spreadsheet, so... up to you.
As we now have December and June yields, you're right that it would be relatively simple to implement a semi-annual model that better matches semi-annual coupon payments of real-life bonds while preserving the self-correcting property of the model. We're in September and I'm very busy. So, I'll give myself until the early 2021 annual update to implement this new model using the new yields.

For simplicity (and for reference), it would be nice if you wrote a new post similar to your previous post about selected yields, but also include FRED links (that's my preference, as it's easier to use a single reliable source even if it's an indirect source). I'd like to use these links to easily collect my own copy of the selected yields (this way we'll be able to compare our spreadsheets to detect transcription errors). What do you think?
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Sun Sep 13, 2020 12:08 pmSo, I'll give myself until the early 2021 annual update to implement this new model using the new yields.
Seems like a great goal to aim at. Cool.
longinvest wrote: Sun Sep 13, 2020 12:08 pmFor simplicity (and for reference), it would be nice if you wrote a new post similar to your previous post about selected yields, but also include FRED links (that's my preference, as it's easier to use a single reliable source even if it's an indirect source). I'd like to use these links to easily collect my own copy of the selected yields (this way we'll be able to compare our spreadsheets to detect transcription errors). What do you think?
I updated the reference post with FRED links.

For the "Yield Curves" inputs, I will tally the June and December yields from the book (and will also add the 7-years yields while I'm at it, for consistency with the CMT series). And if one day, we find our way into the corresponding CRSP yield series database, it will be an easy update (I'm ready to bet the numbers are essentially the same).
dcabler
Posts: 4544
Joined: Wed Feb 19, 2014 10:30 am
Location: TX

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by dcabler »

siamond wrote: Sat Sep 12, 2020 12:57 pm Ok, I went for ((1 + D/2) x (1 + J/2) - 1) for now. Seemed more logical. I updated my historical spreadsheet accordingly.

PS. this is an NBER series, it's just that FRED made it available on its Web site, so we probably shouldn't assume this is the same logic as FRED CMT series. The best description I have (which isn't much) can be found here. I'll see if I can find a more detailed paper somewhere...
Siamond - "historical spreadsheet" above is a dead link....
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

dcabler wrote: Mon Sep 14, 2020 6:10 amSiamond - "historical spreadsheet" above is a dead link....
Hm, sorry about that, was a copy and paste mistake. Here is a working link. I fixed the post. Tx.
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

I updated again the reference post for the new historical data sources, adding the following:
siamond wrote: Thu Sep 10, 2020 5:18 pmI also created a Yields Curve spreadsheet, where I tallied the 1941+ monthly series from the 'Yield Curves' book and added some simple logic to fill a few gaps.
Those numbers (Estimated Par Bond Yields) are Bond Equivalent Yields (aka BEY), this is made clear in the book.
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

siamond wrote: Sun Sep 13, 2020 12:42 pm
longinvest wrote: Sun Sep 13, 2020 12:08 pmSo, I'll give myself until the early 2021 annual update to implement this new model using the new yields.
Seems like a great goal to aim at. Cool.
longinvest wrote: Sun Sep 13, 2020 12:08 pmFor simplicity (and for reference), it would be nice if you wrote a new post similar to your previous post about selected yields, but also include FRED links (that's my preference, as it's easier to use a single reliable source even if it's an indirect source). I'd like to use these links to easily collect my own copy of the selected yields (this way we'll be able to compare our spreadsheets to detect transcription errors). What do you think?
I updated the reference post with FRED links.
Ping! ;-)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Thu Sep 10, 2020 5:18 pm 1941-1961

* 1y, 2y, 3y, 4y, 5y, 7y, 10y, 15y, 20y yields from 'Yield Curves' book (end-of-month tables, estimated PAR bond yields, fully taxable)

* 30y proxied to 'Long Maturity' yields from 'Yield Curves' book (end-of-month tables, estimated PAR bond yields, fully taxable)
siamond wrote: Thu Sep 10, 2020 5:18 pm 1962-Now
* 30y proxied to 'Long Maturity' yields from 'Yield Curves' book to fill the 1962 to 1977 gap
Are these sources available online? I don't have the "Yield Curves" book.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Wed Dec 23, 2020 4:21 pm
siamond wrote: Thu Sep 10, 2020 5:18 pm 1941-1961

* 1y, 2y, 3y, 4y, 5y, 7y, 10y, 15y, 20y yields from 'Yield Curves' book (end-of-month tables, estimated PAR bond yields, fully taxable)

* 30y proxied to 'Long Maturity' yields from 'Yield Curves' book (end-of-month tables, estimated PAR bond yields, fully taxable)
siamond wrote: Thu Sep 10, 2020 5:18 pm 1962-Now
* 30y proxied to 'Long Maturity' yields from 'Yield Curves' book to fill the 1962 to 1977 gap
Are these sources available online? I don't have the "Yield Curves" book.
In the 'reference post' above, I included a link to a spreadsheet where I provided such data. This isn't available online.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Wed Dec 23, 2020 4:15 pm Ping! ;-)
Siamond,

Could you check the returns calculated in the (temporarily) linked spreadsheet? It only implements a few funds, but I'd like them to be validated before continuing. And, while at it, could you tell me which funds you actually need?

Details:

1) I've implemented a new semester-based calculation model and I've reorganized the calculations to simplify creating new funds with very few operations starting from the 30 year ladder.

2) The new model derives a zero-coupon yield curve from the (approximated) bond-equivalent yield curve, and uses it to calculate bond prices as their maturity changes. This is equivalent to recovering the effective (non-par) yield of a bond as it progresses towards maturity and using it to calculate a "more precise" price, relative to using a par yield to calculate the price of a non-par bond.

3) With a semester-based model and the improved pricing model, 6 month yields become important.

4) I now use "Bond Fund 10 to 1 Year" terminology to describe a bond fund which buys 10-year bonds and sells them when they reach 1-year to maturity. I use "Bond Ladder 30 Years" to describe a bond ladder which buys 30-year bonds and keeps them until maturity. (The self-correcting property is maintained; cash flows are precise as long as buy and sell yields are precise.)

5) The temporary spreadsheet currently only uses FRED daily data, starting in 1962. When adding data for earlier years, I'd like to keep using the same maturities: 6 mo, 1 year, 2 year, 3 year, 5 year, 7 year, 10 year, 20 year, and 30 year par yields and lineally approximating other maturities (e.g. not use 4 year and 15 year yields from the Yield Curves book). Of course, when a maturity is missing, I'll linearly approximate it (or extend the last available yield).

6) I'd like to avoid mixing different data sets during a period of time. For example, starting in 1962, I wouldn't mix yields from the Yield Curves book with FRED yields. In other words, I'd only use Yield Curve book yields from 1941 to 1961, and only FRED-NBER yields before 1941.
Last edited by longinvest on Mon Jan 04, 2021 8:03 pm, edited 1 time in total.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

I've just updated the temporary spreadsheet linked in my previous post to include the weighted-average YTM and duration of the funds. Feel free to provide feedback.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Tue Dec 29, 2020 3:55 amCould you check the returns calculated in the (temporarily) linked spreadsheet? It only implements a few funds, but I'd like them to be validated before continuing. And, while at it, could you tell me which funds you actually need?
Will take me a little while to get my mind back into those calculations, but yes, I'll look at it in the coming few days. Let's focus on the core mechanics to begin with, we can refine the yield sources later.

Let me answer your 2nd question first though, as it's easy to list the models I currently use in Simba (all Total Return except the last one):
- Bond fund simulator (30-11 model)
- Bond fund simulator (10-6 model)
- Bond fund simulator (10-4 model)
- Bond fund simulator (10-2 model)
- Bond fund simulator (5-2 model)
- Bond fund simulator (3-2 model)
- Bond fund simulator (EqPar Zero 30-11 model)
- Bond fund simulator (10-4 model, Capital Return)

I also have 15-2 in there, but it doesn't seem to be actually used. Must be a test I ran at some point. Will delete.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Tue Dec 29, 2020 2:57 pm it's easy to list the models I currently use in Simba (all Total Return except the last one):
- Bond fund simulator (30-11 model)
- Bond fund simulator (10-6 model)
- Bond fund simulator (10-4 model)
- Bond fund simulator (10-2 model)
- Bond fund simulator (5-2 model)
- Bond fund simulator (3-2 model)
- Bond fund simulator (EqPar Zero 30-11 model)
- Bond fund simulator (10-4 model, Capital Return)
OK, I'll model these funds. But, let me warn you about the "Capital Return". I now break annual returns into three pieces:
  • Coupon return: The sum of received coupons during the year divided by the initial fund balance.
  • Price return: The final fund balance (assuming coupons aren't reinvested during the year) divided by the initial fund balance. (This theoretically involves internal reinvestment of matured/sold bonds and rebalancing to keep weights similar to having reinvested all coupons into the newly bought bond).
  • Reinvestment return: The additional return from reinvesting coupons during the year.
I think that the idea of thinking in term of "income return" and "capital return" is broken. What's important is total return.

I suggest not waiting too long to provide input... I'm not retired. :wink:
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Tue Dec 29, 2020 3:07 pmI think that the idea of thinking in term of "income return" and "capital return" is broken. What's important is total return.
I don't disagree, but I do have a couple of price return series in Simba for people to better appreciate the exact point you're making (what matters is the total return). Capital return was just the terminology from your previous spreadsheet, what I need is indeed the price return as you defined it.
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Tue Dec 29, 2020 3:55 am2) The new model derives a zero-coupon yield curve from the (approximated) bond-equivalent yield curve, and uses it to calculate bond prices as their maturity changes. This is equivalent to recovering the effective (non-par) yield of a bond as it progresses towards maturity and using it to calculate a "more precise" price, relative to using a par yield to calculate the price of a non-par bond.
Hm. I was going to compare the results of my own per-semester spreadsheet (which essentially mimics the logic of your previous bond fund spreadsheet, just using semester periods and yields instead of annual periods and yields) with your new model while using the same data sources, but... it appears that you changed the approach in subtle ways... Am I correct assuming that your new model will NOT be exactly compatible with the old one (besides the change of periodicity)? Mind elaborating?
longinvest wrote: Tue Dec 29, 2020 3:55 am3) With a semester-based model and the improved pricing model, 6 month yields become important.
Why is that? Wouldn't all bonds in a bond fund model sold before (or when) they reach 1-year to maturity?
longinvest wrote: Tue Dec 29, 2020 3:55 am4) I now use "Bond Fund 10 to 1 Year" terminology to describe a bond fund which buys 10-year bonds and sells them when they reach 1-year to maturity. [...]
Ah, that's nice. MUCH clearer. To state the obvious, I was using the old terminology (10-2 in the example you provided) to document the models used in Simba.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

I've just uploaded a new version of the temporary spreadsheet which contains (using the new naming scheme):
  • Bond Fund 10 to 1 Year
  • Bond Fund 10 to 3 Years
  • Bond Fund 10 to 5 Years
  • Bond Fund 5 to 1 Year
  • Bond Fund 3 to 1 Year
  • Bond Fund 30 to 10 Years
  • Bond Fund 30 to 1 Year
  • Bond Ladder 30 Years
  • EqPar Zero Bond Fund 30 to 20 Years
siamond wrote: Tue Dec 29, 2020 4:32 pm Hm. I was going to compare the results of my own per-semester spreadsheet (which essentially mimics the logic of your previous bond fund spreadsheet, just using semester periods and yields instead of annual periods and yields) with your new model while using the same data sources, but... it appears that you changed the approach in subtle ways... Am I correct assuming that your new model will NOT be exactly compatible with the old one (besides the change of periodicity)? Mind elaborating?
Effectively, the new model changes things in subtle ways. It's more precise.

Let's pick a 1-year bond in the following "par yield curve" environment: 1 year: 2%, 6 months: 1%.

The bond has a face value of $100 and will pay two $1 coupons, one in 6 months, and one in 1 year along with its $100 face value.

I could, equivalently, buy two zero coupon bonds, for a total of $100. To get a $1 payment in 6 months, I would have to pay ($1 / (1 + (1% / 2))) = $0.995024875 for a 6-month zero coupon bond. Which means that I would be left with ($100 - $0.995024875) = $99.00497513 to buy a 1-year zero-coupon bond with a face value of $101. This means that the zero-coupon bond has a 1.00251253% return per 6 months, or a 2.00502506% bond-equivalent yield (BEY).

So, as you can see, the yield curve of zero-coupon bonds differs from the par yield curve. This is true of other non-par bonds, too. The new calculations use the zero yield curve to recover accurate prices and yields for non-par bonds.

Note that the fund buys all bonds at par, but bonds fluctuate as they progress toward maturity, making them non-par bonds. Previously, I simply used par yields to calculate prices of non-par bonds. I thought that there was no point caring about such small differences when I was introducing possibly-bigger differences using annual coupons. Anyway, I didn't know that FRED yields were par yields. Now that we've moved to a semester-based model, and that I've learned about the fact that FRED's yields are par yields (expressed as BEY), I thought it would be nice to do more accurate calculations.
siamond wrote: Tue Dec 29, 2020 4:32 pm Why is that? Wouldn't all bonds in a bond fund model sold before (or when) they reach 1-year to maturity?
Because even a 30-year bond has a coupon upcoming in 6 months. This coupon carries a 6 month yield.

When you think about it, a 30 year bond ladder is a collection of 1,890 future payments (1,830 coupons + 60 principals) over 60 distinct maturities.

All I need, at this point, is some kind of high-level sanity check about the calculated returns, before I modify the spreadsheet to add older historical yields.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Tue Dec 29, 2020 5:01 pm [*] EqPar Zero Bond Fund 30 to 20 Years
I thought this was a typo, but no, you actually fixed an old lingering issue, right? Vanguard EDV (LT STRIPS) tracks Bloomberg Barclays 20-30Y Treasury Strips TR USD. So it was a tad dubious to use a 30-10 (aka 30-11) model for it. Good catch, agreed!
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Tue Dec 29, 2020 5:01 pmAll I need, at this point, is some kind of high-level sanity check about the calculated returns, before I modify the spreadsheet to add older historical yields.
Fair enough. I did a quick check for the 30-10, 10-5 and 5-1 models, for which I have my own per-semester model comparing to the results from the previous (annual) simulator, a much simpler model (Tyler's approach, per semester) and the corresponding Barclays benchmark index. The data sources are based on FRED GSx and DGSx end-of-semester rates. So I added your annual results in there:
- for the 10-5 and 5-1 models, the Telltale lines (against the benchmark) are basically impossible to distinguish (although the exact values are a tad different) between your new per-semester model and mine.
- for the 30-10 model, the Tellltale lines are a tiny bit different, but not that much, hard to say which one is better.

Also the Telltale lines (your new model or mine) relative to the benchmark are significantly improved for the 30-10 model, compared to your old annual simulator. We still have a poor match in the 70s, but this isn't new. The thing which is a bit annoying is that, for the 10-5 and 5-1 models, the Telltale line (your new model or mine) relative to the benchmark actually displays a small skew over time (going up slowly), which wasn't there with your old annual simulator. Tyler's model has a similar issue, albeit a tad less pronounced.

I think we should park the last point for future analysis. For now, the key conclusion is that, yes, according to this quick sanity check, your new model seems to be working fine.
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Tue Dec 29, 2020 3:55 am 5) The temporary spreadsheet currently only uses FRED daily data, starting in 1962. When adding data for earlier years, I'd like to keep using the same maturities: 6 mo, 1 year, 2 year, 3 year, 5 year, 7 year, 10 year, 20 year, and 30 year par yields and lineally approximating other maturities (e.g. not use 4 year and 15 year yields from the Yield Curves book). Of course, when a maturity is missing, I'll linearly approximate it (or extend the last available yield).
I don't have a strong opinion here. Not too sure why we would ignore known data points (notably the 15y series to get the 30-10 model better anchored), but... As you prefer.
longinvest wrote: Tue Dec 29, 2020 3:55 am 6) I'd like to avoid mixing different data sets during a period of time. For example, starting in 1962, I wouldn't mix yields from the Yield Curves book with FRED yields. In other words, I'd only use Yield Curve book yields from 1941 to 1961, and only FRED-NBER yields before 1941.
In general, I agree, with one possible exception... This recommendation I made about 1962-Now:
* 30y proxied to 'Long Maturity' yields from 'Yield Curves' book to fill the 1962 to 1977 gap

My concern is that we have a weird discontinuity (1962 to 1977) where we don't have any maturity over 20 years. While having corresponding data points before 1962 and after 1977. Which means that the average maturity for the 30-11 LT bonds model (and also the 30-20 EDV model) would be strongly distorted for 15 years. I am keenly aware that the suggested fix isn't perfect, but why not take what we have...

On the other hand, the practical impact seems limited from the few tests I ran. And I see the point of keeping a consistent source across the yield curve for a given year. So I am a little torn... Maybe start as you suggested and let's run a couple more experiments...
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Tue Dec 29, 2020 10:52 pm
longinvest wrote: Tue Dec 29, 2020 3:55 am 5) The temporary spreadsheet currently only uses FRED daily data, starting in 1962. When adding data for earlier years, I'd like to keep using the same maturities: 6 mo, 1 year, 2 year, 3 year, 5 year, 7 year, 10 year, 20 year, and 30 year par yields and lineally approximating other maturities (e.g. not use 4 year and 15 year yields from the Yield Curves book). Of course, when a maturity is missing, I'll linearly approximate it (or extend the last available yield).
I don't have a strong opinion here. Not too sure why we would ignore known data points (notably the 15y series to get the 30-10 model better anchored), but... As you prefer.
longinvest wrote: Tue Dec 29, 2020 3:55 am 6) I'd like to avoid mixing different data sets during a period of time. For example, starting in 1962, I wouldn't mix yields from the Yield Curves book with FRED yields. In other words, I'd only use Yield Curve book yields from 1941 to 1961, and only FRED-NBER yields before 1941.
In general, I agree, with one possible exception... This recommendation I made about 1962-Now:
* 30y proxied to 'Long Maturity' yields from 'Yield Curves' book to fill the 1962 to 1977 gap

My concern is that we have a weird discontinuity (1962 to 1977) where we don't have any maturity over 20 years. While having corresponding data points before 1962 and after 1977. Which means that the average maturity for the 30-11 LT bonds model (and also the 30-20 EDV model) would be strongly distorted for 15 years. I am keenly aware that the suggested fix isn't perfect, but why not take what we have...

On the other hand, the practical impact seems limited from the few tests I ran. And I see the point of keeping a consistent source across the yield curve for a given year. So I am a little torn... Maybe start as you suggested and let's run a couple more experiments...
For 5), I probably wrote that too quickly. My main concern was consistency: it looks weird to lose precision when going forward in time (e.g. 4-year and 15-year yields in later years). But, I agree with you; we should do with what we have for each period. I'll include them.

For 6), for the overlapping period of 20-year yield availability between the two data sets (1962 to 1979), there are big-enough differences to make me quite uncomfortable with the idea of mixing the Yield Curves book's Long Maturity yields and FRED's 20-year yields from 1962 to 1976. I think that 20-year and 30-year yields are usually close enough; that we should simply use available FRED 20-year yields as approximation of 30-year yields starting in 1962.

For December 1979, for example, the Yield Curves book reports 10.20% and 10.03% respectively as 20-year and 30-year yields. FRED reports 10.16% and 10.11%. Had the 30-year yield been missing, I would be more comfortable with 10.16% and 10.16% than with 10.16% and 10.03%.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Wed Dec 30, 2020 10:08 am For 5), I probably wrote that quickly. My main concern was consistency: it looks weird to lose precision (e.g. 4-year and 15-year yields in later years). But, I agree with you; we should do with what we have for each period. I'll include them.
Seems to me that the weird thing is why don't we have access to 4y and 15y CMT yield series via FRED... :wink:

Cool. I still don't fully understand your new pricing model (this will take me a bit of time), but my gut feeling is that it should benefit from having more granular maturities.
longinvest wrote: Wed Dec 30, 2020 10:08 am For 6), in the DGS20 version I retrieved from FRED on December 28, 2020, daily 20-year yields are available starting from early January 1962. Maybe there was a gap in the past, but there isn't now. The gap is only for 30-year yields which start in 1977 in FRED's data set. For the overlapping period of 20-year yield availability between the two data sets, there are big-enough differences to make me quite uncomfortable with the idea of mixing the Yield Curves book's Long Maturity yields and FRED's 20-year yields from 1962 to 1976. I think that 20-year and 30-year yields are usually close enough; that we should simply use available FRED 20-year yields as approximation of 30-year yields starting in 1962.
Yes, the gap I was speaking was the 30-year yield. And I do understand your point. Why don't you include ALL the end-of-semester yields (until Dec-79) in a 'Yield Curve Book' tab of your new spreadsheet, and for now only make concrete use of the ones until Dec-61. Then one can easily play around and experiment a bit to see if referencing the 1962-1977 period helps or hurts in any shape or form. It seems pretty likely that your conclusion will hold, but it is always good to leave data available to play around and reproduce a valid test.

Another test I had in mind was to use the Yield Curve data until Dec-79 in its entirety and see what goes. I didn't run such a test so far, but it's another valid question. I am NOT suggesting this is the way to go, just that it's a valid question and it's good to have the data source handy to run a quick test...
Last edited by siamond on Wed Dec 30, 2020 10:49 am, edited 1 time in total.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Wed Dec 30, 2020 10:37 am Why don't you include ALL the end-of-semester yields (until Dec-79) in a 'Yield Curve Book' tab of your new spreadsheet, and for now only make concrete use of the ones until Dec-61. Then one can easily play around and experiment a bit to see if referencing the 1962-1977 period helps or hurts in any shape or form. It seems pretty likely that your conclusion will hold, but it is always good to leave data available to play around and reproduce a valid test.

Another test I had in mind was to use the Yield Curve data until Dec-79 in its entirety and see what goes. I didn't run such a test so far, but it's another valid question. I am NOT suggesting this is the way to go, just that it's a valid question...
OK.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

One last thing. I received feedback by PM making the case to use some of the older 'Yield Curve' data (e.g. the tax-exempt yields from 1925 to 1942). I'll have to dig that up and we should discuss it at some point. But one thing at a time...
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

Siamond, why is the data you've collected from the Yields Curves book broken into distinct tabs? I'd prefer something simpler.

Take, for example, the sheet I constructed from FRED's data. It provides clearly identified sources. In it, I use FRED's recommended citation format. I didn't create one sheet per maturity, with all 250+ dates per year; I constructed a single sheet which only provides June 30 and December 31 yields for various maturities over the 1962- period. More precisely (I'm citing the what I wrote in the spreadsheet): "The table shows the yield on the last open day of the semester, or the yield on the first open day of the next semester when previous data is unavailable." The text is simple, clear, and it provides sufficient information for someone to find the data and verify it.

Could you build a similar sheet for yields extracted from the Yields Curve book? It's important to provide an adequate notice about the sources of data. If possible and necessary, table numbers and page numbers could be included. At a minimum, a proper reference to the book (including ISBN) must be provided. I would include your sheet in the simulator spreadsheet and attribute its construction to you. This sheet shouldn't do any calculation; if averages (between different months/dates) have to be calculated, it will be done elsewhere. It should only report "raw data" of interest for the simulator.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Wed Dec 30, 2020 11:16 am Siamond, why is the data you've collected from the Yields Curves book broken into distinct tabs? I'd prefer something simpler.

Take, for example, the sheet I constructed from FRED's data. It provides clearly identified sources. In it, I use FRED's recommended citation format. I didn't create one sheet per maturity, with all 250+ dates per year; I constructed a single sheet which only provides June 30 and December 31 yields for various maturities over the 1962- period. More precisely (I'm citing the what I wrote in the spreadsheet): "The table shows the yield on the last open day of the semester, or the yield on the first open day of the next semester when previous data is unavailable." The text is simple, clear, and it provides sufficient information for someone to find the data and verify it.

Could you build a similar sheet for yields extracted from the Yields Curve book? It's important to provide an adequate notice about the sources of data. If possible and necessary, table numbers and page numbers could be included. At a minimum, a proper reference to the book (including ISBN) must be provided. I would include your sheet in the simulator spreadsheet and attribute its construction to you. This sheet shouldn't do any calculation; if averages (between different months/dates) have to be calculated, it will be done elsewhere. It should only report "raw data" of interest for the simulator.
The various tabs (e.g. 1940-1943, 1944-1947) reflect the scanning and OCR process, page by page. Then I collated everything in the first tab. Columns C to N provide collated raw results without any further massaging, just assembling numbers for the entire time period. I don't want to change this structure, which reflects how the data was assembled.

Columns S to AB in the first tab add some interpolation and so on, but you don't have to use it. I would advise that you do ponder what I did in those columns though, as some of the formulas interpolate in a more complex manner, due to the varying maturity length of the "long" series and some holes in the data series (mostly in 20y, but there are a few more). If you just take the raw data without considering column N (the maturity of the 'long' bonds in column M), you will lose important information.

I could add another tab extracting end-of-semester data, why not. Seems to me this belongs to another level, but that's fine, why not. Will do later today.

Yes, agreed, I need to better document the source material. Will do.

PS. I just double-checked, there is no 6 months maturity data.
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

siamond wrote: Wed Dec 30, 2020 12:13 pmI could add another tab extracting end-of-semester data, why not. Seems to me this belongs to another level, but that's fine, why not. Will do later today.
Ok, done. The new 'Periodic' tab allows to extract data from the 'Fully Taxable 1941-1979' tab, while specifying a periodicity (see yellow cell). I set it to 6 months by default. Will improve data sources documentation later on.

Same link for download: click here.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Wed Dec 30, 2020 2:53 pm Ok, done. The new 'Periodic' tab allows to extract data from the 'Fully Taxable 1941-1979' tab, while specifying a periodicity (see yellow cell). I set it to 6 months by default.
Thanks. This is useful to me. I've quickly looked at how you filled the gaps (in the non-raw table); this is another topic for another time. :wink:
siamond wrote: Wed Dec 30, 2020 12:13 pm PS. I just double-checked, there is no 6 months maturity data.
That would have effectively been nice. We'll manage with what we have.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

siamond wrote: Wed Dec 30, 2020 2:53 pm
siamond wrote: Wed Dec 30, 2020 12:13 pmI could add another tab extracting end-of-semester data, why not. Seems to me this belongs to another level, but that's fine, why not. Will do later today.
Ok, done. The new 'Periodic' tab allows to extract data from the 'Fully Taxable 1941-1979' tab, while specifying a periodicity (see yellow cell). I set it to 6 months by default. Will improve data sources documentation later on.

Same link for download: click here.
Hm, sorry, I just noticed that a couple of values were missing due to a scanning/OCR issue I had missed so far. Please download again.
=> practical consequence: besides the first few rows, interpolation is only required for various holes in the 20y series

I also documented the data source in more details. And tweaked the name of the file to foresee the possible creation of another file for Fully Tax Exempt data series (1930-1942) or Partially Tax-Exempt (1926-1932).
Alpha4
Posts: 176
Joined: Tue Apr 17, 2012 8:47 pm

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by Alpha4 »

siamond wrote: Wed Dec 30, 2020 10:54 am One last thing. I received feedback by PM making the case to use some of the older 'Yield Curve' data (e.g. the tax-exempt yields from 1925 to 1942). I'll have to dig that up and we should discuss it at some point. But one thing at a time...
Siamond,

What I was referring to in the PM was the use of "partially tax-exempt" yields rather than fully tax-exempt ones.

I would really not be in favor of comparing fully tax-exempt yields to taxable ones; for starters, the highest Federal tax brackets for the years 1932 to 1942 were so high (when combining normal tax and surtax; more on the surtax thing later in this post) that they might make a comparison between taxable and fully tax-exempt yields nearly worthless. If the Treasury bond market is truly efficient (and I have no evidence that in the late 30s or early 40s it wasn't....well, aside from The Fed pegging long-term rates from 1941 to 1951) then an affluent individual investor in, say, the 65% marginal tax bracket during this period would bid up the price on tax-exempt Treasury bonds until the after-tax equivalent yield on these bonds was roughly equal to that of a comparable taxable Treasury security. If you are wondering why said investor would not simply hold the taxable Treasuries inside a corporation (because the top corporate tax rate in the early 1930s was only 13.75%; even when it was raised in 1935 it was only 15%; on top of this dividends were tax-free from 1914 to 1953 with the exception of the years 1936 to 1939 when they were taxed at the investor's regular marginal rate) rather than paying high individual income tax rates on them the answer is that since 1921 a Federal law taxing excess accumulated corporate earnings at regular individual marginal rates was in effect--and in fact still is today--and to the extent this was not a deterrent enough the Personal Holding Company Act was passed in 1934 which taxed passive income by corporations at regular (which, again, were much higher than corporate income tax rates) individual marginal rates if said corporations' assets consisted mostly of securities--stocks, bonds, preferreds, etc--designed to generate passive income; with this law even a corporation whose accumulated net income or profits was small enough to escape the excess accumulated earnings tax could still be taxed at the higher individual rates.

Given the above it seems to me that comparing totally tax-exempt Treasury bonds to regular taxable ones would be more like comparing municipal bonds to Treasury bonds than comparing two roughly equivalent Treasury securities to each other. As such, I'm not sure how useful yield curve data for fully tax-exempt Federal bonds would be; there may/might appear a sudden huge jump in yields (which would not be a "real" jump in yields but rather an artifact created by switching from a tax-free yield to a fully taxable one) when we cut over to a fully taxable yield series in late 1941.

Instead of using fully tax-free Treasury yields from 1932 (or 1930...the post at viewtopic.php?p=5477177#p5477177 has partially-tax exempt (PTE) yields ending in 1932 but also has fully tax-exempt yields starting as of 1930) may I humbly suggest we might try using partially tax exempt yields instead for the early 1930s until late 1941? The Federal Reserve published a HUGE book--it's now available for free in PDF format--titled "Banking and Monetary Statistics, 1914-1941" which has weekly average yields for PTE Treasury bonds from 1934-1941 and monthly average yields for said bonds from 1919-1941. This yield series was constructed as follows (and this is taken straight from the document itself):

"United States Government Bonds:

Yields — (Tables 128 and 129). Source, United States Treasury Department. Long-term Government yields from January, 1, 1919, through October 14, 1925, are unweighted averages of yields of all outstanding partially tax-exempt Government bonds due or callable after eight years, and those from October 15, 1925, to date, of all such bonds due or callable after twelve years. Averages for the two sets of bonds were identical from October 15, 1925, through July 16, 1928. The maturity of issues included in the average was changed because of significant differences in recent years between yields of medium- and long-term Government bonds, which impaired the usefulness of the earlier series as a measure of long-term Government security yields. Beginning with the week ended October 25, 1941, Table 129 shows also average yields on taxable bonds (first issued in March 1941) due or callable after twelve years.

Each new bond with a period to call or to maturity of more than the minimum is included upon date of issue and dropped when its period to call or to maturity reaches the minimum. The yields are figured to call date when the issue is selling above par and to maturity date when the issue is selling below par. Monthly and weekly data are averages of daily figures. Further details appear in the Federal Reserve Bulletin for December 1938, page 1045.
"

Now, in case anyone is wondering how "partially tax-exempt" (PTE) Treasury securities differed from fully tax exempt ones....as far as I can tell (and I may be dead wrong on this) the difference is that PTE securities were totally exempt from normal income tax but (apparently) the interest on them on any principal amount above $5,000 of securities was fully taxable for surtax purposes. The relevant Federal law is as follows:

https://www.law.cornell.edu/cfr/text/26/1.61-7

"(2) Partially tax-exempt interest. Interest earned on certain United States obligations is partly tax exempt and partly taxable. For example, the interest on United States Treasury bonds issued before March 1, 1941, to the extent that the principal of such bonds exceeds $5,000, is exempt from normal tax but is subject to surtax. See sections 35 and 103, and the regulations thereunder."

Which references "Section 103"; Section 103 is available at:

https://www.govinfo.gov/content/pkg/CFR ... -103-4.pdf and it states as follows:

"§ 1.103–4 Interest upon United States obligations.
(a) Issued before March 1, 1941. (1) Interest upon obligations of the United States issued on or before September 1, 1917, is exempt from tax. In the case of obligations issued by the United States after September 1, 1917, and in the case of obligations of a corporation organized under act of Congress, if such corporation is an instrumentality of the United States, the interest is exempt from tax only if and to the extent provided in the acts authorizing the issue thereof, as amended and supplemented. (2) Interest on Treasury bonds issued before March 1, 1941, is exempt from Federal income taxes except surtaxes imposed upon the income or profits of individuals, associations, or corporations. However, interest on an aggregate of not exceeding $5,000 principal amount of such bonds is also exempt from surtaxes. Interest in excess of the interest on an aggregate of not exceeding $5,000 principal amount of such bonds is subject to surtax and must be included in gross income. (3) Interest credited to postal savings accounts upon moneys deposited before March 1, 1941, in postal savings banks is wholly exempt from income tax.

(b) Issued on or after March 1, 1941. (1) Under the provisions of sections 4 and 5 of the Public Debt Act of 1941 (31 U.S.C. 742a), interest upon obligations issued on or after March 1, 1941, by the United States, or any agency or instrumentality thereof, shall not have any exemption, as such, from Federal income tax except in respect of any such obligations which the Federal Maritime Board and Maritime Administration (formerly United States Maritime Commission) or the Federal Housing Administration has, before March 1, 1941, contracted to issue at a future date. The interest on such obligations so contracted to be issued shall bear such tax-exemption privileges as were at the time of such contract provided in the law authorizing their issuance. For the purposes hereof, under section 4(a) of the Public Debt Act of 1941, a Territory and a possession of the United States (or any political subdivisions thereof), and the District of Columbia, and any agency or instrumentality of any one or more of the foregoing, shall not be considered as an agency or instrumentality of the United States"

I am assuming by "an aggregate of not exceeding $5,000 of principal amount" the tax law means on a total of $5,000 of Treasury bonds rather than on $5,000 of any one bond (the difference being that in the latter case a person could own two otherwise virtually identical Treasury bonds that matured only, say, three months apart and could own $5,000 of each and be free of any surtax on them whereas if the former is the case then the surtax would be due on any interest income earned on any amount of Treasury bonds over $5,000 regardless of what types/maturities/issue dates/etc the person owned) but I might be wrong on this....any tax lawyers, tax historians, or accountants on here who can help to clarify this point?

I would also note that for most of the period that PTE bonds were issued (late 1917 to early 1941) that the surtax wasn't merely a minor incidental extra tax but in fact tended to bite far harder than the actual "regular" income tax rate (especially for the upper-middle class, the affluent, and the truly wealthy)! A surtax might not seem like such a big deal if one thinks of surtaxes like Clinton's 1993 "10% of the tax" (i.e. made the 36% rate into 39.6% since 3.6% is 10% of 36%) surtax on the top one or two brackets, or Nixon's similar 10% surtax in 1969 that raised the top bracket temporarily to 77% (since the top bracket at the time was 70% and 10% of 70% is 7% and 70% + 7% is 77%) to help finance the Vietnam War....but from 1918 to 1944 the surtax on any decent level of income ranged from roughly equal to the regular tax at merely "affluent" income levels (say around $210K or $220K in today's dollars) to being upwards of fifteen to eighteen times the regular tax rate (on incomes equal to or above several million dollars in today's money) during the late 1930s through the mid-1940s when the US raised taxes to fund first the defense buildup and then fund the actual fighting of WWII; also keep in mind that unlike the late 1960s and early 1990s "surtaxes" these surtaxes were on total taxable income rather than merely calculated as a percent of tax paid/owed (i.e. if you owed, say, $8,000 of tax on a $32,000 income then a 10% surtax on the tax amount itself would only amount to $800 but a 10% surtax on the total income would amount to $3,200). In 1944 the law was changed to consolidate the surtax and the regular tax but this is kind of a moot point since we have good data for fully taxable long-term yields from several years before that point anyhow. Wikipedia has a good overview of the surtax rates during this period; see:


https://en.wikipedia.org/wiki/Revenue_Act_of_1918

https://en.wikipedia.org/wiki/Revenue_Act_of_1921

https://en.wikipedia.org/wiki/Revenue_Act_of_1924

https://en.wikipedia.org/wiki/Revenue_Act_of_1926

https://en.wikipedia.org/wiki/Revenue_Act_of_1932

https://en.wikipedia.org/wiki/Revenue_Act_of_1934

https://en.wikipedia.org/wiki/Revenue_Act_of_1936

https://en.wikipedia.org/wiki/Revenue_Act_of_1940

https://en.wikipedia.org/wiki/Second_Re ... ct_of_1940

https://en.wikipedia.org/wiki/Revenue_Act_of_1941

In short, "partially-tax exempt" for any reasonably wealthy investor (and keep in mind that very few working-class, middle-class, or not-so-wealthy investors probably bought regular tradable/marketable Treasury bonds during this time period for two main reasons; the first is memories of the bad experiences with the WWI Liberty Loans where the government basically all but promised the bonds wouldn't trade below par and then lo and behold they did end up trading below par as interest rates rose and if you had sold at the bottom in 1920 you would've lost almost 15% of your investment in nominal terms and more than that in inflation-adjusted terms; the second reason is that there was no way--no TreasuryDirect, no Treasury bond mutual funds, no Treasury bond ETFs, etc--to buy a diversified Treasury portfolio and buying an individual Treasury bond at a $500, $1,000, or $10,000 face value would likely be out of the reach of many households at this income level; as such, they generally put their "safe" savings money into bank accounts, S&L or building and loan accounts, credit union share accounts, whole life policies, postal savings, or--after 1935 when US Savings Bonds that were guaranteed to never trade below par were first issued--regular non-marketable US savings bonds) wasn't really very tax exempt at all. Given that such is the case the partially tax-exempt rates should theoretically not be that different from the regular fully taxable bond interest rates but hey, when the reality contradicts the theory, change the theory, don't change the reality. The only data we have comparing the two directly is from the Federal Reserve's Banking and Monetary Statistics 1914-1941 which for late October 1941 to end-of-December 1941 gives a weekly comparison of yields for fully taxable bonds and partially tax-exempt bonds and the PTE bonds did indeed have rather lower yields than the fully taxable ones. However, there is one final wrinkle in all of this that may explain a decent amount of the yield variation between the PTE bond series and the fully taxable bond series and it has nothing to do with taxability (or tax-exemption) at all.

When the Federal Reserve compiled their PTE and fully taxable series they chose--for the time period after October 1925, anyway--to only include Treasury bonds with at least 12 years until maturity or call (see detailed explanation earlier in this post) and this was the case for both the PTE yield series and the fully taxable yield series. Some of the Treasury securities issued from 1929 to 1936 were in fact totally tax-exempt but this only applied to short and medium term issues (mostly rolling over existing short/medium-term debt, refinancings, short-term operational debt, and the like); as per the article "Major Trends In The Market For Tax-Exempt Securities"--which was published in the May, 1954 issue of the Journal of Finance but is now available for free at https://www.jstor.org/stable/2976314 -- only short-term Treasury securities were fully tax exempt; Congress passed laws that specifically limited long-term Treasury securities to being only partially tax-exempt; also, note from Table 1 in The Ownership of Tax-Exempt Securities, 1913-1953 (this whole document is available for free from NBER at https://www.nber.org/system/files/chapt ... /c6454.pdf ) that from the mid-1930s onward the amount of PTE Federal debt securities begins to dwarf the amount of totally tax-exempt ones. As such, the yield series' for the PTE and the fully taxable Treasury bonds likely/presumably consisted of the following bonds (all data for when these bonds were issued, were callable, and matured if not called were taken from the November 1941 and December 1941 "Statement of The Public Debt Of The United States"; both of these are available for free as PDFs on TreasuryDirect.gov's website):

Bonds presumably included in the PTE Treasury yield series (including interest rate at original issue, date to first callability, and date to maturity if not called):

2.875% issued in March 1935; callable March 1955; maturity March 1960
2.750% issued September 1936; callable September 1956; maturity September 1959
2.750% issued June 1938; callable June 1958; maturity June 1963
2.750% issued December 1938; callable December 1960; maturity December 1965
2.250% issued July 1940; callable June 1954; maturity June 1956

Bonds presumably included in the fully taxable Treasury yield series (including interest rate at original issue, date to first callability, and date to maturity if not called):

2.500% issued June 1941; callable March 1956; maturity March 1958
2.500% issued October 1941; callable September 1967; maturity September 1972

Assuming by "unweighted averages" they mean "not weighted by total amount of debt outstanding per each issue but rather simply equal weighted by each bond in the series" then you can see right away that we aren't comparing apples and oranges; the PTE series has an average time to first call or maturity of just over 15 years while the fully taxable series has an average time to call or first maturity of just over 20 years! FWIW, it looks like we are trying to compare a 15-year bond's yield to a 20-year bond's yield. This is an issue because (assuming Treasury yield curves sloped the same way corporate bond yield curves did in the early 1940s) you can clearly see from the chart on pg 477 of "Banking and Monetary Statistics, 1914-1941" that for the year 1941 corporate bond yields decreased 22 basis points on average as you went from a 20-year bond to a 15-year one.

In other words, yes, the fully taxable bond yields are higher vs the PTE ones but that may not be purely 100% due to their tax liability; some of it could be due to essentially trying to compare a longer-term bond equivalent to a somewhat shorter-term one. In order to get truly equivalent as possible yields one would probably need to create a yield series that only included the longer-term PTE bonds (I'd recommend either an average of both the 2.750% bonds from 1938 or perhaps just use the one from that year that was first callable in 1960....in the first case you are basically looking at an 18-year equivalent bond; in the second case you are looking at a just over 19 year equivalent bond); this would give you a series that was roughly equal in time until first callability/maturity to the roughly 20-year bond equivalent provided by the fully taxable yield series.

Of course, in order to do the above we would have to go into old issues of the WSJ (or maybe the Bank and Quotation Record...the Federal Reserve has that paper monthly from the early 1900s to the mid-1960s IIRC on its website for free) and extrapolate yields on those bonds from quoted prices on them.
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

Alpha4 wrote: Fri Jan 01, 2021 9:10 pmWhat I was referring to in the PM was the use of "partially tax-exempt" yields rather than fully tax-exempt ones. [...]

Instead of using fully tax-free Treasury yields from 1932 (or 1930...the post at viewtopic.php?p=5477177#p5477177 has partially-tax exempt (PTE) yields ending in 1932 but also has fully tax-exempt yields starting as of 1930) may I humbly suggest we might try using partially tax exempt yields instead for the early 1930s until late 1941? The Federal Reserve published a HUGE book--it's now available for free in PDF format--titled "Banking and Monetary Statistics, 1914-1941" which has weekly average yields for PTE Treasury bonds from 1934-1941 and monthly average yields for said bonds from 1919-1941.
Thank you for the extensive historical background, this is very impressive. I certainly see your point that using Partially Tax-Exempt data series would be highly preferable. Unfortunately, in the Yield Curves book of reference we're using (where we found the Fully-Taxable yield series for 1941+), the PTE yield series only go from 1925 to 1932. I couldn't find a clear explanation of why, but this was THE book of reference for yield curves for decades (Historical US Treasury Yield Curves, Thomas Coleman and al.), so I assume the practical availability of PTE bonds was too limited between 1933 and 1941 to get a meaningful yield curve. One could certainly dive in the Federal reserve PDF publication you mentioned (I got a copy), but I suspect the conclusion would be similar... :(
User avatar
siamond
Posts: 6010
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Wed Dec 30, 2020 5:53 pm
siamond wrote: Wed Dec 30, 2020 2:53 pm Ok, done. The new 'Periodic' tab allows to extract data from the 'Fully Taxable 1941-1979' tab, while specifying a periodicity (see yellow cell). I set it to 6 months by default.
Thanks. This is useful to me. I've quickly looked at how you filled the gaps (in the non-raw table); this is another topic for another time. :wink:
siamond wrote: Wed Dec 30, 2020 12:13 pm PS. I just double-checked, there is no 6 months maturity data.
That would have effectively been nice. We'll manage with what we have.
Looking at this again, I actually figured out two things:

a) the various gaps for 20y yields are ALWAYS due to the longest bond available having a maturity of less than 20 years. So I made my 'gap filling' formula a tad complicated, it would have been good enough to simply take the "Long" value as a proxy for 20y when the 20y yield is missing... In practice, the results are the same.

b) when I said there is no 6m maturity data, I was looking at PAR yield tables. But the book also provides SPOT (zero-coupon) yield tables. Which do include 6m series for all years of interest. And well, with biannual distributions, it makes sense that PAR tables are restricted to at least one year of maturity...

Addressing the second point is a pain though, as I have to scan a whole new set of tables... And right now, with the new year, I have plenty of other things to do. Why don't you create the structure assuming we do have 6m data, equating it with 1y yields for now, and I'll provide the missing 6m yields later on. Fair enough?
Alpha4
Posts: 176
Joined: Tue Apr 17, 2012 8:47 pm

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by Alpha4 »

siamond wrote: Sat Jan 02, 2021 12:08 am
Alpha4 wrote: Fri Jan 01, 2021 9:10 pmWhat I was referring to in the PM was the use of "partially tax-exempt" yields rather than fully tax-exempt ones. [...]

Instead of using fully tax-free Treasury yields from 1932 (or 1930...the post at viewtopic.php?p=5477177#p5477177 has partially-tax exempt (PTE) yields ending in 1932 but also has fully tax-exempt yields starting as of 1930) may I humbly suggest we might try using partially tax exempt yields instead for the early 1930s until late 1941? The Federal Reserve published a HUGE book--it's now available for free in PDF format--titled "Banking and Monetary Statistics, 1914-1941" which has weekly average yields for PTE Treasury bonds from 1934-1941 and monthly average yields for said bonds from 1919-1941.
Thank you for the extensive historical background, this is very impressive. I certainly see your point that using Partially Tax-Exempt data series would be highly preferable. Unfortunately, in the Yield Curves book of reference we're using (where we found the Fully-Taxable yield series for 1941+), the PTE yield series only go from 1925 to 1932. I couldn't find a clear explanation of why, but this was THE book of reference for yield curves for decades (Historical US Treasury Yield Curves, Thomas Coleman and al.), so I assume the practical availability of PTE bonds was too limited between 1933 and 1941 to get a meaningful yield curve. One could certainly dive in the Federal reserve PDF publication you mentioned (I got a copy), but I suspect the conclusion would be similar... :(
But if you look at the NBER document I mentioned in my post you see that if anything, the opposite was the case...from the mid-1930s onward there begin to again be much more PTE bonds outstanding (in terms of total $ amount) than fully tax exempt ones; furthermore, by late 1941 or early 1942 the amount of PTE bonds outstanding was over seven-and-a half times that of fully tax-exempt directly issued (and/or Federally guaranteed) Federal Treasury debt; in fact, for the entire period 1922-1942 the amount of PTE debt was never less than roughly twice that of fully tax exempt debt (this nadir of the value of PTE bonds outstanding to fully tax exempt bonds oustanding was during the early to mid-1930s when Federal debt began increasing rapidly during the Depression....but all of the increase in fully tax exempt securities was in shorter to intermediate-term debt instruments, not Treasury bonds...those were PTE) and in some periods of this time span it was almost nine times the amount of fully tax-exempt Federal debt outstanding!

There should've been plenty of PTE bonds for use in constructing a long-term yield series (remember, the law at the time mandated that long-term securities were only to be PTE and not fully tax-exempt); if there was an issue I would presume it would be with the shorter to mid-term maturities (i.e. the range between, say, 180 day Bills and eight or nine year Bonds) as from 1929 to the 1936 or so the Federal Government issued plenty of Treasury securities in the 2-7 year range that were tax free (i.e. totally tax-exempt); in fact, the Banking and Money Statistics 1941 document from the Federal Reserve quite clearly and explicitly mentions that their 3-5 year Treasury Note series from August 1932 to December 1941 is tax-exempt (although they do provide a taxable series for this maturity range starting in late 1940 and going through December 1941 as well). I do have to wonder if there were absolutely no taxable securities in this maturity range over the entire 1933-1940 period; mostly for the seemingly obvious reason that wouldn't some of the longer-term PTE bonds (ones that in late 1932 had a time remaining to maturity or earliest callability of say 7, 8, 10, 12, 13, 15, 16, etc years.... i.e. similar to longer intermediate-term or shorter long-term maturities) become in effect intermediate-term PTE bonds as they got closer to their maturity or call dates over the period from early 1933 to the end of 1941? Wouldn't this enable the accurate construction of a PTE yield curve?

Unless we want to be stuck between the "rock" of suddenly switching from fully tax-exempt yields to fully taxable ones (in early 1941) or the "hard place" of not having a full yield curve for the PTE series, we may need to try and crowdsource our own PTE series for the period January 1933 to March or April 1941. From June 1934 onwards the monthly "Statement of The Public Debt Of The United States" (which came out on the last day of every month) shows when Treasury securities were issued, when they are callable or mature, and what their tax-exemption status is; the ones from before that date do give the month and date the bond was issued and when it is due or callable but they don't tell what its level of tax exemption is; this leaves 1933 through May 1934 but if we have, say, a bond that in July of 1934 has three years remaining until call or maturity then, is PTE, and was issued in July 1929 then we can know that it was a roughly 8 year bond at date of issue and a 5 year bond in July 1932. After determining which securities we want to use to construct a PTE yield curve series then we can go to The Bank & Quotation Record and find said securities' prices (the B&QR was typically issued in the first or second week of the month and prices given for a security are as close as possible to its price at the closing day of the previous month; for example, for March 1941 the prices shown would be for February 28th, 1941 or the nearest weekday before that if Feb 28th was a weekend) and using a combination of the price and accrued interest extrapolate a yield curve and total return series.

If we don't want to do that we may just have to deal with a sudden jump in yields in 1940 or 1941 as we switch from a fully tax-exempt bond series to a fully taxable one....this may very well cause our simulated bond fund to show a loss that year (due to an increase in yields that was an artifact of going from a FTE to a fully taxable series that wasn't actually a real true "rate increase") that didn't actually happen in real life.
Post Reply