Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

Alpha4 wrote: Sat Jan 02, 2021 4:36 amBut if you look at the NBER document I mentioned in my post you see that if anything, the opposite was the case...[...]

There should've been plenty of PTE bonds for use in constructing a long-term yield series (remember, the law at the time mandated that long-term securities were only to be PTE and not fully tax-exempt); if there was an issue I would presume it would be with the shorter to mid-term maturities (i.e. the range between, say, 180 day Bills and eight or nine year Bonds) as from 1929 to the 1936 or so the Federal Government issued plenty of Treasury securities in the 2-7 year range that were tax free (i.e. totally tax-exempt) [...]
Understood about the sheer volume of such securities (as described by the NBER document), but we need a decent mix of maturities to get something meaningful and I find hard to believe Coleman, Ibbotson and al. didn't have a really good reason to skip the 1933-1941 time period...

Why don't you sample a little bit, say taking 1935 as a year to study and see what data you can unearth?
Alpha4
Posts: 176
Joined: Tue Apr 17, 2012 8:47 pm

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by Alpha4 »

siamond wrote: Sat Jan 02, 2021 11:13 am
Alpha4 wrote: Sat Jan 02, 2021 4:36 amBut if you look at the NBER document I mentioned in my post you see that if anything, the opposite was the case...[...]

There should've been plenty of PTE bonds for use in constructing a long-term yield series (remember, the law at the time mandated that long-term securities were only to be PTE and not fully tax-exempt); if there was an issue I would presume it would be with the shorter to mid-term maturities (i.e. the range between, say, 180 day Bills and eight or nine year Bonds) as from 1929 to the 1936 or so the Federal Government issued plenty of Treasury securities in the 2-7 year range that were tax free (i.e. totally tax-exempt) [...]
Understood about the sheer volume of such securities (as described by the NBER document), but we need a decent mix of maturities to get something meaningful and I find hard to believe Coleman, Ibbotson and al. didn't have a really good reason to skip the 1933-1941 time period...

Why don't you sample a little bit, say taking 1935 as a year to study and see what data you can unearth?

Alright, after seeing what was available in PTE bonds for the 1933 to roughly the 1935 or early 1936 period I perhaps see why Coleman/Fisher/Ibbotson would've decided to go with a fully tax-exempt series rather than try to find PTE security yield information for the shorter end of the range (T-bills up to maybe 5 year Notes/Bonds).

Such a series could not be completely constructed with pure Treasury securities at all; the shortest term remaining (until maturity or callability) PTE Treasury security with a definite guaranteed and known fully in advance call or maturity date (the Second Conversion Liberty Loan, Third Conversion Liberty Loan, and Fourth Liberty Loans were PTE bonds issued to finance WWI and by the early-mid 1930s appear somewhat unique as regards whether or not they would be called and when; more on them below) would be the Treasury Bond issued in July 1928 with a 3.375% rate at issue and with a first callability date in June 1940; this would on 1-1-1935 be a roughly five and a half year bond. After that there are PTE Treasury bonds that have first call or maturity in March 1941, August 1941, June 1943, October 1943, April 1944, December 1944, March 1946, June 1946, October 1947, December 1949, and the longest term remaining one that was first callable in September 1951 (which would make it a roughly sixteen year and nine month bond....there were no twenty or thirty years remaining PTE bonds at this point; the non-callable Panama Canal funding bonds issued by the US Treasury in 1911 and due in 1961 were fully tax exempt). The above bonds were issued over the period from 1922 to late 1934 so by 1-1-1935 would've been openly traded and available in the secondary market.

Now on to the Liberty Loans. The First Liberty Loan was fully tax-exempt and so is irrelevant for purposes of the discussion here. The two PTE Conversion Liberty Loans were actually theoretically callable by the Treasury any time after June 15th, 1932. Given that these bonds paid interest (at 4.00% and 4.25% respectively) at what were somewhat above market rates (from mid-1932 through '33 and '34) at the time on equivalent bonds it would've made sense for the Treasury to call them in as soon as they had the right to do so. However, apparently from mid-1932 to most of 1933, whether due to either the Depression, the nation's financial system being more or less one step away from a financial crisis and freeze-up, or the Treasury being afraid of being unable to raise enough money at reasonable rates (by selling new Treasuries in order to get cash to pay off the Conversion Liberty Loans as soon as they were callable) to pay off the Conversion Liberty Loan bondholders (Treasury was forbidden by law during this period from directly issuing any debt--of any term or maturity--paying more than 4.25%; Treasury officials were apparently worried that in order to raise enough cash to pay off the several billion $ in Conversion Liberty Loans all at once on the first call date they might have to offer a rate higher than 4.25% in order for the market to be willing to clear and absorb all the new debt; they were of course forbidden to do this and perhaps that's why they didn't call them in the minute they were callable), these were so far as I can tell not called in until June of 1935 (the February 28th, 1935 "Statement of the Public Debt of the United States" just says these two Liberty Loans were "callable after June 15, 1932; the March 31st, 1935 version of this document--which was likely published the first few days of April 1935--indicates that these bonds were to be called for redemption as of June 15th, 1935; by the June 1935 version of the document these bonds aren't even listed so presumably Treasury did call them in by then as planned). So we are left with the question of on 1-1-1935 did these bonds trade as regular bonds (since they weren't due to actually mature until June 1947) or were they unofficially more or less considered "under the gun" to be called even before Treasury officially announced in the spring of 1935 its intention to call them by mid-June 1935 and they as such traded as very short-term almost T-Bill like debt?

The Fourth Liberty Loan was also PTE and was technically callable any time on or after October 15, 1933. However, Treasury apparently saw the same issues with immediately calling this bond as it saw with the two Conversion Liberty Loans and so it wasn't called as soon as it became callable either. Rather than just waiting until it was confident in being able to raise enough funds at reasonable rates to call this issue all at once the Treasury started calling them in by lots depending on what their serial numbers ended in! The June 1934 Statement of the Public Debt of the United States notes that these bonds (the Fourth Liberty Loan) with final digits 1, 9, and 0 were called for redemption on Apr 15, 1934, and bonds with final digits 2 and 8 were to be called for redemption Oct 15, 1934; the October 1934 version of this document notes that bonds with final digits of 5, 6, and 7 were to be called in for redemption on April 15, 1935. The May 1935 version of this document has the earliest mention of the remaining bonds being called in; it notes that the bonds with serial numbers with final digits 3 and 4 were to be called in as of October 15, 1935.

Despite the above it seems the case that Treasury made its intentions clear before mid-1934 or mid-1935 that it was calling these bonds in (and that it gave the date said call was to happen); they apparently would let it be known at least six or seven months before the call date that they were calling the bonds in; if you go to https://www.jta.org/1933/10/16/archive/ ... e-maturity you will see that in mid-October 1933 the US Treasury gave public notice that it was calling in the Fourth Liberty Loan bonds with serial numbers ending in 1, 9, or 0 (while additionally noting that any of these bonds bearing serial numbers other than those designated--i.e. other than 1 , 9, or 0--were not included in or affected by this call for partial redemption). Also; apparently the second lot of bonds called (the ones with serial numbers ending in 2 and 8 that were called in on October 15, 1934) was pre-announced as well; pg 170 of the document at https://play.google.com/store/books/det ... AAJ&rdot=1 states that the notice that Treasury was calling in the bonds with serial numbers ending in 2 or 8 was made on April 13, 1934; Treasury's notice that the third lot of these bonds (with serial numbers ending in 5, 6, or 7) was to be called in as of April 15, 1935 came on October 13, 1934; see the Indianapolis Times, Volume 46, Number 133, Indianapolis, Indiana, 13 October 1934 edition. The final call for redemption (for the bonds with serial numbers ending in 3 or 4) came on April 13, 1935; see https://fraser.stlouisfed.org/title/fed ... rity-16221 . This means in early January 1935 that the third lot theoretically would've traded as being basically PTE T-bills (with a remaining time until call of just over 3.5 months); whether or not people would've suspected/guessed/presumed in January 1935 that the Treasury was planning to call in the remaining bonds (the fourth lot) I cannot say as I do not know; if they did this would mean that these bonds would've traded as roughly six to nine month bills depending on whether people thought they might've been called in during the summer or called in during the fall. I'll have to check on bond quotes (from the WSJ, C&FC, and B&QR) and see what prices these bonds exhibited as they traded.

if we do have two T-Bill equivalents as above as of early 1935 that just leaves us with trying to fill in the space roughly between 180 or 270 days at the shortest maturity and roughly five or five and a half days at the longest maturity (since we have actual Treasury Bonds for maturities from that point all the way up to just under seventeen years until maturity). As I noted earlier in this post, Treasury issued plenty of Notes during this time; as of January 1935 there are maturities ranging from due in the spring of 1935 to due in June 1939...but all of these are fully tax-exempt rather than PTE.

If we want to use fully 100% explicitly US Treasury guaranteed Agency debt (RFC and HOLC mostly) yields from this period as a sub for the non-existent PTE treasury yields for the 1-4 year term we might have enough data for bonds that as of January 1935 matured in 1-4 years to use in creating a semblance of yield curve (as of 1-1-1935 there were RFC Notes maturing on Dec 15th 1935, June 10th 1936, and July 1st 1937; there were HOLC Notes maturing on August 15th of 1936, Aug 15th of 1937, and Aug 15th of 1938.....and as of June 1st 1935 there was also issued a HOLC Note maturing in June of 1939). All of these RFC and HOLC agency securities were non-callable. I will have to do further research to find out what kind of secondary market existed for these bonds and what yields they traded at (or indeed if they traded much at all).
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

I have uploaded version 2.0 of the Bogleheads Bond Fund Simulator spreadsheet.

It can be used online or downloaded using the links in the first post of this thread.

This is a major redesign of the spreadsheet. The main changes are:
  • Replace the annual calculation model with a more precise semi-annual calculation model.
  • Combine historical yields from FRED/NBER, Yield Curves Book, and FRED DGS(X) series.
  • Calculate synthetic returns from 1857 to 2020.
  • Calculate the average YTM and the average duration of bond funds.
  • Split annual total returns into three components: coupon, price, and reinvestment returns.
  • Remove obsolete funds.
As usual, comments are welcome.

Enjoy!
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

I think that switching a bond fund from one set of yields to another, as the simulator currently does, is flawed. One doesn't buy a municipal bond and sell it as a Treasury bond a few years later! Instead, a separate simulation should be run for each yield set to calculate returns. The switch from one data set to another would happen by selling a municipal bond fund, for example, to buy a Treasury fund.

It will take me some time to modify the bond fund simulator to implement this cleaner return calculation model.

The current flaw affect returns for periods that span over distinct data sets. More precisely, annual returns calculated for 1899, 1918, 1941, and 1961 need to be corrected.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

I have uploaded version 2.1 of the Bogleheads Bond Fund Simulator spreadsheet.

It can be used online or downloaded using the links in the first post of this thread.

The main changes are:
  • Make the data set selectable. Bond fund returns are automatically calculated for the time period covered by the selected data set.
As usual, comments are welcome.

Enjoy!
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Tue Jan 05, 2021 7:46 amIt will take me some time to modify the bond fund simulator to implement this cleaner return calculation model.
longinvest wrote: Tue Jan 05, 2021 4:28 pm I have uploaded version 2.1 of the Bogleheads Bond Fund Simulator spreadsheet. [...]
[*] Make the data set selectable. Bond fund returns are automatically calculated for the time period covered by the selected data set.
Does version 2.1 address the issue you described above, or not yet?
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Tue Jan 05, 2021 7:19 pm
longinvest wrote: Tue Jan 05, 2021 7:46 amIt will take me some time to modify the bond fund simulator to implement this cleaner return calculation model.
longinvest wrote: Tue Jan 05, 2021 4:28 pm I have uploaded version 2.1 of the Bogleheads Bond Fund Simulator spreadsheet. [...]
[*] Make the data set selectable. Bond fund returns are automatically calculated for the time period covered by the selected data set.
Does version 2.1 address the issue you described above, or not yet?
Yes, it resolves the issue. The solution was simple; avoid mixing data sets. The inconvenience: returns for the collection of funds can only be calculated one data set at a time. It was relatively simple and quick to implement with the improved organization of the redesigned spreadsheet.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Tue Jan 05, 2021 7:48 pm Yes, it resolves the issue. The solution was simple; avoid mixing data sets. The inconvenience: returns for the collection of funds can only be calculated one data set at a time. It was relatively simple and quick to implement with the improved organization of the redesigned spreadsheet.
Hm. The practical implication seems to be that one has to copy & paste the resulting returns for each data set setting, then splice (e.g. the most 'recent' data set setting having the highest priority) to get a full historical data series of returns for a given fund model. Or duplicate the base spreadsheet (one per data set setting) and assemble a top-level spreadsheet which automates the splicing process. Please confirm?

This also makes me realize that I should beef up a bit the 'Yield Curves' spreadsheet to go till the end of the available raw data (1992). I stopped scanning in 1979 just because I didn't think it was useful to keep going, but if one wants to focus on this specific data set, then better have it all handy. I'll do it when I'll add the 6-months maturity data.

Minor point of feedback: I can easily follow what each dataset is made of, if only because I did the historical research for corresponding data sources and possible ways to combine them, but you may want to add more descriptive information in the spreadsheet for regular users (e.g. what data sources does a given dataset use and what interpolation logic is added).
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Wed Jan 06, 2021 11:27 am The practical implication seems to be that one has to copy & paste the resulting returns for each data set setting, then splice (e.g. the most 'recent' data set setting having the highest priority) to get a full historical data series of returns for a given fund model.
Effectively. As there are periods of overlap between data sets, it's up to the user to decide what to do: splice, use a weighted average, etc. Also, it allows for comparing data sets over overlapping periods, but I haven't had time to do so.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

This discussion has made me too curious. I had to look at the growth of $1 to see how well the data sets overlap for the 10 to 1 year bond fund.

The first data set starts at $1. Each subsequent data set starts at the same value as the previous data set.

Image
Last edited by longinvest on Sat Jan 09, 2021 12:16 pm, edited 1 time in total.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by siamond »

siamond wrote: Wed Jan 06, 2021 11:27 amThis also makes me realize that I should beef up a bit the 'Yield Curves' spreadsheet to go till the end of the available raw data (1992). I stopped scanning in 1979 just because I didn't think it was useful to keep going, but if one wants to focus on this specific data set, then better have it all handy. I'll do it when I'll add the 6-months maturity data.
I updated my Yield Curves spreadsheet with yields going to 1992 (the book's last edition was issued in 1993).

I did NOT scan the SPOT rates pages to get the 6m yields though. It's a lot of work to do so (the OCR process being a tad haphazard) and I find hard to believe this would make a significant change to the results. Please correct me if I'm wrong.

Side note: I would suggest to rename the LT Bonds model "NBER Munis + Long-Term Bonds" or something like that. Bit misleading otherwise.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Fri Jan 08, 2021 5:01 pm I updated my Yield Curves spreadsheet with yields going to 1992 (the book's last edition was issued in 1993).
OK. I'll add them to the simulator.
siamond wrote: Fri Jan 08, 2021 5:01 pm I did NOT scan the SPOT rates pages to get the 6m yields though. It's a lot of work to do so (the OCR process being a tad haphazard) and I find hard to believe this would make a significant change to the results. Please correct me if I'm wrong.
I'll serve you the argument you served me, earlier in this thread: "Not too sure why we would ignore known data points". :wink:
siamond wrote: Fri Jan 08, 2021 5:01 pm Side note: I would suggest to rename the LT Bonds model "NBER Munis + Long-Term Bonds" or something like that. Bit misleading otherwise.
Are the three NBER sets of yields for Municipal bonds? If yes, I'd suggest to call them: NBER Municipal Bonds (set 1), NBER Municipal Bonds (set 2), and NBER Municipal Bonds (set 3). Would that be better?
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Fri Jan 08, 2021 6:35 pm I'll serve you the argument you served me, earlier in this thread: "Not too sure why we would ignore known data points". :wink:
True... Maybe later... Not for now.
longinvest wrote: Fri Jan 08, 2021 6:35 pm
siamond wrote: Fri Jan 08, 2021 5:01 pm Side note: I would suggest to rename the LT Bonds model "NBER Munis + Long-Term Bonds" or something like that. Bit misleading otherwise.
Are the three NBER sets of yields for Municipal bonds? If yes, I'd suggest to call them: NBER Municipal Bonds (set 1), NBER Municipal Bonds (set 2), and NBER Municipal Bonds (set 3). Would that be better?
Hm, no. What you call "Municipal Bonds (set 1)" are actually New England munis (by then, the US was largely centered on New England). IG Munis are self-explanatory (at a time where the rest of the US was developing). I think we should call those models "New England Municipal Bonds" and "Investment Grade Municipal Bonds".

LT Bonds aren't munis, from what I can tell. You've seen the FRED description. Here is the NBER description (rather succinct!):
https://data.nber.org/databases/macrohi ... 13033a.txt
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

So, how about: NBER (set 1), NBER (set 2), and NBER (set 3)?
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Fri Jan 08, 2021 7:01 pm So, how about: NBER (set 1), NBER (set 2), and NBER (set 3)?
I'd personally prefer more descriptive names, but... up to you. In any case, please document somewhere what each model (or set) is made of.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Fri Jan 08, 2021 7:10 pm
longinvest wrote: Fri Jan 08, 2021 7:01 pm So, how about: NBER (set 1), NBER (set 2), and NBER (set 3)?
I'd personally prefer more descriptive names, but... up to you. In any case, please document somewhere what each model (or set) is made of.
The names would be consistent with "Yield Curves Book" and "FRED DGSX".

In the NBER sheet, the complete FRED/NBER references are provided. There's a single set of 6-month yields and three sets of 10-year yields. So, it's pretty easy to figure out how the three combined (6 months and 10 years) yield sets are constructed. Users of the spreadsheet can look at the simple formulas of "calculated" tables for details or they can ask on this thread for clarifications.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

I have uploaded version 2.2 of the Bogleheads Bond Fund Simulator spreadsheet.

It can be used online or downloaded using the links in the first post of this thread.

The main changes are:
  • Update Yield Curves Book data with yields until 1992.
I've also updated the overlap growth chart of this post to reflect the additional data.

As usual, comments are welcome.

Enjoy!
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

I updated the Simba backtesting spreadsheet with the numbers from the v2.2 bond fund simulator. I ran various sanity tests, this seems to add up. Detailed update described here: viewtopic.php?p=5728638#p5728638
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

I have uploaded version 2.3 of the Bogleheads Bond Fund Simulator spreadsheet.

It can be used online or downloaded using the links in the first post of this thread.

The main changes are:
  • Add June 30 and December 31, 2021 yields from FRED daily data.
As usual, comments are welcome.

Enjoy!
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

Here's a 165-years growth chart for bonds (the simulator's Bond Fund 10 to 1 Year using 5 data sets and Vanguard's Total Bond Market Index Fund) from 12/31/1856 to 12/31/2021:

Image
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Alpha4
Posts: 176
Joined: Tue Apr 17, 2012 8:47 pm

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by Alpha4 »

longinvest wrote: Tue Jan 04, 2022 4:36 pm I have uploaded version 2.3 of the Bogleheads Bond Fund Simulator spreadsheet.

It can be used online or downloaded using the links in the first post of this thread.

The main changes are:
  • Add June 30 and December 31, 2021 yields from FRED daily data.
As usual, comments are welcome.

Enjoy!
Thank you for uploading this updated version (the "BogleheadsBondFundSimulator" v 2.3)

I have downloaded it, though, and I have a quick comment and question as follows:

I was trying to find monthly ITT return data back to the mid-1950s (IIRC the bond fund simulator you posted only has data back to 1962 and that only annually and twice yearly i.e. for Jan to end of June and for July to end of Dec each year; the simba/siamond spreadsheet Rev 21a--the Backtest-Portfolio-Returns one--itself only has it annually).

I then recalled that there had been uploaded on this very board some monthly return for ITTs (and LTTs for that matter) spreadsheets back to 1955 that were used in computing the monthly leveraged Treasury ETF returns for use in backtesting HFEA type portfolios. I went through my portable hard drive where I vaguely remembered having saved these a while back and started looking over them again.

I was looking at the annual returns for ITTs in the latest simba/siamond spreadsheet (Rev21a) and noticed that for some years in the 1950s and 1960s that they didn't generally match the annual returns (generated from the monthly returns) for the plain unleveraged ITTs in the "LETF Monthly Simulator Siamond" and "Simple Bond Fund Calculator Monthly" spreadsheets as mentioned above.

Do you happen to know why this may be the case?

Also, do the intermediate-term bond returns in the "BogleheadsBondFundSimulator" you just uploaded mimic:

A. a 3-10 year ladder with equal weights for each year,

B. a 3-7 year ladder,

C. a 7-10 year ladder,

D. a blended average of 50/50 3-7 year ladder and 7-10 year ladder?

I am thinking it is "A" as above since the tab in your Bond Fund Simulator spreadsheet is titled "Bond Fund 10 to 3 Years" but I just wanted to be sure.

Just FWIW, I don't think the difference in returns I saw i.e. the simba spreadsheet (Rev21a) and Bond Fund Simulator spreadsheet annual returns not exactly matching the annual returns from the "LETF Monthly Simulator Siamond" and the "Simple Bond Fund Calculator Monthly" have to do with being pure ITT vs being total aggregate bond market (because before 1973 we have no actual TBM data since we have no IT corporate bond returns data before then) but this still leaves me thoroughly confused. Do you have any idea why the differences (sometimes slight; sometimes not so slight) in returns between these spreadsheets? If one is comparing a 3-10 ITT ladder with an identical 3-10 ITT ladder the annual returns should not differ much if at all, should they?

I am just sitting here wondering exactly what (probably simple) thing I am missing that could account for this. Any ideas?
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

Alpha4 wrote: Thu Jan 20, 2022 12:08 am IIRC the bond fund simulator you posted only has data back to 1962 and that only annually and twice yearly i.e. for Jan to end of June and for July to end of Dec each year
Alpha4, the simulator contains 5 distinct yield sets (with some overlapping years) spread from 1857 to 2021 (165 years). The yield set used for simulating bond funds is selected on the first sheet. The last data set (FRED DGSX, 1962-2021) is selected by default in version 2.3. Other data sets must be selected on the first sheet for earlier periods.

The spreadsheet effectively uses December 31 and June 30 yields for simulating bond funds on a semiannual basis (corresponding to the semiannual frequency of bond coupon payments). For convenience, annual total returns are summarized and decomposed into their 3 components: coupon return, price return, and reinvestment return. The weighted-average yield to maturity (YTM) and weighted-average duration of bond funds at the start of year are also provided.

I developed the bond fund simulator spreadsheet. I can answer (mostly mathematical) questions about it. Forum member Siamond did the impressive work to research and identify the historical yield sets used in the simulator. I don't have answers to questions about the other spreadsheets mentioned in your post.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by siamond »

Alpha4 wrote: Thu Jan 20, 2022 12:08 amI then recalled that there had been uploaded on this very board some monthly return for ITTs (and LTTs for that matter) spreadsheets back to 1955 that were used in computing the monthly leveraged Treasury ETF returns for use in backtesting HFEA type portfolios. I went through my portable hard drive where I vaguely remembered having saved these a while back and started looking over them again.

I was looking at the annual returns for ITTs in the latest simba/siamond spreadsheet (Rev21a) and noticed that for some years in the 1950s and 1960s that they didn't generally match the annual returns (generated from the monthly returns) for the plain unleveraged ITTs in the "LETF Monthly Simulator Siamond" and "Simple Bond Fund Calculator Monthly" spreadsheets as mentioned above.
[...]
I am just sitting here wondering exactly what (probably simple) thing I am missing that could account for this. Any ideas?
Alpha4, let me suggest you post questions about LETF simulations in the corresponding thread.

In short, there are probably several reasons:
- variable definitions of ITT (5-10, 3-10, etc); Fidelity and Vanguard and others aren't consistent in this respect (yeah, annoying)
- the LETF simulator uses monthly returns, which is a distinct/simpler simulator from what longinvest developed (biannual, more complex)
- you may have used an old file for the LETF simulator (I improved my monthly model last year)
Alpha4
Posts: 176
Joined: Tue Apr 17, 2012 8:47 pm

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by Alpha4 »

longinvest wrote: Thu Jan 20, 2022 6:32 am
Alpha4 wrote: Thu Jan 20, 2022 12:08 am IIRC the bond fund simulator you posted only has data back to 1962 and that only annually and twice yearly i.e. for Jan to end of June and for July to end of Dec each year
Alpha4, the simulator contains 5 distinct yield sets (with some overlapping years) spread from 1857 to 2021 (165 years). The yield set used for simulating bond funds is selected on the first sheet. The last data set (FRED DGSX, 1962-2021) is selected by default in version 2.3. Other data sets must be selected on the first sheet for earlier periods.

The spreadsheet effectively uses December 31 and June 30 yields for simulating bond funds on a semiannual basis (corresponding to the semiannual frequency of bond coupon payments). For convenience, annual total returns are summarized and decomposed into their 3 components: coupon return, price return, and reinvestment return. The weighted-average yield to maturity (YTM) and weighted-average duration of bond funds at the start of year are also provided.

I developed the bond fund simulator spreadsheet. I can answer (mostly mathematical) questions about it. Forum member Siamond did the impressive work to research and identify the historical yield sets used in the simulator. I don't have answers to questions about the other spreadsheets mentioned in your post.
Hello longinvest,

I have another question regarding the math and the "behind the scenes" formulas, etc used for the annual returns and regarding annual vs semi-annual total return computation in the spreadsheet. I tried to PM you but the forum software says your private message receipt was disabled.

Anyway, the question basically boiled down to this: Is it possible to configure this spreadsheet to out-put semi-annual total return data (since it already has semi-annual yield data as an input) in addition to just annual total return? I.E. say just for example, total returns from 12-31-1949 to 6-30-1950, then from 6-30-1950 to 12-31-1950, then from 12-31-1950 to 6-30-1951, then 6-30-1951 to 12-31-1951, etc)? I know that it pulls its raw yield data for its computations on a semester (half-year) basis but then it gives the total return on an annual basis.....is there any way in the next edit of it it could also be set up to show total returns on a half-year (end-of-Dec to end of June and then end-of-June to end-of-Dec) basis in addition to showing total returns on an annual basis as it currently does? If it would be too much trouble or too much of a pain in the neck to set up don't worry about it but if it would only require a few changes to certain cells/formulas/etc then would it be possible to make this spreadsheet show semi-annual returns too?
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

Alpha4 wrote: Fri May 27, 2022 4:59 am Anyway, the question basically boiled down to this: Is it possible to configure this spreadsheet to out-put semi-annual total return data (since it already has semi-annual yield data as an input) in addition to just annual total return?
Alpha4, return calculations are done on a semi-annual basis. It's for convenience that annual total returns are calculated from semi-annual returns and summarized at the top left of bond fund sheets.

Let's check this.

First, a data set must be selected on the first sheet (called "Par Yields"). I'll select the "FRED DGSX" data set which contains yields from the end of the second semester of 1961 (12/31/1961) to the end of the second semester of 2021 (12/31/2021):

Image

Then, a bond fund must be selected. I'll choose the "Bond Fund 10 to 1 Year" by simply clicking on the appropriate sheet tab:

Image

The top left of the "Bond Fund 10 to 1 Year" sheet displays annual returns. Clicking on cell B5 reveals its formula:

Image

For 1962, the formula can be simplified* to ((1+$DH$66)*(1+$DH$67)-1) which combines the (non-annualized) returns of the first and second semesters of 1962 into an annual return.

* It's easy to simplify because (A5-$A$5) is necessarily equal to zero in row 5.

This tells us the exact location of semi-annual returns. They're in the DH column.

In the following image, I've hidden columns BV:DF so that we can clearly see the year and end-of semester row identification in BT:BU along with semi-annual returns in DH:

Image

Note that semi-annual returns aren't annualized. They simply correspond to the cumulative growth (including coupons and capital gains or losses) from the end of a semester to end of the next semester.

We can check this. At the end of the second semester of 1961, the bond fund was worth $5,131.47. At the end of the first semester of 1962, it was worth $5,261.06 (including just received coupons). This corresponds to a (($5,261.06 / $5,131.47) - 1) = 2.53% return.

We can also check the annual return shown at the top left of the "Bond Fund 10 to 1 Year" sheet is correct using the value of the fund at the end of the second semesters of 1961 and 1962: (($5,412.48 / $5,131.47) - 1) = 5.48%.

I've disabled private messages so that questions like yours are asked in the public forum. I'm not retired; I don't have much time for the forum. Public replies are likely to help other forum members and, as a result, reduce the number of questions I get.

In summary, the spreadsheet already provides the information you're asking for. You won't have to wait for a new version. Isn't that nice?
Last edited by longinvest on Fri May 27, 2022 2:49 pm, edited 2 times in total.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Alpha4
Posts: 176
Joined: Tue Apr 17, 2012 8:47 pm

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by Alpha4 »

Longinvest,

Yes, that is very nice indeed! Your post was exactly what I was looking for....thank you very much for this.

I was just trying to create a monthly ITT TR series back to the late 1940s (for comparison to an intermediate corporate bond total return series I am working on for 1946 to 1972) and I needed something with more granularity than just annual in order to start running regressions and interpolations to try and get the best-fit monthly TR data I can (FWIW I'm aware of the SBBI ITT TR data and in fact have it from one of the more recent SBBI yearbooks; however, it has more than a few issues of its own especially as you go further back beyond the early 50s or late 40s. I also have monthly ITT TR data from the 1X/2X/3X leveraged Treasury ETF spreadsheets from this very forum but it doesn't appear to be wholly based on a pure 3-10 year ladder quasi-index like your bond fund simulator is; IIRC it may be a 3-7 year or 3-9 year or 5-7 year maturity range or something else entirely but whatever it is based on, the 1X annual ITT fund numbers it gives differ from the annual numbers from your simulator which are the same ones--when they have an ER added--in the Portfolio Backtest v21 spreadsheet).

With the info from your post I will be able to get the semesterly (twice yearly from 12-31 to 6-30 and then from 6-30 to 12-31 and so on for each year) TR data. Thanks again for your help with this and for the speedy reply.
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

I have uploaded version 2.4 of the bond fund simulator.

It can be used online or downloaded using the links in the first post of this thread.

The main changes are:
  • Add June 30 and December 31, 2022 yields from FRED daily data.
As usual, comments are welcome.

Enjoy!
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

Here's a growth chart for bonds (the simulator's Bond Fund 10 to 1 Year using 5 data sets and Vanguard's Total Bond Market Index Fund) from 12/31/1856 to 12/31/2022 (a total of 166 years of growth):

Image
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by siamond »

longinvest wrote: Thu Jan 12, 2023 8:19 pm I have uploaded version 2.4 of the bond fund simulator.
Ah cool, thank you. I'll include the 2022 numbers in the Simba February update.
Alpha4
Posts: 176
Joined: Tue Apr 17, 2012 8:47 pm

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by Alpha4 »

siamond wrote: Thu Sep 10, 2020 5:18 pm Time to reach closure on those new historical data sources. Here is my complete proposal (year XXXX means beginning of year in the following, all series are monthly, either monthly average or end-of-month values):

1871 to 1940

* 1 year rate from FRED Commercial Paper 4-6m rates
=> see the APY/BEY discussion in following posts

* 10+ years rate from Prof. Shiller's FRED sources for his 'RLONG' series (i.e. New England municipals until 1899; High-Grade municipals until 1918; Long-Term United States Bonds 1919+ series afterwards)

1941-1961

* 1y, 2y, 3y, 4y, 5y, 7y, 10y, 15y, 20y yields from 'Yield Curves' book (end-of-month tables, estimated PAR bond yields, fully taxable)

* 30y proxied to 'Long Maturity' yields from 'Yield Curves' book (end-of-month tables, estimated PAR bond yields, fully taxable)

1962-Now

* FRED CMT Daily series when possible (Dec 31st rates)

* 30y proxied to 'Long Maturity' yields from 'Yield Curves' book to fill the 1962 to 1977 gap

I updated my NBER/FRED spreadsheet, adding a 'Yields for Simulator' tab where I tallied the FRED series. Note that the 'FRED CMT Daily' numbers were copied by value from the 1.25 simulator.

I also created a Yields Curve spreadsheet, where I tallied the 1941+ monthly series from the 'Yield Curves' book and added some simple logic to fill a few gaps.

Longinvest, if you agree with my proposal, the next step is up to you... Feel free to recompute directly from FRED sources if you prefer.
Just a thought before the next update:

We actually have reasonably PTE good yield curve data (with end-of-month datapoints) for all of 1929 through 1940 from Stephen Cecchetti's "The Case of the Negative Nominal Interest Rates: New Estimates of the Term Structure of Interest Rates During the Great Depression" which is available free as a PDF on SSRN.

I would recommend using this data for the 1929 to 1940 period as follows:

Actual yields (YTMs) as from this paper for the following points on the yield curve:

1 year, 2 year, 3 year, 4 year, 5 year, 10 year, and 15 year (note that 7 year is not included in the paper but can reasonably be interpolated as being between 5 year and 10 year).

Midpoints between the years (say, for the 8 year and 9 year between the 10 year and 7 year) can be interpolated as being equidistant from the two points the paper does provide data for and equidistant from each other. Please correct me if I'm wrong but isn't this exactly what the current bond fund simulator does for the 1919 to 1941 period anyhow (albeit it does it in a much cruder manner; it uses the 10-year yield and the commercial paper yield and then simply assumes that each year's maturity is evenly spaced between these two points; at least with the yields from this paper we won't have to crudely interpolate so many data points in lieu of actual yields)?

Note that this paper does have "20-year" yields as well but I can think of two good reasons we may not want to use those:

One, in some cases these seem to be an entirely synthetic construct, or seem to be based on yields of a PTE bond with anywhere from 15.5 to 19.8 years remaining until maturity or call, and/or based more closely on the yield of the fully tax-exempt Panama Canal 3s due in 1961 and/or the fully tax-exempt Conversion Bonds due in 1946 and 1947,

Two, in keeping with actual boots-on-the-ground reality there were several periods during the mid-1920s to mid-1930s where there was no PTE Treasury bond with at least twenty years until maturity or call date (whether going by either "the maturity date is assumed to be the call date if the bond is callable of the maturity date if it is not callable" or by "the maturity data is assumed to be the call date if the bond is trading above par but is assumed to be the actual maturity date if it is trading below par" ) whereas there always was--as per the work done by board member McQ here on Bogleheads--at least one PTE Treasury bond with at least 15 years until maturity during this period.

As such, we should IMO perhaps just use the same yield for 15 years for everything from 16 to 30 years for the 1929 to 1940 period (which is what the bond fund simulator now does for these years anyway but it does the same thing only starting from ten years which is even cruder and arguably less accurate). Anyway, we could try it out with both the above method (no 20-year yield and use the same yield for everything 15 years and over) and then re-do it with the setup similar but actually use the 20-year yield for everything 20 years and above and then interpolate yields evenly between 15 and 20 years for the 16-year, 17-year, etc yields and see how much the results differ.

Finally, not entirely related to the above but why does the bond fund simulator use commercial paper yields before 1941 or 1942? We have actual T-Bill yields from 1929 onward (from FRED and other Federal Reserve sources), 6-12 month Treasury yields from Jan 1920 to year-end 1928 (from the 1941 Banking and Monetary Statistics PDF from the Federal Reserve), short term (one year or less) Treasury Certificate of Indebtedness (C of I) yields from the various MSPDs and from the Hall-Payne-Sargent database for 1917 to 1919, and finally we have prime 90-day and 6-month banker's acceptance yields (banker's acceptances were much less risky than commercial paper and thus would better fit the "risk-free rate" description in the pre-Treasury era) for 1915 to 1916 from various sources (monthly and annual Federal Reserve Bulletins for these years for yields on these for Jan 1915 to late 1915 and then the Commercial & Financial Chronicle--available free on the Federal Reserve's FRASER site--has them from August or Sept 1915 onward until 1917 but at that point--1/1/1917 and on--we can use short-term Treasury C of I yields from the aforementioned sources anyway). Why not use the yields on these instead of commercial paper yields for the "short-term" end of the simulator?

EDITED TO ADD: I am aware that doing everything above would leave us with nothing for a PTE Treasury yield curve for 1-1-1941 to 12-31-1941 (anything after 1-1-1942 doesn't matter since by that point we have data from the Coleman-Fisher-Ibbotson historical yield curves book that lets the simulator switch to fully taxable bonds from 1-1-1942 on); presuming we don't want to switch back to the "US long-term yields" from FRED for just that one year it should technically be feasible to calculate a PTE yield curve using actual PTE Treasury bond prices from the Bank & Quotation Record and/or the N Y Times since by the start of 1941 there were PTE Treasury securities maturing (or callable, if trading at above par) at every point from late 1941 to the early 1960s.
Alpha4
Posts: 176
Joined: Tue Apr 17, 2012 8:47 pm

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by Alpha4 »

longinvest wrote: Thu Jan 12, 2023 8:19 pm I have uploaded version 2.4 of the bond fund simulator.

It can be used online or downloaded using the links in the first post of this thread.

The main changes are:
  • Add June 30 and December 31, 2022 yields from FRED daily data.
As usual, comments are welcome.

Enjoy!
Hi longinvest,

What do you think of the idea of using the data from Cecchetti (please see my recent post in this thread a day or two ago) for 1929 to 1940?

I checked the data in the Ibbotson/Coleman/Fisher yield curves book and the year end 1928/start of 1929 YTMs (for Partially-Tax Exempt Treasuries i.e. PTE Treasuries which pretty much was every Treasury security with a maturity greater than five years issued between late 1919 and early 1941) match very well with the 1 yr, 2 yr, 10 yr, 15 yr, and 20 yr (well, 20 yr for the Cecchetti data; "long" for the yield curves book data) and pretty well with the 3 yr YTMs.

Therefore, might I suggest the following revisions in the next update of the bond fund simulator?

For 1915 to 1917 - The same as currently for the long end--high grade municipal bond yields from FRED as per https://fred.stlouisfed.org/series/M13023USM156NNBR as used by Shiller--but use prime 90 day or six month banker's acceptance yields rather than commercial paper yields for the short end; see my post for reasons why (actually we might be able to use short-term Treasury yields for 1917 since the Hall-Payne-Sargent database has yields for these from late 1916 onward and so do the MSPDs from the Treasury department).

For 1/1/1918 to 12/31/1925 - use Treasury bill yields and/or three to six month Treasury note/C of I yields (not commercial paper yields) for the short end and "US long-term bond yields" from FRED for the long end (which is what it currently uses for the long end as of right now); note that I indicate it here as starting in 1918 rather than 1919--despite the FRED data series for long-term US bonds starting in January 1919--since I know of a way to easily and accurately extend this data back one more year (basically, the FRED data series for this one uses YTMs--or yields to call if the bond is trading above par--for any PTE Treasury or Liberty bond maturing or callable within eight years or more; starting in late 1917 there were at least two Treasury Bonds/Liberty Bonds that met this description and by the end of 1918 there were four that met this description; we can use calculated YTMs for these--obtained by taking prices for them from the Bank and Quotation Record and/or the N Y Times Times Machine online archive and using this and the coupon rate to calculate the yield to maturity--and/or yield to call as applicable if trading above par--for an equal weighted portfolio of the two/three/four PTE bonds outstanding on 12/31/1917, 6/30/1918, and 12/31/1918).

For 1/1/1926 to 12/31/1928 - use yields (i.e. YTMs) from the yield curves for PTE Treasury bonds as taken from the yield curves book

For 1/1/1929 to 12/31/1940 - use yields (i.e. YTMs) from the Cecchetti paper (the reason/s we can't use yields from the yield curves book for this period is/are twofold; one, it doesn't provide a fully equivalent and non-discontinuous yield curve series from 1929 to 1941; it switches from PTE to fully tax-exempt to fully taxable; and two, the yield curve data it does show for PTE bonds is hot garbage (i.e. laughably wrong and I can demonstrate why and how if you wish) for the period from early or mid 1931 to the end of 1932 (at which point the PTE yield curve data stops).

For 1941 - Calculate yield curves as of 6/30/1941 and 12/31/1941 for PTE securities using Cecchetti's methodology; this should be simple enough since by early 1941 there were PTE Treasury bonds maturing/callable anywhere from late 1941 to the early 1960s

For 12/31/1941 or 1/1/1942 onward - No revisions or changes needed as the bond fund simulator uses fully taxable Treasury bond yield curve data from that point on anyway.

Your thoughts?
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

Alpha4 wrote: Sat May 27, 2023 4:11 pm Hi longinvest,

What do you think of the idea of using the data from Cecchetti (please see my recent post in this thread a day or two ago) for 1929 to 1940?

I checked the data in the Ibbotson/Coleman/Fisher yield curves book and the year end 1928/start of 1929 YTMs (for Partially-Tax Exempt Treasuries i.e. PTE Treasuries which pretty much was every Treasury security with a maturity greater than five years issued between late 1919 and early 1941) match very well with the 1 yr, 2 yr, 10 yr, 15 yr, and 20 yr (well, 20 yr for the Cecchetti data; "long" for the yield curves book data) and pretty well with the 3 yr YTMs.

Therefore, might I suggest the following revisions in the next update of the bond fund simulator?

For 1915 to 1917 - The same as currently for the long end--high grade municipal bond yields from FRED as per https://fred.stlouisfed.org/series/M13023USM156NNBR as used by Shiller--but use prime 90 day or six month banker's acceptance yields rather than commercial paper yields for the short end; see my post for reasons why (actually we might be able to use short-term Treasury yields for 1917 since the Hall-Payne-Sargent database has yields for these from late 1916 onward and so do the MSPDs from the Treasury department).

For 1/1/1918 to 12/31/1925 - use Treasury bill yields and/or three to six month Treasury note/C of I yields (not commercial paper yields) for the short end and "US long-term bond yields" from FRED for the long end (which is what it currently uses for the long end as of right now); note that I indicate it here as starting in 1918 rather than 1919--despite the FRED data series for long-term US bonds starting in January 1919--since I know of a way to easily and accurately extend this data back one more year (basically, the FRED data series for this one uses YTMs--or yields to call if the bond is trading above par--for any PTE Treasury or Liberty bond maturing or callable within eight years or more; starting in late 1917 there were at least two Treasury Bonds/Liberty Bonds that met this description and by the end of 1918 there were four that met this description; we can use calculated YTMs for these--obtained by taking prices for them from the Bank and Quotation Record and/or the N Y Times Times Machine online archive and using this and the coupon rate to calculate the yield to maturity--and/or yield to call as applicable if trading above par--for an equal weighted portfolio of the two/three/four PTE bonds outstanding on 12/31/1917, 6/30/1918, and 12/31/1918).

For 1/1/1926 to 12/31/1928 - use yields (i.e. YTMs) from the yield curves for PTE Treasury bonds as taken from the yield curves book

For 1/1/1929 to 12/31/1940 - use yields (i.e. YTMs) from the Cecchetti paper (the reason/s we can't use yields from the yield curves book for this period is/are twofold; one, it doesn't provide a fully equivalent and non-discontinuous yield curve series from 1929 to 1941; it switches from PTE to fully tax-exempt to fully taxable; and two, the yield curve data it does show for PTE bonds is hot garbage (i.e. laughably wrong and I can demonstrate why and how if you wish) for the period from early or mid 1931 to the end of 1932 (at which point the PTE yield curve data stops).

For 1941 - Calculate yield curves as of 6/30/1941 and 12/31/1941 for PTE securities using Cecchetti's methodology; this should be simple enough since by early 1941 there were PTE Treasury bonds maturing/callable anywhere from late 1941 to the early 1960s

For 12/31/1941 or 1/1/1942 onward - No revisions or changes needed as the bond fund simulator uses fully taxable Treasury bond yield curve data from that point on anyway.

Your thoughts?
Alpha4, thanks for your detailed post. I'll have to take the time to read it carefully. But, here's a preliminary answer.

The simulator supports multiple yield sets. I could add a new Cecchetti yield set, if that's what you're suggesting early in your post.

As for the finer changes, it would be best to get Siamond's opinion first, as he is the one who suggested the current 5 yield sets of the simulator.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Sat Jan 14, 2023 10:39 am
Siamond, what's your opinion about Alpha4's suggestions?
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by siamond »

Alpha4 wrote: Sat May 27, 2023 4:11 pm What do you think of the idea of using the data from Cecchetti (please see my recent post in this thread a day or two ago) for 1929 to 1940?
longinvest wrote: Tue Jun 06, 2023 5:25 pm
siamond wrote: Sat Jan 14, 2023 10:39 am
Siamond, what's your opinion about Alpha4's suggestions?
Well, to be honest, I just don't have an opinion on such matters at the present time. It is clear that Alpha4 put a lot of thinking into this, but my own head has been quite far from such considerations for a while and it would take me a significant effort to re-establish context and build any vaguely informed opinion, and this isn't on my personal agenda for the time being. Gone fishin'... :wink:

Alpha4, I apologize for deflecting the question, but it seems to me that folks with a much more solid bonds background than me should opine first. If a clear opinion prevails based on a solid rationale (while keeping the proposed strategy for a spliced series reasonably simple), I'm happy to update Simba (say the 2024 update) with the results, but I'm afraid this would be my only possible contribution for the time being.

This being said, before discussing splicing, it would be great if bonds experts on this forum (e.g. McQ, Kevin_M and others) would opine on how authoritative they think this Cecchetti research is (say, compared to the other sources we've used so far). Personally, I have no clue...
User avatar
Kevin M
Posts: 15787
Joined: Mon Jun 29, 2009 3:24 pm
Contact:

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by Kevin M »

Historical data sets are more McQ's thing I think. Sorry I can't offer more help. Whoever is most interested could PM him.
If I make a calculation error, #Cruncher probably will let me know.
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

A few tidbits about the Cecchetti paper. According to SSRN, it was only cited 4 times in other publications. Hm. But, according to ResearchGate, it was cited 96 times, which is pretty good. And this includes fairly recent publications.

Now if some of our bond experts could opine on the methodology used in this research paper, this would be highly appreciated...
McQ
Posts: 1425
Joined: Fri Jun 18, 2021 12:21 am
Location: California

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by McQ »

Sorry for the delayed reply, I missed the query at first.

I’ve been advising Alpha4 offline on the long term project they have undertaken regarding historical yields. By this point Alpha4 knows more about early and mid-20th century bond yields than I do (bond *returns*, not yields, and the 19th century, have been my primary focus).

Cecchetti’s paper was written in the late 1980s and published in a top peer-reviewed journal: https://www.journals.uchicago.edu/doi/a ... 086/261580, which has been cited 149 times. His academic credentials are impeccable.

*aside: for citation counts, I always start with scholar.google.com. Many working papers are way stations on the way to peer reviewed publication, and later scholars typically cite the journal article (if available; not all working papers make it any further). There may be updates to the data or more data in the citing papers, these can be examined by those interested.

No reason not to follow the path Alpha 4 laid out, if the energy to do so exists.
You can take the academic out of the classroom by retirement, but you can't ever take the classroom out of his tone, style, and manner of approach.
Alpha4
Posts: 176
Joined: Tue Apr 17, 2012 8:47 pm

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by Alpha4 »

McQ wrote: Sat Jun 10, 2023 4:17 pm Sorry for the delayed reply, I missed the query at first.

I’ve been advising Alpha4 offline on the long term project they have undertaken regarding historical yields. By this point Alpha4 knows more about early and mid-20th century bond yields than I do (bond *returns*, not yields, and the 19th century, have been my primary focus).

Cecchetti’s paper was written in the late 1980s and published in a top peer-reviewed journal: https://www.journals.uchicago.edu/doi/a ... 086/261580, which has been cited 149 times. His academic credentials are impeccable.

*aside: for citation counts, I always start with scholar.google.com. Many working papers are way stations on the way to peer reviewed publication, and later scholars typically cite the journal article (if available; not all working papers make it any further). There may be updates to the data or more data in the citing papers, these can be examined by those interested.

No reason not to follow the path Alpha 4 laid out, if the energy to do so exists.
Thank you for your kind words; I guess I know quite a bit about yields after some of the work I've done but I'm not sure I'd say I know more than you do....and I freely admit when it comes to TRs rather than yields I can't hold a candle to what you know. That said, certainly it could be argued that--given the...how shall I say...."quality" (and I use that term somewhat loosely and sarcastically) of some of the GFD, Ibbotson, Siegel, and Shiller yield data sets (e.g. see the background on what each calls a "10-year" yield before about the late 1930s or early 1940s)--that what I do know is (hopefully) more accurate than some of the data from those four sources.

In any event, I will go ahead and download the simulator spreadsheet tab for the relevant NBER series and modify it as follows (I will do the data for each point at 1 ,2, 3, 4, 5, 6, 7, 10, 15, 20 years, etc....longinvest will have to use their formulas to space out the yields evenly between those points):

1. 1 to 20 year yields (using appropriate blends of Liberty Bonds and risk-free ST rate as needed) for 12-31-1917, 6-30-1918, and 12-31-1918 for the year 1918,

2. Yield curve data using the same "US long-term bond yield" FRED data for 1919 to the present; the only real change here will be using rates for the short end of the simulator that in my opinion--and feel free to critique this or suggest why this would not be a good idea if they wish to do so, I am open to feedback--will more fairly approximate risk-free rates on the short end than the current "4-6 month commercial paper" rates the spreadsheet now uses (see items 6 to 9 below),

3. Yield curve data from Ibbotson-Coleman-Fisher yield curves book for 1926-1928

4. Yield curve data from Cecchetti for 1929-1940 (with the exception of the 20-year data; he admits that his 20-year data is the result of a Nelson-Siegel style yield curve equation rather than actually based purely on observed prices and--here I quote the paper directly--"This implies that the estimates are likely to be unreliable at maturities past twenty years, and have only limited accuracy past fifteen years"). His 15 year data aren't that far from actual 15 year PTE bond yields (or close enough...say things like a 15 year and 4 month, or 14 year and 10 month, etc....very rarely was a bond with exactly 15 years to call/maturity always on hand) when available; the 20-year data are sometimes off by 20 to perhaps 35 basis points (generally by being too high). Happily, the NY Times provides YTMs (or YTCs for bonds trading above par) and generally there is always one that is anywhere from 16.8 years or so to around 23 or 24 years until maturity; we can use these (and/or some combination/blend of them as appropriate to equal the duration of a 20-year bond) for the 20-year yields and anything above that (i.e. for the semesters from 40 and above....which would basically be the same thing proxied by the "long" yields in the Ibbotson-Coleman-Fisher book.

5. Yield curve data for 1941 as per Cecchetti's equation for seeing what the actual positive nominal yield on Treasury securities less than 5 or 6 years maturity was; this will be necessary for the semester ending June 30, 1941, and December 31, 1941 (since his yield data only goes back to 12-31-1940); for anything over 5 or 6 years I won't worry about it because--as per the paper--even at 5 years the yield results for the adjusted vs unadjusted yields were similar and for 7 years and above the results were virtually identical. I may need some help double checking to be sure my math was done properly, though. The good thing is that by 1941 there were enough PTE bonds to construct a yield curve from 6 months to 20+ years so no fully tax-exempt bonds will have to be used (the higher tax rates of 1941 and later being the reason Cecchetti didn't complete a PTE yield curve for those years; those higher taxes meant that from that point on there was an actual significant difference in the yields of PTE vs fully tax exempt securities....not that this apparently stopped, say, CRSP from using the 1947-maturing fully tax exempt Treasury Panama Canal Conversion Bond as a "5-year Treasury" at some point in 1942 for their data set, though).

6. Using 60-90 day banker's acceptance yields for 1915 and 1916 instead of commercial paper to anchor the short end,

7. Using Treasury C of I yields for 1917 to mid-1929 instead of commercial paper to anchor the short end,

8. Using T-Bill yields from late 1929 to 1941 instead of commercial paper to anchor the short end,

9. Would it also not be wise to explore (for the 1890 to 1914 period) maybe using a 50/50 blend of call money rates ( see https://fred.stlouisfed.org/series/M13001USM156NNBR ) and commercial paper rather than just pure commercial paper? IIRC call money averaged almost 1 percent lower over this whole period (1890-1914) than commercial paper. Until the early 1920s the commercial paper rate (at least on anything besides double name prime paper...which the FRED and NBER data are not) was not really a true "risk-free" rate and as such it included a risk premium as well; the call money rate on call money secured by good collateral was a lot closer to a risk-free rate than the commercial paper rate IMO. However, the call money rate also wildly varied at times--i.e. it was a good bit more volatile than the commercial paper rate--which is why I suggested a 50/50 blend of the two. Actually, come to think of it I know a of an even better proxy for the "risk-free" short term (i,e. maybe 3 months to 1.5 years or so) rate but it will be a nice chunk of work to collect data for; not hard, mind you; just rather tedious. Given what Siegel said--in his paper on real interest rates from 1800-1990--I think short-term high-grade municipal rates would make a very acceptable proxy for the closest thing to a "risk-free" rate before there were Treasury bills/Treasury C of Is and before there were banker's acceptances (well, actually there were banker's acceptances issued back to the late 1800s, at least....but actual reliable yield data we have on them only starts in very late 1914 or very early 1915; this is simply because before the end of November 1914 the Federal Reserve--which itself had just come into being around this time--was allowed neither to discount them nor to buy them for its own book; as such there is no official rate data on them before roughly January 1915).
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Fri Jun 09, 2023 2:58 pm
McQ wrote: Sat Jun 10, 2023 4:17 pm
Siamond and McQ, thanks for the feedback.
Alpha4 wrote: Fri Jun 16, 2023 8:34 pm In any event, I will go ahead and download the simulator spreadsheet tab for the relevant NBER series and modify it as follows (I will do the data for each point at 1 ,2, 3, 4, 5, 6, 7, 10, 15, 20 years, etc....longinvest will have to use their formulas to space out the yields evenly between those points):
Alpha4, for your information, the spreadsheet allows for easily adding new yield sets. The yield set table is located in the Par Yields sheet cell BL8. To take new rows into account, the validation rule of cell B8 (that's the yield set selection cell) must be updated accordingly. New data sets can sit anywhere (including in new sheets), as long as the location is provided in the BL8 table.

Given the positive feedback received, it's my intention to add Cecchetti’s yields (and most probably also include your other suggestions) into a new version of the simulator.
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

Copying these posts to preserve them:
McQ wrote: Sat Jun 10, 2023 4:17 pm I’ve been advising Alpha4 offline on the long term project they have undertaken regarding historical yields. By this point Alpha4 knows more about early and mid-20th century bond yields than I do (bond *returns*, not yields, and the 19th century, have been my primary focus).

Cecchetti’s paper was written in the late 1980s and published in a top peer-reviewed journal: https://www.journals.uchicago.edu/doi/a ... 086/261580, which has been cited 149 times. His academic credentials are impeccable.

*aside: for citation counts, I always start with scholar.google.com. Many working papers are way stations on the way to peer reviewed publication, and later scholars typically cite the journal article (if available; not all working papers make it any further). There may be updates to the data or more data in the citing papers, these can be examined by those interested.

No reason not to follow the path Alpha 4 laid out, if the energy to do so exists.
Alpha4 wrote: Fri Jun 16, 2023 8:34 pm Thank you for your kind words; I guess I know quite a bit about yields after some of the work I've done but I'm not sure I'd say I know more than you do....and I freely admit when it comes to TRs rather than yields I can't hold a candle to what you know. That said, certainly it could be argued that--given the...how shall I say...."quality" (and I use that term somewhat loosely and sarcastically) of some of the GFD, Ibbotson, Siegel, and Shiller yield data sets (e.g. see the background on what each calls a "10-year" yield before about the late 1930s or early 1940s)--that what I do know is (hopefully) more accurate than some of the data from those four sources.

In any event, I will go ahead and download the simulator spreadsheet tab for the relevant NBER series and modify it as follows (I will do the data for each point at 1 ,2, 3, 4, 5, 6, 7, 10, 15, 20 years, etc....longinvest will have to use their formulas to space out the yields evenly between those points):

1. 1 to 20 year yields (using appropriate blends of Liberty Bonds and risk-free ST rate as needed) for 12-31-1917, 6-30-1918, and 12-31-1918 for the year 1918,

2. Yield curve data using the same "US long-term bond yield" FRED data for 1919 to the present; the only real change here will be using rates for the short end of the simulator that in my opinion--and feel free to critique this or suggest why this would not be a good idea if they wish to do so, I am open to feedback--will more fairly approximate risk-free rates on the short end than the current "4-6 month commercial paper" rates the spreadsheet now uses (see items 6 to 9 below),

3. Yield curve data from Ibbotson-Coleman-Fisher yield curves book for 1926-1928

4. Yield curve data from Cecchetti for 1929-1940 (with the exception of the 20-year data; he admits that his 20-year data is the result of a Nelson-Siegel style yield curve equation rather than actually based purely on observed prices and--here I quote the paper directly--"This implies that the estimates are likely to be unreliable at maturities past twenty years, and have only limited accuracy past fifteen years"). His 15 year data aren't that far from actual 15 year PTE bond yields (or close enough...say things like a 15 year and 4 month, or 14 year and 10 month, etc....very rarely was a bond with exactly 15 years to call/maturity always on hand) when available; the 20-year data are sometimes off by 20 to perhaps 35 basis points (generally by being too high). Happily, the NY Times provides YTMs (or YTCs for bonds trading above par) and generally there is always one that is anywhere from 16.8 years or so to around 23 or 24 years until maturity; we can use these (and/or some combination/blend of them as appropriate to equal the duration of a 20-year bond) for the 20-year yields and anything above that (i.e. for the semesters from 40 and above....which would basically be the same thing proxied by the "long" yields in the Ibbotson-Coleman-Fisher book.

5. Yield curve data for 1941 as per Cecchetti's equation for seeing what the actual positive nominal yield on Treasury securities less than 5 or 6 years maturity was; this will be necessary for the semester ending June 30, 1941, and December 31, 1941 (since his yield data only goes back to 12-31-1940); for anything over 5 or 6 years I won't worry about it because--as per the paper--even at 5 years the yield results for the adjusted vs unadjusted yields were similar and for 7 years and above the results were virtually identical. I may need some help double checking to be sure my math was done properly, though. The good thing is that by 1941 there were enough PTE bonds to construct a yield curve from 6 months to 20+ years so no fully tax-exempt bonds will have to be used (the higher tax rates of 1941 and later being the reason Cecchetti didn't complete a PTE yield curve for those years; those higher taxes meant that from that point on there was an actual significant difference in the yields of PTE vs fully tax exempt securities....not that this apparently stopped, say, CRSP from using the 1947-maturing fully tax exempt Treasury Panama Canal Conversion Bond as a "5-year Treasury" at some point in 1942 for their data set, though).

6. Using 60-90 day banker's acceptance yields for 1915 and 1916 instead of commercial paper to anchor the short end,

7. Using Treasury C of I yields for 1917 to mid-1929 instead of commercial paper to anchor the short end,

8. Using T-Bill yields from late 1929 to 1941 instead of commercial paper to anchor the short end,

9. Would it also not be wise to explore (for the 1890 to 1914 period) maybe using a 50/50 blend of call money rates ( see https://fred.stlouisfed.org/series/M13001USM156NNBR ) and commercial paper rather than just pure commercial paper? IIRC call money averaged almost 1 percent lower over this whole period (1890-1914) than commercial paper. Until the early 1920s the commercial paper rate (at least on anything besides double name prime paper...which the FRED and NBER data are not) was not really a true "risk-free" rate and as such it included a risk premium as well; the call money rate on call money secured by good collateral was a lot closer to a risk-free rate than the commercial paper rate IMO. However, the call money rate also wildly varied at times--i.e. it was a good bit more volatile than the commercial paper rate--which is why I suggested a 50/50 blend of the two. Actually, come to think of it I know a of an even better proxy for the "risk-free" short term (i,e. maybe 3 months to 1.5 years or so) rate but it will be a nice chunk of work to collect data for; not hard, mind you; just rather tedious. Given what Siegel said--in his paper on real interest rates from 1800-1990--I think short-term high-grade municipal rates would make a very acceptable proxy for the closest thing to a "risk-free" rate before there were Treasury bills/Treasury C of Is and before there were banker's acceptances (well, actually there were banker's acceptances issued back to the late 1800s, at least....but actual reliable yield data we have on them only starts in very late 1914 or very early 1915; this is simply because before the end of November 1914 the Federal Reserve--which itself had just come into being around this time--was allowed neither to discount them nor to buy them for its own book; as such there is no official rate data on them before roughly January 1915).
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
User avatar
siamond
Posts: 6008
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by siamond »

Longinvest, any hope of getting the usual annual update this week-end? I'll work on Simba by then...
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

siamond wrote: Fri Jan 05, 2024 11:12 am Longinvest, any hope of getting the usual annual update this week-end? I'll work on Simba by then...
Siamond, working on it...
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

I have uploaded version 2.5 of the bond fund simulator.

It can be used online or downloaded using the links in the first post of this thread.

The main changes are:
  • Add June 30 and December 31, 2023 yields from FRED daily data.
As usual, comments are welcome.

Enjoy!
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Topic Author
longinvest
Posts: 5682
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - From Rates to Returns [Bond Fund Simulator]

Post by longinvest »

Here's a growth chart for bonds (the simulator's Bond Fund 10 to 1 Year using 5 data sets and Vanguard's Total Bond Market Index Fund) from 12/31/1856 to 12/31/2023 (a total of 167 years of growth):

Image
Variable Percentage Withdrawal (bogleheads.org/wiki/VPW) | One-Fund Portfolio (bogleheads.org/forum/viewtopic.php?t=287967)
Post Reply