Long(ish) term storage for computer files?
-
- Posts: 1967
- Joined: Sat Apr 09, 2016 5:06 pm
- Location: NYC
Long(ish) term storage for computer files?
Looking for an external solution to store our tax returns and other financial documents so that they aren't on our computers. I had been using an encrypted disc on my Macs (through disc utility) that syncs with everything else to iCloud, but had a little scare where some files in there got corrupted.
I've read that flash drives and external SSDs are no good because of the electrical charges needed to maintain flash storage, and that external hard drives are also prone to failures if they aren't booted up and used frequently. CD-ROM is not an option because computers don't come with disc drives anymore.
What are other folks using? Ideally I'd like to get the files off my computers entirely so that if a laptop or device is lost, someone can't access those files.
I've read that flash drives and external SSDs are no good because of the electrical charges needed to maintain flash storage, and that external hard drives are also prone to failures if they aren't booted up and used frequently. CD-ROM is not an option because computers don't come with disc drives anymore.
What are other folks using? Ideally I'd like to get the files off my computers entirely so that if a laptop or device is lost, someone can't access those files.
Re: Long(ish) term storage for computer files?
Print them out and store them somewhere dry.
Even if you do find long term electronic storage, it is possible that technological changes will make the data impossible to read. Like all the data on mag tapes insurance companies had that couldn't be read after 10 years because the computer equipment and programs to read them had be thrown out.
Even if you do find long term electronic storage, it is possible that technological changes will make the data impossible to read. Like all the data on mag tapes insurance companies had that couldn't be read after 10 years because the computer equipment and programs to read them had be thrown out.
Re: Long(ish) term storage for computer files?
It’s a common scenario
Really, I’d consider printing hard copies if you mean really long term storage.
Hard drives and flash drives are acceptable if replaced every few years and it is not the only copy.
Really, I’d consider printing hard copies if you mean really long term storage.
Hard drives and flash drives are acceptable if replaced every few years and it is not the only copy.
Re: Long(ish) term storage for computer files?
No one serious used tapes (or carts) as the only storage mechanism.
You have to split backup and retention. Lots of solutions for backups but very few for long-term retention, especially at large scale. If your just after tax returns for your lifetime (not sure why you want stuff after the limit since I think that could technically cause issues) then paper or film (not digital) is the best option. Otherwise you need a plan to move from tech to tech at regular intervals. If you concern is just invalid backups... then you need a multi location approach with testing and recovery to ensure they are good copies.
|
Rob |
Its a dangerous business going out your front door. - J.R.R.Tolkien
Re: Long(ish) term storage for computer files?
Encrypt the computer hard drive if that's the main concern. That's what government and businesses do.NYCaviator wrote: ↑Sat Aug 13, 2022 5:22 pm What are other folks using? Ideally I'd like to get the files off my computers entirely so that if a laptop or device is lost, someone can't access those files.
Mac and Windows both support encryption.
Re: Long(ish) term storage for computer files?
External USB CD Writer for about $20.
Re: Long(ish) term storage for computer files?
There's no reason to worry about failures if you always have the data in 2 places - that's the definition of a "backup." If one drive fails, you replace it as soon as you discover it is dead (you get to see at least every year when you file and save/backup a tax return). Both drives should be encrypted if containing sensitive info.
-
- Posts: 1967
- Joined: Sat Apr 09, 2016 5:06 pm
- Location: NYC
Re: Long(ish) term storage for computer files?
Not lifetime, just the 7 years or so you are supposed to keep them. From what I've read, USB drives and flash storage can degrade even after a year and lose data.rob wrote: ↑Sat Aug 13, 2022 5:37 pmNo one serious used tapes (or carts) as the only storage mechanism.
You have to split backup and retention. Lots of solutions for backups but very few for long-term retention, especially at large scale. If your just after tax returns for your lifetime (not sure why you want stuff after the limit since I think that could technically cause issues) then paper or film (not digital) is the best option. Otherwise you need a plan to move from tech to tech at regular intervals. If you concern is just invalid backups... then you need a multi location approach with testing and recovery to ensure they are good copies.
- tuningfork
- Posts: 885
- Joined: Wed Oct 30, 2013 8:30 pm
Re: Long(ish) term storage for computer files?
The backup experts suggest the 3-2-1 strategy for important data. At least 3 copies, at least 2 different kinds of media, at least 1 offsite.
Offsite might mean in the cloud, or in a safe deposit box at the bank, at a trusted friend's house, etc.
By having at least 2 different kinds of media, you're somewhat protected from a media becoming obsolete over time. However, for all kinds of media (digital as well as paper) it's best to check it periodically to make sure it's still readable. It's a good idea to replace digital media every 5 years or so with any newer mainstream technology that is likely to remain supported for the next 5-10 years.
Offsite might mean in the cloud, or in a safe deposit box at the bank, at a trusted friend's house, etc.
By having at least 2 different kinds of media, you're somewhat protected from a media becoming obsolete over time. However, for all kinds of media (digital as well as paper) it's best to check it periodically to make sure it's still readable. It's a good idea to replace digital media every 5 years or so with any newer mainstream technology that is likely to remain supported for the next 5-10 years.
Re: Long(ish) term storage for computer files?
We store our tax records since 2000 on an external SSD that is fairly new; modern external SSDs tend to be more reliable than spinning hard drives. Twice a year or so, we copy the entire contents of that SSD to an external hard drive normally stored in our safe deposit box. That drive in the safe deposit box is not only a second copy for tax records, but a failsafe for other important documents and digital photos if our house burns down. We also keep several previous copies of backups on that drive, so that most files are saved in several places. In addition, we store the current and previous two years of tax records in a Microsoft OneDrive encrypted vault.
-
- Posts: 5774
- Joined: Mon Sep 22, 2014 4:47 pm
Re: Long(ish) term storage for computer files?
Not quite right. Flash and SSDs store data as electrons on chunks of conductor with a surrounding insulator (How many electrons? Several hundred, depending). These electrons can "tunnel" away, and do so more rapidly as the insulators wear.NYCaviator wrote: ↑Sat Aug 13, 2022 5:22 pm I've read that flash drives and external SSDs are no good because of the electrical charges needed to maintain flash storage,
In my opinion, redundant hard drives are the best straightforward solution for long-term storage. Redundant tape drives would be even better, but are rarely used by consumers.
A secure backup system includes not just backup, but verification and off-site duplication.
Re: Long(ish) term storage for computer files?
It's better NOT to move your long-term storage offline. Media fail over time. If your files are offline you won't notice the failure until you need the docs and then it will be too late. Better to keep them online, backed up regularly with periodic tests of the backups. You can use a program like Duplicati to back up the files end-to-end encrypted to some cheap or even free cloud storage.
Re: Long(ish) term storage for computer files?
I make multiple backups and store folders rolled up into one archive along with the OS also rolled up into one archive and store the backups:
(1) local PC usually a 2nd drive
(2) a NAS in the home,
(3) a USB flash drive,
(4) a 2.5" HDD (via USB),
(5) a 3.5" HDD (via USB)
I generate a 256-bit checksum on the local copy and then after copying the backup onto the 4 other drives, I generate a 256-bit checksum on those copies, sort and compare the checksums. If there is an issue I take care of right away. The reason is simple. Once you create a backup and copy it, sometimes something goes wrong with the copy and you don't want to find out when you need the backup.
The bottom line is the more places you can store the backup the better off you are. I use 2 sets of everything and rotate them every time I do backups which varies from 30-60 days so the oldest backup on the media is never more than 60 days old.
I also run Windows in a Virtual machine and backup those up as part of the monthly backup and I have had to use them several times mostly on Windows 10 as sometimes their patch would break my machine. 20 minutes later I am restored the machine to the state it was before I did the patch. Usually by the next month the patch has been revised and is not an issue.
Two of the backups are stored in different fireproof safes. There is also a NAS whose only job is to backup the other 3 NAS's in the home and it has it's own dedicated USB storage to store older NAS backups (this process is fully automated).
I don't trust the cloud with personal information. It's far too easy for a hacker to gain access to this data and happens to often.
I wrote my own backup program that does all this and its far more reliable than the commercial software that I have used in the past. We are fortunate and still have DVD burners in all the PC;s except for the newest laptop and 3.5" USB connected floppy drive (for when we find that 3.5" disk that seemed important in the 90's.)
I still have a 5.25" floppy drive and an even older 8" floppy drive for a now 42 year old CP/M system that still works. It amazes my children as do long playing records and 8-track tapes, as well as the "rotary" dial phone (which no longer works)
You can't have too many backups and if you re-use/rotate them you don't have to worry about them going bad from disuse, although they may eventually fail but that's another reason for having backups on multiple media types.
The hardest part of this is plugging in the 3 USB flash drives and running the backup program: $ ./bkup
Backups are done right before patch Tuesday for the machines that are running Windows in a virtual machine, so if something goes wrong it's easy to recover instead of after the patch shreds your machine.
Hope this helps
(1) local PC usually a 2nd drive
(2) a NAS in the home,
(3) a USB flash drive,
(4) a 2.5" HDD (via USB),
(5) a 3.5" HDD (via USB)
I generate a 256-bit checksum on the local copy and then after copying the backup onto the 4 other drives, I generate a 256-bit checksum on those copies, sort and compare the checksums. If there is an issue I take care of right away. The reason is simple. Once you create a backup and copy it, sometimes something goes wrong with the copy and you don't want to find out when you need the backup.
The bottom line is the more places you can store the backup the better off you are. I use 2 sets of everything and rotate them every time I do backups which varies from 30-60 days so the oldest backup on the media is never more than 60 days old.
I also run Windows in a Virtual machine and backup those up as part of the monthly backup and I have had to use them several times mostly on Windows 10 as sometimes their patch would break my machine. 20 minutes later I am restored the machine to the state it was before I did the patch. Usually by the next month the patch has been revised and is not an issue.
Two of the backups are stored in different fireproof safes. There is also a NAS whose only job is to backup the other 3 NAS's in the home and it has it's own dedicated USB storage to store older NAS backups (this process is fully automated).
I don't trust the cloud with personal information. It's far too easy for a hacker to gain access to this data and happens to often.
I wrote my own backup program that does all this and its far more reliable than the commercial software that I have used in the past. We are fortunate and still have DVD burners in all the PC;s except for the newest laptop and 3.5" USB connected floppy drive (for when we find that 3.5" disk that seemed important in the 90's.)
I still have a 5.25" floppy drive and an even older 8" floppy drive for a now 42 year old CP/M system that still works. It amazes my children as do long playing records and 8-track tapes, as well as the "rotary" dial phone (which no longer works)
You can't have too many backups and if you re-use/rotate them you don't have to worry about them going bad from disuse, although they may eventually fail but that's another reason for having backups on multiple media types.
The hardest part of this is plugging in the 3 USB flash drives and running the backup program: $ ./bkup
Backups are done right before patch Tuesday for the machines that are running Windows in a virtual machine, so if something goes wrong it's easy to recover instead of after the patch shreds your machine.
Hope this helps
Last edited by enad on Sat Aug 13, 2022 10:01 pm, edited 1 time in total.
What Goes Up Must come down -- David Clayton-Thomas (1968), BST
Re: Long(ish) term storage for computer files?
I would make multiple backups and replace the media periodically.
Re: Long(ish) term storage for computer files?
I use Dropbox, it’s encrypted. 2fa on account, secret codes only (no sms)
I keep files locally mirrored, and the hard drive is encrypted.
The I also use Backblaze to cloud backup my entire PC. 2fa on account, secret codes only
Passwords and encryption keys are in 1Password. I have a paper recovery kit if I am incapacitated or dead.
I keep files locally mirrored, and the hard drive is encrypted.
The I also use Backblaze to cloud backup my entire PC. 2fa on account, secret codes only
Passwords and encryption keys are in 1Password. I have a paper recovery kit if I am incapacitated or dead.
Re: Long(ish) term storage for computer files?
OP asked a pretty simple question, and we provided a bunch of complex answers.
He just wants to get his records removed from his Apple laptop.
So just be simple: get a USB disc writer, and copy the files to a couple of CD's or DVD's. Do it twice. As long as the next computer has a USB, he'll be good.
If that tech goes out of fashion someday, re-copy onto whatever is current.
He just wants to get his records removed from his Apple laptop.
So just be simple: get a USB disc writer, and copy the files to a couple of CD's or DVD's. Do it twice. As long as the next computer has a USB, he'll be good.
If that tech goes out of fashion someday, re-copy onto whatever is current.
Re: Long(ish) term storage for computer files?
I use CrashPlan to backup all my documents.
Re: Long(ish) term storage for computer files?
Okay - then multi tested backups for 7 years is easy enough.... I would make one of them cloud but either works. Tech is not changing fast enough in 7 years. I was recently able to look at a 25 year old backup on good quality CD but I would not guarantee that as the only leg.NYCaviator wrote: ↑Sat Aug 13, 2022 6:29 pmNot lifetime, just the 7 years or so you are supposed to keep them. From what I've read, USB drives and flash storage can degrade even after a year and lose data.rob wrote: ↑Sat Aug 13, 2022 5:37 pmNo one serious used tapes (or carts) as the only storage mechanism.
You have to split backup and retention. Lots of solutions for backups but very few for long-term retention, especially at large scale. If your just after tax returns for your lifetime (not sure why you want stuff after the limit since I think that could technically cause issues) then paper or film (not digital) is the best option. Otherwise you need a plan to move from tech to tech at regular intervals. If you concern is just invalid backups... then you need a multi location approach with testing and recovery to ensure they are good copies.
I know there has been work around glass laser etching and alike (although glass is technically not a stable solid ).... right now a number of studios put the digital movies back to film and store that somewhere safe (at least some are in salt mines) because it's a better long term storage medium than digital media.
|
Rob |
Its a dangerous business going out your front door. - J.R.R.Tolkien
-
- Posts: 1967
- Joined: Sat Apr 09, 2016 5:06 pm
- Location: NYC
Re: Long(ish) term storage for computer files?
This is kind of what I'm looking for. I just want to make sure I have all of my supporting docs around for the 7 years I'm supposed to keep it, but I don't like having all of that very important info just hanging out in the Documents folder on all my Apple devices. Just seems like too much of a security risk if something got lost or stolen.obgraham wrote: ↑Sun Aug 14, 2022 3:15 pm OP asked a pretty simple question, and we provided a bunch of complex answers.
He just wants to get his records removed from his Apple laptop.
So just be simple: get a USB disc writer, and copy the files to a couple of CD's or DVD's. Do it twice. As long as the next computer has a USB, he'll be good.
If that tech goes out of fashion someday, re-copy onto whatever is current.
It was fine when I had it in an encrypted partition that got backed up to iCloud, but that got corrupted which worried me. I may just keep a paper copy and find a USB-C CD-ROM drive and burn it to that as a backup.
-
- Posts: 1967
- Joined: Sat Apr 09, 2016 5:06 pm
- Location: NYC
Re: Long(ish) term storage for computer files?
I have all of my documents backed up to iCloud, and it syncs across all of my apple devices, but I just don't like having tax returns with account nos., SSNs, etc. just sitting in the documents folder on my computer/iPad/iPhone.
Re: Long(ish) term storage for computer files?
I'd go with 1 or 2 USB flash drives and cloud storage. Since long term reliability is a priority for you, don't buy them from Amazon. Too many counterfeits that will eventually fail. Order them directly from the manufacturer or pick them up at Best Buy.
Re: Long(ish) term storage for computer files?
I have had a lot of luck with Patriot Rage USB flash drives (purchased from NewEgg) which carry a 5-year warranty. They go on sale periodically. Rotate the flash drives.
Don't forget to label the USB drive. Since they are small and easily lost/misplaced, I keep from 1 to 4 in an empty Rx bottle and label the bottle. Much harder to lose.
The only thing I would add is to perform a checksum on the source and on each copy, even if it's a copy to the cloud and then compare the checksums. That will give you a high degree of confidence that what you copied did so without issues. If you use Windows, certutil is built in, if you use Linux you can use sha1sum all the way up to sha256sum. Not sure what is available in the MAC sphere.
What Goes Up Must come down -- David Clayton-Thomas (1968), BST
Re: Long(ish) term storage for computer files?
I'm assuming that this is for personal use and not business.NYCaviator wrote: ↑Sat Aug 13, 2022 5:22 pm Looking for an external solution to store our tax returns and other financial documents so that they aren't on our computers. I had been using an encrypted disc on my Macs (through disc utility) that syncs with everything else to iCloud, but had a little scare where some files in there got corrupted.
I've read that flash drives and external SSDs are no good because of the electrical charges needed to maintain flash storage, and that external hard drives are also prone to failures if they aren't booted up and used frequently. CD-ROM is not an option because computers don't come with disc drives anymore.
What are other folks using? Ideally I'd like to get the files off my computers entirely so that if a laptop or device is lost, someone can't access those files.
Flash drives and external SSDs work for me. I probably have 4 one or two TB drives and several 32 GB thumb drives. I rotate several periodically into and out of a safe deposit box.
Non sensitive files are local and mirrored online by the vendors software.
I don't use backup software; I copy and paste. Two large folders take an hour or two. I don't backup operating system files only files that I create.
The person with the most backups wins.
The person who tests his/her backups gets extra points and likely no tears.
Thanks for starting this discussion; I need to do a summer backup!
If I had sensitive files on a laptop, I would use some kind of industry standard encryption like windows' Bitlocker. I'm sure Apple has an equivalent.
Last edited by hudson on Mon Aug 15, 2022 8:58 pm, edited 1 time in total.
- SmileyFace
- Posts: 9186
- Joined: Wed Feb 19, 2014 9:11 am
Re: Long(ish) term storage for computer files?
I have a NAS drive that keeps everything backed up. I also have 2 external drives and every 6 months I backup all to one of them (100,000 photos plus all financial docs) and swap it with the other one that is in the safety deposit box (if my house burns down I have 20+ years of photos safe in the deposit box). I also have some old internal drives I pulled out of old laptops that I also backup to on occasion. For taxes I also back up to cheap USB fobs. Some of the very old flash drives I use for financial file backupa (15+ years) I have I only reload once a year - never had one fail.
Someone mentioned printouts due to changing technology. In our lifetime its hard to imagine a PDF won't be readable.
Someone mentioned printouts due to changing technology. In our lifetime its hard to imagine a PDF won't be readable.
Re: Long(ish) term storage for computer files?
First of all, I believe this is not really necessary since like other cloud storage services, iCloud has (I believe) versioning, and so you should be able to recover things if a file is corrupted.NYCaviator wrote: ↑Sat Aug 13, 2022 6:29 pmNot lifetime, just the 7 years or so you are supposed to keep them. From what I've read, USB drives and flash storage can degrade even after a year and lose data.rob wrote: ↑Sat Aug 13, 2022 5:37 pmNo one serious used tapes (or carts) as the only storage mechanism.
You have to split backup and retention. Lots of solutions for backups but very few for long-term retention, especially at large scale. If your just after tax returns for your lifetime (not sure why you want stuff after the limit since I think that could technically cause issues) then paper or film (not digital) is the best option. Otherwise you need a plan to move from tech to tech at regular intervals. If you concern is just invalid backups... then you need a multi location approach with testing and recovery to ensure they are good copies.
USB thumb drives are not terribly reliable (although I'd be VERY surprised to see one fail after one year), and I would not trust them for critical documents. However, true solid state drives (internal or external - something like the Samsung T7) are much more reliable than USB thumb drives or traditional hard disk drives.
There's also optical media, but in 7 years you might have trouble finding drives that will read the disc. And the discs can also degrade.
Re: Long(ish) term storage for computer files?
NASA says you need to write/print to a slab of stainless steel.
Just saying.
Just saying.
Old fart who does three index stock funds, baby.
-
- Posts: 88
- Joined: Fri Mar 08, 2019 10:43 pm
Re: Long(ish) term storage for computer files?
Another vote for cloud storage - we use Dropbox, with 2fa and a complicated password stored in Bitwarden. Then we have it set up to sync local copies to our laptops as well.
For something small like tax return PDF's, you probably don't even need to pay a monthly fee. So you're getting enterprise-class security and distributed, backed-up storage for the low low price of $0. It's not really worth it to try to roll your own solution, IMHO.
If you're extra security paranoid, you can encrypt the files locally before uploading them. This can be done for free with 7-Zip (https://www.7-zip.org/), a free archive utility that has excellent AES-256 encryption as an option.
For something small like tax return PDF's, you probably don't even need to pay a monthly fee. So you're getting enterprise-class security and distributed, backed-up storage for the low low price of $0. It's not really worth it to try to roll your own solution, IMHO.
If you're extra security paranoid, you can encrypt the files locally before uploading them. This can be done for free with 7-Zip (https://www.7-zip.org/), a free archive utility that has excellent AES-256 encryption as an option.
Portfolio: 75% VT, 25% BNDW/I-bonds/HYSA
Re: Long(ish) term storage for computer files?
Just a word of caution ... and this could of course happen with any of them: Crashplan sent me a mail some years ago telling me my entire cloud backup was lost due to some hardware failure. They ended up giving me 3 free months and nothing more. It also meant all the versions of files I still had locally were lost. I stopped trusting cloud backup that day and now it's just an extra offiste backup. Nothing beats annual hard drives at the bank --- and only after checksums are verified (ideally to two different, protected offsite locations). I keep a checksum file in the root of my main directories, back them up with the files themselves, and then ensure the checksums match on the media I'm taking to the bank. This also helps guard against bit rot / corruption you'd never notice... like some photo from 2010 that became corrupted on your main source... when would you discover it without some checksum verification?
Yeah, the old hard drive may fail, but that's where the checksums show their value to ensure you haven't lost files through the years... It might tell you that this list of files present in the checksum file is now missing and that these two files have changed since the checksum was generated, for example.
I highly recommend this utility: https://corz.org/windows/software/checksum/
Re: Long(ish) term storage for computer files?
Cloud is safest and most secure. We put everything there. Automatic.
BH Consumer FAQ: |
Car? Used Toyota, Lexus or Miata. |
House? 20% down and 3x salary. |
Vacation house? No. |
Umbrella? $1 million. |
Goods? Costco.
Re: Long(ish) term storage for computer files?
All "uptime" or disaster recovery is a matter of "how many 9s do you want". Each 9 is exponentially more expensive than the previous 9 based on risk averted.
For example - if you're OK with 99% reliability, just keep a copy on your hard drive. Done.
(these are not necessarily accurate)
99.9% - store to a single backup external hard drive, keep at home
99.99% - store to a backup hard drive, keep in a safe deposit box
99.999% - store to 2 backup hard drives, keep in a safe deposit box
99.9999% - store to 2 backup hard drives, keep in 2 separate safe deposit boxes, refresh annually.
99.99999% - geesh
As someone else mentioned, can't you just print all of it out in the off chance you need it?
Also, are you morally opposed to just backing it up on GoogleDrive or Apple or something? If you back it up there and keep an electronic copy on your machine, that is probably a 99.999% level of availability. Anyway - that's what I do.
(and before anyone corrects me, I know this is generally used for "uptime", but the concept is the same > risk of loss)
For example - if you're OK with 99% reliability, just keep a copy on your hard drive. Done.
(these are not necessarily accurate)
99.9% - store to a single backup external hard drive, keep at home
99.99% - store to a backup hard drive, keep in a safe deposit box
99.999% - store to 2 backup hard drives, keep in a safe deposit box
99.9999% - store to 2 backup hard drives, keep in 2 separate safe deposit boxes, refresh annually.
99.99999% - geesh
As someone else mentioned, can't you just print all of it out in the off chance you need it?
Also, are you morally opposed to just backing it up on GoogleDrive or Apple or something? If you back it up there and keep an electronic copy on your machine, that is probably a 99.999% level of availability. Anyway - that's what I do.
(and before anyone corrects me, I know this is generally used for "uptime", but the concept is the same > risk of loss)
-
- Posts: 15368
- Joined: Fri Apr 10, 2015 12:29 am
Re: Long(ish) term storage for computer files?
A micro SD card and USB adapter is more reliable than an integrated thumb drive. A significant source of thumb drive failures is loose connections inside the device. With a micro SD card and USB adapter, you can just replace the adapter if that happens. With 2 of them and a redundant copy, I would trust it for the 7-year horizon of tax documents, and longer, but not for say 30 or 40 years. The primary risk for tax document archiving seems like it is all of the supporting receipts and other supporting docs. I assume you are planning to scan those?
You also can purchase a USB DVD writer drive.
Don't underestimate the risk associated with key management and not having the key when you need to decrypt.
You also can purchase a USB DVD writer drive.
Don't underestimate the risk associated with key management and not having the key when you need to decrypt.
-
- Posts: 9279
- Joined: Sun Dec 26, 2010 11:47 am
Re: Long(ish) term storage for computer files?
External drives are cheap, as are SD cards.
Yes, a drive can fail....so back up critical data on multiple drives. If one fails another will still have your data.
Even paper records can burn, get stolen, or otherwise disappear. Nothing is 100% fail safe.
Yes, a drive can fail....so back up critical data on multiple drives. If one fails another will still have your data.
Even paper records can burn, get stolen, or otherwise disappear. Nothing is 100% fail safe.
-
- Posts: 88
- Joined: Fri Mar 08, 2019 10:43 pm
Re: Long(ish) term storage for computer files?
Also, I'd be very cautious about trusting safe deposit boxes. They're not actually a very reliable way to store stuff, stories about banks losing the contents are pretty common, either through theft, negligence, or losing records in corporate takeovers. The bank isn't held liable for the loss, either.
https://www.nytimes.com/2019/07/19/busi ... theft.html
https://www.nytimes.com/2019/07/19/busi ... theft.html
Portfolio: 75% VT, 25% BNDW/I-bonds/HYSA
- SmileyFace
- Posts: 9186
- Joined: Wed Feb 19, 2014 9:11 am
Re: Long(ish) term storage for computer files?
"safest" is a strong word and "most secure" a strong statement. Safer than the full encrypted drive in my safety deposit box? I guess only time will tell.
Last edited by SmileyFace on Tue Aug 16, 2022 2:05 pm, edited 1 time in total.
- SmileyFace
- Posts: 9186
- Joined: Wed Feb 19, 2014 9:11 am
Re: Long(ish) term storage for computer files?
That article was discussed at length in another thread. Somewhat a FUD clickbait article and what is the chance of my house burning down AND the very rare event my 30+ year old safety deposit box becomes compromised in the same year?stefan_lec wrote: ↑Tue Aug 16, 2022 1:55 pm Also, I'd be very cautious about trusting safe deposit boxes. They're not actually a very reliable way to store stuff, stories about banks losing the contents are pretty common, either through theft, negligence, or losing records in corporate takeovers. The bank isn't held liable for the loss, either.
https://www.nytimes.com/2019/07/19/busi ... theft.html
You say stories are "pretty common" but the only thing I ever see cited is the same ole' 2019 NYT article.
Last edited by SmileyFace on Tue Aug 16, 2022 2:10 pm, edited 2 times in total.
Re: Long(ish) term storage for computer files?
This is news to me. I didn't think flash drives needed anything else to retain information. They were as robust as CDs and/DVDs, I thought. What have I missed?NYCaviator wrote: ↑Sat Aug 13, 2022 5:22 pm [...]
I've read that flash drives and external SSDs are no good because of the electrical charges needed to maintain flash storage, and that external hard drives are also prone to failures if they aren't booted up and used frequently. CD-ROM is not an option because computers don't come with disc drives anymore.
[...]
Re: Long(ish) term storage for computer files?
Nothing. They don't need charges to retain information.an_asker wrote: ↑Tue Aug 16, 2022 2:08 pmThis is news to me. I didn't think flash drives needed anything else to retain information. They were as robust as CDs and/DVDs, I thought. What have I missed?NYCaviator wrote: ↑Sat Aug 13, 2022 5:22 pm [...]
I've read that flash drives and external SSDs are no good because of the electrical charges needed to maintain flash storage, and that external hard drives are also prone to failures if they aren't booted up and used frequently. CD-ROM is not an option because computers don't come with disc drives anymore.
[...]
Re: Long(ish) term storage for computer files?
Thanks! OP had me scratching my head for a bit there!kelway wrote: ↑Tue Aug 16, 2022 2:12 pmNothing. They don't need charges to retain information.an_asker wrote: ↑Tue Aug 16, 2022 2:08 pmThis is news to me. I didn't think flash drives needed anything else to retain information. They were as robust as CDs and/DVDs, I thought. What have I missed?NYCaviator wrote: ↑Sat Aug 13, 2022 5:22 pm [...]
I've read that flash drives and external SSDs are no good because of the electrical charges needed to maintain flash storage, and that external hard drives are also prone to failures if they aren't booted up and used frequently. CD-ROM is not an option because computers don't come with disc drives anymore.
[...]
-
- Posts: 88
- Joined: Fri Mar 08, 2019 10:43 pm
Re: Long(ish) term storage for computer files?
True, it's not a huge risk (though it's definitely not zero). But then, none of the other risks mentioned here are all that common, either.SmileyFace wrote: ↑Tue Aug 16, 2022 2:03 pmThat article was discussed at length in another thread. Somewhat a FUD clickbait article and what is the chance of my house burning down AND the very rare event my 30+ year old safety deposit box becomes compromised in the same year?stefan_lec wrote: ↑Tue Aug 16, 2022 1:55 pm Also, I'd be very cautious about trusting safe deposit boxes. They're not actually a very reliable way to store stuff, stories about banks losing the contents are pretty common, either through theft, negligence, or losing records in corporate takeovers. The bank isn't held liable for the loss, either.
https://www.nytimes.com/2019/07/19/busi ... theft.html
You say stories are "pretty common" but the only thing I ever see cited is the same ole' 2019 NYT article.
I mean, what's the chance of your house burning down, all your laptops being destroyed, and your cloud storage provider losing your data on the same day? Pretty dang small, as well.
Difference is the cloud backup is much cheaper (or free), and can be checked often with very little trouble. How often would people actually drive to the bank and check to see if a hard drive in a safety deposit box was actually (a) there and (b) working? I'm not guessing that would happen more than once or twice a year, if that.
Since none of these things are 100% reliable, you need to use multiple options that don't share failure modes, where it is easy to detect a failure quickly. Cloud storage combined with local storage in your house meets those conditions, and is practical, easy and cheap. Making it harder and more complicated than that isn't really adding any meaningful security or reliability.
Portfolio: 75% VT, 25% BNDW/I-bonds/HYSA
Re: Long(ish) term storage for computer files?
I use the M-Disc.
I don't use cloud storage backup solutions because it's too much of a pain for me to figure out what I'm comfortable storing in the cloud and what I'm not, since I think it's basically a given that it's only a matter of time that these cloud providers get hacked/compromised in some way and the data is all out there.
If you mostly have documents and photos, you can buy a 100 GB M-Disc (you can only write to it once), and then archive everything you want on a couple of them once every year or two years. They are rated to last hundreds of years and don't have many of the drawbacks of external hard drives.
I don't use cloud storage backup solutions because it's too much of a pain for me to figure out what I'm comfortable storing in the cloud and what I'm not, since I think it's basically a given that it's only a matter of time that these cloud providers get hacked/compromised in some way and the data is all out there.
If you mostly have documents and photos, you can buy a 100 GB M-Disc (you can only write to it once), and then archive everything you want on a couple of them once every year or two years. They are rated to last hundreds of years and don't have many of the drawbacks of external hard drives.
-
- Posts: 1967
- Joined: Sat Apr 09, 2016 5:06 pm
- Location: NYC
Re: Long(ish) term storage for computer files?
Not entirely true. Plenty of articles online about how the data is stored as an "electron charge" and over time that charge dissipates and can lead to data loss, hence the recommendation that you plug a thumb drive into a computer to power it up occasionally.kelway wrote: ↑Tue Aug 16, 2022 2:12 pmNothing. They don't need charges to retain information.an_asker wrote: ↑Tue Aug 16, 2022 2:08 pmThis is news to me. I didn't think flash drives needed anything else to retain information. They were as robust as CDs and/DVDs, I thought. What have I missed?NYCaviator wrote: ↑Sat Aug 13, 2022 5:22 pm [...]
I've read that flash drives and external SSDs are no good because of the electrical charges needed to maintain flash storage, and that external hard drives are also prone to failures if they aren't booted up and used frequently. CD-ROM is not an option because computers don't come with disc drives anymore.
[...]
Re: Long(ish) term storage for computer files?
M Discs are practically eternal, and the drives aren't going anywhere.
You can still buy USB floppy drives.
If you torture the data long enough, it will confess to anything. ~Ronald Coase
Re: Long(ish) term storage for computer files?
Lots of people here. Anyone have firsthand experience of a thumb drive failing? I recently pulled one out of a bag in the garage that is from around 2007. It had bounced around in two countries, four states, and temperatures from about 19F to 98F, with all sorts of humidity. The files were just like I left them. I have had similar experiences with several others.
Also have an old hard drive that was really abused, tossed in a bin in a buddy’s garage in 2004 after being used since 1997, got it back in 2010, booted it up in a external power unit last year. No issues.
Two copies, together with the low likelihood of needing anything, seems to be more than sufficient for the time period needed.
Also have an old hard drive that was really abused, tossed in a bin in a buddy’s garage in 2004 after being used since 1997, got it back in 2010, booted it up in a external power unit last year. No issues.
Two copies, together with the low likelihood of needing anything, seems to be more than sufficient for the time period needed.
Re: Long(ish) term storage for computer files?
You’re right. My post was more about I’m generalNYCaviator wrote: ↑Tue Aug 16, 2022 6:59 pmNot entirely true. Plenty of articles online about how the data is stored as an "electron charge" and over time that charge dissipates and can lead to data loss, hence the recommendation that you plug a thumb drive into a computer to power it up occasionally.kelway wrote: ↑Tue Aug 16, 2022 2:12 pmNothing. They don't need charges to retain information.an_asker wrote: ↑Tue Aug 16, 2022 2:08 pmThis is news to me. I didn't think flash drives needed anything else to retain information. They were as robust as CDs and/DVDs, I thought. What have I missed?NYCaviator wrote: ↑Sat Aug 13, 2022 5:22 pm [...]
I've read that flash drives and external SSDs are no good because of the electrical charges needed to maintain flash storage, and that external hard drives are also prone to failures if they aren't booted up and used frequently. CD-ROM is not an option because computers don't come with disc drives anymore.
[...]
Whether they need a charge to retain data, but indeed it’s a risk I now understand better. Thanks ….
“ SSD are reliable because, when powered, they manage the realities of flash. Over time, SSD lower-level firmware will refresh the charges stored on the drive, keeping them from dropping data over the course of the drive’s useful life. Left on a shelf too long, the drive will fail.”
-
- Posts: 6561
- Joined: Tue Jul 26, 2011 1:35 pm
Re: Long(ish) term storage for computer files?
I print out tax returns, and store them in a fire proof safe.
I copy import files like tax returns to a USB drive and copy it onto another computer that we own. Files that are not sensitive I email to a Gmail account.
Important documents are in safety deposit box.
The only thing I was really paranoid about was my doctoral thesis when I was writing/modifying it.
I hired a typist, who kept her own backups, but who would give me a floppy back when I asked for changes. I kept a 2 copies in my office, and a copy at home. The cost of the typist, changes and floppies were trivial when compared with the cost of grad school.
I copy import files like tax returns to a USB drive and copy it onto another computer that we own. Files that are not sensitive I email to a Gmail account.
Important documents are in safety deposit box.
The only thing I was really paranoid about was my doctoral thesis when I was writing/modifying it.
I hired a typist, who kept her own backups, but who would give me a floppy back when I asked for changes. I kept a 2 copies in my office, and a copy at home. The cost of the typist, changes and floppies were trivial when compared with the cost of grad school.
Re: Long(ish) term storage for computer files?
For long term storage, a hard drive offers great protection and these can be purchased with USB interfaces.
SSD’s (like the kind used in a laptop or computers) can also be used but should be refreshed at least once a year and are also susceptible to temperature with each 5 degree C rise can cause the SSD to degrade faster so this is probably not a prudent option.
USB flash drives and SD memory cards are not the same as SSD’s and use a different type of flash memory that is non-volatile and can easily last 10+ years w/o being refreshed, but refreshing restarts the 10-year clock.
Cloud storage has it's own challenges specifically because you don't know the media they are using and you may not know how often the "cloud" gets hacked (it's not really a cloud, rather a server using SSD's or hard drives). Your data gets backed up along with everyone else's backup and if the service you chose is worth their weight they probably store backups in multiple locations, so you don't really know where your data is at; the point is there are many places where it can get hacked and you or the cloud service may never know about it.
For these reasons it makes sense to use multiple media types to protect their long term data:
a. portable USB based Hard Drive,
b. SSD (if you like) but they need to be refreshed periodically and are sensitive to heat,
c. USB flash (up to 10 year shelf life w/o needing to be refreshed),
d. SD card (up to 10 year shelf life w/o needing to be refreshed),
e. CD (limited to 700 MB of storage)
f. DVD (limited to 4.7 GB or 8.5 GB with double sided)
g. Blu-ray (can hold from 25 GB to 100 GB with Quad layer)
h. Cloud storage - may not be risk free because there are many points where you could be hacked and the cloud service may never know. It's one thing to get a letter stating they were hacked, it's another when they get hacked and they don't even realize it, or if they do they realize it 18 months later. To me this is the worst place to store financial data since it is out of my control.
Yes, I still file paper tax returns but I try & limit my returns to less than $50 to the state or the government since these days it can take the government several months to process a paper return.
No matter which methods you use, it's important to validate or create a checksum for the both the source and each copy because if you ever need to recover the data the last thing you want to find out is you stored "corrupted" data. Corruption can take place as a result of copying the data.
On Windows you can use the certutil.exe utility in an elevated command prompt.
On Linux you can use sha1sum or sha256sum (most secure but takes the longest) from a command window
On Apple you can use a launch terminal and run a sha256sum (see this article for help):
https://osxdaily.com/2021/12/17/check-sha256-hash-mac/
This should cover the 3 major operating systems that are popular and in widespread use today. I haven't looked at using Android Smartphones but someone must have an App for it. The problem is you don't know the permissions the app is requesting so I personally avoid using smartphone apps for financial information.
SSD’s (like the kind used in a laptop or computers) can also be used but should be refreshed at least once a year and are also susceptible to temperature with each 5 degree C rise can cause the SSD to degrade faster so this is probably not a prudent option.
USB flash drives and SD memory cards are not the same as SSD’s and use a different type of flash memory that is non-volatile and can easily last 10+ years w/o being refreshed, but refreshing restarts the 10-year clock.
Cloud storage has it's own challenges specifically because you don't know the media they are using and you may not know how often the "cloud" gets hacked (it's not really a cloud, rather a server using SSD's or hard drives). Your data gets backed up along with everyone else's backup and if the service you chose is worth their weight they probably store backups in multiple locations, so you don't really know where your data is at; the point is there are many places where it can get hacked and you or the cloud service may never know about it.
For these reasons it makes sense to use multiple media types to protect their long term data:
a. portable USB based Hard Drive,
b. SSD (if you like) but they need to be refreshed periodically and are sensitive to heat,
c. USB flash (up to 10 year shelf life w/o needing to be refreshed),
d. SD card (up to 10 year shelf life w/o needing to be refreshed),
e. CD (limited to 700 MB of storage)
f. DVD (limited to 4.7 GB or 8.5 GB with double sided)
g. Blu-ray (can hold from 25 GB to 100 GB with Quad layer)
h. Cloud storage - may not be risk free because there are many points where you could be hacked and the cloud service may never know. It's one thing to get a letter stating they were hacked, it's another when they get hacked and they don't even realize it, or if they do they realize it 18 months later. To me this is the worst place to store financial data since it is out of my control.
Yes, I still file paper tax returns but I try & limit my returns to less than $50 to the state or the government since these days it can take the government several months to process a paper return.
No matter which methods you use, it's important to validate or create a checksum for the both the source and each copy because if you ever need to recover the data the last thing you want to find out is you stored "corrupted" data. Corruption can take place as a result of copying the data.
On Windows you can use the certutil.exe utility in an elevated command prompt.
On Linux you can use sha1sum or sha256sum (most secure but takes the longest) from a command window
On Apple you can use a launch terminal and run a sha256sum (see this article for help):
https://osxdaily.com/2021/12/17/check-sha256-hash-mac/
This should cover the 3 major operating systems that are popular and in widespread use today. I haven't looked at using Android Smartphones but someone must have an App for it. The problem is you don't know the permissions the app is requesting so I personally avoid using smartphone apps for financial information.
What Goes Up Must come down -- David Clayton-Thomas (1968), BST
-
- Posts: 15368
- Joined: Fri Apr 10, 2015 12:29 am
Re: Long(ish) term storage for computer files?
If files archived for 40 years are encrypted, how and where will you store the encryption key, and what media will it use for 40-year storage?
Re: Long(ish) term storage for computer files?
Good point, no doubt you'll want a paper copy of the key and store it in a properly labeled envelope within the safe, that is why I prefer USB flash drives and/or SD cards. You can now get a 1TB SD card so you can put all your files on one card making it easier to have a total loss if you lose just that 1 tiny card.Northern Flicker wrote: ↑Wed Aug 17, 2022 1:14 pm If files archived for 40 years are encrypted, how and where will you store the encryption key, and what media will it use for 40-year storage?
What Goes Up Must come down -- David Clayton-Thomas (1968), BST
-
- Posts: 15368
- Joined: Fri Apr 10, 2015 12:29 am
Re: Long(ish) term storage for computer files?
If the media and papercopy of the encryption key are stored in the same safe, you neither need to, nor derive any value from encrypting the data.
There are many other subtle issues. For instance, do you require the capability to know that the data was compromised if there is unauthorized access?
Best practice of system administration for a typical enterprise is that system backups that contain confidential proprietary data (file systems, databases, etc.) should be encrypted. Managing the encryption keys to maintain both a high level of confidentiality of the data and a high level of assurance that the data can be recovered from the backup in the future turns out to be a fairly difficult problem. A common solution is an encryption key server that generates random keys and associates them with backup metadata. Great. Now how do you backup the encryption key server with an encrypted backup?
As a friend and colleague of mine likes to say, "It's turtles all the way down."
The best you can do is to articulate which risks you can bear and which you cannot, and engineer a solution that eliminates the risks you do not wish to bear.
There are many other subtle issues. For instance, do you require the capability to know that the data was compromised if there is unauthorized access?
Best practice of system administration for a typical enterprise is that system backups that contain confidential proprietary data (file systems, databases, etc.) should be encrypted. Managing the encryption keys to maintain both a high level of confidentiality of the data and a high level of assurance that the data can be recovered from the backup in the future turns out to be a fairly difficult problem. A common solution is an encryption key server that generates random keys and associates them with backup metadata. Great. Now how do you backup the encryption key server with an encrypted backup?
As a friend and colleague of mine likes to say, "It's turtles all the way down."
The best you can do is to articulate which risks you can bear and which you cannot, and engineer a solution that eliminates the risks you do not wish to bear.
- squirrel1963
- Posts: 1253
- Joined: Wed Jun 21, 2017 10:12 am
- Location: Portland OR area
Re: Long(ish) term storage for computer files?
Keep at least two backups plus the copy on your desktop. I actually have 3:
* Microsoft OneDrive
* 2 hard drives with backups done every 3 months or so, I keep one drive at home and one drive in the safe box.
What's important to keep in mind is that any single drive can fail at any time, regardless of the technology (good old spinning rust aka HDD or nand-flash technology aka SSD or USB flash drive), hence the reason for multiple copies.
Make sure though to never ever store passwords in clear text in the backups, use at least a password manager for them.
* Microsoft OneDrive
* 2 hard drives with backups done every 3 months or so, I keep one drive at home and one drive in the safe box.
What's important to keep in mind is that any single drive can fail at any time, regardless of the technology (good old spinning rust aka HDD or nand-flash technology aka SSD or USB flash drive), hence the reason for multiple copies.
Make sure though to never ever store passwords in clear text in the backups, use at least a password manager for them.
LMP | Liability Matching Portfolio | safe portfolio: TIPS ladder + I-bonds + Treasuries | risky portfolio: US stocks / US REIT / International stocks