More than one copy > 'longevity' in the marketing. Drives fail, make allowances. Realistically, it only has to last until you get an even bigger drive in a couple of years.
this post was submitted on 21 Jul 2024
39 points (100.0% liked)
datahoarder
6785 readers
2 users here now
Who are we?
We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.
We are one. We are legion. And we're trying really hard not to forget.
-- 5-4-3-2-1-bang from this thread
founded 4 years ago
MODERATORS
i few drives that I have are 10yod and a couple more 5+..
do modern drives meant to last and be replaced every a couple of years?
I dont mind if a drive fails because ill have dupes.. what I want is to minimize the failures.
wd red+ and wd ae make look promising but i dont have an experience with them
Doesn't matter what drives are "meant" to do: shit happens.
The only real insurance against data loss is duplication.
i agree.. but Im trying to find out if the chances of this shit happening is lower with specific hdds
Fair enough. In that case, I'd just go for ones with a 5-year warranty and call it a day. At least you get a replacement if it fails.
If you want longevity and shelf stability, tape drives are the way to go. You can get them in very large capacities, even into the hundreds of TB.
Their benefit is that they have no internal motorized components, they are a lot like VHS videocassettes - two spools with tape. This makes them very shelf-stable, unlike hard drives which can have their spindles seize up over time.
They also have absolutely epic data densities. You could store on one tape the contents of dozens of the largest hard drives currently available.
Their downside is that you need highly specialized hardware to read and record them. And this makes the hardware quite expensive.
So why don’t we use tape drives to store data? Because they store said data linearly - great for writing once, terrible for finding or updating said data - and because they are slow. You want to get to a file 20Tb in? Enjoy scrolling past every single byte up until that point.
But for cold backups, there ain’t nothing better.
We recently researched these for work.
They tick a lot of boxes - lots of space, reasonable speed, great cold storage figures. Reasonably priced tapes. Agree, they're the best thing. The slow read speed isn't quite as bad as expected (They can go extremely fast in seek mode), but definitely something to consider. We were okay with that for our needs.
But damn, the price of the hardware was horrendous - we got priced (I think) close to £20k for a suitable drive that met our needs. Completely killed the project. And remember that if you're doing site replication for DR, you'll need at least two of them. Sadly, it looks like we'll be using external HDD's for a while longer...
exciting but def not for me :)
Is there a go-to budget tape system that's worth looking at? In the past it seemed like getting anything with reasonable density like 10TB+ per tape was so expensive I could buy like 200TB of HDDs for the same price as the drive alone.
Don't buy into tape. It is costly and is inferior to hard drives by most metrics for smaller scale operations. You can easily get 8TB hard drives for less than $20/TB. While tape is cheaper than that, the drive to actually use it is expensive, plus you get all the disadvantages of the tape itself.
Fun fact: you can probably buy a whole server, external sas card and disk shelf for less than the cost of a somewhat modern tape drive.
If you are wanting to store less than 100TB of data, it would probably be cheaper to use drives, then in 3-5 years buy another set of disks and still be ahead compared to tape.
Yeah that's what it seems like, used SAS HDD are around $7/TB right now and I don't think anything is going to be cheaper than that.
Yeah but what about in 9000 years?
Where do you buy these? What's the cost per space?
The hardware to read the tapes are calibrated to the maximum size they are configured to accept. So when you hit up eBay, you will need to know the maximum amount of data you will need, and either the size of the largest tape drive to hold all that - if you are not getting a machine with an auto-loader - or the maximum number of drives that the machine’s autoloader can take, so you can size the tapes properly for the data.
Say you need to back up 25Tb. You are unlikely to ever need more backup than that. So you either look for a machine that takes 25Tb tapes, or you get a machine that can take max. 5Tb tapes, but has an autoloader that can hold at least 4 additional tapes (in addition to the one in the drive) such that all five will automagically cycle through the backup process. That way, all 25Tb will be backed up in either case without your direct and immediate involvement, all you have to do is rotate the tapes off-site after the backups are done, and slot the next ones in for the next backup run.
Obviously, incremental backups are a no-go, as backups are stored off-site. So it’s an all-or-nothing process. And as such, this is usually done both on your entire primary data set (for fast total-disaster restores) once in a while, with a different set of tapes focusing on your local/on-site warm backups and backing up only the atomic/incremental locally-stored backups for the day/week/month.
I have a couple of 8TB Seagate SMR drives. They’re slow as sin but perfect for cold storage backups.
One sits in a safe deposit box and the other is in my backup server and gets zfs snapshots replicated to it automatically with sanoid. I swap them out about every six months. My data is very safe.
Main server zfs array with snapshots replicated hourly to backup zfs server, replicated weekly to HD swapped in SD box every 6 months.
Im curious on why you are using zfs snapshots? (i never used them myself as I handle my backups with duplicity)
Snapshots on the file system are instant and can be rolled back or opened in read only mode if you need an older version of the file.
Sending a snapshot through replication is also much faster as the system just sends the snapshot itself which updates the files system in one pass by sending all of the changes made rather than having to compare file by file.
For comparison, using rsync to update my backup with 250,000 files might take 20-30 minutes because it has to check every file on the source and destination to see if it has been modified. Sending a zfs snapshot is 15 seconds because it just sends the differences in one pass.
do you encrypt them?
Yes they’re all encrypted with LUKS. Using unRAID as my base OS which handles most aspects of the process.
I found a e-waste retailer that is selling old new stock of ST3000DM001 for 25 euro
For server usage they're awful due to the extremely high failure rate (so high that it deserved a dedicated page on Wikipedia) but for cold storage they are perfect as the cost per TB is very good