Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

This question was inspired by http://superuser.com/questions/374386/how-to-store-and-preserve-lots-of-data. There have been other similar questions, but none with the same criteria.

This is two questions in one.

  1. How do you store financial/critical records that should survive anything but a fire and should be available for decades?
  2. Lets say I want to store family photos/videos and want people do be able to find them in storage 100 years from now and still be able to use them. How would this be done?

Criteria

  1. Long term means 30+ years guaranteed. 100+ years average. [If this is not practical, use the closest solution]
  2. High volume means a couple terabytes.
  3. Answers can be 'no-compromise/industrial' solutions or practical solutions for the home office/small business user.
  4. Media will not be active during the timespan. (i.e., if you suggest hard drives, they will not be spinning).
  5. Further, there is no expectation of needing to read these archives. They are there for emergency or "for future generations" purposes.
  6. Should not require maintenance (if at all possible).

My thoughts:

  1. CD-R/DVD-Rs have proven to me, even in the short term, to be a terrible medium for backups. They seem to be very fragile and seem to lose their data a very short time even when in pristine condition.
  2. I can't help but think that storing data on a couple of 1TB hdd's and then expecting them to spin up correctly a decade or two later to be a terrible idea. Am I wrong?
  3. Industrial tape drives seem like a viable option?
share|improve this question
    
I'm no expert, but I'd say tape. This question might be better on Server Fault, but I honestly don't think it fits perfectly on either, so I'll decline to vote. It is a good question and should live somewhere. –  Shinrai Jan 4 '12 at 17:57
    
I agree @Shinrai. I am welcome to moving this somewhere else if someone can comment on where it should live. –  user606723 Jan 4 '12 at 18:01
    
If you want no compromise, there is existing technology that is designed to last at least 40,000 years with no intervention: voyager.jpl.nasa.gov/spacecraft/goldenrec.html –  fixer1234 Feb 5 at 23:20
    
The future is in crystals, it can potentially store 360TB and last a million years. See: 5D 'Superman memory crystal' heralds unlimited lifetime data storage –  kenorb May 21 at 11:03

13 Answers 13

up vote 14 down vote accepted

Paper

Other than archival ink on archival paper in sealed storage, no current medium is proven to last an average 100 years without any sort of maintenance.

Archival Paper

Older papers were made from materials such as linen and hemp, and so are naturally alkaline. or acid free, therefore lasting hundreds of years. 20th century paper and most modern paper is usually made from wood pulp, which is often acidic and does not keep for long periods.

Archival Inks

These permanent, non-fading inks are resistant to light, heat and water, and contain no impurities that can affect the permanence of paper or photographic materials. Black Actinic Inks are chemically stable and feature an inorganic pigment that has no tendency to absorb impurities like other ink pigments can.

Redundant storage

Torvalds once said

Only wimps use tape backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it

Which suggests you should not rely on a single copy on a single medium.

Not magnetic media?

http://www.zdnet.com/blog/perlow/the-bell-tolls-for-your-magnetic-media/9364?tag=content;siu-container

  • Typical example of irretrievable degradation of magnetic media.
  • Issues of hardware and software (and data formats)

Not specialized systems

In 2002, there were great fears that the discs would become unreadable as computers capable of reading the format had become rare and drives capable of accessing the discs even rarer. Aside from the difficulty of emulating the original code, a major issue was that the still images had been stored on the laserdisc as single-frame analogue video,

http://en.wikipedia.org/wiki/BBC_Domesday_Project#Preservation

Long Term Personal storage

http://www.zdnet.com/blog/storage/long-term-personal-data-storage/376

  • both the media AND the format can become unreadable.
  • print on acid-free paper with pigment inks and store in a cool, dry and dark place.
  • The first problem is picking data formats for maximum longevity.
  • Avoid using proprietary formats
  • USCSF is transferring all their original tapes - many in now-obsolete formats like BetaSP and VHS - to the 75Mbit motionJPEG2000 format
share|improve this answer
    
1) Can you provide details about this? Will normal hard copies not last that long? (Photos from 100 years ago seems to be fine, AFAIK). 2) If no current data medium will last this long, I suggest that we use the closet solution possible. It's depressing that decades from now we won't be able to look through old boxes and expect to be able to look at any of our old, forgotten photos, etc. –  user606723 Jan 4 '12 at 18:05
    
@user606723: see updated answer –  RedGrittyBrick Jan 4 '12 at 19:11
    
I've figured that laser printing on acid-free paper would be a good way to store data (a few megabytes per page) that has a high probability of being readable in 100-200 years. The software to read it would be relatively simple, and one presumes that scanners will always be available, so the format (so long as not too convoluted) would never really "go away" beyond the ability of a competent amateur to recover. –  Daniel R Hicks Jan 29 '12 at 16:10
up vote 10 down vote
+100

/EDIT: I will soon post a follow-up with more information I have gathered and better solutions. Meanwhile, you can check about error-correcting codes and file fixity. I also developped a set of opensource and simple python scripts (to be used in complement with other solutions like DVDisaster or MultiPar PAR2).

Short answer

There's currently no failproof and scientifically proven way to guarantee 30+ years of cold data archival. But some projects are aiming to do that like the Rosetta Disks project of the Long Now museum, although they are still very costly and with a low data density (about 50 MB).

In the meantime, you can use scientifically proven resilient optical mediums for cold storage like archival grade DVD+R like Verbatim Gold Archival, or better, BluRay Discs of HTL type like Panasonic's, and keep them in water-tight boxes in a soft spot (avoid high temperature) and out of the light. Make multiple copies of your data, and compute hashes to check regularly that everything is alright, and every few years you should rewrite your data on new disks.

Long answer

I have similar needs as yours, thus I tried to find a maximum of documentation and scientific results about mediums for long-term cold archival of digital data.

Results: I found a lot of documentation, but very few using independent scientific results, and in the end, no perfect solution emerged from those studies.

The first thing to clarify is what you are looking for (following your description):

  • Long-term archival: you don't just want to store backups of your OS, but you want to keep copies of your sensible, irreproducible data. Archiving is fundamentally different than a backup, as well explained here. A short definition of the difference: backups are for data that regularly get updated and thus need to be refreshed into backups, whereas archives are static data that you would likely write only once and just read from time to time. Archives are for intemporal data.

  • Cold storage: you want to avoid maintenance of your archived data as much as possible. This is a BIG constraint, as it means that the medium must use components and a writing methodology that stay stable for a very long time, without any manipulation from your part, and without requiring any connection to a computer or electrical supply.

To meet both criteria is impossible, thus I propose to prioritize cold storage by choosing mediums that maximize the data retention time without any manipulation, and then long-term archiving by choosing methods that should avoid losing data in the long run.

Cold storage mediums

We already defined above what a good cold storage medium should be: it should retain data for a long time without any manipulation required (that's why it's called "cold": you can just store it in a closet and you do not need to plug it into a computer to maintain data).

Hard Drives (HDDs) are known to have an average life span between 3 to 8 years: they do not just degrade over time, they are guaranteed to eventually die. The following curves show this tendency for all HDDs to die at a staggering rate:

Curves showing the evolution of HDD failure rate depending on the error type: Curves showing the evolution of HDD failure rate depending on the error type

Curve showing the evolution of HDD failure rate by merging all error types: Curve showing the evolution of HDD failure rate by merging all error types

From a post on Apple community by PlotinusVeritas.

You can see that there are mainly 3 types of HDDs relatively to their failure: the rapidly dying ones (eg: manufacturing error, bad quality HDDs, head failure, etc.), the constant dying rate ones (good manufacturing, they die for various "normal" reasons, this is the case for most HDDs), and finally the robust ones that live a bit longer than most of HDDs but they eventually die too not too long after the "normal ones" (eg: lucky HDDs, not-too-much used, ideal environmental conditions, etc..). Thus, you are guaranteed that your HDD will die, even if it's robust and if you are lucky, simply because this magnetic technology is not made to guarantee long term retention of data.

In addition, HDDs are known to demagnetize over time if not used (including SSD). Thus, you cannot just store data on a hard disk, store it in a closet and think that it will retain data without any eletrical connection. Thus, HDDs are clearly not a good fit for cold storage. This also rules out magnetic tapes (even if they are more reliable than HDD, they are still not a good fit for cold storage for the same reasons, as their primary focus is for backups, not long-term archives).

CompactFlash and SecureDigital (SD) cards are known to be quite sturdy and robust, able to survive catastrophic conditions.

The memory cards in most cameras are virtually indestructible, found Digital Camera Shopper magazine. Five memory card formats survived being boiled, trampled, washed and dunked in coffee or cola.

However, as any other magnetic based medium, it relies on an eletrical field to retain the data, and thus if the card runs out of juice, data may get totally lost. Thus, it may be not a good fit for cold storage (as you would need to regularly plug the card, or rewrite on the card, to refresh the electrical field), but it can be a good medium for backups and short-term archival.

Optical mediums: Another possibility is to use optical mediums, like CD, DVD or BluRay (BD). Like any technology, new iterations not only offer bigger density (storage room), but also error correction along other features. The first debate about DVD reliability was between DVD-R and DVD+R, and even if DVD-R are still common nowadays, DVD+R are recognized to be more reliable and precise. DVD have since seen emerged archival grade discs, specifically made for cold storage, and which claim that they can withstand a minimum of about 20 years without any maintenance:

Verbatim Gold Archival DVD-R [...] has been rated as the most reliable DVD-R in a thorough long-term stress test by the well regarded German c't magazine (c't 16/2008, pages 116-123) [...] achieving a minimum durability of 18 years and an average durability of 32 to 127 years (at 25C, 50% humidity). No other disc came anywhere close to these values, the second best DVD-R had a minimum durability of only 5 years.

From LinuxTech.net.

Furthermore, some companies even specialized in very long term DVD archival and extensively market them, like the M-Disc from Millenniata or the DataTresorDisc, affirming that they can retain data for over 1000 years, and verified by some (non-independent) studies (from 2009) among less-scientific others.

This all seems very promising! Unluckily, there's not enough independent scientific studies to confirm these claims, and the few ones available are not so enthusiastic:

Humidity (80% RH) and temperature (80°C) accelerated ageing on several DVDs over 2000 hours (about 83 days) of test with regular checking of readability of data: Humidity and temperature accelerated ageing on several DVDs brands

Translated from the french institution for digital data archival (Archives de France), study from 2012.

The first graph show DVD with a slow degradation evolution. The second one DVD with rapid degradation curves. And the third one is for special "very long-term" DVDs like M-Disc and DataTresorDisc. As we can see, their performance does not quite fit the claims, being lower or on par with standard, non archival grade DVDs!

However, inorganic optical discs such as M-Disc and DataTresorDisc get one advantage: they are quite insensible to light degradation:

Accelerated ageing using light (750 W/m²) during 240 hours: Light accelerated ageing on several DVDs brands

These are great results, but an archival grade DVD such as the Verbatim Gold Archival also achieves the same performance, and furthermore, light is the most controllable parameter for an object: it's quite easy to put DVD in a closed box or closet, and thus removing any possible impact of light whatsoever. It would be much more useful to get a DVD that is very resilient to temperature and humidity than light.

This same research team also studied the BluRay market to see if there would be any brand with a good medium for long term cold storage. Here's their finding:

Humidity and temperature test, under the same parameters as for DVDs: Humidity and temperature accelerated ageing on several BluRays brands

Light test, under the same parameters: Light accelerated ageing on several BluRays brands

Translated from this study from Archives de France, 2012. Two summaries of all findings (in french) here and here.

In fine, the best BluRay disc (from Panasonic) performed similarly to the best archival grade DVD in humidity+temperature test, while being virtually insensible to light! And this BluRay disc isn't even archival grade. Furthermore, BluRay discs use an enhanced error correcting code than DVDs (themselves using an enhanced version relatively to CDs), which further minimizes the risks of losing data. Thus, it seems that some BluRay discs may be a very good choice for cold storage.

And indeed, some companies are starting to work on archival grade, high density storage BluRay discs like Panasonic and Sony, announcing that they will be able to offer 300 GB to 1TB of storage with an average life span of 50 years. Also, big companies are turning themselves towards optical mediums for cold storage (because it consumes a lot less resources since you can cold store them without any electrical supply), such as Facebook which developped a robotic system to use Blu-ray discs as "cold storage" for data their system rarely access.

Long Now archival initiative: There are other interesting leads such as the Rosetta Disc project by the Long Now museum, which is a project to write microscopically scaled pages of the Genesis in every languages on earth the Genesis got translated to. This is a great project, which is the first to offer a medium that allows to store 50 MB for really very long term cold storage (since it's written in carbon), and with future-proof access since you only need a magnifier to access the data (no weird format specifications nor technological hassle to handle such as the violet beam of the BluRay, just need a magnifier!). However, these are still manually made and thus estimated to cost about $20K, which is a bit too much for a personal archival scheme I guess.

Internet-based solutions: Yet another medium to cold store your data is over the net. However, cloud backup solutions are not a good fit, for the primary concern than the cloudhosting companies may not live as long as you would like to keep your data. An alternative is to rent your own private server online, and if possible, choose one with automatic mirroring/backup of your data in case of hardware failure on their side (a few ones even guarantee you against data lost in their contracts, but of course it's more expensive). This is a great solution, first because you still own your data, and secondly because you won't have to manage the hardware's failures, this is the responsibility of your host. And if one day your host goes out of business, you can still get your data back (choose a serious host so that they don't shutdown over the night but notify you beforehand, maybe you can ask to put that onto the contract), and rehost elsewhere.

If you don't want to hassle of setting up your own private online server, and if you can afford it, Amazon offers a new data archiving service, called Glacier. The purpose is exactly to cold store your data for the long-term: thus, it costs a lot to store data on a Glacier, but it costs even more to get back this data, as this service is made to store data out of reach, not to keep data that you want to often access. This means that this service quotes prices for writing data, but also for reading them. This service has a huge cost, but it may be a good deal for some of your most sensible data (ie: if you have a few text files or images that are VERY sensible, since this kind of data is usually of small size, it won't cost you very much to store in a Glacier).

Shortcomings of cold storage: However, there is a big flaw in any cold storage medium: there's no integrity checking, because cold storage mediums CANNOT automatically check the integrity of the data (they can merely implement error correcting schemes to "heal" a bit of the damage after corruption happened, but it cannot be prevented nor automatically managed!) because, contrariwise to a computer, there's no processing unit to compute/journalize/check and correct the filesystem. Whereas with a computer and multiple storage units, you could automatically check the integrity of your archives and automatically mirror onto another unit if necessary if some corruption happened in an data archive (as long as you have multiple copies of the same archive).

Long-Term Archival

Even with the best currently available technologies, digital data can only be cold stored for a few decades (about 20 years). Thus, in the long run, you cannot just rely on cold storage: you need to setup a methodology for your data archiving process to ensure that your data can be retrieved in the future (even with technological changes), and that you minimize the risks of losing your data.

There's no foolproof rules here, but there are some well accepted advices:

  • Redundancy principle: this is not an advice, this is based on information theory. To keep data, you need to duplicate this data. Error codes are exactly an automatic application of the redundancy principle. However, you also need to ensure that your data is redundant: multiple copies of the same data on different discs, multiple copies on different mediums (so that if one medium fails because of intrinsic problems, there's little chances that the others on different mediums would also fail at the same time), etc. On this topic, a software called DVD Disaster offers to add error correction codes (in addition of those already implemented in storage mediums) to your data to ensure redundancy. I didn't try it (yet), but the idea of allowing the user to choose the percentage of error-correction is certainly enticing and should be promoted, as error-correction codes are VERY efficient (potentially a lot more than just duplicating your data on the same disc, see for example convolution error codes). For example, using TurboCodes (a recursive convolutional error code, used for 3G and 4G), you can expect to correct a lot more using the same amount of data (see redundancy rate and F merit rate, a very good introductory book being this one in French, or this one in english, both by the TurboCodes author).

  • Verify the integrity of your files before (re-)copying them: Hash your files, and check them before and after making copies/archives (because your files copies can get silently corrupted, and if you then copy the copies that have been tampered, you will end up with totally corrupted files). This point is very important, because cold storage mediums CANNOT automatically check the integrity of the data (they can merely implement error correcting schemes to "heal" a bit of the damage after corruption happened, but it cannot be prevented nor automatically managed!) because, contrariwise to a computer, there's no processing unit to compute/journalize/check and correct the filesystem. There's a software called CDCheck specifically made for this purpose, but you can just use any GNU hashing software, they are easy to script to batch process your whole data collection.

  • Store your archives mediums on different locations (and not only in your house!) to avoid for real life catastrophic events like flood or fire.

  • Store in specific containers with controlled environmental parameters: for optical mediums, store away from light and in a water-tight box to avoid humidity. For hard drives and sd cards, store in anti-magnetic sleeves to avoid residual electricity to tamper the drive.

  • Use good hardwares and check them beforehand (eg: when you buy a SD card, test the whole card with softwares such as HDD Scan to check that everything is alright before writing your data). This is particularly important for optical drives, because their quality can drastically change the quality of your burnt discs, as demonstrated by the Archives de France study (a bad DVD burner will produce DVDs that will last a lot less).

  • Store alongside your data archives every softwares and specifications that are needed to read the data. Remember that specifications change rapidly, and thus in the future your data may not be readable anymore, even if you can access the file. Thus, you should prefer open source formats and softwares, and store the program's sourcecode along your data so that you can always adapt the program from sourcecode to launch from a new OS or computer.

  • Lots of other methods and approaches are available here, here and in various parts of the Internet.

Conclusion

You should use whatever technology you see fit and which you are the most confortable with. Also, ensure that the hardware you use is of good quality (ie: the quality of the DVD/BD burner impact as much as the quality of the blank DVD/BD you will use to store your data, as demonstrated by the Archives de France study).

Personally, I would (will) primarily use BluRay HTL discs or archival grade DVD discs stored in water-tight boxes and away from light. I will also maintain copies on SD cards and HDDs (mainly for data that changes over time, like working documents) that I will regularly burn on multiple BluRay discs. I would also store those discs in multiple locations: one in my own house, one at my relatives, and maybe one at a safe deposit box at the bank for really sensitive data.

More importantly, I would regularly checkup on the integrity of the data stored on those discs by checking the MD5 or SHA-1 hash I computed on the files before burning them to the disc. If you can checkup twice a year, I feel that would be the best, but if not, I think that using a combination of optical and magnetic mediums and multiple copies you can still safely check only every few years (like every 3 to 5 years) without having too much to worry about.

On a final note, I really do hope that more R&D will be put on this problem. This is a major issue for our current society, having more and more data digitalized, but without any guarantee that this mass of information will survive more than a few years. That's quite depressing, and I really do think that this issue should be put a lot more in the front, so that this becomes a marketing point for constructors and companies to make storage devices that can last for future generations.

Automating long-term cold store management?

Companies managing big volumes of data need to be able to automatically manage archives and errors, and automatically ensure their integrity (like for example Facebook with the automated BluRay based cold storage facility).

However, to my knowledge, there's nothing like that for home consumers. Indeed, one of the big hassle in long-term storage, is that even if you can use cold storage mediums, you still have to manually check the integrity regularly. Wouldn't it be great if it would be done automatically, and alert you if one of the data archives failed? Even more, wouldn't it be great if such a system could then automatically use other copies of the corrupted archived data to create a new copy (ie: burning on a new BluRay disc or any other medium of choice), thus ensuring that you always keep x copies of your archived data at any given time?

Unfortunately, such a system does not exists. But, we could easily imagine a simple script running on a minimal computer like a Raspberry Pi to be the processing unit of this system. The only remaining problem would be to make a mechanical system to automatically load/unload and seek through a physical library of optical discs. Or you could do that with magnetic tapes such as LTO4. Of course, you would still need to manually dispatch some of those copies in off-house sites to account for natural catastrophic events, but it would be quite foolproof for any other type of event, as long as you maintain your Raspberry Pi on electrical supply, and that you replace its components when they wear off (like the disc burner).

share|improve this answer
    
Outstanding answer! This needs far more upvotes. –  DragonLord Feb 4 at 1:24
    
You plan to add MORE information? Consider publishing it as a textbook. :-) –  fixer1234 Mar 21 at 18:56
    
@fixer1234 yes I plan to add more information and, more importantly, more pertinent and reliable information. There are a lot of misconceptions and falsely perceived secure solutions in the field of file fixity, so there's quite a lot to say. I have found so much info after publishing this post that an update is clearly needed, and I already compiled everything in my notes along with references. I'm not sure SuperUser is the best place where to publish all this data but I have no blog of my own :-/ I will try to be as concise as possible. –  gaborous Apr 3 at 20:27

There is no easy solution. The archive maintenance is a process, not one-time job. All three currently-available archival media types have their own pluses and minuses, however these arguments apply to all media types:

  1. Nobody stored DVDs or hard disks for 30 or 100 years, for obvious reasons. So there is no track record and nobody knows how the media will age. Artificial aging tests do not prove much, and you rely on vendor's testing, (not impartial).

  2. You must store the media in the controlled environment for best results (constant temperature/humidity, low light, etc.). Otherwise media life is shortened significantly.

  3. You must maintain the hardware and software that reads the media (e.g. SATA interfaces might not be readily available in 30 years from now).

So, in my opinion, the only viable solution for home users or small businesses is this:

  1. Maintain multiple copies of all data on diverse media types (both hard disks and DVDs)
  2. Maintain multiple copies of all data in multiple locations (at home and in your banks's safety deposit box).
  3. Copy all data to new media every so often (e.g. copy to a new hard disk and new DVD disks every 2 years. As the data density grows, you will probably need fewer disks, too.
  4. Maintain paper copies for all critical data, if possible (e.g. print those yearly general ledgers for your business, print most precious family photos, etc.)
share|improve this answer
1  
I wonder if there is RAID for DVDs.... ie.. if you store DVDs for two years, you might be seansibly sure that 80% of them would be error free, so you might have two parity disks. Hmmmm. usenet uses parity files I think. Might be worth using something like that for DVD/CD/BD archival. –  user606723 Jan 4 '12 at 19:00
1  
@user606723: This is a very good idea! I suggest using something like multi-volume RAR archive (if the original files are really big) with PAR2 parity files... –  haimg Jan 4 '12 at 19:29
3  
Interface compatibility would be a major concern; it's been about 30 years since the IBM XT was introduced, yet how many computers today can in any way interface with a pre-ATA hard disk? How many computers built today can even interface with a PATA hard disk without additional hardware (controller card or USB adapter)? –  Michael Kjörling Jan 30 '12 at 9:48

Up to 5TB (or more?) you can securely store up to 30 years on a magnetic tape aka tape drive. This time is proven. Blue-ray recordables shall safely store your stuff up to 30 years also, but it's capacity is around 100GB.

If you have more money, you'd store it on black/white 35mm film. It's assumed that data can be restored (depending on density) for the next 700 years. (German link to wikipedia)

share|improve this answer
    
For the record, writing to 20-50 blu-ray disks is not out of the question. –  user606723 Jan 4 '12 at 18:14
    
I've never heard of data archival on 35mm, although the principle is obvious I suppose. What's the density like? –  Shinrai Jan 4 '12 at 18:41
    
@Shinrai: I dunno the density of film, sorry –  tuergeist Jan 5 '12 at 12:18
    
You can probably figure a density somewhere between 1 and 10 megabits per frame. –  Daniel R Hicks Jan 29 '12 at 16:05
3  
Nikon's LS-9000 ED scans film at 4000 dpi, giving you 21.4 Mp/frame at 35 mm (24 x 36 mm). If you can use 1/10th of that for actual data storage (allowing for film imperfections, focusing and resolution limitations in the optics at both ends, etc) that's 2 Mb/frame or something like 10 MB for a 36-exposure roll of film and pure black/white. If the scanner's 4000 dpi is the limiting factor, that's 100 MB for a 36-exp roll. Of course, you'd still have to in some other way preserve information on how to read the data, because to the naked eye the frames would likely appear fairly uniformly gray. –  Michael Kjörling Jan 30 '12 at 9:57

I'd go microfilm. I don't know if it is still manufactured, but I'd be surprised if it wasn't. Silver-based negatives last hundreds of years if stored correctly. Of course that is a huge investment, and will take up a whole room for photography and viewing, and that is not counting storage. So that's only if you really MEAN 100 years+ with no maintenance.

If not - and chances are you aren't unless you want to make a time capsule -, just use HDD backups, and copy the whole stuff over to new media in every 10-15 years. Really, there is no better insurance against the aging of the medium than copying the whole thing over every 10 years or so. Better than microfilm, better than clay tablets, better than stone obelisks buried in the desert sand.

share|improve this answer

For that kind of time spans, anything that already is on paper (or can be easily printed without losing information) would be best to store in that form. Just be mindful of the paper and toner you use for the hardcopy.

As for others, I don't know of a currently used digital medium that would last for those spans of time. If you spend time (and thus money) to refresh your collection, then a magnetic tape might be a viable option - but even then you'd need some redundancy, as you might just find out that a single tape has gone bad (or it might be that the tape drive just happens to mangle the tape on reading it).

And even when you can get the actual media to stand the test of time, you'd still be faced with the issue whether any program could read the media at 30 years from now, let alone 100 years from now.

share|improve this answer
    
Magnetic tape is subject to a number of failure modes, from "print through" to demagnetization over time to the oxide simply falling off the tape. –  Daniel R Hicks Jan 29 '12 at 16:12

It's true that common CD-Rs and DVD-Rs are not reliable enough for archiving important data. But you can get DVDs that are not so quick to decay:

https://www.google.com/search?q=archival+dvd-r

share|improve this answer
    
Thank you for pointing this option out, a good alternative to M-Discs that is accessible to just about anyone with a DVD recorder. –  gaborous Feb 2 at 0:32
    
"Verbatim Gold Archival DVD-R [...] has been rated as the most reliable DVD-R in a thorough long-term stress test by the well regarded German c't magazine (c't 16/2008, pages 116-123) [...] achieving a minimum durability of 18 years and an average durability of 32 to 127 years (at 25C, 50% humidity). No other disc came anywhere close to these values, the second best DVD-R had a minimum durability of only 5 years.", linuxtech.net/tips+tricks/best_safe_long-term_data_storage.html –  gaborous Feb 2 at 0:35

I recommend a three inch diameter nickel disk with information microscopically etched onto its surface.

http://rosettaproject.org/blog/02008/aug/20/very-long-term-backup/

share|improve this answer
2  
Does it have to be exactly three inches? I have a 75 mm diameter nickel disk handy... –  Michael Kjörling Dec 26 '13 at 21:44
    
The only problem with this approach is that it can only store still images (scans). But it's currently the best approach for VERY long term storage (up to 2000 years, millenias yey!). Another shortcoming highlighted by some comments on the blog is that it can only store about 50 MB of data. –  gaborous Feb 2 at 0:37

You need to mix different technologies, locations and mediums in order to achieve long life backups:

  • Burn to DVD - Bluray at low speed. Keep them in low light, low temp, low humidity, free of scratch.
  • Keep a copy in a RAID 1, Raid5, Raid6 or Raid10 unit.
  • Keep another copy in an external HDD
  • Keep a copy in the cloud (carbonite, crashplan)
  • Keep a copy on M-Disc technology (Mdiscs and Mdisc burners) are not available at Amazon.com at very good prices. Manufacturer states they can hold data for 1000 years.
share|improve this answer
    
I see three of your five bullet points are really variations of a single theme: magnetic hard drive storage. As for your last point, the issue isn't so much how long the media will retain the data (and at least hard disk manufacturers commonly cite numbers that are far better than reality) but for how long equipment to read the data will be available or knowledge of how to make them will be available. All of your suggested techniques are high-tech. Suppose the Vikings stored data on blu-ray disks; what are the odds we'd have the knowledge how to interpret that data now? –  Michael Kjörling Dec 26 '13 at 21:43
    
@MichaelKjörling Store an additional computer with all the peripherals needed. Use ROM memory if needed. –  QuyNguyen2013 Dec 24 '14 at 3:16

As someone already mentioned there is a new tech called M-Disc. They are very reliable: http://www.zdnet.com/torture-testing-the-1000-year-dvd-7000023203/ We started to use them for securing images of production machines disks. There are already Blu-Rays on the market. Only disadvantage is they are slower than classic B-RDs.

share|improve this answer
    
I have a similar need to OP and after reading about it, I think I'll give it a try for this solution, thank's for pointing out this technology! It only requires to buy a DVD or Blu-ray writer compatible with M-Disc, and LG already produced a lot, so it's also quite accessible and low-cost! –  gaborous Feb 2 at 0:28
    
In fact it seems M-Discs are not as reliable as they pretend. An independent french study by the Archives de France (official France's data archival institution) to find the best data archival support, and they found that M-Discs do not really stand against humidity and temperature (accelerated ageing). I'll post here an answer with more details. –  gaborous Feb 3 at 17:51

I've read that 'M-Disc' have created a DVD which needs a special writer yet is readable on generic DVD readers. They claim an estimatible life span of 1000 years, stating it cannot be accurately tested. Long exposure to the sun, scratches, multiple usage etc and the disc is 100% useable. I'd be interested in any feedback from anyone who's encountered this system.

Here's an excerpt from Dell who maybe installing the M-Disc drive in their new laptops/PC's

M-DISC Ready drives laser-etch data into an inorganic rock-like material to prevent data loss, ensuring your files are safe and can be stored for up to 1000 years, the company claims.

Unlike all other recordable DVDs that use organic dyes to hold data, M Discs won’t fade or degrade over time.

share|improve this answer
    
Instead of reposting with more info, you should have edited your original post. –  Kazark Apr 16 '13 at 19:25
    
Can you cite the quote with a link or something? Also, you can use > to format it as a block quote. –  Kazark Apr 16 '13 at 19:28

Practical long term data storage using the current technology of the year 2014:

...and this is what I am doing.

Get two of the multi-terabyte drives, for example two drives 3 terabytes each. Call one TB-1 and the other TB-2. Back up everything to TB-1. After a year of backing up to TB-1, re-format TB-2 and copy TB-1 to TB-2. Then for the next year, back up everything to TB-2. After that year, re-format TB-1 and copy TB-2 to TB-1 thereby starting the bi-annual cycle again.

The reformatting restores the magnetic strength of the sector markers. And the copying restores the magnetic strength of the data.

The same principle can be applied to tape backup and CD backup, or most any other backup. But CDs are so inconvenient because they can go bad in less than a year, and you need so many of them to back up everything. So, burning copies of all backup CDs every 5 months is just too much work. So far, I can store my whole life on one multi-terabyte drive.

share|improve this answer
    
CD's go bad in less than a year? Are you saying you don't own any CD more than 1 year old? I have data and audio CD's from more than a year I can assure you, and they work fine! –  Dave May 19 '14 at 14:04
    
I have CDs from 1998 which still work fine. Regardless of us knowing this isn't true, what makes you believe this is the case? Can you source your information? Thanks. –  Matthew Williams May 19 '14 at 14:29

If you want to have a method to solve this problem, you should study the Digital Presevation field.

http://en.wikipedia.org/wiki/Digital_preservation

Digital preservation is the method of keeping digital material alive so that they remain usable as technological advances render original hardware and software specification obsolete (wikipedia)

There is also a reference model: OAIS http://en.wikipedia.org/wiki/Open_Archival_Information_System

There are a few open source and commercial solutions to accomply it. Libraries and Archives use this technologies to preserve digitized books for long periods of time.

share|improve this answer
    
Keeping data for a long period of time does not equal the media itself surviving for that long, as has been pointed out in several highly voted answers already. One major part of digital preservation is data migration as media ages and becomes obsolete. –  Michael Kjörling May 21 at 8:54
    
Thank you Michael. Only pointing to OAIS as a method to achieve the real objetive. –  AGM May 22 at 11:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.