Sudden doubts about DVD+R reliability

Question

We periodically write two copies of a DVD image. The images typically contain on the order of 5000 files, with relatively few files in each folder. We write two because these are backups of ongoing work, and one copy is carried to an offsite location for safety. There are presently about 500 of these discs in the offsite box.

We recently had occasion to restore a number of files from these backups, and found that many files could not be read. At least not on the first try. And in many cases they could be read from a different DVD drive. Needless to say, this has caused some concern about the health of these backups.

So we added a checksum step to our image generation, where we use sha1sum to collect checksums of every file on the disc. With a freshly burned disc in hand that includes the sums, we then proceeded to attempt to validate the process using sha1sum --check checksum.txt. Now we are even more concerned.

A typical result is that about the first 1/3 to 2/3 of the files check OK. Then sha1sum starts reporting "permission denied", and continues to do so for nearly every remaining file in the list.

Oddly, if the list is shortened to just the failed files, sha1sum proceeds to read most or all of those OK.

This is consistent with results of using robocopy to copy all of the files off a disc. Robocopy gets most of the files on the first pass, then gets the rest on subsequent passes.

Our fundamental question is do we have a hardware problem with the DVD writer, our DVD readers, or our PCs? Or is this a media problem with the quality of the blank media (the current stack says Memorex)? And how should we tell?

Answer 1

I would guess you are suffering from old DVDs, or the particular box of discs that you bought has a defect (someone left them in the sun for too long before they were put on the shelf).

On the other hand, if your DVD burner is old, it's likely that the laser is toast. Lasers have a hard life and don't last very long. A bad laser could affect the readers and writer, presuming they were bought at similar times. You replace the writer (or reader) and try again.

This ends my official answer. For unsolicited, but highly practical advice, keep reading.

---

Avoid the disposable media du jour

If your data is important, don't trust a 50¢ DVD to store it indefinitely. Additionally, don't trust SD cards, usb flash disks, zip disks, LS120s, or floppy disks, and all for the same reasons:

• Small things get lost, stolen, run through the wash, and stepped on
• Stuff that's archived offsite has to be moved; moving computer media has a fairly significant impact on reliability (see above)
• Manufacturing tolerances are lower for cheap, extremely-mass-produced things
• The reliability of media is always hugely overestimated (DVDs last for 100-200 years!)
• Most of todays computing devices don't have optical drives. In 5 years, you might not have a flash card reader, a USB port, or even a regular computer.

For similar reasons, don't archive data (remove it from a computer and keep it only on removable media) either. If it's important enough to keep, or if you're not sure how important it is, then keep it on a computer. Otherwise, delete it.

Reliable Backups

The only reliable backups that I've ever known were those kept on real hard disks. You don't have a lot of data; 500 DVDs should fit onto a cheap hard disk with room to spare (and will weigh a great deal less).

• Buy the cheapest external drive you can find. 2TB USB drives are about $70. In a few years, bigger (4TB+) external drives will likely be about $70.
• Copy your important data on there, and get another one to do a computer backup (now you have three copies of the data).
• Replace these drives every couple years, depending on your budget.
• This goes without saying, but do not move or drop your backup disks! Leave them on your desk or (for network-attached disks) in your server closet. Moving them risks adding vibration, breaking cables, dropping, etc.

Hard disks are also way easier to use and way faster to write than DVDs (and other media).

(The one exception are tape backups, which can be thousands of times more reliable than hard disks, but not really practical unless you're backing up a bunch of servers).

Offsite Backups

Find an online provider (Backblaze etc.).

Don't consider an internet backup provider to be a permanent storage medium either. Businesses will fail, get hacked, and have IT accidents. Consider it to be an ultra emergency backup, and price that accordingly. Keep in mind that a $70 external drive replaced every two years costs less than $3 a month. Most online backup services are a lot more than that.

Rather than an online provider, get yourself some personal file synchronization software (something like Dropbox or AeroFS). Then set up a computer at your house that's connected to the internet, and use it to be your "online" backup provider. It's cheaper, but has a bit more overhead. It also has the benefit of being able to grab the drive and bring it into the office if there's an emergency (vs. trying to figure out how the online backup's "restore" option works).

Vs. Paper Backups

Paper is not fire, flood (mold), theft, or earthquake-proof. And, it's a bit speedy.

At 6 KB per page square inch, backing up a single (6GB) DVD would take about 6000 pages. Printing that will cost you about $60 in supplies (not including the cost of a high end laser printer or the upfront cost of toner), and take several hours, using about 2 kWh of electricity (releasing about a kg of CO2).

(My old answer was far more entertaining, but I misread per square inch as per page, sorry folks).