Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm intended to package up quite a lot of data in to archives so that I can store them using an on-line file storage service and to various local backups - the intention being that this will be fairly long term static storage.

I'm currently planning to compress them using 7zip into .7z files, but I'd rather compress the folder tree in large chucks, such that each archive will contain ~350Mb-1Gb of data (before compression) and I'm wondering how resilient the format is to damage.

Is the archive structured such that "minor" damage can be corrected?
Or if the damage is "serious", will it destroy the entire archive - or just those files in the specific site of the damage within the archive?

Essentially, the hidden question here is: should I package in to lots of small separate archives or few large ones?

share|improve this question
add comment

3 Answers

up vote 4 down vote accepted

Any compression tool is going to be subject to corruption issues on very large files. Your best bet is probably to use smaller files, but NOT volumes of a larger archive - as separate archives.

AFAIK 7zip will lose the whole archive if you have file damage in a portion of the archive.

share|improve this answer
    
This. While some schemes are better than others, any compression scheme risks killing the whole archive depending on the damage, and the larger the file the more likely this is. –  Shinrai Aug 6 '10 at 17:37
    
That's what I thought. Oh well, lots of small archives and copious checksums it is. Thanks. –  DMA57361 Aug 6 '10 at 18:14
add comment

If you want redundancy in your compression, I would suggest using rar/par/par2 files. This has been a standard for compression redundancy for files sent over newsgroups and a lot of other sources. You split your files into many rar files... and you can even be missing entire rar files and still recover your data. For data that doesn't compress well, this could actually increase the total size, but that is the price you pay for redundancy.

share|improve this answer
    
+1 for RAR/PAR. There's no better way. –  Fosco Aug 6 '10 at 19:35
add comment

Depending on your needs it might be better to introduce the redundancy on another level. What I want to say is you rather think about another complete copy of the files rather than trying to alleviate partial damage. Then you regularly check the checksums of these files and whenever a problem arises you replace the defective hardware and copy from an intact backup again.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.