5

I have around 100,000 small files which I backup every few weeks. Because they are really small, write speed to my external 2.5" USB 3.0 HDD is really slow.

In 7zip, I compressed them using Compression level: Store. That shouldn't compress data, it should only make one big file, right? I did this because transfer speed of one big file is much better than that for a lot of small files.

If there is one bad block on the hard drive, that wouldn't be a problem if I have 100k small files - some will be lost but I will have other 99%. What happens if I have one damaged block in one really big "compressed" file? Can I recover the other 99% of undamaged data from it?

2
  • 3
    PAR (look it up) files can be created to repair damaged archives
    – Ramhound
    Mar 12, 2016 at 20:38
  • I definitely second Ramhound's suggestion. Beware that generation is slow. My memory is that the Par2 format doesn't handle 100k files, you'll have to break it up. Mar 12, 2016 at 21:41

1 Answer 1

3

7zip archives using the "store" compression mode do indeed have no actual compression; you can see all the original data in the 7z file if you open it with a hex editor. The end of one file is immediately adjacent to the beginning of the next. Bookkeeping like filenames, file sizes, CRC32 checksums, and offsets within the archive appear to be kept compressed at the beginning and/or end of the archive.

When I scribbled over a large run in the middle of the file, WinRAR refused to extract only the affected files - the ones that I didn't hit came out fine. 7zip itself also complained about corruption, but got all files out anyway. My scribbling was then visible in the unarchived files, but they were otherwise intact.

If the bookkeeping is damaged, 7zip won't be able to read the archive and you'd have to rescue the surviving data with a hex editor. But as long as the beginning and end of the archive are intact, you'll be able to get all data out by normal means except for the parts affected by the disk corruption. Even so, if this data is important, you should make and store off-site backups.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .