Sign up ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a directory with a huge number of jpeg images (more than 300,000 files) that are part of a project. This is somewhat of a nightmare to backup and move -not because of the size (total ~3 GB) which is rather large but still manageable, but because of the number of files. Copying them takes an insane amount of time and dramatically slows down any backup/copy routine.

Since I must keep those files I thought I'd combine them into some kind of archive. I'm trying WinRar with "store" compression, but it's quite slow. On my dev machine (2xSAS harddrives in Raid-0, dual i7) the estimate is about 100 minutes.

It's not that bad and this time I'm willing to wait (it's a one-time operation), but I'm curious to know if there are faster ways to do this in case I have the same need in future (with a less powerful machine and less time).

share|improve this question

1 Answer 1

up vote 3 down vote accepted

I don't think so. The bottleneck here would be drive bus speed, and file access overheads. That's why copying many files takes so much longer than one file of the same size.

The "store" option in Winrar will definitely speed it up, but creating the archive on the same drive where the files are stored, will slow it down too. Creating an archive on an external device will also hinder performance.

Ideally you'd want to configure your backup scenario with a synchronization tool (like rsync) which will only backup new and changed files, dramatically reducing the time to finish.

If you plan to just keep the files archived on your local machine, then I think your .rar archive is a good call, but I can imaging having to work with those files on your project, out of the archive, could be a nightmare.

For such a large volume of files, you'd need a dedicated storage drive, preferably a fast (10K+rpm) one at that, and preferably inside a dedicated file server.

You might also be interested in the windows port of tar. Tar is a fast archiver that is designed to dump all files into a single .tar archive, without any compression.

share|improve this answer
Bingo! Creating the archive on a different drive makes all the difference. The source drive is not particularly fast (a "normal" 7200 rpm), but the target is very fast (2x 10krpm SAS/Raid-0) and it's like night and day. Thanks! – Francesco De Vittori Dec 9 '10 at 14:51

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.