Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Let's say I have 50 GiB of files that weights around 500 KiB each.

My guess is that having, for example, 5 large files of 10 GiB each with the same content archived in them would be better for hard drive performance. Am I correct?

Will there be a noticeable gain on an NTFS filesystem?

=====================================================================

Finally, which tool could I use to group the files together while retaining the ability to modify the content of the archive with zero or minor performance loss? For example, I like TrueCrypt archiving because after mounting an archive file, it creates a drive which I can use seamlessly as if it was a normal drive. The only thing with TrueCrypt is that I don't need encryption/compression, only archiving.

share|improve this question
    
...then with TrueCrypt, just set the password to "password" (and indicate that this is what it is in the filename). –  Randolf Richardson Jun 29 '11 at 15:56
    
But even if the password is easy and known, it's still gonna encrypt the whole thing, which will be worse for performance than simply leaving the files unarchived. –  asmo Jun 29 '11 at 16:29
    
That's a good point. It sure would be nice if TrueCrypt had a "no encryption" option ("for testing purposese," of course). =) –  Randolf Richardson Jun 29 '11 at 16:48
1  
Doing this gives you the secondary problem that if your container gets corrupted/damaged at all, there's a very significant chance that you'll lose all the contained data rather than a file or two. –  afrazier Jun 29 '11 at 20:06

2 Answers 2

up vote 2 down vote accepted

In Windows 7, you can mount a .VHD as a drive. This is the virtual hard drive format used by virtual machines and by Windows Backup (for Complete PC backup only on Windows client, and for all backups on Windows Server). No compression or encryption. Performance slowdown during ordinary use is minimal. After all, people are running whole virtual machines this way.

NTFS metadata and disk seeking can lead to substantial overhead on small files. For example, copying 10,000 files of under 10 KB each to a USB hard drive will proceed at about 300 files per second. That's 30 seconds to copy the files individually, vs. 10 seconds to copy them in a block. (The difference becomes even more striking with internal or eSATA drives, since the block throughput rate is higher. SSDs are so great at random access that it might not matter either way.)

But 500 KB files are large enough that the impact might be limited. You'd have to benchmark it and see.

share|improve this answer
    
That's what I was looking for. And on top of that, it's included in Windows 7, as you said, so I won't need to install or buy anything. Thanks! –  asmo Jun 30 '11 at 0:05

Combining files

I would expect that a single large file is only better for performance if you usually read all the data, read it sequentially and if the large file is relatively unfragmented.

TrueCrypt

Using any kind of compression or encryption will be much worse for hard drive performance.

Update:

According to an answer to this question "there will be some drop in performance, albeit a slight one." The answer refers to a Tom's Hardware article which says

The benchmark shows varying performance and highly depends on the processor, followed by the drive you are about to encrypt: AES and Twofish provide highest throughput on our Core 2 Duo notebook Dell Latitude D610. Once you start combining multiple encryption algorithms, e.g. Twofish and Serpent, performance drops considerably. While this isn’t noticeable while working with Windows and popular applications, increasing system load—such as may occur during heavy multi-tasking or when taking on intensive workloads such as video transcoding—will reduce system performance considerably.

The Wikipedia article says

When using popular desktop applications in a "reasonable manner", and with only a single encryption algorithm, the performance impact of TrueCrypt on desktop applications is not generally noticeable, though that does depend on the application, and power users may complain. Using a fast multi core processor and a fast system drive, preferably a Flash SSD, makes TrueCrypt almost transparent

I don't know of any evidence that shows Truecrypt is going to significantly be "better for hard drive performance".

share|improve this answer
    
Compression in and of itself isn't necessarily going to hurt anything. As long as you don't have any "solid" archiving options turned on, random access to an individual file won't be all that painful. –  afrazier Jun 29 '11 at 20:05
    
@afrazier: Thanks for pointing that out, I'll update my answer. –  RedGrittyBrick Jun 29 '11 at 20:10

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.