Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Note that I am not asking how many files NTFS can store per folder (that's been covered here and here).

I would like to know what others consider an "optimal" number for enumerating files in NTFS folders. Specifically, I am storing ~20kb images. It seems in the tens of thousands of files, that Windows Explorer begins to have problems (i.e. taking long amounts of time) displaying the file lists.

Should I cap the file quantity at 10k? 5k? Is there some "optimal" number like 4096 which will give me the best performance when navigating folders?

share|improve this question
    
I should add that this pertains to storage of images on a single 500GB hard drive at 7200 RPM. Such storage is temporary until images are migrated elsewhere. –  JYelton Mar 21 '11 at 17:45

2 Answers 2

up vote 2 down vote accepted

There's no magical optimal number, it just takes Explorer longer depending on how many files it needs to look at.

If you're browsing with Explorer, I'd try and keep it under 2000-3000 per folder, but that's a personal preference.

That said, I discourage using Explorer to do anything with a massive amount of files. Stick with the command line if at all possible. We have a guy here that insists on manually browsing through image folders here that have about 900k files apiece, and he's miserable for it.

share|improve this answer
    
I agree, I usually use command line or scripts to deal with these folders, but we have users who occasionally want to browse, and I'd like to offer the best performance I can. My curiosity was whether performance is simply n based on file quantity n, or if thresholds exist. i.e. Opening an explorer window with 1 to 4096 files is a given speed, but 4096...16k is some other (slower) speed. –  JYelton Mar 21 '11 at 18:02
    
No, it's fairly linear. Explorer has to open and process the metadata on each and every file in the folder to provide its happy little columns/icons/etc, which is why it takes so long with lots of files. The upper limit for performance is probably either RAM or a hardcoded limit that I've never encountered before. Either way, if you hit a limit like that it'll just crash or not work as opposed to going even slower. –  Hyppy Mar 21 '11 at 18:07
    
Thanks @Hyppy that's what I was wanting to know. I'll use a round number that seems to give decent performance, then. –  JYelton Mar 21 '11 at 18:11

Well you asked a bigger question than you might think...the short answer is no there isn't an optimal number as you've presented the question.

It's apparent it's important enough that you would like some optimization but not important enough that you care to give many more details. I'd say don't worry about it if most other variables remain unchanged and all you want to experiment with is number of files in an NTFS folder. because the amount of optimization will be hardly noticeable to you if you are simply using explorer. If you really don't want to change anything I'd say you should focus on things like disabling windows disk indexing which is very intensive bookeeping which slows down your PC (though there might be certain reasons to use it). maybe a great alternative for you is using a program like picasa, which is really good at handling large numbers of image files and accessing them efficiently.

without other details like how often you update files, how long before migration, how often are they accessed before migration, exactly why are you accessing such a large repository of image files using NTFS explorer, etc., all sorts of other details that would help in optimizing your answer, it's hard to say what would best help you. if you really want to make a serious stride in optimizing file access set up an inexpensive RAID configuration. good luck hope i said soimething that may be useful.

.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.