up vote 8 down vote favorite
2
share [g+] share [fb]

I am using windows 7 64-bit on q6600 with 4gb ram.

I have a folder with 2 million files which are 10kB on average.

windows 7 explorer stops responding when I open the folder.

how can I overcome this?

link|improve this question
12  
By not storing two million files in the same folder... – Daniel Beck Jan 23 at 16:02
To be a little more clear than that comment, I'm not an expert on this but I believe NTFS is simply not designed well for that many files in one folder. It doesn't scale very well and simply enumerating the contents of the folder can take forEVER. – Shinrai Jan 23 at 16:15
2  
You know whats even worse.. trying to delete that folder.. good luck :( – ppumkin Jan 23 at 16:20
1  
@Shinrai, as Roger notes in his answer, NTFS is fine, it's Explorer that has a lot of work to do when you open the folder. – dsolimano Jan 23 at 16:56
1  
@dsolimano - That's why I left answering to folks more knowledgeable than myself. :) – Shinrai Jan 23 at 17:55
show 4 more comments
feedback

2 Answers

up vote 10 down vote accepted

You can overcome it by using a more sensible structure for your files. While Explorer will support directories with millions of files, it isn't designed for this sort of scenario and performance will be slow.

For more info, see this TechNet article:

http://technet.microsoft.com/en-us/magazine/hh395477.aspx

link|improve this answer
I like Raymond's explanation. I would be angry if Microsoft spent money testing Explorer on such a niche situation. – surfasb Jan 24 at 9:31
imho, official guidelines and/or system warnings would be more appropriate than a (non-)apologetic technet article. – eyaler Jan 25 at 17:14
It just the filesytem restriction.. it was flawed since NTFS was made, then they tried fixing ti with indexing.. but is sitl rubbish, ZFS forward – ppumkin Jan 25 at 17:30
It's not a filesystem restriction - it's a display problem. NTFS can store an unlimited number of files per volume. The issue is that it's very difficult to build a UI that will display a directory of two million files - and why should MSFT worry about this? Who's going to page through so many files? – Roger Jan 26 at 19:20
feedback
up vote 8 down vote

Sorry.. Windows can not handle more than 80 thousand files in a folder, neatly and efficiently... (On a Zeon 2.8Ghz with 4GB RAM - Server)

It starts to crash and burn near 60k.

At 200k it takes 2 days to enumarte the folder.. we had this problem with badmail folder using windows exchange. The only way to overcome it is to make thousands of folders.. that it can handle fine.. with files inside those folders with more folders..

Otherwise you should seriously consider dumping that data into a SQL file, using SQLExpress which is free or MySQL which is also free.. Write a small program to handle it and you will be good to go.. that's what we did with our 200k files.. and now we have over 12 million emails sorted, with quick access.

link|improve this answer
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.