I work (create, read, copy, delete, count the number of) with millions of files under Windows XP (and NTFS). Are there any other filesystems or OS better optimized for dealing with millions of files? Can you provide any suggestions on how I can significantly speed up things (say, copy/delete/count)? Any useful utilities?
I use FAR Manager when dealing with large numbers of files. It's ideally suited for this sort of task because you can set up filters to limit the number of files you see at one time:
FAR Manager (free and open source)
Both 32-bit and 64-bit editions are available, which is particularly important when dealing with really large numbers of files since 64-bit processing will yield a definite advantage for you.
DiskBoss is an automated, rule-based file and disk management solution allowing one to perform disk space utilization analysis, file classification and categorization, duplicate files detection and removal, high-speed file synchronization and data migration, disk change monitoring, rule-based file organizing, data wiping and much more.
- It sounds a design issue. You should revise your solution.
- Format your drive(s) with a lower cluster size to save some space.
Total commanderis a good utility for this purpose. Maybe you need to develop your own