I have about 65000 files in 3000 folders in c:\foo\root (ie, c:\foo\root\folder1, c:\foo\root\folder2 etc). I want to copy all of those files to a single folder c:\All\.

I've tried Robocopy but it just does a full copy (directories included). If during the copying process it comes across files with the same name, I need it to rename the new one as temp(2).file or similar, not just ignore it or overwrite.

Anyone know how to do this with a batch file/command prompt? Even some C# would be great.

  • 65K in a single directory. Let us know how that affects things like opening in Explorer, listing, searching ... Would be interesting, I'd think. – therube May 28 '11 at 6:35
  • Initially opening the directory isn't a problem, but trying to move, rename or delete any file(s) causes a massive slowdown. When I do that, Explorer resorts the list like 1, 10, 100, 1000, 11, 110, 1100 etc, THEN it resorts it in the logical 1, 2, 3 manner. Sorting and resorting takes a long time. Supposedly the NTFS file system can handle up to 4 billion files (I assume they can all be in the same directory). Good luck ever doing anything with that. I think around 10000 files per folder is the limit if you want things to run smoothly. – Alex May 28 '11 at 9:39

Not thinking clearly ATM, so this is a round-about method ...

If you were to archive the tree (using a non-compress method would be faster) you could work it in two ways. Archive the tree (so subdirectories included) & extract with an 'E' (Extract) rather then with an 'X' (eXtract files with full path), the extracted files would be placed into a single directory. Other way to do the same would be on archive creation, to tell it to "exclude paths from names". Then the created archive would be barren of paths, i.e. all files stored in a single directory in the archive, so when extracted, they would extract that way too. (Thinking the former method makes more sense.)

(ARJ can do this. Would think that 7z & RAR & ZIP could too?)

"Flatten" was a word I was trying to think of earlier. And with that, a backup/synchronization program likely can be set to "flatten" a directory structure.

So other thoughts ... Tar & DD? Maybe that will help others come up with a better answer that I'm not thinking of ATM.

(Been a long time since I've used Tar, so I may off base, but something in the nature of...)

 tar -cvf c:/foo/roo | tar -xvf -

An archiver & backup/synchronizer should be able to handle any dups in an automated fashion.
ARJ can. Don't know about tar?

| improve this answer | |
  • Wow I'd never heard of ARJ. I tried this with 7z but no such luck. Anyway, I tried a temporary solution using C#- simply recursively run through every folder in the root, and put the full path of every file found into a text file. Then run through the text file with a File.Copy(), and name each file 1.pdf, 2.pdf, 3.pdf, etc. I didn't really need the original file names anyway, so this works and I don't have to worry about overwriting files. Just finished the program and it's running- ~35000 files left to copy! I'll post the effect of 65000 files in one folder when done. Thanks! – Alex May 28 '11 at 7:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.