I have a folder which should at any time have less than 500k folders. These folders are created dynamically, and when the files (images) inside them are deleted by some script, the folder itself should also be deleted. Due to a bug, the deletion of the empty folders was not done, and we now ended up with millions of empty folders...
So at the moment I have about 500k folders with files in them and besides them millions of empty folders.
Because there are so many dirs in there even a simple ls | wc -l takes hours so can't even find the exact number.
I could run a find command: find . -maxdepth 1 -type d -empty; and then pipe it with a rmdir; but again, the find takes ages (or even use the -delete option that find offers)
I then got the idea to use rmdir --ignore-fail-on-non-empty /path/to/huge/folder/*; but I am not sure on its efficiency. Basically this command only deletes the empty directories in my folder, which means the ones which have files should be safe and not deleted. Then the '--ignore-fail-on-non-empty' option ignores the hundreds of thousands or warnings for the dirs that are actually not empty.
I have quickly created about 100k empty folders on my local, then 'touched' a few files in some of them, and then used the rmdir command. While it took 2 minutes to create 100k empty folders, it took 2 seconds(!) to delete the empty ones. So my hopes are high, but I'm not sure how it will handle about 10 million folders... (I guess I could use ionice to stop the system from locking down in case it would actually take ages)
If anyone has any ideas/advice, or faced something like this before, can you please point me to the best way of approaching my issue. Or even to tell me which way is better to go: find or rmdir...
EDIT: I forgot to mention the folder structure before. The folders within the main folder that have files in them, have no more folders in them. The other folders are empty. So all those millions of folders cam either be empty, or have files in them, but no more folders. So basically there is only one level of folders in the tree structure.