fast Linux directory tree removal

Question

rm -rf mydir

is painfully slow for a directory tree with 100000 files (in sub-directories) on a reiserfs file system.

Any ideas for faster removal of directory trees (even risking rare corruption) ?

Answer 1

The only solution I can think of is to have all your files on a separate files system. The file system can live on disk partition or in a file.

Instead of deleting the files you could wipe out the partition or delete the file.

I can sympathize with you because I have project with > 200 000 files on NTFS and deleting the tree is really a pain. If I could, I would

• use another file system (reiserfs, in your case, is pretty good with lots of files anyway IIRC)
• avoid having so many files (e.g. use a database)

Answer 2

It's pretty much always a bad idea having a zillion files in a directory. But it happens to me all the time. Old filesystems got unusable because delete was O(n) in the number of files. I don't think any current Linux filesystems are bad that way. (Not positive about ReiserFS, but I'd be surprised if it was). But even with a good filesystem, the shell tools do too much work when removing files. They're stating files, explicitly testing permissions, creating large command lines, etc.

One workaround is to do a very low level delete, just calling the unlink() system call. Here's some quick-and-dirty Python that has let me delete a million files when rm failed me:

files = os.listdir('.')
for f in files:
  try: os.unlink(f)
  except Exception, e: print e

Answer 3

XFS does deletes a lot faster. ext{2,3,4} are the worst, I don't know where reiserfs is between them.