Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top
rm -rf mydir

is painfully slow for a directory tree with 100000 files (in sub-directories) on a reiserfs file system.

Any ideas for faster removal of directory trees (even risking rare corruption) ?

share|improve this question
    
Not tested, but you could try find mydir -delete to remove it. find is heavily optimized - maybe it's faster than rm -rf ? – Johannes Schaub - litb Sep 15 '09 at 16:52
    
Update with times: rm -rf took 0m18.688s real time, while find -delete took 1m38.126s real time. I guess rm wins :) Although i did the rm test afterwards - i wonder whether it matters? (it was an exact copy, and i did a restart of the box after creating the trees). User and sys times are interesting: rm took user 0m0.130s sys 0m11.913s while find took user 0m0.057s sys 0m0.597s. the hierarchy was around 7000 files and that many directories, consisting of depth 2 (files are in subdirs). – Johannes Schaub - litb Sep 15 '09 at 17:27

The only solution I can think of is to have all your files on a separate files system. The file system can live on disk partition or in a file.

Instead of deleting the files you could wipe out the partition or delete the file.

I can sympathize with you because I have project with > 200 000 files on NTFS and deleting the tree is really a pain. If I could, I would

  • use another file system (reiserfs, in your case, is pretty good with lots of files anyway IIRC)
  • avoid having so many files (e.g. use a database)
share|improve this answer
    
That's exactly what people do e.g. on Gentoo because the package manager has many many small files that it reads and writes. Just put /usr/portage on a separate file system dd if=/dev/zero of=/opt/portagefile bs=1k seek=1024k count=1; mkfs.reiserfs -f /opt/portageFile gentoo-wiki.info/TIP_Speeding_up_portage#Why_It_Works – j13r Mar 19 '12 at 14:15

It's pretty much always a bad idea having a zillion files in a directory. But it happens to me all the time. Old filesystems got unusable because delete was O(n) in the number of files. I don't think any current Linux filesystems are bad that way. (Not positive about ReiserFS, but I'd be surprised if it was). But even with a good filesystem, the shell tools do too much work when removing files. They're stating files, explicitly testing permissions, creating large command lines, etc.

One workaround is to do a very low level delete, just calling the unlink() system call. Here's some quick-and-dirty Python that has let me delete a million files when rm failed me:

files = os.listdir('.')
for f in files:
  try: os.unlink(f)
  except Exception, e: print e
share|improve this answer
    
that is exactly what rm -rf does, calling unlink. rm is just a synonym for unlink. Python will not be faster than C. How has rm failed you? What's supposed to be the difference? – j13r Mar 19 '12 at 14:02

XFS does deletes a lot faster. ext{2,3,4} are the worst, I don't know where reiserfs is between them.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .