Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.
rm -rf mydir

is painfully slow for a directory tree with 100000 files (in sub-directories) on a reiserfs file system.

Any ideas for faster removal of directory trees (even risking rare corruption) ?

share|improve this question
    
Not tested, but you could try find mydir -delete to remove it. find is heavily optimized - maybe it's faster than rm -rf ? –  Johannes Schaub - litb Sep 15 '09 at 16:52
    
Update with times: rm -rf took 0m18.688s real time, while find -delete took 1m38.126s real time. I guess rm wins :) Although i did the rm test afterwards - i wonder whether it matters? (it was an exact copy, and i did a restart of the box after creating the trees). User and sys times are interesting: rm took user 0m0.130s sys 0m11.913s while find took user 0m0.057s sys 0m0.597s. the hierarchy was around 7000 files and that many directories, consisting of depth 2 (files are in subdirs). –  Johannes Schaub - litb Sep 15 '09 at 17:27

3 Answers 3

The only solution I can think of is to have all your files on a separate files system. The file system can live on disk partition or in a file.

Instead of deleting the files you could wipe out the partition or delete the file.

I can sympathize with you because I have project with > 200 000 files on NTFS and deleting the tree is really a pain. If I could, I would

  • use another file system (reiserfs, in your case, is pretty good with lots of files anyway IIRC)
  • avoid having so many files (e.g. use a database)
share|improve this answer
    
That's exactly what people do e.g. on Gentoo because the package manager has many many small files that it reads and writes. Just put /usr/portage on a separate file system dd if=/dev/zero of=/opt/portagefile bs=1k seek=1024k count=1; mkfs.reiserfs -f /opt/portageFile gentoo-wiki.info/TIP_Speeding_up_portage#Why_It_Works –  j13r Mar 19 '12 at 14:15

It's pretty much always a bad idea having a zillion files in a directory. But it happens to me all the time. Old filesystems got unusable because delete was O(n) in the number of files. I don't think any current Linux filesystems are bad that way. (Not positive about ReiserFS, but I'd be surprised if it was). But even with a good filesystem, the shell tools do too much work when removing files. They're stating files, explicitly testing permissions, creating large command lines, etc.

One workaround is to do a very low level delete, just calling the unlink() system call. Here's some quick-and-dirty Python that has let me delete a million files when rm failed me:

files = os.listdir('.')
for f in files:
  try: os.unlink(f)
  except Exception, e: print e
share|improve this answer
    
that is exactly what rm -rf does, calling unlink. rm is just a synonym for unlink. Python will not be faster than C. How has rm failed you? What's supposed to be the difference? –  j13r Mar 19 '12 at 14:02

XFS does deletes a lot faster. ext{2,3,4} are the worst, I don't know where reiserfs is between them.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.