Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Removing what seems to me to be a moderate number of files takes a very long time. For example:

$ pv linux-3.0.16.tar.bz2 | tar xj
73.2MB 0:00:16 [4.55MB/s]
$ pv linux-3.0.16.tar.bz2 | tar xj
73.2MB 0:03:16 [ 382kB/s]

The partition is formatted ext4, mounted relatime,error=remount-ro. There is no processor utilization spike during removal of files. Does any one have any idea what might be going on?

Update: Running either the above commands or rm -rf as root performs as expected... update: figured it out. The sysadmin for the box had installed a ridiculous amount of audit rules globally on the box. Removing them fixes it.

share|improve this question
    
The content seems to be tar extract and file-write performance not rm performance as in the title and first line - which is it? –  RedGrittyBrick Sep 26 '12 at 16:15
    
That is remove performance. The difference between the first and second command is that each file is replaced instead of created. rm -rf has the same problem though, and would have been a better example, I just happened to have those commands up on my terminal. –  Seamus Sep 26 '12 at 16:19
    
Remove files in random order is very expensive as each file remove operation removes an entry from an arbitrary position on a list. –  David Schwartz Sep 26 '12 at 16:37
    
Did you run a 'sync' between the commands, so that the data is flushed to disk? –  mikebabcock Sep 26 '12 at 17:04

1 Answer 1

When you replace or delete a file it simplistically does this:

  1. Find the inode for the file.
  2. Check permissions, user and group of the individual file.
  3. Remove the files in the directory as well if the file were a directory (repeat 1 - 4 for each file).
  4. Unlink the file and its inode.

This is obviously going to slow performance a lot, because it's much harder than just adding a new inode and writing a file to the disk. This is a side effect of it being an inode table list, if it were a inode tree instead it could just delete a whole branch instead. Although this would not change a thing as you are untarring, as untarring will do it file by file regardless the way in which inodes are stored...

Running as root cuts out step 2, which might or might not be a huge difference.


Have you tried mounting back in ext3 and comparing the execution time?

Because ext4 deletes faster due to the use of extents. It would be remarkable if it were slower...

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.