Tell me more ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.
up vote 0 down vote favorite

I did a rm -rf foldername in an attempt to delete a folder containing 10,000s of 25KB files.

Question Why is it taking so long (Almost 30 minutes)?

Why did the server became extremely slow, even though htop shows very low CPU % utilization? The process is taking up only 0.0% CPU and 0.2% MEM? But somehow the Load average is 10!

htop

enter image description here

iotop

enter image description here

share|improve this question
1  
high hard drive activity? – D0rf Mar 14 at 1:07
Is there a way to check how much IO % the process is using? iotop? Anyway to speed up the deletion of all files in a directory? – Nyxynyx Mar 14 at 1:09
1  
Some file systems handle small files better than others. A smaller blocks size (set when formatting) for the file system will generally improve operations on small files. As an example I can mention that having the gentoo portage tree (~128 thousand files under 100kb) on a reiserfs3 filesystem has improved the time it takes to resolve package dependencies. – Ярослав Рахматуллин Mar 14 at 1:33

2 Answers

up vote 1 down vote

You need to nice the process to reduce CPU load and ionice the process to reduce I/O load on the disk:

ionice -c 3 nice -n 19 echo foo

This should create a process that runs with the least favorable CPU and I/O scheduling.

You can confirm with:

ionice -p 19574
share|improve this answer
You cannot combine "nice" priority by means of "ionice -c 3 -n", since that "-n" affects the priority inside the I/O priority class 3, "idle" (which in my linux accepts 0-7). I am afraid you need to use two commands instead (ionice and [re]nice). – esperanto Mar 19 at 15:14
up vote 0 down vote

lots of small files are typically difficult for filesystems/hard drives (the read/write heads have to thrash all over the place just to locate each file, and with them being so small, it extrapolates this effect). Either that or you have a lot your HDD is already doing, or it is failing.

nice -n 19 rm -rf directory

this will run 'rm' at the lowest priority.

share|improve this answer
How can I reduce the IO priority of the process thats deleting the many files and hogging the IO? – Nyxynyx Mar 14 at 1:17
1  
nice -n 19 rm -rf directory – SnakeDoc Mar 14 at 1:19
Thanks, from iotop is appears that jbd2 is eating up alot of the IO, is there a way to prevent this? Like disable jbd2 for a directory where many small files are deleted/purged every day? Or will this have a negative performance impact in other areas? – Nyxynyx Mar 14 at 1:28
you could put them on their own partition and disable the journal for it (probably good anyways for stuff that is more or less "temp" files) check into noatime for your fstab file also, make sure you are on the most updated kernel for your distro/release. I think there have been some kernel patches regarding ext4 and journaling thrashing hard drives. – SnakeDoc Mar 14 at 1:30

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.