8

I have a cache folder with minimum of 15000 files.

I tried this:

find cache* -mtime +30 -exec rm {} \;

But this made my server load fly to the skies!

Is there any faster/better solution?

Or can I limit speed or iterations of this command?

  • 1
    If cache* expands to many files, you may want to try something like find . -name 'cache*' -mtime +30 -exec rm {} \; instead. – Jaap Eldering Nov 6 '12 at 10:15
10

I like to use tmpwatch for these things, this is for the last time the file was modifiyed. It's simple and works well in many cases:

tmpwatch -m 720 /path/to/cache

For Ubuntu, check tmpreaper instead.

If you want to check the last time the file was accessed than you use the following:

tmpwatch -a 720 /path/to/cache

You are not able to use tmpwatch -a on file systems mounted with noatime. you can still use -m

| improve this answer | |
  • I'm using Ubuntu 10.04.2 LTS and that command doesn't exists.. – Kristian Nov 6 '12 at 10:47
  • 2
    @Kristian Check for tmpreaper instead. – slhck Nov 6 '12 at 11:07
  • Any pitfalls when using these to manage filesystems mounted with noatime option? – AnonymousLurker Nov 6 '12 at 11:28
  • @AnonymousLurker I have modifyed the answer for you – WojonsTech Nov 6 '12 at 19:42
6

You could avoid the spawning of a new process for each file by using

find cache* -mtime +30 -delete
| improve this answer | |
3

Try running the above with nice:

nice -n 39 find cache* -mtime +30 -exec rm -f {} ';'

That way the huge load will only appear if nothing else needs to run, otherwise the other processes will take precedence (if their niceness is lower than 19 i.e. the maximum).

Note that the argument to the -n option is added to the default niceness which varies between -20 and 19. I used 39 so that it will be very nice regardless of what original niceness there was.

| improve this answer | |
2

As commented by chiborg, the load is due to starting rm for every file found. I noticed the answer where tmpwatch is suggested as an alternative, which I'm sure works well. However, it is not necessary.

Find can run the command given to exec once, if you tell it to accumulate the found files into a list of arguments like so:

find /path -name "*.moo" -exec rm {} \+

This may sometimes fail to work because the argument list may grow larger (in bytes) than the maximum allowed by the shell (getconf ARG_MAX). This may be solved by xargs with the -L option.

consider this example:

$ echo 0 > /tmp/it; 
$ for i in {0..15000};do echo $i;done  |\
    xargs --no-run-if-empty -L 5000 ./tmp/xr.sh 
Iteration=0; running with 5000 arguments
Iteration=1; running with 5000 arguments
Iteration=2; running with 5000 arguments
Iteration=3; running with 1 arguments

$ cat tmp/xr.sh 
#!/bin/sh
IT=`cat /tmp/it`
echo Iteration=$IT\; running with $# arguments
let IT=IT+1
echo $IT > /tmp/it

So there is no need install extra software, all you need is in gnu-findutils:

find /path -mtime +30 -print0 | xargs -0 -L 5000 rm 
| improve this answer | |
  • some d00d thought it would be useful to add -print0 and -0 to find and xargs. Those arguments force their commands use the null char (\0) instead of a new-line (\n) to separate the arguments. This is helpful when the filename has spaces or new-lines in it. – Ярослав Рахматуллин Nov 7 '12 at 16:11
  • (1) You don’t need to say \+; plain + is good enough.  (2) This won’t “fail to work because the argument list may grow large…”.  find  … -exec … {} + will do the same thing that xargs will do; it will run the command as many times as it needs to.  For example (as in your example), if you have between 15001 and 20000 files, find would run the -exec’d program (rm) four times. – G-Man Says 'Reinstate Monica' Apr 26 '18 at 17:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.