47

I'm trying to keep a directory full of log files manageable. Nightly, I want to delete all but the 10 most recent. How can I do this in a single command?

migrated from stackoverflow.com Apr 8 '11 at 14:36

This question came from our site for professional and enthusiast programmers.

  • 3
    have a look at logrotate – knittl Apr 8 '11 at 15:26
  • duplicate: stackoverflow.com/questions/6024088/… – michael May 31 '15 at 10:29
  • @michael How can mine be a duplicate of that one, if my discussion was created first? – dev4life Jul 6 '17 at 16:46
  • @ovaherenow "yours"? I don't see your name on either question, so I'm not sure which you mean. My comment doesn't even indicate which is a duplicate of the other. But it doesn't matter -- it's just a comment, with a link. It could be added to either question, or both questions. It's up to someone else -- an admin -- to mark one or the other as a duplicate. No nefarious sleight was intended nor should any be perceived. More important than which question was first, is which question has the best answer -- again, the link facilitates this discussion, it doesn't determine the answer. – michael Jul 7 '17 at 1:31
  • @michael wow. That is really odd. This thread was opened by me but that's obviously not my user name. But I am getting notified about this thread. – dev4life Sep 12 '17 at 18:12

11 Answers 11

57

For a portable and reliable solution, try this:

ls -1tr | head -n -10 | xargs -d '\n' rm -f --

The tail -n -10 syntax in one of the other answers doesn't seem to work everywhere (i.e., not on my RHEL5 systems).

And using $() or `` on the command line of rm runs the risk of

  1. splitting file names with whitespace, and
  2. exceeding the maximum commandline character limit.

xargs fixes both of these problems because it'll automatically figure out how many args it can pass within the character limit, and with the -d '\n' it will only split at the line boundary of the input. Technically this can still cause problems for filenames with a newline in them, but that's far less common than filenames with spaces, and the only way around the newlines would be a lot more complicated, probably involving at least awk, if not perl.

If you don't have xargs (old AIX systems, maybe?) you could make it a loop:

ls -1tr | head -n -10 | while IFS= read -r f; do
  rm -f "$f"
done

This will be a bit slower because it spawns a separate rm for each file, but will still avoid caveats 1 and 2 above (but still suffers from newlines in file names).

  • @HaukeLaging: From the head(1) man page: -n, --lines=[-]K print the first K lines instead of the first 10; with the leading '-', print all but the last K lines of each file – Isaac Freeman May 15 '13 at 6:06
  • On Solaris 10 head and xargs give errors. Correct options are head -n 10 and xargs rm -f. Full command is ls -1tr ./ | head -n 10 | xargs rm -rf – DmitrySandalov Oct 31 '14 at 13:36
  • 1
    Note the ls -1tr is the number ONE not the letter "L". – shonky linux user Aug 27 '15 at 7:51
  • 1
    It should also be noted that newlines are rare but valid characters in ext filenames. This means that if you have the files "that", and "that\nthing", if this script hits "that\nthing", it'll delete "that", error when it couldn't delete "thing", and leave "that\nthing" (the intended target) unaffected. – Synthead Apr 15 '16 at 23:26
  • 1
    @IsaacFreeman I've deleted my bad advice, cheers. – Walf Aug 8 '18 at 0:42
20

The code you'd want to include in your script is

 rm -f $(ls -1t /path/to/your/logs/ | tail -n +11)

The -1 (numeric one) option prints each file on a single line, to be safe. The -f option to rm tells it to ignore non-existent files for when ls returns nothing.

  • Original poster, here. I tried this command, but it doesn't work--I get the error "rm: missing operand" – user75814 Apr 10 '11 at 13:50
  • 2
    Did you use the correct path? Obviously /path/to/your/logs is just made up for you to enter the correct path. To find the bug execute the parts ls -t /path/... and ls -t /path/... | tail -n +11 alone and check if the result is correct. – Daniel Böhmer Apr 11 '11 at 9:01
  • 1
    @HaukeLaging I am quite sure you got something wrong. Mind the + before the number. It makes tail not print the last n elements but all elements beginning from the n'th. I checked in again and the command should definitely remove only elements older than 10 most recent files under all circumstances. – Daniel Böhmer Mar 14 '13 at 14:01
  • @halo Indeed, sorry. – Hauke Laging Mar 14 '13 at 17:41
  • 2
    I think your answer is very dangerous ls prints a file name, not a path so your rm -f will try to remove the files in the current directory – Jürgen Steinblock Jun 20 '17 at 11:37
13

Apparently parsing ls is evil.

If each file is created daily and you want to keep files created within the last 10 days you can do:

find /path/to/files -mtime 10 -delete

Or if each file is created arbitrarily:

find /path/to/files -maxdepth 1 -type f -printf '%Ts\t%P\n' | sort -n | head -n -10 | cut -f 2- | xargs rm -rf
  • 4
    -mtime 10 -delete only deletes files modified exactly 10 days ago. Older files will not be deleted. -mtime +11 -delete will delete files that have been modified 11 or more days ago leaving the last 10 days of log files. – Markus Jun 19 '18 at 8:54
  • 1
    I think the use of %p instead of %P is needed on the printfso that the files got have the full path. Otherwise the rm -rf does not work. – jalopaba Sep 5 at 10:49
9

A tool like logrotate does this for you. It makes log management much easier. You can also include additional cleanup routines shuch as halo sugggested.

2

I modified Isaac's approach a little bit.

It now works with a desired path:

ls -d -1tr /path/to/folder/* | head -n -10 | xargs -d '\n' rm -f
1

From here:

#! /bin/sh
# keepnewest
#
# Simple directory trimming tool to handle housekeeping
# Scans a directory and deletes all but the N newest files
#
# Usage: cleanup <dir> <number of files to keep>
#
# v 1.0 Piers Goodhew 1/mar/2007. No rights retained.


if [ $# -ne 2 ]; then
  echo 1>&2 "Usage: $0 <dir> <number of files to keep>"
  exit 1
fi

cd $1
files_in_dir=`ls | wc -l`
files_to_delete=`expr $files_in_dir - $2`
if [ $files_to_delete -gt 0 ]; then
  ls -t | tail -n $files_to_delete | xargs rm
  if [ $? -ne 0 ]; then
    echo "An error ocurred deleting the files"
    exit 1
  else
    echo "$files_to_delete file(s) deleted."
  fi
else
  echo "nothing to delete!"
fi
  • 1
    Thanks for providing this answer. While providing code like this is helpful, it also helps to provide some additional information about how it might be used or what the code does. You can use the edit button to make future improvements to your post. – nhinkle Mar 21 '13 at 5:23
1

In Bash you can try:

ls -1t | (i=0; while read f; do
  if [ $i -lt 10 ]; then
    ((i++))
    continue
  else
    rm -f "$f"
  fi
done)

This skips the 10 newest als deletes the rest. logrotate may be better, I just want to correct the wrong shell related answers.

1

not sure this will help anyone but to avoid potential issues with wonky characters i just used ls to perform the sort but then use the files inode for the delete.

For current directory, this keeps the most recent 10 files based on modification time.

ls -1tri | awk '{print $1}' | head -n -10 | xargs -n1 -t -Iinode find . -inum inode -exec rm -i {} \;

0

I needed an elegant solution for the busybox (router), all xargs or array solutions were useless to me - no such command available there. find and mtime is not the proper answer as we are talking about 10 items and not necessarily 10 days. Espo's answer was the shortest and cleanest and likely the most unversal one.

Error with spaces and when no files are to be deleted are both simply solved the standard way:

rm "$(ls -td *.tar | awk 'NR>7')" 2>&-

Bit more educational version: We can do it all if we use awk differently. Normally, I use this method to pass (return) variables from the awk to the sh. As we read all the time that can not be done, I beg to differ: here is the method.

Example for .tar files with no problem regarding the spaces in the filename. To test, replace "rm" with the "ls".

eval $(ls -td *.tar | awk 'NR>7 { print "rm \"" $0 "\""}')

Explanation:

ls -td *.tar lists all .tar files sorted by the time. To apply to all the files in the current folder, remove the "d *.tar" part

awk 'NR>7... skips the first 7 lines

print "rm \"" $0 "\"" constructs a line: rm "file name"

eval executes it

Since we are using rm, I would not use the above command in a script! Wiser usage is:

(cd /FolderToDeleteWithin && eval $(ls -td *.tar | awk 'NR>7 { print "rm \"" $0 "\""}'))

In the case of using ls -t command will not do any harm on such silly examples as: touch 'foo " bar' and touch 'hello * world'. Not that we ever create files with such names in real life!

Sidenote. If we wanted to pass a variable to the sh this way, we would simply modify the print:

print "VarName="$1

to set the variable VarName to the value of $1. Multiple variables can be created in one go. This VarName becomes a normal sh variable and can be normally used in a script or shell afterwards. So, to create variables with awk and give them back to the shell:

eval $(ls -td *.tar | awk 'NR>7 { print "VarName="$1 }'); echo "$VarName"
0

I agree that parsing ls is evil!

Ensure only the last 5 files are kept in the /srv/backups path.

find /srv/backups -maxdepth 1 -type f -printf '%Ts\t%P\n' \
    | sort -rn \
    | tail -n +6 \
    | cut -f2- \
    | xargs -r r
0

Based on Isaac Freeman response, but working on any directory:

ls -1tr $PATH_TO_DELETE/* | head -n -10 | xargs -d '\n' rm -f --

You can also set a prefix, like $PATH_TO_DELETE/testFi*

If you use * after the PATH ls will output absolute file names, at least in "my" bash :)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.