59

I'm trying to keep a directory full of log files manageable. Nightly, I want to delete all but the 10 most recent. How can I do this in a single command?

6
  • 3
    have a look at logrotate
    – knittl
    Apr 8 '11 at 15:26
  • duplicate: stackoverflow.com/questions/6024088/…
    – michael
    May 31 '15 at 10:29
  • @michael How can mine be a duplicate of that one, if my discussion was created first?
    – dev4life
    Jul 6 '17 at 16:46
  • @ovaherenow "yours"? I don't see your name on either question, so I'm not sure which you mean. My comment doesn't even indicate which is a duplicate of the other. But it doesn't matter -- it's just a comment, with a link. It could be added to either question, or both questions. It's up to someone else -- an admin -- to mark one or the other as a duplicate. No nefarious sleight was intended nor should any be perceived. More important than which question was first, is which question has the best answer -- again, the link facilitates this discussion, it doesn't determine the answer.
    – michael
    Jul 7 '17 at 1:31
  • @michael wow. That is really odd. This thread was opened by me but that's obviously not my user name. But I am getting notified about this thread.
    – dev4life
    Sep 12 '17 at 18:12

13 Answers 13

68

For a portable and reliable solution, try this:

ls -1tr | head -n -10 | xargs -d '\n' rm -f --

The tail -n -10 syntax in one of the other answers doesn't seem to work everywhere (i.e., not on my RHEL5 systems).

And using $() or `` on the command line of rm runs the risk of

  1. splitting file names with whitespace, and
  2. exceeding the maximum commandline character limit.

xargs fixes both of these problems because it'll automatically figure out how many args it can pass within the character limit, and with the -d '\n' it will only split at the line boundary of the input. Technically this can still cause problems for filenames with a newline in them, but that's far less common than filenames with spaces, and the only way around the newlines would be a lot more complicated, probably involving at least awk, if not perl.

If you don't have xargs (old AIX systems, maybe?) you could make it a loop:

ls -1tr | head -n -10 | while IFS= read -r f; do
  rm -f "$f"
done

This will be a bit slower because it spawns a separate rm for each file, but will still avoid caveats 1 and 2 above (but still suffers from newlines in file names).

12
  • @HaukeLaging: From the head(1) man page: -n, --lines=[-]K print the first K lines instead of the first 10; with the leading '-', print all but the last K lines of each file May 15 '13 at 6:06
  • On Solaris 10 head and xargs give errors. Correct options are head -n 10 and xargs rm -f. Full command is ls -1tr ./ | head -n 10 | xargs rm -rf Oct 31 '14 at 13:36
  • 1
    Note the ls -1tr is the number ONE not the letter "L". Aug 27 '15 at 7:51
  • 1
    It should also be noted that newlines are rare but valid characters in ext filenames. This means that if you have the files "that", and "that\nthing", if this script hits "that\nthing", it'll delete "that", error when it couldn't delete "thing", and leave "that\nthing" (the intended target) unaffected.
    – Synthead
    Apr 15 '16 at 23:26
  • 1
    @IsaacFreeman I've deleted my bad advice, cheers.
    – Walf
    Aug 8 '18 at 0:42
21

The code you'd want to include in your script is

 rm -f $(ls -1t /path/to/your/logs/ | tail -n +11)

The -1 (numeric one) option prints each file on a single line, to be safe. The -f option to rm tells it to ignore non-existent files for when ls returns nothing.

7
  • Original poster, here. I tried this command, but it doesn't work--I get the error "rm: missing operand"
    – user75814
    Apr 10 '11 at 13:50
  • 2
    Did you use the correct path? Obviously /path/to/your/logs is just made up for you to enter the correct path. To find the bug execute the parts ls -t /path/... and ls -t /path/... | tail -n +11 alone and check if the result is correct. Apr 11 '11 at 9:01
  • 1
    @HaukeLaging I am quite sure you got something wrong. Mind the + before the number. It makes tail not print the last n elements but all elements beginning from the n'th. I checked in again and the command should definitely remove only elements older than 10 most recent files under all circumstances. Mar 14 '13 at 14:01
  • @halo Indeed, sorry. Mar 14 '13 at 17:41
  • 2
    I think your answer is very dangerous ls prints a file name, not a path so your rm -f will try to remove the files in the current directory Jun 20 '17 at 11:37
19

Apparently parsing ls is evil.

If each file is created daily and you want to keep files created within the last 10 days you can do:

find /path/to/files -mtime 10 -delete

Or if each file is created arbitrarily:

find /path/to/files -maxdepth 1 -type f -printf '%Ts\t%P\n' | sort -n | head -n -10 | cut -f 2- | xargs rm -rf
3
  • 7
    -mtime 10 -delete only deletes files modified exactly 10 days ago. Older files will not be deleted. -mtime +11 -delete will delete files that have been modified 11 or more days ago leaving the last 10 days of log files.
    – Markus
    Jun 19 '18 at 8:54
  • 2
    I think the use of %p instead of %P is needed on the printfso that the files got have the full path. Otherwise the rm -rf does not work.
    – jalopaba
    Sep 5 '19 at 10:49
  • -delete also kills the directory not only the files within
    – Max
    Jul 21 '20 at 9:27
9

A tool like logrotate does this for you. It makes log management much easier. You can also include additional cleanup routines shuch as halo sugggested.

2

I modified Isaac's approach a little bit.

It now works with a desired path:

ls -d -1tr /path/to/folder/* | head -n -10 | xargs -d '\n' rm -f
1

From here:

#! /bin/sh
# keepnewest
#
# Simple directory trimming tool to handle housekeeping
# Scans a directory and deletes all but the N newest files
#
# Usage: cleanup <dir> <number of files to keep>
#
# v 1.0 Piers Goodhew 1/mar/2007. No rights retained.


if [ $# -ne 2 ]; then
  echo 1>&2 "Usage: $0 <dir> <number of files to keep>"
  exit 1
fi

cd $1
files_in_dir=`ls | wc -l`
files_to_delete=`expr $files_in_dir - $2`
if [ $files_to_delete -gt 0 ]; then
  ls -t | tail -n $files_to_delete | xargs rm
  if [ $? -ne 0 ]; then
    echo "An error ocurred deleting the files"
    exit 1
  else
    echo "$files_to_delete file(s) deleted."
  fi
else
  echo "nothing to delete!"
fi
1
  • 1
    Thanks for providing this answer. While providing code like this is helpful, it also helps to provide some additional information about how it might be used or what the code does. You can use the edit button to make future improvements to your post.
    – nhinkle
    Mar 21 '13 at 5:23
1

In Bash you can try:

ls -1t | (i=0; while read f; do
  if [ $i -lt 10 ]; then
    ((i++))
    continue
  else
    rm -f "$f"
  fi
done)

This skips the 10 newest als deletes the rest. logrotate may be better, I just want to correct the wrong shell related answers.

1

not sure this will help anyone but to avoid potential issues with wonky characters i just used ls to perform the sort but then use the files inode for the delete.

For current directory, this keeps the most recent 10 files based on modification time.

ls -1tri | awk '{print $1}' | head -n -10 | xargs -n1 -t -Iinode find . -inum inode -exec rm -i {} \;

1

I agree that parsing ls is evil!

Ensure only the last 5 files are kept in the /srv/backups path.

find /srv/backups -maxdepth 1 -type f -printf '%Ts\t%P\n' \
    | sort -rn \
    | tail -n +6 \
    | cut -f2- \
    | xargs -r r
1
  • This is the best solution here simply because it doesn't parse the output of ls. (It does have a problem if you have newlines in your filenames though, eek!)
    – bobbogo
    Feb 6 '20 at 16:03
0

I needed an elegant solution for the busybox (router), all xargs or array solutions were useless to me - no such command available there. find and mtime is not the proper answer as we are talking about 10 items and not necessarily 10 days. Espo's answer was the shortest and cleanest and likely the most unversal one.

Error with spaces and when no files are to be deleted are both simply solved the standard way:

rm "$(ls -td *.tar | awk 'NR>7')" 2>&-

Bit more educational version: We can do it all if we use awk differently. Normally, I use this method to pass (return) variables from the awk to the sh. As we read all the time that can not be done, I beg to differ: here is the method.

Example for .tar files with no problem regarding the spaces in the filename. To test, replace "rm" with the "ls".

eval $(ls -td *.tar | awk 'NR>7 { print "rm \"" $0 "\""}')

Explanation:

ls -td *.tar lists all .tar files sorted by the time. To apply to all the files in the current folder, remove the "d *.tar" part

awk 'NR>7... skips the first 7 lines

print "rm \"" $0 "\"" constructs a line: rm "file name"

eval executes it

Since we are using rm, I would not use the above command in a script! Wiser usage is:

(cd /FolderToDeleteWithin && eval $(ls -td *.tar | awk 'NR>7 { print "rm \"" $0 "\""}'))

In the case of using ls -t command will not do any harm on such silly examples as: touch 'foo " bar' and touch 'hello * world'. Not that we ever create files with such names in real life!

Sidenote. If we wanted to pass a variable to the sh this way, we would simply modify the print:

print "VarName="$1

to set the variable VarName to the value of $1. Multiple variables can be created in one go. This VarName becomes a normal sh variable and can be normally used in a script or shell afterwards. So, to create variables with awk and give them back to the shell:

eval $(ls -td *.tar | awk 'NR>7 { print "VarName="$1 }'); echo "$VarName"
0

Based on Isaac Freeman response, but working on any directory:

ls -1tr $PATH_TO_DELETE/* | head -n -10 | xargs -d '\n' rm -f --

You can also set a prefix, like $PATH_TO_DELETE/testFi*

If you use * after the PATH ls will output absolute file names, at least in "my" bash :)

0

On my Synology NAS I slightly modified @h0tw1r3's answer to make it work in a scheduled task and to handle possible blank and/or newline characters in the file name by zero-terminating the generated strings:

find /path/to/backup/folder -maxdepth 1 -type f -printf '%Ts\t' -print0 \
| sort -rnz \
| tail -n +11 -z \
| cut -f2- -z \
| xargs -0 -r rm -f

IMPORTANT: This will delete any matching files! I strongly recommend doing a test run first by replacing the last command xargs -0 -r rm -f with xargs -0 which will echo the matching files instead of deleting them.

A short explanation of each step:

  1. find /path/to/backup/folder -maxdepth 1 -type f -printf '%Ts\t' -print0

    Find all regular files (-type f) in the backup folder without traversing any subfolders (-maxdepth 1), print (-printf) the Unix time (%Ts) of the last modification followed by a tab character (\t, used in step 4) and the full file name followed by a null character (-print0).

  2. sort -rnz

    Sort the zero-terminated items (-z) from the previous step using a numerical comparison (-n) and reverse the order (-r). The result is a list of all backups sorted by their last modification date in descending order.

  3. tail -n +11 -z

    Print the last lines (tail) from the previous step starting from line 11 (-n +11) considering each line as zero-terminated (-z). This excludes the newest 10 files (by modification date) from the remaining steps.

  4. cut -f2- -z

    Cut each line from the second field until the end (-f2-) treating each line as zero-terminaded (-z) to obtain a list containing the full path to each backup older than 10 days.

  5. xargs -r -0 rm -f

    Take the zero-terminated (-0) items from the previous step (xargs), and, if there are any (-r avoids running the command passed to xargs if there are no nonblank characters), force delete (rm -f) them.
-1

You can delete file with specific name pattern and numbers of day old using

find /Users/test/delete/*.zip -mtime +10 -exec rm {} \;

Above command deletes files matching pattern /Users/test/delete/*.zip and which are 10 days older.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.