7

I have this bash script which nicely backs up my database on a cron schedule:

#!/bin/sh

PT_MYSQLDUMPPATH=/usr/bin
PT_HOMEPATH=/home/philosop
PT_TOOLPATH=$PT_HOMEPATH/philosophy-tools
PT_MYSQLBACKUPPATH=$PT_TOOLPATH/mysql-backups
PT_MYSQLUSER=*********
PT_MYSQLPASSWORD="********"
PT_MYSQLDATABASE=*********
PT_BACKUPDATETIME=`date +%s`
PT_BACKUPFILENAME=mysqlbackup_$PT_BACKUPDATETIME.sql.gz
PT_FILESTOKEEP=14

$PT_MYSQLDUMPPATH/mysqldump -u$PT_MYSQLUSER -p$PT_MYSQLPASSWORD --opt $PT_MYSQLDATABASE | gzip -c > $PT_MYSQLBACKUPPATH/$PT_BACKUPFILENAME

Problem with this is that it will keep dumping the backups in the folder and not clean up old files. This is where the variable PT_FILESTOKEEP comes in. Whatever number this is set to thats the amount of backups I want to keep. All backups are time stamped so by ordering them by name DESC will give you the latest first.

Can anyone please help me with the rest of the BASH script to add the clean up of files? My knowledge of bash is lacking and I'm unable to piece together the code to do the rest.

6 Answers 6

9

First, be sure you are in right folder:

if [ -z $PT_MYSQLBACKUPPATH ]; then
 echo "No PT_MYSQLBACKUPPATH set. Exit"
 exit 1
fi
cd $PT_MYSQLBACKUPPATH
if [ $? != 0 ]; then
 echo "cd to PT_MYSQLBACKUPPATH failed. Exit"
 exit 1
fi

You can remove files older than n, in your case:

find -mtime +14 -delete

Deletes files older than 14 days.

More complicated (definitely not optimal, though) solution for your question:

# Get list of newest files. If newest files are first, use head -n 14 instead of 
# head.
files=(`ls | sort | tail -n 14`)
# Loop over all files in this folder
for i in *; do 
 preserve=0; 
 #Check whether this file is in files array:
 for a in ${files[@]}; do 
  if [ $i == $a ]; then 
   preserve=1; 
  fi; 
 done; 
 # If it wasn't, delete it (or in this case, print filename)
 if [ $preserve == 0 ]; then 
  echo $i; # test first, then change this to "rm $i"
 fi;
done
6
  • This isnt what I was really after but actually will work better than what I had in mind. Before I test how do I make sure this is run in PT_MYSQLBACKUPPATH. Don't want to run this outside of that otherwise end up deleteing a whole load of files. Could be a disaster...
    – Scott
    Mar 21, 2011 at 12:46
  • @Brady: I added example for checking environment.
    – Olli
    Mar 21, 2011 at 12:51
  • 2
    Do not ever use for i in $(ls) or var=($(ls)) (Hint: for i in *) unless you can be 400% sure that the filenames will never contain spaces or anything like that. Mar 21, 2011 at 13:02
  • @Olli - Ok that script is now working without error however I won't be able to tell if the 14 day deletion is working untill I have some old files in there. Will re-visit in 14 days if there is an issue. Thanks for the help.
    – Scott
    Mar 21, 2011 at 13:10
  • 1
    ls | sort sorts by name. Use ls -t | head -n 14 (it will still fail, though, for filenames that contain whitespace). Mar 22, 2011 at 0:14
11

You could try this one:

ls -r1 $PT_MYSQLBACKUPPATH/ | tail -n +$(($PT_FILESTOKEEP+1)) | xargs rm

ls -r1 will list all files in reverse order, one file per line.

tail -n +$number filters the first $number-1 files of the list out (resp. displays all files beginning from $number till the last one).

xargs will execute rm with all file names from standard input.

2
  • Thanks for your reply bmk but I have gone with using find -mtime +14 -delete provided by Olli
    – Scott
    Mar 22, 2011 at 14:49
  • +1 because this very precisely answers the question.
    – DevSolar
    Jun 28, 2012 at 14:58
3

Here is my usage of inspiration from this post:

#!/bin/bash
# Thu Jun 28 13:22:53 CEST 2012
# ${DESTDIR}/files2keep.sh
# Keep the 3 yungest files
# mra at miracleas.dk , deployed on RHEL 6.
InitValues(){
TODAY=`date +"%Y%m%d"`
NOW=`date +"%H%M"`
DESTDIR=/mnt/dbdmp
LOGFILE=?{0}-${TODAY}-${NOW}.log
}
BackupFileMaintenance(){
KEEPFILES=(`ls -lrt ${DESTDIR}/*mysqldump.sql.gz| tail -n 3| awk '{print $9}'`)
    for i in `ls -lrt ${DESTDIR}/*mysqldump.sql.gz | awk '{print $9}'`; do
    preserve=0 
    #Check whether this file is in files array:
        for a in ${KEEPFILES[@]}; do
                if [ $i == $a ]; then
                preserve=1 
                fi 
        done 
    if [ $preserve == 0 ]; then
    echo $i; # then change this to "rm -f $i" after test
    fi
    done
}
InitValues
BackupFileMaintenance
exit
3

Regarding answer from bmk, when available ls -t1 is safer than -r1 (sort by modification time rather than arbitrary file order)

Also on some versions of tail syntax is tail -n +$number (option -n is needed)

As a bonus, combining both find and ls, here is a way to remove files older than 30 days but keep at least 15 files:

ls -t1 $PT_MYSQLBACKUPPATH/|tail -n +16| xargs -n1 basename|xargs -n1 -I{} find $PT_MYSQLBACKUPPATH/ -mtime +30 -name {} -delete
0

sorry to reopen this older thread, but i had recently a similar issue and could not find a good solution.

In the end i solved it like this:

cd /directory_where_things_need_removing
ls -tr1dQ * | head -n -31 | xargs rm -rf

important parts are ls -tr1dQ where the Q quotes, solving the spaces issue, and 1 which gives 1 solution per line. head -n -31 which OMITS the first 31 lines (in my case = 1 month). please note the - before n and before the number of lines you want to keep. tested on scientific linux 6.5

-1
ls -t /path/* | grep -v "$(ls -t /path/* | head -6)" | xargs rm -f
1
  • Some explanation of the code you're posting would be great. Can you edit your answer?
    – slhck
    Aug 12, 2013 at 18:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .