Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have this bash script which nicely backs up my database on a cron schedule:

#!/bin/sh

PT_MYSQLDUMPPATH=/usr/bin
PT_HOMEPATH=/home/philosop
PT_TOOLPATH=$PT_HOMEPATH/philosophy-tools
PT_MYSQLBACKUPPATH=$PT_TOOLPATH/mysql-backups
PT_MYSQLUSER=*********
PT_MYSQLPASSWORD="********"
PT_MYSQLDATABASE=*********
PT_BACKUPDATETIME=`date +%s`
PT_BACKUPFILENAME=mysqlbackup_$PT_BACKUPDATETIME.sql.gz
PT_FILESTOKEEP=14

$PT_MYSQLDUMPPATH/mysqldump -u$PT_MYSQLUSER -p$PT_MYSQLPASSWORD --opt $PT_MYSQLDATABASE | gzip -c > $PT_MYSQLBACKUPPATH/$PT_BACKUPFILENAME

Problem with this is that it will keep dumping the backups in the folder and not clean up old files. This is where the variable PT_FILESTOKEEP comes in. Whatever number this is set to thats the amount of backups I want to keep. All backups are time stamped so by ordering them by name DESC will give you the latest first.

Can anyone please help me with the rest of the BASH script to add the clean up of files? My knowledge of bash is lacking and I'm unable to piece together the code to do the rest.

share|improve this question

6 Answers 6

up vote 9 down vote accepted

First, be sure you are in right folder:

if [ -z $PT_MYSQLBACKUPPATH ]; then
 echo "No PT_MYSQLBACKUPPATH set. Exit"
 exit 1
fi
cd $PT_MYSQLBACKUPPATH
if [ $? != 0 ]; then
 echo "cd to PT_MYSQLBACKUPPATH failed. Exit"
 exit 1
fi

You can remove files older than n, in your case:

find -mtime +14 -delete

Deletes files older than 14 days.

More complicated (definitely not optimal, though) solution for your question:

# Get list of newest files. If newest files are first, use head -n 14 instead of 
# head.
files=(`ls | sort | tail -n 14`)
# Loop over all files in this folder
for i in *; do 
 preserve=0; 
 #Check whether this file is in files array:
 for a in ${files[@]}; do 
  if [ $i == $a ]; then 
   preserve=1; 
  fi; 
 done; 
 # If it wasn't, delete it (or in this case, print filename)
 if [ $preserve == 0 ]; then 
  echo $i; # test first, then change this to "rm $i"
 fi;
done
share|improve this answer
    
This isnt what I was really after but actually will work better than what I had in mind. Before I test how do I make sure this is run in PT_MYSQLBACKUPPATH. Don't want to run this outside of that otherwise end up deleteing a whole load of files. Could be a disaster... –  Brady Mar 21 '11 at 12:46
    
@Brady: I added example for checking environment. –  Olli Mar 21 '11 at 12:51
2  
Do not ever use for i in $(ls) or var=($(ls)) (Hint: for i in *) unless you can be 400% sure that the filenames will never contain spaces or anything like that. –  grawity Mar 21 '11 at 13:02
    
@Olli - Ok that script is now working without error however I won't be able to tell if the 14 day deletion is working untill I have some old files in there. Will re-visit in 14 days if there is an issue. Thanks for the help. –  Brady Mar 21 '11 at 13:10
1  
ls | sort sorts by name. Use ls -t | head -n 14 (it will still fail, though, for filenames that contain whitespace). –  Dennis Williamson Mar 22 '11 at 0:14

You could try this one:

ls -r1 $PT_MYSQLBACKUPPATH/ | tail +$(($PT_FILESTOKEEP+1)) | xargs rm

ls -r1 will list all files in reverse order, one file per line.

tail +$number filters the first $number-1 files of the list out (resp. displays all files beginning from $number till the last one).

xargs will execute rm with all file names from standard input.

share|improve this answer
    
Thanks for your reply bmk but I have gone with using find -mtime +14 -delete provided by Olli –  Brady Mar 22 '11 at 14:49
    
+1 because this very precisely answers the question. –  DevSolar Jun 28 '12 at 14:58

Here is my usage of inspiration from this post:

#!/bin/bash
# Thu Jun 28 13:22:53 CEST 2012
# ${DESTDIR}/files2keep.sh
# Keep the 3 yungest files
# mra at miracleas.dk , deployed on RHEL 6.
InitValues(){
TODAY=`date +"%Y%m%d"`
NOW=`date +"%H%M"`
DESTDIR=/mnt/dbdmp
LOGFILE=?{0}-${TODAY}-${NOW}.log
}
BackupFileMaintenance(){
KEEPFILES=(`ls -lrt ${DESTDIR}/*mysqldump.sql.gz| tail -n 3| awk '{print $9}'`)
    for i in `ls -lrt ${DESTDIR}/*mysqldump.sql.gz | awk '{print $9}'`; do
    preserve=0 
    #Check whether this file is in files array:
        for a in ${KEEPFILES[@]}; do
                if [ $i == $a ]; then
                preserve=1 
                fi 
        done 
    if [ $preserve == 0 ]; then
    echo $i; # then change this to "rm -f $i" after test
    fi
    done
}
InitValues
BackupFileMaintenance
exit
share|improve this answer

Regarding answer from bmk, when available ls -t1 is safer than -r1 (sort by modification time rather than arbitrary file order)

Also on some versions of tail syntax is tail -n +$number (option -n is needed)

As a bonus, combining both find and ls, here is a way to remove files older than 30 days but keep at least 15 files:

ls -t1 $PT_MYSQLBACKUPPATH/|tail -n +16| xargs -n1 basename|xargs -n1 -I{} find $PT_MYSQLBACKUPPATH/ -mtime +30 -name {} -delete
share|improve this answer

sorry to reopen this older thread, but i had recently a similar issue and could not find a good solution.

In the end i solved it like this:

cd /directory_where_things_need_removing
ls -tr1dQ * | head -n -31 | xargs rm -rf

important parts are ls -tr1dQ where the Q quotes, solving the spaces issue, and 1 which gives 1 solution per line. head -n -31 which OMITS the first 31 lines (in my case = 1 month). please note the - before n and before the number of lines you want to keep. tested on scientific linux 6.5

share|improve this answer
ls -t /path/* | grep -v "$(ls -t /path/* | head -6)" | xargs rm -f
share|improve this answer
    
Some explanation of the code you're posting would be great. Can you edit your answer? –  slhck Aug 12 '13 at 18:18

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.