-1

We are facing what I'm certain is an incredibly common problem: we do daily backups that result in compressed archive files that have the date in the file name, and we don't want to keep all of them forever.

Some requirements that I don't think are too outlandish:

  • delete all but the last X backups/days
  • keep the Sunday backups
  • keep the each months last backup
  • alert someone if the newest backup is size 0/more than 50% bigger/smaller than the last one

I have tried to google, but only found a thousand scripts, many of them with glaring issues and bugs. Are there no 'proper' solutions to this problem that are well tested and have a community ? Am I doomed to copy and paste a script from SO or SU that will not be maintained or get bugfixes and will some day probably eat all our backups ?

  • google 'incremental backup solutions linux'. Take your pick. – Tetsujin Aug 23 at 14:10
  • Well, if you've built a custom backup system, I suppose you'll end up building a custom pruning system as well... – grawity Aug 23 at 14:11
  • Your problem sounds like a relatively simple set of shell scripts and cron jobs would do it. Might suggest instead of googling for a script that works for someone else's situation you google for how to learn how to do shell scripting. – ivanivan Aug 23 at 14:33
  • 1
    GFS (Grandfather-Father-Son) method of retention is probably a good answer here. Couple that with some well timed crons like @ivanivan mentions. – Smock Aug 23 at 14:55
0

GFS (Grandfather-Father-Son) method of retention is probably a good answer here. Couple that with some well timed crons like @ivanivan mentions.

Basically you take backups daily, weekly, and monthly (The crons will control which these are).

Then you can decide retention policies

  • Daily backups would normally be kept for minimum of 1 week (but sometimes 2 to 3 weeks)
  • Weekly backups would normally be kept for 5 weeks (but potentially 8 weeks)
  • Monthly backups would normally be kept for 6 or 12 months (depending on yearly/half yearly backup)
  • Yearly/Half Yearly backups could also be taken but might only be useful for accounting, or where you need to keep logs

The crons would then also clear down the old archives that are outside the policy window

As ivanivan suggests:

Your problem sounds like a relatively simple set of shell scripts and cron jobs would do it. Might suggest instead of googling for a script that works for someone else's situation you google for how to learn how to do shell scripting.

you might want to look at bash scripting - it's a great thing to learn :)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.