Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am working on an approach to archive our website (dynamically generated) periodically (say every month) and keep it versionned so that I can go back and pull a page at a certain period.

My initial approach is to crawl the site recursively and commit it to a subversion repository so that I can use subversions history and export feature.

Is there any other optimal solution which uses less space as possible? Also, I am not sure how long a subversion commit of an entire site would take so a faster solution is also desired.

share|improve this question
Is there anything wrong with just creating a gzipped tarball of the site in regular intervals? Space shouldn't be the primary concern when taking backups. – slhck Aug 23 '13 at 14:51
Why don't you just put the site itself in a version control system and when you make a change, update the deployed site from your repository? – Oliver Salzburg Aug 23 '13 at 14:51
@OliverSalzburg I should have mentioned in the question. The site is dynamically generated. – Balaji Natarajan Aug 23 '13 at 15:03
If having a ready to be seen archive, I would suggest wget and the suggestion of tarballing the output. – Doktoro Reichard Aug 23 '13 at 15:03
Dyanamically generated static pages? If so then dynamically generated doesn't really matter. Gzipped tarball per slhck and a cron job. If dynamic at runtime then a database snapshot and standard source control – ToddB Aug 23 '13 at 15:05

Use 7zip on a cron job to periodically zip with recursive and a datetime stamp - then test and mv to a fail-over cluster - always test your backup periodically or else you will have corrupt data and not find out until its critical. I believe you can use a 7zip test switch as well. We've been using this approach for 7 years with mission critical data and zip 16 times a day with no failures whatsoever.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .