Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to create a script that

  • must run with a CRON every minute (Q: Is it wise to run CRON FTP/wget every minute?)
  • compares local directory with remote directory (Q: do I use FTP or wget and how do I create

  • obtains all the newer/changed XML files from the remote directory (Q: How do I throw that into one variable to use in loop later)

  • mail notify the administrator as soon as this process fails

  • copy newer/changed files_this_date to a directory_this_date

Is it possible or recommended to segment the scripts into,,, At the end I have to process the XML files with PHP

I know this is many questions (some parts has been solved) but for now I want to get an

  • 1) opinion on viability/analysis
  • 2) how to get $variable = (ls newer/changed files?)
share|improve this question

It would be a lot easier if you use rsync instead of wget as rsync has been designed exactly to give an answer to your need among other tasks. You could use:

rsync -raz -e ssh root@ 

To check if the process fails simply check the exit code $? and if the value of this variable is distinct from zero "0" then something went wrong and you can flag an e-mail communication

To copy the changed files use a timestamp to know which is the last file you have before syncronizing and then you know which the new ones are.

You can create a date based mask in ISO format for your dirs with something like newdirmask=$(date +%Y%m%d''%H%M%S) and use it later to mkdir like mkdir /home/$newdirmask.

share|improve this answer
Exactly what I was hoping for. Learning new and better ways to do this, but the remote is a client's Windows NT server with only a FTP Server. Can this be used to sync a working copy of my application between my localhost and development server? – Wil Sep 3 '13 at 14:25
Indeed, rsync is much easier, and will also work with remote Windows machines when mounting that as a local disk, @Wil. But rather than comparing timestamps, consider using --link-dest. (See my answer here.) – Arjan Sep 3 '13 at 14:46

You can install cygwin ( in your Windows server and have all the functionality of rsync in your Windows environment. Don't be afraid of the Cygwin website look&feel Cygwin is well known software and has been more than tested into production from my part.

share|improve this answer
Thank you for this – Wil Sep 9 '13 at 14:05

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.