I am transferring files over the network using rsync in cron job that runs every 1 one minute, task is whenever new file come it has to start uploading on remote server:

Since I am new so doing big mistake here:

* * * * * rsync -azP /local_path [email protected]:/remote_path > /dev/null

rsync create temp file (at remote end) and on next minute it creates temp file again (at remote end) after few minutes my networks starts getting choke and no upload bandwidth left, then I kill all rsync processes and copy at at a time.

Need this automated please help.


  • Find out how long the rsync really takes, then schedule at least the quadruple of this time. Every minute is overdoing it - your computer will do nothing else but rsync.
    – harrymc
    Aug 15, 2018 at 20:01

2 Answers 2


Looks like this is because the command takes longer than a minute to run, so the commands pile up until your system is starved of resources.

You can ensure that only one instance of that rsync runs by using a lock file. util-linux includes a file locking wrapper called flock, which you can use in your crontab like so:

* * * * * /usr/bin/flock -n '/tmp/example.lock' -c 'rsync -azP /local_path [email protected]:/remote_path' > /dev/null

In the above example, the lock file path is /tmp/example.lock, but you can set it to be anything sensible. -n will prevent flock commands from piling up because it'll exit immediately instead of wait for the lock to be released to continue.

Alternatively, you can also prevent your cron command from piling up by making a script with a fairly simple lock that ensures only one instance of the script can be running at a time:



if [ -f "$PIDFILE" ]
        PID="$(cat "$PIDFILE")"
        ps -p $PID > /dev/zero 2>&1
        if [ $? -eq 0 ]
                echo "Instance of this script is still running as PID $PID"
                exit 1

echo $$ > $PIDFILE
if [ $? -ne 0 ]
        echo "Could not create PID file: $PIDFILE"
        exit 1

rsync -az /local_path use[email protected]:/remote_path

rm -f "$PIDFILE"

Locking mechanism inspired by Preventing duplicate cron job executions by Benjamin Cane

Note that the main limitation in the script is if the PID file isn't removed for whatever reason and another process has the same PID, the script will erroneously quit.

Save this script somewhere, like in /usr/local/bin/example.sh, chmod +x /usr/local/bin/example.sh, and then call it from your crontab like this:

* * * * * /usr/local/bin/example.sh > /dev/null
  • 1
    Dangerous script. If machine would forcibly stopped, or case with power outage then saved PID file will live forever preventing rsync start. Also, rm command in such script must be executed via trap to be make sure in case for example restart or shutdown event then PID file would be removed. Also, encourage students to do at least part of their works instead of giving them free solutions
    – Alex
    Aug 15, 2018 at 17:26

First of all remove -P option, cron really won't to see progress.

You need to implement some locking mechanism, don't call rsync directly from cron, but make some script instead that would be used in cron. In the script, check first condition if rsync already running (grep ps output for example), if it running then simply exit script, otherwise run rsync.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .