2

I am a developper, and I have to replicate some code to our test server to run my code.

I am using Ubuntu 18.04.2.

I have developped a mix of inotify-tools and rsync :

rsyncWrap(){
    rsync -avzq --delete --exclude-from="$HOME/.rsyncignore" "$source" "$target"
}   

rsyncWrap
while inotifywait -r -e modify,create,delete,move "$source"; do
    rsyncWrap
done

Note that I rely on openssh to handle the underlaying private key authentication.

but I have to make sure to launch it every time I start the computer (I tend to be quite distracted, and it seems @reboot doesn't work in my cron ...).

Furthermore, that scripts tends to behave badly in case of error (I sometimes find it in "defunct" state in ps aux).

What is the correct tool to accomplish that ?

  • Can't you just create the SSH tunnel when you call rsyncWrap? – Kinnectus Jul 29 at 10:31
  • I don't know what is a SSH tunnel. – Pierre-Antoine Guillaume Jul 29 at 10:32
  • You've said you have to run OpenSSH (ssh) to be able to run your rsync command...you could call your ssh command when your rsyncWrap() function is called. This means you won't need to keep the SSH open all the time (unles it's necessary to have it open all the time?) – Kinnectus Jul 29 at 10:36
  • When I say I use openssh to handle the password, I mean the variable $target is something like : remote-server:path, and I have a private key associated with that connection in my $HOME/.ssh. Is that different from what I said ? – Pierre-Antoine Guillaume Jul 29 at 10:39
2

What is the correct tool to accomplish that ?

What you are doing is correct. rsync is among the best tools to use for file synchronization accross different machines, unless you have a repository you can push and pull.

but I have to make sure to launch it every time I start the computer (I tend to be quite distracted, and it seems @reboot doesn't work in my cron ...).

I'd suggest you to use systemd here, as you can configure systemd services to run after the network starts and after you have connection. @reboot might not work exactly because you don't have a network connection yet (which rsync relies upon).

This is an example systemd configuration you can use:

[Unit]
Description=Some description
After=network-online.target

[Service]
ExecStart=/path/to/script
Type=oneshot
User=<user to run the script with>

[Install]
WantedBy=multi-user.target

You save that configuration in /etc/systemd/system/<service_name>.service and run the following command:

systemctl daemon-reload

Now if you want to enable that script to run on boot:

systemctl enable <service_name>.service

If you want to start the script manually, you can do it in your usual way or use systemd again:

systemctl start <service_name>.service

Furthermore, that scripts tends to behave badly in case of error (I sometimes find it in "defunct" state in ps aux).

That might be because of rsync segmentation faults or faulty exits (which actually tends to happen quite often).

I will suggest you two things here:

1. Add a timeout for the rsync command, so that it doesn't hang.
2. Exit (?) if rsync doesn't return with exit status 0 or at least log a message so you know about it. Or add -e flag to /bin/bash so it exits once an error is received.

A short example on how to achieve this:

rsyncWrap(){
    timeout 30 rsync -avzq --delete --exclude-from="$HOME/.rsyncignore" "$source" "$target"
}   

rsyncWrap
rsync_exit_code=$?
if [ ! rsync_exit_code -eq 0 ]; then
    echo "rsync exited with exit code ${rsync_exit_code}"
done

while inotifywait -r -e modify,create,delete,move "$source"; do
    rsyncWrap
    if [ ! rsync_exit_code -eq 0 ]; then
        echo "rsync exited with exit code ${rsync_exit_code}"
    done
done

timeout 30 will make sure the following command doesn't run more than 30 seconds.

References:
1. How to run a single command at startup using systemd
2. systemd.Service wiki
3. What is bash -e flag

  • Thanks, but then I have another issue : service are run as root, correct ? my private key is in my $HOME/.ssh, how do I connect the two ? – Pierre-Antoine Guillaume Jul 29 at 10:46
  • 1
    @Pierre-AntoineGuillaume They run as root by default. As I've added in the configuration, there is a parameter User=, which tells systemd which user to run the script as. Via that User= parameter you can tell systemd to run the service with your own user (or any other user). – Fanatique Jul 29 at 10:50
  • great. Is there a way to support the restart as well ? – Pierre-Antoine Guillaume Jul 29 at 10:50
  • @Pierre-AntoineGuillaume which restart do you mean? Reboot of the machine? – Fanatique Jul 29 at 10:51
  • 1
    Ok so it turned out I am better off with Type=simple instead of oneshot ! thanks for your input, it helped tremendously – Pierre-Antoine Guillaume Jul 29 at 12:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.