87

Scenario: An IDE is set up on a Linux desktop box, editing PHP files locally. Every time I save a file, I want this change to appear on the linux server where Apache is running. The server has ssh (and samba and nfs for that matter).

As a reference, when I edited files on Windows, I finally came over WinSCP as the exact tool I needed - WinSCP have just this feature present, with initial synch and then continuous update, using the filesystem watch service: "Keep Remote Directory up to Date".

On Linux, one could argue that sshfs could be employed to sidestep the need for synchronization entirely. On windows, a samba-share would do the same. However, I want the IDE to work with local files (on a SSD disk!), not having to go over the network to do PHP indexing and whatnots, which takes ages.

But sshfs might be a part of the solution nevertheless - so that the continuous synchronization just needed to be done between two local directories.

Any ideas or pointers?

4
  • consider using the rsync tool, or sharing a folder in the webserver's document root so you could operate on the files directly under windows Aug 1, 2011 at 14:43
  • 2
    rsync is "one go". I need continous updates, that is the entire point here - I edit a file, save it, and the product/system/idea I request would pick this save-action up and upload the new version immediately. NB: Both sides are Linux. NB2: I want to edit on local files, or else sshfs itself would cut it.
    – stolsvik
    Aug 1, 2011 at 14:44
  • 7
    Off-topic voters, this is a boundary case, but I think it falls fairly clearly under "tools commonly used by programmers" in the FAQ. Aug 1, 2011 at 20:44
  • While it's not a perfect match, you might look at overlayfs
    – phs
    Jul 12, 2016 at 21:10

7 Answers 7

68

You can also use inotifywait from the inotify-tools package.

inotifywait -r -m -e close_write --format '%w%f' /tmp | while read MODFILE
do
    echo need to rsync $MODFILE ...
done
5
  • 16
    On the inotify-tools website, there's a pretty good example of using inotify-wait to trigger an rsync. Aug 1, 2011 at 20:41
  • I was looking for this exact thing last night! ahhh I love superuser Aug 1, 2011 at 21:27
  • Is there a way to do this but keep the socket open? rsync/scp has an annoying delay while establishing the ssh connection.
    – Flash
    Jul 9, 2017 at 5:50
  • 2
    @Flash yes, you can use SSH multiplexing to keep the socket open. Apr 5, 2018 at 17:40
  • 1
    unfortunately this gets me auto firewall IP banned xD
    – hanshenrik
    Apr 22, 2022 at 7:51
41

Lsyncd would be a good solution for this.

Lsyncd watches a local directory trees event monitor interface (inotify or fsevents). It aggregates and combines events for a few seconds and then spawns one (or more) process(es) to synchronize the changes. By default this is rsync. Lsyncd is thus a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or blockdevices and does not hamper local filesystem performance.

Bottom-line, it uses the same kind of tools to do the job (inotify and rsync) as suggested by other answers, but it's easier to set up for someone not familiar with shell scripting.

3
  • not only easier, it also tries to handle situations like moving directory properly, without rsyncing.
    – Ciantic
    Dec 4, 2013 at 17:18
  • 11
    Beware: This program has some problematic features. As of writing this: 1. It deletes remote files not present at the source per default. 2. "remote:" refers to "remote:/" instead of the home folder. 3. It daemonizes, so you do not know what is going on. 4. It does not immediately respect the TERM signal.
    – Friedrich
    Dec 7, 2017 at 14:03
  • Also, I did not manage to compile it. Aug 14, 2022 at 10:15
35

I need this a lot since my code needs to run on remote boxes and I write code on local machine. I found a nice tool which you can use to continuously monitor your local folders and sync them to remote or local folder: https://github.com/axkibe/lsyncd

A simple command to continuously sync a local dir with remote machine over ssh will be:

lsyncd -log all -nodaemon -rsyncssh <local_path> <user>@<ip> <remote_path>

Just like with any other rsync command, make sure you give the folder path right and check before you run the command. I almost had killed one of my remote machine because I missed to give a correct destination directory. Make sure yo don't miss out the remote path and don't use '/' unless you know what you are doing.

0
2

If you need to observe filesystem, then inotify is the way to do it. I would write a simple python script using pyinotify to perform sync when filesystem get changed. See documentation. You might also checkout the autosync.py for some inspiration. Have fun.

1

What I did once is have a bash script running ls -l in a loop (with some sleep) and comparing to the previous output. If it changed, do your synchronization.

#!/bin/bash

listcommand="ls -l $*"

newfilelist=$( $listcommand )
while true
do
   if [[ $oldfilelist != $newfilelist ]]
   then
      oldfilelist=$newfilelist
      # run your synchronization tool
   fi
   sleep 10 || exit 2 
   newfilelist=$( $listcommand )
done

Start this script in a new terminal with the file names as arguments (after putting in your synchronization tool).

(I used this to start a compilation, not to synchronize, but this would work a similar way.)

1
  • This gets my ip banned under a firewall after sometime.
    – diffracteD
    Jun 22, 2022 at 1:20
1

Another option: https://github.com/mikkorantalainen/rsync-continuous

This script uses bash, inotifywait and rsync over ssh to create very fast one-way sync because only modified files need to be transferred and if only part of the file has been modified, rsync will only transfer the modified part.

If you run mkdir -p ~/.ssh/connections and add following to the start of ~/.ssh/config the performance will be much better:

ControlMaster auto
ControlPath   ~/.ssh/connections/%r@%h:%p
ControlPersist 1

Note that the above configuration causes extra output for every case you exit an ssh connection due bug https://bugzilla.mindrot.org/show_bug.cgi?id=3220

If you keep an open ssh connection to the remote host used with the above script, files should be syncronized with roughly 100 ms delay.

-2

Much more simple approach:

Export your /var/www with samba (or nfs) and work on the files directly on the server.

Another solution: most IDE's allow various deployment configuration - check if there is one that suits you.

Or set a Source Code Management system like Git, Bazaar, etc.

Good luck !!

1
  • 3
    Thanks for answering. However, I specifically excluded this very approach in the question.
    – stolsvik
    Aug 3, 2011 at 17:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .