0

I have a few time consuming tasks that I like to spread across several computers. These tasks require running an identical ruby or python script (or series of scripts that call each other) on each machine. The machines will a separate config file telling the script what portion of the task to complete.

I want to figure out the best way to syncronize the scripts on these machines prior to running them. Up until now, I have been making changes to a copy of the script on a network share and then copying a fresh copy to each machine when I want to run it. But this is cumbersome and leaves a chance for error ( e.g missing a file on the copy or not clicking "copy and replace").

Lets assume the systems are standard windows machines that are not dedicated to this task and I don't need to run these scripts all the time (so I don't want a solution that runs 24/7 and always keeps them up to date, I'd prefer something that pushes/pulls on command).

My thoughts on various options:

  • Simple adaptation of my current workflow: Keep the originals on the network drive, but write a batch file that copies over the latest version of the scripts so everything is a one-click operation. Requires action on each system, but that's not the end of the world (since each one usually needs their configuration file changed slightly too).
  • Put everything in a Mercurial/Git reposotory and pull a fresh copy onto each node. Going straight to the repo from each machine would guarantee a current version (and would have the fringe benefit of allowing edits to the script to be made from any machine). Cons would be that it requires VCS to be installed on each machine and there might be some pains dealing with authentication since I wouldn't use a public repo.
  • Open up write access on a shared folder and write a script to use rsync (or similar) to push the changes out to all of the machines at once. This gets a current version on every machine (though you would have to change the script if you want to omit a machine or add a new one). Possible issue would be that each computer has to allow write access.
  • Dropbox is a reasonable suggestion (and could work well) but I dont want to use an external service and I'd prefer not to have to have dropbox running 24/7 on systems that would normally not need it.

Is there something simple that I am missing? Some tool designed expressly for doing this kind of thing? Otherwise I am leaning toward just tying all of the systems into Mercurial since, while it requires extra software, it is a little more robust than writing a batch file (e.g. if I split part of a script into a separate module, Mercurial will know what to do whereas I would have to add a line to the batch file).

2 Answers 2

1

I use Jenkins (CI Server) and Capistrano 2 (Ruby build/deploy). You don't necessarily need Jenkins for this, but a CI server is great for this kind of work. Capistrano is great because I can do one thing/many machines easily and can handle checking out from the SCM (git or mercurial as you asked) if you want. Fabric (Python) also is a good "deploy" framework.

Workflow:

  • Capistrano checks out a git branch/master containing script on local machine
  • Capistrano pushes script file out to all machines via scp or sftp. (:deploy_via, :copy)
  • Capistrano executes script on all "app" nodes, in parallel or sequentially, and reports success or failure.

You can substitute Capistrano with (Fabric|bash|Chrome|chef recipe|puppet config)

Now if you have a CI server in play, you can manage SSH keys and adminstration from one central point.

I can safely assume you are using Windows for local development, if these are also Windows machines you are deploying to... you may want to consider Powershell commands run from Capistrano rather than the "unix" way I outlined here. The challenge is how X logs into box Y and performs Z, a different question.

0

Just have an initial "loader" script that grabs the latest version of everything from the network. Store config data outside of this tree (which you should be doing anyway). The loader can be anything from a simple batch file calling xcopy/robocopy up to a DVCS pull or DropBox/OwnCloud sync.

This way it doesn't matter how it's run, the first action will be to update to latest before execution.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.