1

Let's say I have a common directory on a Windows server such as: \\servername\location\common. During the day, any user can come and add/remove files to that Windows directory.

How would I setup a nightly schedule (always 9pm) to transfer all files in that directory over to a Unix server at a pre-defined path (that does not change)?

I'm not too concerned about hiding passwords as everyone knows them.

  • Is there some well-known script to do this?
  • If not, does anyone have any tips on how I could implement this?

I have some knowledge of .bat files and shell.

7
  • Transfer how? FTP, SMB...?
    – m0skit0
    Feb 8, 2012 at 9:58
  • @m0skit0 - that is part of the question, I'm open to optinos.
    – toop
    Feb 8, 2012 at 10:20
  • Are the files text or binary?
    – m0skit0
    Feb 8, 2012 at 10:40
  • text (i believe ascii win format with crlf linebreaks)
    – toop
    Feb 8, 2012 at 11:20
  • Are they the same files that change content or totally new different files?
    – m0skit0
    Feb 8, 2012 at 15:01

7 Answers 7

4

This can be done using a Windows task, and WinSCP.

First, go and install WinSCP http://winscp.net/eng/index.php Once that is installed, click New, then enter the information for your server. If you are using keys, locate the key files, otherwise enter the password. Click save. In the dialog that comes up, be sure to check "Save password" if you aren't using key files. Remember the profile name that you chose.

Next Create a file, config.txt in the same location you installed WinSCP in the file insert the following, replacing the ** with things specific to you.

option batch on
option confirm off
open *NAME OF PROFILE* synchronize local -mirror *Local path: \\servername\location\common* *Remote Path*
exit

Press Windows Key + R

Then type taskschd.msc /s click okay. In the right hand side of the window that opens select Create Task... Enter a Name: Folder Auto Upload (or whatever suits you)

Click on the Triggers tab then click New Select the Daily radio button There will be a time listed, Adjust the time to when you want the upload to happen. Click OK

Click on the Actions tab click New in the Program/script field enter the location of the WinSCP.exe (or browse to where you installed it and select it.) in the Add arguments (optional): field enter /console /script=config.txt then click OK click OK again, and you should be all set.

3

I would either use a free software like "AlwaySync" to send my files over SFTP/SSH. Or I would set up an ISCSI/SMB share on the remote system, and back up that way. Or I would install Cygwin, and set up rsync to backup to the Linux server on a cron.

These are just concepts to get your brain juices flowing!

1
  • bounty up for grabs if you can provide more detail and a recommendation.
    – toop
    Mar 11, 2012 at 8:10
2
+50

You could install putty, and use pscp with schedule tasks. If security wasn't a priority, you can do the same with the built in ftp client on windows.

In my case, i use putty and pscp from the gow distribution of unix tools

A basic file transfer would be of an entire directory would be something like 'pscp -pw password -r "path/to/source/" [email protected]:/path/to/destination/' - there's some problems with this approach, such as having your password in plain text

Using pagent would be a better approach - you'll have to set it up, and replace -pw password with -agent i believe.

IIRC if you wanted to go the rsync route deltacopy might be a good option, it does scheduled incremental backupsh

4
  • thanks for the ideas. The idea is not to do backups but instead run some SQL contained in those files against the unix server.
    – toop
    Feb 9, 2012 at 9:20
  • in which case, cygwin might be a good option since it has scp, and a cli ssh client for easier scripting
    – Journeyman Geek
    Feb 9, 2012 at 15:11
  • bounty up for grabs if you can provide more detail and a recommendation.
    – toop
    Mar 11, 2012 at 8:10
  • Answered, though not as tested as i'd like with respect to pagent.
    – Journeyman Geek
    Mar 11, 2012 at 10:22
1

Since you have the thing available over SMB, I would handle this from the Unix side. That means I would set up a cron job (probably using an non-root user account) which does the transfer using either

  • rsnapshot -- which is a good backup utility (and comes with instructons about how to set up cron)
  • rsync -- if your problem is more file-synch rather than backup.
1

You could share the folder and file on windows. Install Samba on linux mount the folder and run a cron job on linux to transfer the files where needed

1

If they're text/script files, I would go for a versioning system, like SVN, GIT or Mercurial. This way the server won't be too loaded (only differences between files are kept) and you can easily check differences between each sync operation, and even know who/when changed what file and what was changed. You can also add comments on each sync operation, which is definitely very useful.

2
  • How would you transfer the files between svn and a specified directory on the unix server?
    – toop
    Mar 13, 2012 at 10:31
  • The SVN server would be on the UNIX server, there you can specify any directory you want it to keep the files in. On the SVN client side, you just do a SVN commit on that SVN repository and all changes (and only the changes) are updated to the server, which make upload faster and with less bandwith consumption (and with optional comments as well). It's just one line using SVN command line client, which can be programmed as a Windows task to do the commit anytime you want automatically.
    – m0skit0
    Mar 13, 2012 at 13:29
1

In my opinion rsync is the only way to go - what you are asking to do is synchronise a directory on one filesystem to another and the tool to do this in the most reliable way is rsync. To try to justify this further I think it's good to look at the history of the rsync project - it was created in 1996, continuously developed and used widely throughout the unix community. It's last release was in 2008 - that means for the last 4 years it has been considered to do a near-perfect job of synchronising two directories. So the question is how to set this up.

I figured this must be quite a common thing to do so had a quick google and uncovered a very good post on stack overflow - bash backup script for samba shares using rsync. I'd already got in mind the steps necessary (which I'll outline below) but the script goes above and beyond by creating log files, multiple shares, mounting unmounted filesystems etc; it really is a great script. All you'll need to do is modify it to contain the directories/ip address for windows share and the destination for the files and logs.

The only remaining thing to do is to schedule the script to run at 9pm. This can be done with cron or crontab - Read cron or crontab tutorial if you're unsure how to do this.

2
  • Do I need root access to install rsync or does it come packaged in some linux distros?
    – toop
    Mar 15, 2012 at 9:59
  • rsync is often installed with standard installations of both desktop and server releases. If you chose a minimal install of the distribution rsync is far from necessity so it's likely not to be installed.
    – James
    Mar 15, 2012 at 15:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .