How to sync large, disconnected file sets with a small USB flash drive?

Question

I want to keep a large file set (700 GB) synchronized between two computers at two different locations (A and B). The computers are not networked in any way. The file sets will be synchronized initially at one location.

Then I want to be able to work on various files in location A, and transfer the gist of the file system changes (files added, changed, deleted, renamed or moved) to a small, 8 or 16 GB USB flash drive. Then plug the flash drive into the computer at location B, and have those changes applied there prior to commencing additional work. Then repeat the same process at location B back to location A.

Obviously, the system will be limited by the size of the flash drive, which in my case should be very infrequent. The goal is to keep disconnected file sets synchronized without having to carry the entire 700 GB back-and-forth between locations or rely on a network connection. These are Windows computers. Any ideas of how to accomplish this?

Answer 1

This is a slightly odd idea, but it would work. So I'm throwing it out there: Use a version control system and export patches to USB. This would probably depend on your VCS being able to produce reasonably efficient binary patches though.

Also, why use such a small USB stick? It's possible to buy 750 GB and 1 TB external 2.5" hard drives. That would let you sync the entire dataset.

Answer 2

Create a set of directories on the USB that correspond to the hard-disk directories to be synced, then use one of the zillion free directory-sync products.

My favorite is SyncBack Freeware which for a sync job has the option of "Ignore file that have not been modified/created within the last [N] days".

Set N to a few days back, and create a batch file to clean up the USB once it fills up.

Answer 3

you can use something like:

• format media_drive: /q

• xcopy A_machine_data_path media_path /m /v /e /r /h

• move disk to machine B and copy media to B_machine

• format media

• copy B to media

• copy media to A

note /m will copy only altered files and reset their A- attribute, so first time you should use big media to transfer data (first time will be copied entire data set)

you can use del, xdel commands to erase media instead of formatting

Answer 4

I'm relatively new to all this, but I'll give it a shot.

I have written several batch files that "sync" (mostly copy) different hardware I have. It recognizes the serial numbers for my external hard-drive, my flash drives, and my mini-SD memory card.

With my mini-SD card, since it is in my phone, I copy the files both ways, but since I don't want to constantly fill up the tiny memory card, I wrote in the batch file to only synchronize files after the last date the file was run. This enables me to delete files on either side, but still use the same folders as the sources and destinations.

Basically, the batch file reads a txt file with the last date the batch was run. Then the batch file copies files modified on or after that date, based off of %Last_run_date%, and then calls another batch file to update the %Last_run_date%.

---

@echo off 
set /p Last_run_date=<"C:\Users\Owner\Desktop\Stuff I don't use\Shortcuts\Batch Files\lastrun.txt"
@echo.
@echo.
@echo.
@echo Last Update %Last_run_date%
@echo.
@echo Sync card to folder!
xcopy "F:\" "C:\Users\Owner\Desktop\Random\Phone Sync Folder" /s /y /h /r /e /c /i /d:%Last_run_date%


@echo.
@echo.
@echo.
@echo Sync folder to card!
echo     Videos
xcopy "C:\Users\Owner\Desktop\Random\Phone Sync Folder\Video" "F:\Video" /s /y /h /r /d:%Last_run_date% /e /c /i
echo     Audio
xcopy "C:\Users\Owner\Desktop\Random\Phone Sync Folder\Audio" "F:\Audio" /s /y /h /r /d:%Last_run_date% /e /c /i
echo     Pictures
xcopy "C:\Users\Owner\Desktop\Random\Phone Sync Folder\Picture" "F:\Picture" /s /y /h /r /d:%Last_run_date% /e /c /i
@echo.
@echo Update LastRun.txt
"C:\Users\Owner\Desktop\Stuff I don't use\Shortcuts\Batch Files\lastrun.bat - Shortcut.lnk"
@echo off 
set /p Last_run_date=<"C:\Users\Owner\Desktop\Stuff I don't use\Shortcuts\Batch Files\lastrun.txt"
echo.
echo Current Update %Last_run_date%

---

Here's the lastrun.bat.

---

@echo off
set DD=%date:~7,2%
set MM=%date:~4,2%
set YY=%date:~10,4%
echo %MM%-%DD%-%YY%>"C:\Users\Owner\Desktop\Stuff I don't use\Shortcuts\Batch Files\lastrun.txt"

---

Finally, I also have another batch file that synchronizes one of my flash drives that I use for school. It deletes and remakes the print folder directory so that I don't constantly fill up my flash drive

---

echo Empty Print Folder (F)
rmdir /s /q "F:\Stuff that needs printed"
mkdir "F:\Stuff that needs printed"
echo Sync to F Drive
xcopy "C:\Users\Owner\Desktop\Random\4GB Flashdrive" "F:" /s /y /h /r /d /e /c /i

---

Also, little side note, if any of this looks familiar, some of the pieces of code I found via Google and then I re-arranged to suit my needs.

Answer 5

Microsoft has a pretty simple, free program called Synctoy which will keep files and folders up-to-date as you have described.

You can find this program on the Microsoft Downloads Site and try it out.

I use this one quite a bit to keep my folders of pictures