whole volume private Dropbox with Windows 7 [closed]

Question

I want to setup full volume file mirroring from a client computer to a server computer. In my imagination the end result is something slightly comparable to both RAID1 and dropbox.

I have listed a lot of constraints, so probably there isn't a perfect solution. Let's just see what gets closest. My reasons are both academic and practical.

Context:

• a modern workstation computer with 5.0TB of storage (client)
• older workstation computer with 5.5TB of storage (HTPC/server)
• both running Windows 7
• 1Gbit LAN
• cheap consumer disks

Purpose:

• redundancy against mechanical disk failure
• data availability from both systems even if the other is offline

Constraints:

• can handle non-system volumes
• at least one way synchronization (client -> server mirroring)
• near real-time, like dropbox
• good utilization of available network bandwidth
• can handle large number of small files, e.g. full Cygwin installation with 365k files and 21k folders
• can handle large files up to tens of gigabytes (hd video)
• files stored on plain NTFS (no images or zips)
• no versioning by default

Bonus objectives:

• can handle the system volume
• two way synchronization
• open source
• integrated solution
• filesystem level solution (not block level)
• manual versioning of special locations for backups (e.g. only C drive)
• can handle open files (using shadow copy?? not 100% familiar with what it can do)
• support for commercial cloud storage
• encryption
• updates based on file change (vs. periodic backups)

Okay, so I'm asking for a lot. ;)

Do you think anything comes even close?

The closest thing I can think of is rsync running on a continuous loop. I'm hoping for something "more elegant" though.

Answer 1

You are not really asking for too much. You are asking for a disk-to-disk backup/sync tool.

I guess that the remaining question is whether the two computers are on the same local area network or do you need to traverse a WAN (e.g. Internet)?

If they are on the same LAN, there are dozens of tools that will do most of what you need. Although the optional requirement to sync the system drive is a little more tricky if you want to be able to restore directly from the backup.

You should always be looking for a couple of key features:

• Shadow Copy support. Without this, any files that are still open when the backup runs will not be backed up
• Differential file copy. This allows the system to only backup the changes to files but you still get the full file on both ends
• File change watching. This gives you the near real-time backup without massive overheads on the system. It uses the filing system change monitoring capabilities of Windows/NTFS to notify the application that it needs to do something.

Here are a couple of examples

• SyncBackSE - not free but I've used this for years. There is a free version but it doesn't quite do all of your requirements. There is a pro version too with additional features. SyncBack is really powerful.
• Allway Sync - Not tried this one but it looks OK
• Paragon Backup & Recovery - this one DOES do full drive backups, free for personal use
• Bacula - Open source but I think you need to purchase the Windows version to do what you want

Most of these will also work over a WAN connection but there are more issues to consider such as transmission security, bandwidth and so on.

One last warning: A file sync tool is NOT a backup! Backups are protected from change, your proposal would leave you with a COPY. Useful but not protected necessarily. Also, backups should be in at least 2 locations and preferably on two different media (though that is less common now with the reliability and size of hard disks). Location is key. If the sync/backup is in a similar location to the original, a disaster striking one will also strike the other.

Note that if you want a good backup strategy, you could do a lot worse than to use CrashPlan with one of their cloud plans. The basic CrashPlan tool (free) allows you to backup to another computer you own - even over the Internet (e.g. friends/family). Full encrypted, differential copies, deduplication and other good features. Adding the cloud facility allows that all important 2nd location if you cant use family/friends etc. I keep a local backup for quick restores and a cloud backup for security.