Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a 2TB internal HDD, with a single NTFS(Windows 7) partition used for storage purposes. I want to backup/mirror/sync it to an external 2TB HDD.

  1. From what I know that mirroring can be done only if the external HDD is connected all the time to the PC. If this is the case, I will exclude this option.

  2. Again, regarding backups, I know that they usually use a proprietary container file that makes an image of the drive. This would also pose some issues when updating the backup. If it's incremental(lack of space, due to the fact that you backup 2TB on 2TB, and there's no spare space for incremental containers), if you can access the data easily in case you need to, etc.

  3. Syncing, this seems for the moment the best option for me(ease of access in case of emergencies, less time involved, since it only copies the modified data, etc), but I have some doubts here also. How does the software decide what needs to be sync-ed and what doesn't need to?Does it start a CRC32 or byte comparison(this would take huge amounts of time for 2TB drives)?

Does the software have some options to verify that the sync-ed files are consistent with the source?

If I were to choose by myself, I'd go with syncing, but I hope that this isn't the case, and that you will share your experience and advice with me.

How do you backup such large amounts of data? What strategies and software do you use?

share|improve this question

1 Answer 1

For home / small office use, your backup strategy should probably not involve backing up the entire operating system. Data can loosely be categorized into three rough categories:

  • Data which is completely irreplaceable; that is, if you lose the copy on your HDD, it is gone forever and there is no easy way to get it back. Examples include documents you have written, pictures you've drawn or taken with a camera, music that isn't backed up elsewhere, proprietary program files you downloaded online which are no longer available, and so on. This data tends to take up a lot of space. These files tend to reside within your user folder, e.g. C:\Users\, 99% of the time.

  • Data which is replaceable with some annoyance. For example: device drivers, application configuration settings, bookmarks, installed application program files (for applications that you can easily re-download and re-install), and so on. If you lose this data, it's not gone forever, but it can involve long downloads and a lot of hunting around the internet to find the right download to re-obtain the data. This category also involves data that requires some degree of expertise to create, such as hand-edited .ini or .xml configuration files for certain programs, and so on. These files tend to reside either within hidden folders (AppData) in your user folder, or they reside outside your user folder entirely.

  • Data which is trivially replaceable with little effort. For example: the Windows operating system files, service packs, freeware and open source programs, and so on. If you lose this data, you just shrug and install Windows again. These files tend to reside in C:\Windows, C:\Program Files, or other paths outside of your user folder.

Now: the purpose of doing a FULL DISK backup (every block on your entire hard disk) is to be able to quickly get running again in case of a disk failure. This is critical in enterprise and server environments where any downtime results in a huge loss of customers. But of course, doing full disk backups comes with increased costs in terms of storage, backup time, and the amount of effort required to devote to the task.

A full disk backup will cover all three types of data because it backs up the entire disk. If you are into full disk backup, one possible way to improve your operating system's resiliency is to run two hard drives in RAID-1 (mirrored) mode. Make sure the hard drives were not produced at the same factory at the same time, as defective hard drives are often produced in a series together, and they will tend to fail almost simultaneously, causing you to lose all of your data. But the disks should be the same capacity.

If you want to go that way, then all you have to do is decide on an appropriate RAID-1 solution, whether it's dedicated hardware, fake (BIOS) RAID, or software RAID. But many people will tell you that RAID-1 is not equivalent to a proper backup, because it's not off-site, and any damage (e.g. flood, fire) or theft of the physical computer involving both disks will result in full data loss.

A true resilient backup for "completely irreplaceable" data would consist of something like the following:

  • Automated, nightly backups (or more frequent if the data changes significantly more often than that)
  • Backups are transmitted to multiple remote locations, e.g. in the cloud (for example, some cloud services keep 9 backups of every file in multiple datacenters on multiple continents, and they keep backups of backups around for a while in case you accidentally delete something)
  • Frequent pruning of the backups to delete data that is truly no longer needed... if you can look at the data and say "there's no way I would ever want/need it", delete it to save space and reduce complexity.
  • Backups that capture all of your completely irreplaceable data, and some subset of your replaceable-with-annoyance data -- the kind of data you want to replace in the latter category is something like, if you had to search the internet for hours (or ask SuperUser) for help writing an email server configuration file, you should probably include that in the backup.
  • Backups do not capture trivially-replaceable data. It is wasteful and can result in exhaustion of your backup resource or paying more than needed.

Your backup strategy has to principally involve an analysis and understanding of

  • Exactly which files are on your computer
  • Where those files came from
  • How hard it would be to replace them (impossible, annoying, or trivial)
  • Patterns in where different files of different replacement-difficulties reside; e.g. if irreplaceable files tend to reside in C:\Users\You\Documents, then you would always want to backup that directory
  • Maintenance of a list of directories/files to backup, and a list of directories/files to NOT backup in case there are false positives; for example you may store gigabytes of easily-replaceable data in C:\Users\You\Documents\dev (for software compilation from source code, for instance) and you'd want to exclude that.
  • How much money / bandwidth / time you're willing to invest in getting a proper backup solution that meets criteria that are stringent enough for you to feel that your data is secure

I can't list specific services or options or solutions here other than the theoretical, due to SU rules, but if you have a specific question about a specific product or service you're trying to use, you may be able to ask that here if it's within the scope of the site.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.