27

I am overwhelmed by the ROBOCOPY documentation. I want to do an incremental backup of my local files to the network drive (M). I want it to run as quickly as possible, with no log file and with as little text as possible to the screen. My files are all somewhere inside one folder (MyFolder) that has many nested subfolders. By "incremental" I mean "only copy what is new or changed." I don't want to delete any historical files in the destination, but if I've changed a file I only want the newer version. And if I have not changed a file, then I don't want to copy it over the existing backed-up file. Is below correct? (Running Win7 Enterprise.)

robocopy C:\MyFolder M:\MyFolder /z /np /xo /e

Please, no suggestions for anything but robocopy. I'm not allowed to install anything. And I don't care about security stuff. I have people for that (whether I want them or not). ;-)

  • 1
    You need a /s or /e option to copy subdirectories. – David Marshall Sep 19 '14 at 21:31
  • @DavidMarshall: oops. I actually knew that. Added. So, am I good now? – dmm Sep 19 '14 at 21:51
  • Yes. I've been using something similar. – David Marshall Sep 19 '14 at 21:59
  • Testing it now. There is still a bunch of stuff being written to the screen. How do I get rid of it, in future runs? – dmm Sep 19 '14 at 22:19
  • 2
    One option that can make a big difference performance wise is to have multithreaded transfers. /MT[:N] I have a 4 core CPU so I typically use something in the 8-16 rage for my number of threads. I figure 2-4 threads per core. You should certainly test with different values on your hardware. Multithreaded helps the most when you have a lot of small files. If you have a small number of large files it won't help nearly as much. – Zoredache Sep 19 '14 at 23:10
36

I have been successfully using a variant of the following script for a few years now:

robocopy C:\source M:\destination /MIR /FFT /R:3 /W:10 /Z /NP /NDL

Parameters explained

  • The /MIR option (equivalent to /E /PURGE) stands for "mirror" and is the most important option. It regards your source folder as the "master", causing it to overwrite any changes on the target side, which is a useful setting for a backup.

  • /FFT is a very important option, as it allows a 2-second difference when comparing timestamps of files, such that minor clock differences between your computer and your backup device don't matter. This will ensure that only modified files are copied over, even if file modification times are not exactly synchronized.

  • /R:3 specifies the number of retries, if the connection should fail, and /W:10 specifies a wait time of 10 seconds between retries. These are useful options when doing the backup over a network.

  • /Z copies files in "restart mode", so partially copied files can be continued after an interruption.

  • /NP and /NDL suppress some debug output, you can additionally add /NS, /NC, /NFL to further reduce the amount of output (see the documentation for details). However, I would suggest to print some debug output during the first runs, to make sure everything is working as expected.

Additional useful parameters mentioned by other users

  • /XJD excludes "junction points" for directories, symbolic links that might cause problems like infinite loops during backup. See Brian's comments for details.

  • /MT[:N] uses multithreading and can speed up transfers of many small files. For N, a value of 2-4 times the number of cores should do on a normal machine. Commented by Zoredache on the original question.

Edit in response to Granger's comment:

If you really want to keep files that exist on the destination, but not on the source side, simply replace the /MIR option with /E. However, I would strongly suggest to use /MIR when you want to use the destination for incremental backups. Otherwise any files that have been renamed or moved at the source will clutter up the destination, meaning you get duplicates. I usually create a subfolder "backup" on the destination which contains a 1:1 copy of my source folder tree. That way you can still keep around historical files next to the backup folder and remove or reorganize them safely later on.

  • Does this command allow to copy files with permissions? robocopy C:\source M:\destination /MIR /FFT /R:3 /W:10 /Z /NP /NDL – Shabeer Sher Jan 8 '15 at 15:34
  • 3
    Except /MIR (/PURGE) will delete files at the destination which no longer exist at the source. The question asked explicitly stated that was not wanted. – Granger Jan 20 '15 at 17:57
  • @Shabeer No, Robocopy by default only copies data (D), attributes (A) and timestamps (T). Other values have to be specified with the "/copy:<value>" parameter, e.g. "/copy:DATS" for including ACL permissions. But since I copy to a Linux-based NAS, that doesn't make much sense for me. – pederpansen Jan 21 '15 at 11:43
  • 1
    @Brian Thanks for the hint. I've never had problems with junction points, but I guess it won't hurt to avoid them just in case. Added it in my answer. – pederpansen Apr 26 '16 at 17:07
  • 1
    @pederpansen: It tends to happen when copying user folders (or copying an entire drive). Usually the application data folder is the culprit, as it has a junction point for application compatibility reasons. Try running "cd C:\Users\[USERFOLDER]\AppData\Local\Application Data\application data\application data\" from a command prompt if you want to see a sample infinite path. This works even on Windows 10. – Brian Apr 26 '16 at 17:50
8

I like to use the following:

robocopy "C:\Users\<user>" "F:\robocopy\<user>" /XJD /R:0 /XA:SH /E /ZB /XO /XD "Downloads" "AppData" /LOG:robocopy.log /TEE

I run this as Administrator so backup mode (/b option) can make copy of files in use.

Other options not related to incremental nature of backup are:

/XD to exclude directories from backup.

/XJD to exclude junction points ("My Music").

/R:0 to set retry on failed attempts to 0.

/XA:SH to skip hidden and system files.

To remove as much logging info as possible, append the following options /NP /NS /NDL /NFL /NC.

protected by Community Jan 19 '16 at 19:17

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.