0

I have inherited a Transfer job at work that is causing me some issues. We are using RoboCopy to transfer files from location to another location. The script does copy the files fine but is set to scan the source folder 15 times for files. The source location has approx. 200 GB of files and Folders. Daily there may be 1 GB of changes that need to be transferred.

The task never seems to compete. Looking at the job log I can see that is still scanning folder for changes. I have changed the amount of retried from 15 to 5 to see if this would change but faced with the same issue.

My question based on this is Scanning 200 GB of files with Robocopy. Does it scan each directory for new changes or can it be set to only scan time stamp changed folders ?

  • 1
    Perhaps you could post a copy of the RoboCopy batch, and let people see it for a better analysis. – the original mike western Jan 21 '16 at 9:53
0

I recommend you try the Robocopy GUI utility to reconfigure the schedule to make sure there is no configuration issue: https://technet.microsoft.com/en-us/magazine/2009.04.utilityspotlight.aspx

To know how does it work, check this blog: https://technet.microsoft.com/en-us/magazine/ee851678.aspx

0

Yes, robocopy will traverse the whole tree and it will enter and scan directories even if the "last modified" timestamp on both source and destination counterparts is the same. It has some timestamp-based copying logic, but it's only for files.

Given that you said "location to location", your bottleneck is almost certainly not with the scan per se, but with the traversal of the remote tree. What you may want to do is to look at robocopy alternatives that can either cache the contents of the destination tree between the runs or that work similar to rsync, whereby there are two cooperating instances running at both ends of the connection.

For the former option I think SuperFlexibleFileSynchronizer used to have this feature and I know that Bvckup 2 has it. And for the latter, perhaps you can just try and use rsync itself?

0

With robocopy /???, you get an explanation of all options.

My backup script is using the following parameters which might be helpful for you:

 /FFT /E /R:5 /W:30 /XO
  • /FFT takes timestamps with 2s granularity
  • /E copy subdirectories
  • /R:5 maximum of 5 retries
  • /W:30 wait 30s between retries
  • /XO only copy newer files

It might help to suspend your Anti-Virus software before starting robocopy. This can have quite an effect on the overall performance.

  • In your example woult that be wait (An in Pause) for 30s the whole procsess then retry ? – Mr Os Jan 21 '16 at 11:35
  • The retry happens, if robocopy cannot access source or destination. This might be due to networking or device access problems. By delimiting the number of retries, you can prevent endless loops. The retry wait time is 30s by default. Depending on your environment, other retry parameters might make more sense. The retry does not repeat the whole process but only the current file copy operation. – Axel Kemper Jan 21 '16 at 11:51

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.