2

I am evaluating if Robocopy will be a good option for me for my backup script that will take backup of only newer files from C drive (NTFS) to a Pendrive (exFAT).

I have this command that I am running. It does the job but seems there's incorrect logging when the destination is a removable USB pen drive and that too exFAT formatted. This issue does not occur if destination is FAT or NTFS.

robocopy C:\Temp\F1 D:\F1 /XO /E /FFT /LOG:C:\Temp\robo.txt /NP  /NDL /R:1 /W:3

In the above command D: was the pendrive letter and the command or .BAT file was always run as administrator on Windows 7 Ultimate 64.

The issue appears with Case 2 as described below.

Case 1 - See the log screenshot. This appears correct. All copied file names are logged and copy stats are correct. 3 Files are copied.

enter image description here

Case 2 - I add one more file in source. Now it only copies this new file but shows all files in log and stats are wrong. It says 4 files copied.

enter image description here

This type of inconsistent logging happens only when the destination is exFAT formatted pendrive. No issues with FAT or NTFS.

OS - Windows 7 Ultimate 64.

Questions.

  1. Is this some issue or bug in Robocopy logging when destination is exFAT pen drive?
  2. If not, am I missing any option in the command that should fix this?

Would greatly appreciate further clarity on this.


Edit

Case 3 - No change, still it list all 4 files in log file.

enter image description here

/FFT or absence thereof does not change the log data.

I checked using Free File Sync, both directories are in Sync when it comes to file size, timestamp and actual contents. I believe it's not copying but still logging.

enter image description here


Edit 2

I placed 2 large files in source together 312 MB. It takes 42 seconds to copy then to USB 2 pendrive destination. Log is fine.

enter image description here

Now I run the command again. It finishes in 0 seconds but still logs 2 files and stats show 2 files copied. I am certain this is not possible with 312MB data on USB 2.0 pendrive.

enter image description here

14
  • 1
    how does the log look like after the third try without additional files?
    – Albin
    Oct 17, 2019 at 18:04
  • 1
    How sure are you that it "is only copying this one file"? The log may be copying all of them (and logging) but only flagging the one as "new." in which case the logging is fine. This would jibe with @albin suggesting a timestamp (i.e. decision process) problem.
    – Yorik
    Oct 17, 2019 at 18:14
  • @Yorik good point, I assumed rajeev checked that through a third verification, but I fear that's just my mind playing tricks on itself ;)
    – Albin
    Oct 17, 2019 at 18:17
  • I have edited the question with further information and try-outs. Thanks.
    – rajeev
    Oct 17, 2019 at 18:25
  • 1
    @MrEthernet my bad, you are absolutely right, only /e is called seems I got confused. Yeah I don't expect /copy:dat to make any difference as well, but you never know how it's implemented and if there's some strange error with the default value... after all it's M$ ... ;) just a shot in the dark
    – Albin
    Oct 17, 2019 at 22:20

2 Answers 2

2

Robocopy version on my Windows 7 is 6.1.7601.23403.

That version of Robocopy is from 2009. It's 10 years out of date.

I tried copying Robocopy from Windows 10 (64) PC on to my Windows 7 (64) but when command is put in .BAT, it errors out stating that it’s not a valid Win32 application

Unfortunately, Windows 7 lacks certain prerequisites required by the current Robocopy executable, so the latest executable cannot simply be copied across from a Windows 10 system:

Even copying from Windows 8 is not going to work as underlying components have to support it.

Robocopy is just a utility that will call into underlying file system components.

enter image description here

I was unable to recreate this issue on my Windows 10 1903 system, which has the latest version of Robocopy.

There's no question that the problem is with the logs rather than with the copy process itself. Robocopy actually does exactly what it's supposed to here, it just incorrectly reports it.

The instantaneous copies we're seeing here are impossible. If copying a file from one volume to another takes 42 seconds the first time around, the same process can't take 0 seconds the second time!

Whatever bottlenecks limited the speed of the initial file copy would affect subsequent copies in exactly the same way (i.e. USB bandwidth and flash drive write speed).

This can be easily demonstrated by performing a copy job that includes a relatively large file and observing how long it takes, then deleting the large file from the destination drive and re-running the same job. Subsequent copies across two different volumes will take approximately the same time.

Log discrepancies:

  • Green = true.

  • Red = false.

enter image description here

Just to clarify what's shown in the first green box: I added the "No 100% New File" lines to show where the log was correct to show blank space. Had these file copies really taken place, then "100%" and "New File" would have been shown beside each successfully copied file.

These file copies never happened. The OP could put 20 GB of data in there and Robocopy would still report an instantaneous transfer!

Conclusion:

The OP will not be able to upgrade his version of Robocopy as Windows 7 cannot use anything newer than the 2009 version.

His immediate options are to either use XCOPY or some other file-copy utility.

When the OP eventually upgrades to a newer version of Windows, such as Windows 10, he will have a recent version of Robocopy where this old bug has been patched and this logging glitch will no longer occur.

7
  • 1
    Wow.... This whole time I totally overlooked OP was about Windows 7. No wonder I cannot reproduce. LOL Oct 17, 2019 at 22:32
  • I never miss a clue! 😂 Oct 17, 2019 at 22:37
  • WTH.... For some reason, my Windows 10 home 1903 version on this upstairs home PC is Version XP010showing as if it's from the 2003 resource kit per wikipedia or whatever. I checked another Win 10 1903 on a downstairs PC and it did not show this from command line robocopy /?. So apparently I tested and could not reproduce on this version. Oct 17, 2019 at 22:49
  • 1
    Roboopy version on my Windows 7 is 6.1.7601.23403. I tried copying Robocopy from Windows 10 (64) PC on to my Windows 7 (64) but when command is put in .BAT, it errors out stating that it’s not a valid Win32 application. It seems such copying may not help. See here stackoverflow.com/questions/32896279/… For now, I am checking a free alternative called XXCOPY ,unfortunately discontinued, but logging looks good. It’s correctly logging only the copied file list to the log file which suffices my requirement.
    – rajeev
    Oct 18, 2019 at 23:38
  • 1
    Thank you all for all your help. I have already upvoted both answers for the clarity that they bring to this issue.
    – rajeev
    Oct 18, 2019 at 23:44
1

Robocopy is known to cause issues when transfering files between different files systems e.g. due to the fact NTFS is a 64 bit time stamp, ex fat uses 3 separate fields to store the time stamp with one byte being the time zone for UTC time.

And there are serveral examples where the summary does not display the correct information eg here. I would guess the summary calculation is (loosly speanking) not directly integrated into the copying procedure, so there's some kind of bug. But I didn't find any "official" document verifying this. You might also wanna verfiy which one is actually correct, the log or the summary.

3
  • he did - it's in the log files as an option
    – mael'
    Oct 17, 2019 at 18:04
  • "I would guess the summary calculation is (loosely speaking) not directly integrated into the copying procedure." That's a good way of describing how this calculation error was able to creep in. Fortunately, this obscure bug seems to be confined to certain older Windows versions of Robocopy. Oct 19, 2019 at 3:02
  • interessting, yeah I assumed updating woud have been the first thing he would have tried... it's always good to ask. Are the current version still compatible with Windows 7?
    – Albin
    Oct 19, 2019 at 8:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .