25

I've read (somewhere) that Robocopy skips copying over existing files by default.

But... I can't find any Command Line switches for that.

1
  • 1
    If the program behaves a certain why by default, this means that no switches are required to invoke that particular behavior.
    – Lumberjack
    Dec 11 '19 at 19:17
20

By default, Robocopy skips copying existing files if the specific metadata of the files match then those files will be skipped from the "file" copy operation (/COPY:DAT).

Robocopy Defaults

  • Skips file copy if last write time, file name, and file size match
  • Copies file if last write time, file name, or file sizes don't match

As indicated by @mklement0, the implied default /COPY:DAT will not copy over files with different data if the timestamp [LastWriteTime] and the file size is the same and thus this is a default skip.

So if for some reason you have two files you are syncing that have matching file size, file name, and last modified attributes even though the data is different, it will not copy the source file.

Robocopy Default Options: /COPY:DAT /R:1000000 /W:30

As confirmed by log file or command window running robocopy c:\source c:\dest *


Robocopy or Robocopy /?

 /COPY:copyflag[s] : What to COPY (default is /COPY:DAT)
                      (copyflags : D=Data, A=Attributes, T=Timestamps
                       S=Security=NTFS ACLs, O=Owner info, U=aUditing info).
6
  • From what I can tell, the skip-if-same logic is unrelated to the (implied) /COPY argument. Specifically, only a file's size and its last-modified time stamp ever seem to matter.
    – mklement0
    Aug 8 '18 at 22:25
  • Yes: 2 files that have the same last-modified timestamp and the same file size are considered identical, even if their content differs (an exotic scenario, to be clear). Whatever you pass to /COPY doesn't matter in terms of determining sameness - it only applies to what will be copied if the destination file has been deemed to be different.
    – mklement0
    Aug 8 '18 at 22:40
  • @mklement0 - Let me ask you this.... if two files are the same size and the same date last modified to the second or microsecond or whatever, what would you recommend the taken action be based on that condition? Should this be "a case by case basis" or should "the default copy to destination"? Aug 8 '18 at 22:50
  • 1
    @mklement0 Okay, so I tested and I confirmed that you are correct the same $file.FullName.ForEach{(Get-Item $_).LastWriteTime=$(Get-Date)} for files that are exactly the "same size" and "same file name" with one having aaaaa2 and the other having aaaaa5 content are not copied. Using no options with the Robocopy command the implied default of /COPY:DAT which is accurate does not copy files when the data is different when the timestamp and file size is the same. The default is still /COPY:DAT as the documentation states and the output results of the command shows regardless. Aug 8 '18 at 23:40
  • 1
    Let me try to phrase it the opposite way: it doesn't matter what you pass to /COPY or whether you use the default (DAT): The only pieces of metadata that are invariably considered are the last-modified time stamp and the file size. To give a specific example: the default /COPY value includes file attributes, yet Robocopy doesn't consult attributes to determine sameness. I believe my answer to tell the full story, and I invite you to vet it.
    – mklement0
    Aug 9 '18 at 2:39
10
robocopy SOURCE DESTINATION FILE(S) /IS

Where IS stands for Include Same file(s). Using this switch causes to overwrite existing files. See below:

::
:: File Selection Options :
::
                 /A :: copy only files with the Archive attribute set.
                 /M :: copy only files with the Archive attribute and reset it.
    /IA:[RASHCNETO] :: Include only files with any of the given Attributes set.
    /XA:[RASHCNETO] :: eXclude files with any of the given Attributes set.

 /XF file [file]... :: eXclude Files matching given names/paths/wildcards.
 /XD dirs [dirs]... :: eXclude Directories matching given names/paths.

                /XC :: eXclude Changed files.
                /XN :: eXclude Newer files.
                /XO :: eXclude Older files.
                /XX :: eXclude eXtra files and directories.
                /XL :: eXclude Lonely files and directories.
                /IS :: Include Same files.
                /IT :: Include Tweaked files.
0
8
  • Unfortunately, the official documentation doesn't describe the logic behind what files are skipped by default.

  • The unofficial ss64.com documentation, however, provides the crucial pointer (emphasis added):

    • By default Robocopy will only copy a file if the source and destination have different time stamps or different file sizes.

Note: time stamp refers to the last-modified time stamp (only).

In other words: Robocopy considers two files to be the same based only on whether their last-modified time stamps and file sizes are identical and therefore skips copying in that event.

From what I can tell:

  • This behavior is not affected by modifying what aspects of a file / directory to copy (parameters /copy / /dcopy)

    • These arguments are only applied if RoboCopy has fundamentally deemed a given file / directory in need of copying, based on sameness / inclusion logic.
  • If you want to extend the default sameness detection (same last-modified timestamp and same file size) to the following properties, use the /it (Include Tweaked) option:

    • file attributes (represented as A for the /copy parameter)
    • ACL (represented as S for the /copy parameter)
    • file ownership (represented as O for the /copy parameter)
    • auditing information (represented as U for the /copy parameter)
  • Robocopy seems offers no option to detect sameness of files based on their content (which is typically implemented via a cryptographic hash function).
    Therefore, in the rare event that there could be files with identical last-modified time stamps and identical file sizes that nonetheless differ in content, your only option is to use the /is (Include Same) option:

    • This causes the files that are considered the same to be copied unconditionally - and therefore potentially unnecessarily.

    • Caveat: Curiously, files that are the same with respect to last-modified file stamp and file size but differ with respect to the /it-related properties mentioned above are not included by default with /is alone; therefore, to cover all eventualities, use /is and /it combined.


The following Pester tests demonstrate the behaviors; save the code as Tests.ps1 and invoke it as Invoke-Pester ./Tests.ps1:

Describe RoboCopySkippedFilesTests {
  BeforeAll {
    Push-Location TestDrive:
  }
  AfterAll {
    Pop-Location
  }
  BeforeEach {
    # Set up a source and a destination folder and make their
    # Content the same.
    if (Test-Path dest) { Remove-Item -Force -Recurse dest }
    $null = New-Item -Type Directory -Force src
    'file1' > src\file1
    'file2' > src\file2
    Copy-Item -Recurse src dest
  }

  It "Does not copy anything with identical folders." {
    robocopy.exe src dest
    $LASTEXITCODE | Should Be 0
  }
  It "Does recognize a changed last-modified filestamp" {
    (gi src\file1).LastWriteTime = [datetime]::now
    robocopy.exe src dest # | Out-Host
    $LASTEXITCODE | Should Be 1
  }
  It "Does recognize a change in file size" {
    '!' | Add-Content dest\file1
    robocopy.exe src dest # | Out-Host
    $LASTEXITCODE | Should Be 1
  }
  It "Does not recognize a change in file content, with size and last-modified date identical" {
    'file!' > dest\file1
    (Get-Item dest\file1).LastWriteTime = (Get-Item src\file1).LastWriteTime
    robocopy.exe src dest # | Out-Host
    $LASTEXITCODE | Should Be 0
  }
  It "Does not recognize a change in file attributes, with size and last-modified date identical" {
    (Get-Item dest\file1).Attributes = 'System'
    robocopy.exe src dest # | Out-Host
    $LASTEXITCODE | Should Be 0
  }
  It "Does recognize an attribute-modified-only file as tweaked (/IT)" {
    (Get-Item dest\file1).Attributes = 'System'
    robocopy.exe src dest /IT # | Out-Host
    $LASTEXITCODE | Should Be 1
  }
  It "Does not include an attribute-modified-only file with /IS" {
    Remove-Item src\file2, dest\file2
    (Get-Item dest\file1).Attributes = 'System'
    robocopy.exe src dest /IS # | Out-Host
    $LASTEXITCODE | Should Be 0
  }
}

You should see something like the following, indicating that all tests passed (as of Robocopy.exe with file version 10.0.16299.15 (WinBuild.160101.0800)):

Describing RoboCopySkippedFilesTests
 [+] Does not copy anything with identical folders. 231ms
 [+] Does recognize a changed last-modified filestamp 112ms
 [+] Does recognize a change in file size 68ms
 [+] Does not recognize a change in file content, with size and last-modified date identical 69ms
 [+] Does not recognize a change in file attributes, with size and last-modified date identical 83ms
 [+] Does recognize an attribute-modified-only file as tweaked (/IT) 65ms
 [+] Does not include an attribute-modified-only file with /IS 70ms
Tests completed in 589ms
Passed: 7 Failed: 0 Skipped: 0 Pending: 0 Inconclusive: 0
1
  • 1
    Great info. I guess the /is argument not comparing files before copying makes sense; the overhead of comparing the 2 files' content would likely be as time consuming as copying the file (except in cases where write IO is significantly slower than the network latency between source and destination).
    – JohnLBevan
    Jul 20 '21 at 9:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.