0

When I start copying a very large folder (say 40 GB) in Windows 7 File Explorer, the rate of progress indicates that it may take hours to complete. So I leave it overnight, only to discover in the morning that it still claims to be trying to copy (it has failed but still appears to be in progress).

Is there a way to copy a large folder with high confidence that it will be successfully completed, or at least that it will conclude and display an error message if there is a problem?

  • Last time I used Windows, it didn't even know if the destination had sufficient space for the file until it returned a 'disk full' error, half way through. Hopefully that has improved since Win7... but ... – Tetsujin Nov 21 '18 at 19:30
  • @Tetsujin it really is a pain to get a large folder to copy via file explorer. This issue is not acknoledged too well. Using File explorer for large folders should be a no no to put it blunetly. – zar Nov 21 '18 at 19:53
  • 1
    You've got a legit question buried in this, but it's worded in an off-topic way. It invites discussion, invites dispute about whether File Explorer is actually problematic, and asks for opinion on what's best. I'm voting to close on that basis, but will retract the vote if you can improve the question. Just state the problem you're facing and ask for a solution to it. – fixer1234 Nov 24 '18 at 1:16
  • @fixer1234 I have changed the title since. – zar Nov 24 '18 at 16:14
  • Copying from one drive to another drive will be much faster than copying to the same drive. When copying to the same drive, the drive needs to keep finding, and moving between, the source and destination locations. If the folder contains a huge number of tiny files, it also needs to store directory information for each one. So that will take much longer than copying a few extremely large files. Huge folders won't be copied instantaneously, but 40 GB shouldn't take hours, even under the worst conditions (unless you are doing another drive-intensive, (cont'd) – fixer1234 Nov 24 '18 at 23:52
0

Using windows file explorer to copy a large folder is, indeed, not very helpful for two reasons.

  • The main reason is that the file copy may fail for one reason or another, but it never returns and it will appear as if copying is still in progress where, in fact, it'ts just stuck. This can waste a lot of time of user.
  • If it failed due to error, it doesn't tell why it failed, what was the problem.

For example a file copy may fail because amid 40 GB data, there was one invalid shortcut where the target was missing. This will fail the copy but user will neer know and it will appear as if the copying is still in progress. There could be other reasons of failures that we we wouldn't know.

Solutions:

  1. It's best to copy large folders on command line and use `xcopy'. The benfit is that if it fails, it will return immediatly and you will know. That said it will still fail with above situation (invalid shortcut) but will error out correctly.
  2. The best Selection is to deploy a blind copy which doesn't care about the files but copy bits to bits. The best tools are scp or rsync but they are not readily available on Windows.

Using SCP on Windows

Install cygwin on Windows and use scpcommand from its shell the following way assuming to copy c:\BigFolder to a USB drive F:\BigFolderBackup/

scp -r /cygdrive/c/BigFolder/ /cygdrive/f/BigFolderBackup/

Where -r is for recurrsive copy which means copy all subfolders and their files.

Update 1

As noted by Scott in comments, since its copy on local machine so cp will do as well. Syntax is essentially identical.

Another scenario I came across is when a folder contain a file where I didn't had permssions to read. Our IT department doesn't allow users to run CCleaner utility so read permission is disabled on it.

I used cp this time and it performed nicely. It reported appropriate error on this file (didn't copy it) but didn't halt the copy operation and it continued with copying everything else. Another reason that this approach wins.

Update 2

I discovered scp (and probably cp) modifies file ownership which can cause problem with read/write access. Adding p flag should address that as noted in this article. Another way is to use rsync with -o and -g flags as noted in this answer to preserve ownership of files.

  • If you have Cygwin, and you're not copying from one machine to another, why no just use cp (instead of scp)? – Scott Nov 25 '18 at 20:38
  • @Scott I guess that too, might be better in this case. It just didn't come on my radar since I had only used scp in the past – zar Nov 27 '18 at 19:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.