10

Is there a directory copying method for NTFS that can copy a directory 100% accurately, every time, with every single file, sub-directory, permission, attribute, and symbolic link / junction intact without skipping a single file? I'm open to any suggestions, including command line, boot disc tools, partial disc imaging (as long as you can restore to a disc with data on it and not wipe it), raw copying, or anything else that might solve this.

I run into this problem far too often. I need to copy a tree of directories to another location but am foiled every time by some file that refuses to copy for one reason or another (access denied, file in use, or the best one, no reason given, it just refuses to copy.) Many methods don't copy NTFS permissions. Along with that, all the file copying tools are woefully inadequate when it comes to symbolic links and junctions. I want to copy the symbolic link itself, not the contents of the referenced location.

Copying certain directories in Windows is almost always a recipe for failure. Even tools like FreeFileSync that use shadow copying fail. Because of that, I prefer command line or boot disc solutions. (I boot to command line using Windows install disc.) Acronis True Image file backup/restore doesn't copy symbolic links or junctions and seems to skip several files as well (and runs incredibly slow from the boot CD.) Even Xcopy and Robocopy fail to copy 100% of the files, and neither can copy junctions, hardlinks, or directory symbolic links.

For example, I am currently trying to copy a Windows 8 system's C:\Users and C:\ProgramData to drive D. Regardless of the options used, booting to the command line, Robocopy still fails to copy several files, just skipping them without any reason given. Xcopy doesn't seem to get them all either. And because Windows uses a number of directory junctions in these folders, you're forced to rebuild them manually one by one.

Improve this question
4
  • imagex (from WAIK, the Windows Automated Installation Kit) might address your needs, but the license terms are fairly restrictive.
    – Harry Johnston
    Nov 4, 2012 at 20:43
  • Come to think of it, have you tried Windows Backup?
    – Harry Johnston
    Nov 5, 2012 at 22:59
  • 1
    @Harry Johnston: Re: imagex. That looks interesting. I'm going to have to dig deeper to figure out if that can be used on an individual folder and also extract the image to a drive without wiping the contents. Re: Windows Backup. I had not, but I looked at it and it doesn't seem to allow one to backup a specific folder. It also has a notice that system files will not be backed up, which are exactly the type of files I want to back up. The image option would back up the data along with everything else on the drive, but I'd have to be able to extract a folder from it and keep it all in tact...
    – Guavaman
    Nov 6, 2012 at 1:34
  • Although I don't recall ever trying it, the command-line reference for imagex says it can both capture and apply to a directory rather than an entire volume. It will however delete any existing contents in the target directory. You must create the target directory first. You're probably right about Windows Backup now that I think about it, I was thinking of the older versions (pre-XP) which were more flexible.
    – Harry Johnston
    Nov 6, 2012 at 2:03

4 Answers 4

Reset to default
3

I know this question is old, but I think there is a new answer that might be helpful since this question has not been answered with any upvotes, and I've never gotten a good answer to similar questions.

I just found FastCopy for Windows available free under GPLv3. https://en.wikipedia.org/wiki/FastCopy

FastCopy is still maintained by Shirouzu Hiroaki (@shirouzu) and was last updated on August 8, 2018 (as of this answer). It supports:

  • copying path names well over the 260 character limit
  • avoiding OS cache (by default) to improve speed
  • can copy ACL with Extended Attributes and alternate streams
  • will optionally copy HardLinks, Junctions, and SymLinks
  • copy verify via various hashes
  • differential copying based on size/date, etc.

The download is linked from his website (with source code available) here.

It is an EXE installer (ugh) but during install allows a portable "Extract Only" installation to the directory of your choice.

I was amazed by the speed today and it will save me hours on a repeated copy task that I have implemented. I have tried other frequently referenced alternatives like Robocopy, TeraCopy, xxcopy, Ultracopier, Copy Handler, etc., but I've never seen anything perform like this.

Improve this answer
4
  • 2
    Thanks for the suggestion. It sounds like a good program. I tried it out and immediately ran into access denied errors on files open by other applications. Part of my criteria is that I need to be able to copy all files without skipping a single one. This program can't do that because it must be run under Windows and can't copy files that are in use.
    – Guavaman
    Sep 17, 2018 at 18:34
  • 1
    @Allen - Kind of what I've been researching and not finding good solutions for.
    – Alex S
    Oct 10, 2019 at 9:05
  • 1
    @Guavaman I think what you need is a shadow copy because otherwise you will always have the "files in use" problem. This worked for me: blog.johnwray.com/post/2016/05/06/…
    – Allen J
    Dec 23, 2019 at 4:47
  • this app github.com/candera/shadowspawn make it easy to create a shadow and run what ever command you gave it
    – Badr Elmers
    Jul 6, 2020 at 4:03
1

Robocopy might be able to copy those after all:

http://technet.microsoft.com/en-us/library/cc733145(v=ws.10).aspx

/sl Copies the symbolic link instead of the target.

Improve this answer
7
  • 4
    I mentioned I've tried to use Robocopy in my question without success. I have used it with many, many permutations of options, the most common being /copyall /e /xj and sometimes /b. #1, it still skips a handful of files. #2, it does not copy symbolic links or junctions and they have to be manually recreated. If you leave out /xj, it will follow the junctions and copy that data. That is not a true mirror copy of a directory.
    – Guavaman
    Nov 2, 2012 at 8:12
  • could the /sl switch be the answer? not sure about it.
    – TheUser1024
    Nov 2, 2012 at 8:16
  • 1
    I just did a test with /copyall /e /sl and it followed the junction and copied junction as a directory and copied the target dir's contents as well. Did the same for a symlinkD. :( Thanks for helping though!
    – Guavaman
    Nov 2, 2012 at 8:20
  • I just tested and /SL does work for SYMLINK (files) only. Does not work for junctions, directory symlinks, or hard links.
    – Guavaman
    Nov 2, 2012 at 8:29
  • 1
    @Alix Axel: Yes, NTFS supports Symlinks (files only), Directory Symlinks, Hard Links (files only I think), and Directory Junctions. en.wikipedia.org/wiki/NTFS_symbolic_link
    – Guavaman
    Nov 2, 2012 at 22:49
1

I've used Macrium Reflect (free) and had no issues. Requires to boot from a CD, so that everything is gotten if you are wanting to do a full system backup/restore if you move to a new HDD/SSD. Very easy to use. https://www.macrium.com/reflectfree

Improve this answer
0
1

You can also use the Smart Copy or Smart Mirror functions in Link Shell Extension. It also has a command line tool.

Improve this answer
1
  • 1
    Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
    – Community Bot
    Sep 20, 2022 at 6:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .