9

How can I copy or move a sparse file from one NTFS volume to another NTFS volume while maintaining the sparseness using the native tools included with Windows? If there is no way to do this with the native tools, then what is a free application from a trusted vendor that will do this? Surely there is a command prompt command or powershell command that will do this.

I have a file that represents 250GiB that is taking up 20GiB of disk space and I would like it to continue only taking up 20GiB when I move it. Thanks.

2
  • 1
    What kind of sparse file is it? Would you be able to use a utility to shrink it after it's moved? If it is a VM Image, it should be fine to copy and retain just the space that is already allocated.
    – kobaltz
    Oct 12, 2013 at 2:00
  • 2
    It is a file system level sparse file, as in an NTFS file with the 'P' extended attribute set on it. It is not an application level sparse file which adds sparse support (and overhead) regardless of whether the file system already supports sparseness or not such as with a vmdk file. You're suggesting I transfer 250GiB, then spend the CPU to compress that, and then uncompress back to a 250GiB file whenever I want to actually access its contents. I'd rather move the small 20GiB file and have it stay 20GiB and remain accessible to me in its destination. Much less network, CPU, and storage use. Oct 12, 2013 at 3:44

4 Answers 4

5

You cannot do it with standard Windows tools.

But there is an utility. Go to http://www.flexhex.com/docs/articles/sparse-files.phtml, download and unpack sparse.zip.

Then run from from the command line:

cs.exe c:\src.dat d:\dest.dat

The file will be copied preserving sparseness.

3

Use Cygwin and mount the filesystem with the sparse option. Then Unix commands that support sparse files, such as cp, dd conv=sparse, and rsync -S, will correctly create or copy the file as a sparse one. As one would expect, simple output redirection will not create a sparse file.

See the following demonstration.

$ mount -o sparse D: /tmp/mnt
mount: warning - /tmp/mnt does not exist.
$ cd /tmp/mnt
$ dd conv=sparse if=/dev/zero seek=10000 of=sparse count=1
1+0 records in
1+0 records out
512 bytes copied, 0.0101909 s, 50.2 kB/s
$ ls -lh sparse
-rw-rw-r--+ 1 dds None 4.9M Sep  5 13:05 sparse
$ du -h sparse
0       sparse
$ cp sparse sparse-cp
$ dd conv=sparse if=sparse of=sparse-dd
10001+0 records in
10001+0 records out
5120512 bytes (5.1 MB, 4.9 MiB) copied, 0.0500354 s, 102 MB/s
$ rsync -S sparse sparse-rsync
$ cat sparse >sparse-fail-cat
$ cat sparse | dd conv=sparse of=sparse-cat-dd
$ ls -lh sparse*
-rw-rw-r--+ 1 dds None 4.9M Sep  5 13:05 sparse
-rw-rw-r--+ 1 dds None 4.9M Sep  5 13:15 sparse-cat-dd
-rw-rw-r--+ 1 dds None 4.9M Sep  5 13:06 sparse-cp
-rw-rw-r--+ 1 dds None 4.9M Sep  5 13:11 sparse-dd
-rw-rw-r--+ 1 dds None 4.9M Sep  5 13:19 sparse-fail-cat
----rw----+ 1 dds None 4.9M Sep  5 13:06 sparse-rsync
$ du -h sparse*
0       sparse
0       sparse-cat-dd
0       sparse-cp
0       sparse-dd
4.9M    sparse-fail-cat
0       sparse-rsync
1
  • I'll surely try this in the next few days, when I'll have to try to move a good lot of sparse files from a host to another. Would you happen to know if this sparse mount would also work onto a network mapped dir (basically, could I achieve the same by mounting the drive of another computer in the network)? Of course, I'm ready to move the new HDD to the same system and do it, if I have to, but that could save a hardware move. Jul 21, 2019 at 7:09
0

On the Windows side I ended up using WinHex to do this or rather its forensic brother X-Ways Forensics. The GUI has an option in the menus to copy files sparsely.

0

I wrote a little script copy-sparse-windows.sh to copy a file, making it as sparse as possible. This needs msys2 or similar, specifically sh, truncate, and rsync.

It uses the windows fsutil tool to set the sparse flag, truncate to make a large empty file, and rsync to copy the non-zero content.

As I understand, some of the other answers do not create new sparse regions. I wanted to reduce the size of a large VM disk image, by adding sparse areas everywhere possible.

#!/bin/sh -euv
from=$1 to=$2
touch "$to"
fsutil sparse setflag "$to"
truncate --reference "$from" "$to"
rsync --archive --inplace --no-whole-file --verbose --progress "$from" "$to"
1
  • To make this run on WSL, changed the fsutil invocation to fsutil.exe sparse setflag "$(wslpath -w "$to")". But I'm still ironing out the difficulties--it seems to work copying sparse files to and from a Windows directory but not from a WSL directory to a Windows directory.
    – piojo
    Oct 23, 2021 at 8:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .