Why can't I create sparse files on my NTFS partition under Linux?

Question

I have a very large sparse file on an EXT4 partition, and I want to copy that file to an NTFS partition. The file has a size of about 2 TB, but since it is a sparse file it uses only 700 GB on disk.

The NTFS partition where I want to copy that file to only has about 1.3 TB free space. So if I can copy the sparse file there, it should fit comfortably; but as non-sparse file this won't fit.

I tried to copy the file with a simple cp -a command, but it failed in the end with "no space left on device". Interestingly it failed while about 570 GB were still reported as free.

By checking the file size and its reported disk usage I can see that the file on NTFS partition is not sparse.

I also tried to create a sparse file manually, with either of these commands:

• dd if=/dev/zero of=test1.bin bs=1 count=1 seek=100000
• truncate -s 100000 test2.bin

On an EXT4 partition both of the created files were sparse (du -h shows that they use 0 bytes on disk). But on the NTFS partition each file actually uses 100 KB of disk space, according to du -h and also according to ls -lsk.

So: why can't I create sparse files on the NTFS partition? And what can I do to copy the large file as sparse file to the NTFS partition?

The system where this happens:

• Debian 11 (Bullseye) x86-64
• kernel package 5.10.92-1
• ntfs-3g package 1:2017.3.23AR.3-4+deb11u1

According to mount, the NTFS partition is mounted like this:
/dev/sdg1 on /mnt/loop2 type fuseblk (rw,relatime,user_id=0,group_id=0,allow_other,blksize=4096)

My understanding is that the partition is mounted using the ntfs-3g driver - is that correct? And according to man ntfs-3g that driver supports sparse files. I also did not find any option to enable or disable that feature.

I did not create the NTFS partition myself, but used the existing partition that was on the drive when I bought it.

Answer 1

Normally you can’t in Windows copy or create a sparse file if its nominal size exceeds the amount of available free space. In addition, Windows files may need to be flagged as sparse by the fsutil sparse command, or they are not sparse.

I suggest doing the copy from inside Windows, perhaps using a utility such as sparse.zip, found at the end of the article NTFS Sparse Files For Programmers.

The syntax of this command is simple:

cs.exe from-path to-path

If this can't be done on Windows because the file is only on Linux, try to tar the file on Linux, then untar it on the target Windows system using Windows Subsystem for Linux (WSL).