0

I have a very large sparse file on an EXT4 partition, and I want to copy that file to an NTFS partition. The file has a size of about 2 TB, but since it is a sparse file it uses only 700 GB on disk.

The NTFS partition where I want to copy that file to only has about 1.3 TB free space. So if I can copy the sparse file there, it should fit comfortably; but as non-sparse file this won't fit.

I tried to copy the file with a simple cp -a command, but it failed in the end with "no space left on device". Interestingly it failed while about 570 GB were still reported as free.

By checking the file size and its reported disk usage I can see that the file on NTFS partition is not sparse.

I also tried to create a sparse file manually, with either of these commands:

  • dd if=/dev/zero of=test1.bin bs=1 count=1 seek=100000
  • truncate -s 100000 test2.bin

On an EXT4 partition both of the created files were sparse (du -h shows that they use 0 bytes on disk). But on the NTFS partition each file actually uses 100 KB of disk space, according to du -h and also according to ls -lsk.

So: why can't I create sparse files on the NTFS partition? And what can I do to copy the large file as sparse file to the NTFS partition?

The system where this happens:

  • Debian 11 (Bullseye) x86-64
  • kernel package 5.10.92-1
  • ntfs-3g package 1:2017.3.23AR.3-4+deb11u1

According to mount, the NTFS partition is mounted like this:
/dev/sdg1 on /mnt/loop2 type fuseblk (rw,relatime,user_id=0,group_id=0,allow_other,blksize=4096)

My understanding is that the partition is mounted using the ntfs-3g driver - is that correct? And according to man ntfs-3g that driver supports sparse files. I also did not find any option to enable or disable that feature.

I did not create the NTFS partition myself, but used the existing partition that was on the drive when I bought it.

1
  • There's one thing you can try -- ntfstruncate. I'm not sure if it is in ntfs-3g or ntfsprogs on your distro though (before some point the latter was still a thing). But it's a bit difficult to use / lame. Here it takes a block device and an inode number and works on only an unmounted filesystem (so you will also need to use touch to create the file beforehand). Maybe a little test with it will give some more hints. Usage might differ with the version you got.
    – Tom Yan
    Mar 7 at 18:11

1 Answer 1

0

Normally you can’t in Windows copy or create a sparse file if its nominal size exceeds the amount of available free space. In addition, Windows files may need to be flagged as sparse by the fsutil sparse command, or they are not sparse.

I suggest doing the copy from inside Windows, perhaps using a utility such as sparse.zip, found at the end of the article NTFS Sparse Files For Programmers.

The syntax of this command is simple:

cs.exe from-path to-path

If this can't be done on Windows because the file is only on Linux, try to tar the file on Linux, then untar it on the target Windows system using Windows Subsystem for Linux (WSL).

2
  • Unfortunately I don't have a Windows system available (and I don't want to install one just for this, TBH). But I guess doing this from Windows is generally a good idea nonetheless - might be helpful for others who also have this problem.
    – oliver
    Mar 7 at 17:18
  • tar supports sparse files since some years now, so it could be the mechanism of transfer.
    – harrymc
    Mar 7 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.