11

I'm thinking about using compressed NTFS on my external USB harddisk. It should be used for backups only.

  • Is it possible to access it from Linux?
  • Is it reliable?
  • According to Microsoft it's slower than normal filesystem. Given the transfer rate (30MB/s) and the processor (2.8 GHz Phenom II X4) I think the opposite may be true. What do you think?
  • According to this question not all files get compressed. How can I avoid this?
Improve this question

3 Answers 3

Reset to default
7

1) Yes, it is possible to access it from Liunx. As others explained, ntfs-3g is capable of handling compressed ntfs.

2) reliability: compression is in ntfs for a long time now (since Windows NT, see http://www.ntfs.com/ntfs-compressed.htm). I don't see any reliability problems on windows. I'm not that sure about non-windows implementations (e.g. linux). If your only concern is reading the data on linux, this is poses no reliability problem (mounting read-only is a good idea when restoring backups anyway).

Also data is compressed on cluster level which is usually 4 kiB. A physically corruption of the disk only affects single clusters, not entire files. In this regard compressed ntfs should be just as reliable as non-compressed.

3) Performance: The KB article you cite says performance might be affected on a server system, where the CPU is already saturated. On a desktop system with a current CPU as you describe it, using compressed ntfs for backups, should not have any significant performance impact. On the contrary, if you store compressible data, you might actually gain performance as you have less I/O. That is especially true, if the interface (USB 2.0) is slow compared to the CPU. I guess your CPU should easily be capable of saturating a USB 2.0 link writing or reading compressed ntfs.

4) If you set the compression flag for the entire filesystem you shouldn't have the problem of non-compressed moved files.

Improve this answer
1
  • 4
    NTFS never transfers files compressed, in either the windows or the Linux driver. That optimization doesn't exist, sadly. You'd have to go extremely low level to achieve that: Create the file and metadata, pre-allocate the space, then raw write the compressed data to the MFT and allocated clusters. So in normal use, you won't gain anything I/O-wise.
    – SilverbackNet
    Jan 27, 2014 at 2:01
1

  1. I think it is accessible from Linux, but I'm not sure.

  2. Reliable in what sense? It's easier for your data to get corrupted, but I'm not sure if that's what you mean.

  3. It's slower processor-wise, but faster I/O-wise. If your processor is much faster than your disk, then it might be worth it to read less data but do some processing, instead of reading huge files but avoiding postprocessing.

  4. You can't avoid this, it depends on how the applications were programmed. I remember seeing a setting in Windows that had to do something with this, but I don't think it does what you need -- I'll update this if it turns out it does.

Improve this answer
4
  • Ad 2: I mean if there are no bugs in the filesystem or alike. Ad 4: Actually, I'm going to copy files from another non-compressed drive, probably using rsync. I really don't see why an application should care about the internals of the filesystem, IMHO it's the job of the FS to do all the work.
    – maaartinus
    Mar 14, 2011 at 4:05
  • To pt#1 -> NTFS can be accessed @Mehrdad
    – Sathyajith Bhat ♦
    Mar 14, 2011 at 4:05
  • 1
    @maartinus: #2: Well, I have no hard data on this, but personally I would trust the Windows version but not the Linux version. #4: It's simply a matter of keeping the default settings, not a matter of the application caring. When a 2-GB file moves from a non-compressed folder to a compressed folder, there's no reason for the file system to spend a long time compressing it -- it just leaves it uncompressed, hence the problem. I don't think there's much you can do about it. @Sathya: Ah thanks.
    – user541686
    Mar 14, 2011 at 4:27
  • I suppose this leaving file uncompressed happens only when files get moved from an uncompressed part of the same disk, which will not be the case. So everything is fine.
    – maaartinus
    Mar 15, 2011 at 12:15
1

  1. The ntfs-3g driver supports reading, appending and (recently) modifying compressed files.

    Currently reading compressed files is supported by all ntfs-3g versions. Creating new compressed files, clearing contents, and appending data to existing compressed files are supported since ntfs-3g-2009.11.14. Modifying existing compressed files by overwriting existing data (or existing holes) are supported since ntfs-3g-2010.8.8.

    — NTFS-3G Advanced: Data Compression

  2. The filesystem is as reliable as its usual Linux counterparts, ext3/ext4.

    The ntfs-3g driver handles everything really well. (It might still have some bugs in modifying compressed files; as the above quote says, it has only been added in version 2010.8.8.)

  3. (no answer)

  4. When this is caused by programs creating an uncompressed file elsewhere and moving it to its intended location later, the workaround is easy: Re-enable compression on those files.

Improve this answer
1
  • ntfs-3g -V: 2013.1.13AR.1 As of Ubuntu 14.10, kernel 3.16, I still can not vouch for ntfs-3g's ability to even properly read compressed files on my Win8 GPT partition. Even copying a file over results in a different md5sum.
    – Marcos
    Oct 4, 2014 at 22:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .