17

From my observations as a Java developer working on Windows workstations, NTFS is slow compared to Linux filesystems. Question is, is there anything in the NTFS driver that can be manually tuned, for example give it more memory for cache? Enable some experimental algorithms? If that's not available, is there perhaps another filesystem that can be used on Windows, maybe even commercial, that's faster than NTFS?

To be clear, I'm not looking to improve compilation speeds for Maven projects, I'd like to get an overall improvement for the OS. I get a feeling that NTFS is long outdated and slow compared to Linux filesystems. It strikes me as weird that the most popular OS on planet has only one filesystem which still requires manual defragmentation. Perhaps there is an alternative?

Update: Here is what's slow according to my observations. I'm building/packaging a project, which means lots of read/write operations on disk. The build system is cross-platform (Java, Maven), so I can perform exactly the same actions when booted to Ubuntu, for example.

On Linux my builds are at least 1/3 faster. Hence the question about filesystem. I'm sorry if it's misplaced.

8
  • 7
    In what way is NTFS slow? How are you measuring its speed to know that it is slower than any other filesystem for exactly the same operations? NTFS is the only natively supported filesystem (apart from FAT which has limitations) that Windows itself can boot from, though do I believe that you may be able to get an ext3 filesystem driver for non-boot partitions.
    – Mokubai
    Feb 2, 2011 at 8:38
  • Yeah, please enlighten us, how is NTFS slow?
    – JL.
    Feb 2, 2011 at 8:51
  • 1
    But what is the question here? Is it how you can speed up NTFS or whether Windows can use a different file system?
    – Ivo Flipse
    Feb 2, 2011 at 8:57
  • 1
    @ivo-flipse: I'm sorry for causing confusion here, I'm interested in learning something about both questions. Like, is there no proven alternative to NTFS on Windows? And if not, would it be possible to generally increase NTFS' performance, like give more memory to some caches or enabling some experimental mechanisms? Feb 2, 2011 at 9:14
  • 1
    @sinni800 That's just rubbish, sorry. Whilst I admit that as a Desktop OS, Linux is far from perfect, as a Server OS it is far better than Windows for nearly all workloads. There is no question of the 96.6% of top 1 million websites using a beta quality operating system.
    – mjaggard
    Sep 9, 2016 at 10:55

5 Answers 5

4

While I'd love to see something like ZFS available for Windows hosts, NTFS isn't a horrible filesystem. It supports most "modern" filesystem features (extended attributes, journaling, ACLs, you name it), but it's hampered by Explorer and most other apps not supporting any of these.

One thing that will absolutely kill its performance is having "too many" entries in a directory. Once you pass a couple thousand entries in one directory, everything slows to a crawl. Literally the entire machine will halt waiting for NTFS to create or remove entries when this is happening.

I used to work with an app that generated HTML-based docs for .NET assemblies; it would create one file per property, method, class, namespace, etc. For larger assemblies we'd see 20+k files, all nicely dumped into a single directory. The machine would spend a couple of hours during the build blocked on NTFS.

In theory, Windows supports filesystem plugins, which would make native ZFS, ext3 or whatever (even FUSE) possible. In practice, the APIs are undocumented, so you're completely on your own.

Now, since you're doing Java development, could you install a different OS on your machine, or use a VM on top of Windows?

Also, you might want to try some platform-independent filesystem benchmarks (iozone, bonnie... there are probably more modern ones I don't know off the top of my head, maybe even a few written in Java) to see if it's actually the filesystem holding you back, or if it's something else. Premature optimization and all that...

3
  • The Installable File System has been documented at least since 2000. Lack of plugins was always down to difficulty and lack of interest. It's still difficult, but FUSE and FSDK are now available.
    – user165568
    Jun 7, 2019 at 7:38
  • I have used Ext2 on Windows for years. Works fine!
    – oligofren
    Feb 24, 2020 at 9:18
  • On topic of fuse: there is a FUSE analog for windows, it called Dokan Library This repo appears to be the latest fork: github.com/dokan-dev/dokany I seen it in use to provide network access for a game mod (which originally have no network access).
    – Krypt
    Apr 27, 2022 at 14:53
5
  • Turn off Last Access time
  • Turn off Short File names
  • Turn off Delete Notification
  • Turn off Indexing
  • Disable journeling
  • Disable shadow copy and previous versons and quotas and shares.
  • Enable bypass traverse checking

I guess that the real solution would be to rewrite your build system so that it used the native windows file system API, instead of the unix API (fopen etc) itself underneath the portability framework. But that's not going to happen, so basically you are stuck with whatever level of performance they thought was acceptable.

Often when using systems not originally written for Windows, you find that directory traversal has been handled very poorly, so make sure you have a very flat directory tree.

2
  • 2
    This would be a better answer if it gave some more information. 1. Some measure of how much time might be saved for each and 2. How to do these changes.
    – mjaggard
    Sep 9, 2016 at 10:40
  • 3
    Yeah, how do you do this? This is all fine and dandy on Linux, just modifying the fstab, but on Windows?
    – oligofren
    Feb 24, 2020 at 9:19
3

There is one file system which is supported by new Windows OS AND is faster than NTFS. It's exFAT. There's a possibility to use it for the system drive. But it's unknown what complications it might have.

You can certainly use it for other partitions though. It's faster with random read/write operations. Perfect for an SSD for example.

3
  • 2
    Tried it just now, and exFAT seems slower. First build on NTFS: 36.083s, second: 19.884s. First build on exFAT: 41.160s, second: 27.291s. Feb 2, 2011 at 12:42
  • Huh. exFAT is the only other file system that is now natively supported by Windows and COULD be used as the system drive (permissions are present). I saw a test once, online, and it said exFAT was faster with random reads.
    – sinni800
    Feb 2, 2011 at 13:01
  • The reason that test said exFAT was faster was probably because they didnt test it on a directory containing >32768 entries. Also, newer versions of Windows could have slowed down further due to internal changes. God (MS) knows what happens in a opendir() (or equivalent) call.
    – MattBianco
    May 10, 2017 at 12:53
3

Do you have an anti virus program running that's doing on access/write checking for your project folder?

Compiling involves reading and writing lots of small files in quick succession, which can overwhelm the virus scanner. Add your project folder to the list of excluded folders and see if that improves the situation.

On Linux you (probably) don't have any anti virus software....

2
  • Defender is off, AV is shutdown (or uninstalled completely, as case may be). Feb 2, 2011 at 11:57
  • @Yuri - oh well.
    – ChrisF
    Feb 2, 2011 at 11:58
0

Historical note: There was WinFS, which added a more structured layer on top of NTFS to enable Windows working the the content of the files more efficiently.

The "real" successor of NTFS is ReFS. Its focus is resiliency. It is only availble in Windows 10 Enterprise and Windows Pro for Workstations. A good read is NTFS vs. ReFS – How to Decide Which to Use.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .