Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

From my observations as a Java developer working on Windows workstations, NTFS is slow compared to Linux filesystems. Question is, is there anything in the NTFS driver that can be manually tuned, for example give it more memory for cache? Enable some experimental algorithms? If that's not available, is there perhaps another filesystem that can be used on Windows, maybe even commercial, that's faster than NTFS?

To be clear, I'm not looking to improve compilation speeds for Maven projects, I'd like to get an overall improvement for the OS. I get a feeling that NTFS is long outdated and slow compared to Linux filesystems. It strikes me as weird that the most popular OS on planet has only one filesystem which still requires manual defragmentation. Perhaps there is an alternative?

Update: Here is what's slow according to my observations. I'm building/packaging a project, which means lots of read/write operations on disk. The build system is cross-platform (Java, Maven), so I can perform exactly the same actions when booted to Ubuntu, for example.

On Linux my builds are at least 1/3 faster. Hence the question about filesystem. I'm sorry if it's misplaced.

share|improve this question
7  
In what way is NTFS slow? How are you measuring its speed to know that it is slower than any other filesystem for exactly the same operations? NTFS is the only natively supported filesystem (apart from FAT which has limitations) that Windows itself can boot from, though do I believe that you may be able to get an ext3 filesystem driver for non-boot partitions. –  Mokubai Feb 2 '11 at 8:38
    
Yeah, please enlighten us, how is NTFS slow? –  JL. Feb 2 '11 at 8:51
    
Rather show us a benchmark of how slow it is with the specs of your system :-) –  Ivo Flipse Feb 2 '11 at 8:54
1  
But what is the question here? Is it how you can speed up NTFS or whether Windows can use a different file system? –  Ivo Flipse Feb 2 '11 at 8:57
1  
@ivo-flipse: I'm sorry for causing confusion here, I'm interested in learning something about both questions. Like, is there no proven alternative to NTFS on Windows? And if not, would it be possible to generally increase NTFS' performance, like give more memory to some caches or enabling some experimental mechanisms? –  Yuri Ushakov Feb 2 '11 at 9:14

5 Answers 5

up vote 2 down vote accepted

While I'd love to see something like ZFS available for Windows hosts, NTFS isn't a horrible filesystem. It supports most "modern" filesystem features (extended attributes, journaling, ACLs, you name it), but it's hampered by Explorer and most other apps not supporting any of these.

One thing that will absolutely kill its performance is having "too many" entries in a directory. Once you pass a couple thousand entries in one directory, everything slows to a crawl. Literally the entire machine will halt waiting for NTFS to create or remove entries when this is happening.

I used to work with an app that generated HTML-based docs for .NET assemblies; it would create one file per property, method, class, namespace, etc. For larger assemblies we'd see 20+k files, all nicely dumped into a single directory. The machine would spend a couple of hours during the build blocked on NTFS.

In theory, Windows supports filesystem plugins, which would make native ZFS, ext3 or whatever (even FUSE) possible. In practice, the APIs are undocumented, so you're completely on your own.

Now, since you're doing Java development, could you install a different OS on your machine, or use a VM on top of Windows?

Also, you might want to try some platform-independent filesystem benchmarks (iozone, bonnie... there are probably more modern ones I don't know off the top of my head, maybe even a few written in Java) to see if it's actually the filesystem holding you back, or if it's something else. Premature optimization and all that...

share|improve this answer

There is one file system which is supported by new Windows OS AND is faster than NTFS. It's exFAT. There's a possibility to use it for the system drive. But it's unknown what complications it might have.

You can certainly use it for other partitions though. It's faster with random read/write operations. Perfect for an SSD for example.

share|improve this answer
    
That's interesting, thank you. –  Yuri Ushakov Feb 2 '11 at 12:00
1  
Tried it just now, and exFAT seems slower. First build on NTFS: 36.083s, second: 19.884s. First build on exFAT: 41.160s, second: 27.291s. –  Yuri Ushakov Feb 2 '11 at 12:42
    
Huh. exFAT is the only other file system that is now natively supported by Windows and COULD be used as the system drive (permissions are present). I saw a test once, online, and it said exFAT was faster with random reads. –  sinni800 Feb 2 '11 at 13:01

In order to answer more specifically, I would need to know more about how you use NTFS. For instance, are you using NTFS in a server or in a desktop computer? Are there some tasks for which you would specifically want to optimise your system for?

Also I have to disagree in that NTFS is the causing a system to slow down. From where I come from, a much more common bottleneck in a system is the lack of memory, which is the primary reason a system needs to re-read something from a hard drive in the first place.

share|improve this answer
    
I was comparing performance to a linux on the same hardware, for the same sequence of actions. Updated the description. –  Yuri Ushakov Feb 2 '11 at 9:30
    
@Yuri Ushakov: Thank you for the clarification! However: what makes you think NTFS is the bottleneck? Are you sure the slowing down is not caused by some other factor? –  jsalonen Feb 2 '11 at 9:34
    
No, I was jumping to a conclusion, perhaps mistakenly. There is nothing special to my Windows configuration, the swap is enabled, there is at least 1G of free RAM at all times, the antivirus is shut down, no other actively running processes. I just assumed it was the filesystem. Slow builds on Windows is what I observed for the past five years. Colleagues in my company were switching to Linux just for faster development. I cannot think of anything else other than NTFS, except such mystical areas as OS memory management. –  Yuri Ushakov Feb 2 '11 at 9:42
2  
@Yuri Ushakov Oh right, I get it now. What I suggest you to do is to rephrase your question into something like "How to speed up Java compilation in Windows", since with this question, you are already implying that the problem is actually in NTFS (which may contribute to it, but may not be the causing factor). –  jsalonen Feb 2 '11 at 9:50

Do you have an anti virus program running that's doing on access/write checking for your project folder?

Compiling involves reading and writing lots of small files in quick succession, which can overwhelm the virus scanner. Add your project folder to the list of excluded folders and see if that improves the situation.

On Linux you (probably) don't have any anti virus software....

share|improve this answer
    
Defender is off, AV is shutdown (or uninstalled completely, as case may be). –  Yuri Ushakov Feb 2 '11 at 11:57
    
@Yuri - oh well. –  ChrisF Feb 2 '11 at 11:58
  • Turn off Last Access time
  • Turn off Short File names
  • Turn off Delete Notification
  • Turn off Indexing
  • Disable journeling
  • Disable shadow copy and previous versons and quotas and shares.
  • Enable bypass traverse checking

I guess that the real solution would be to rewrite your build system so that it used the native windows file system API, instead of the unix API (fopen etc) itself underneath the portability framework. But that's not going to happen, so basically you are stuck with whatever level of performance they thought was acceptable.

Often when using systems not originally written for Windows, you find that directory traversal has been handled very poorly, so make sure you have a very flat directory tree.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.