Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

It seems to me that most network "file sharing" protocols are one or more of old, slow, and insecure. The most used protocols seem to be SMB, NFS, and WebDAV.

I'm sitting here, looking at iTunes trying to scan a media library over SMB, and it's going at about 2 MB per second. It's hooked up through wired gigabit networking, and the share lives on a RAID array that can do 50x as much throughput, even when searching. This is ludicrous!

Some file sharing protocols from the past might include:

  • Andrew File System
  • 9P/Styx protocol from Plan9 / Inferno
  • RFS from old System V
  • AppleShare protocol
  • Novell Netware protocol

My requirements are reasonably simple:

  • Modern security -- ideally, uses public/private keys like SSH. Tunneling over TLS would be great.
  • High performance -- scanning a file system or reading bulk data should run at the speed that the server and network can support.
  • Native clients for Windows and Linux -- others would be gravy but not interesting to me.
  • File locking compatible with Linux and Windows.
  • Change notification compatible with Linux and Windows.
  • Ideally, high quality open source implementations are available, but I'd be OK with a small license fee. (And, no, not enterprise-wide "small" -- I'm a regular guy with a family and a mortgage)

Am I missing something?

share|improve this question

migrated from stackoverflow.com Dec 19 '11 at 5:50

This question came from our site for professional and enthusiast programmers.

2  
Two things: (1) this isn't a programming question, and (2) you have competing requirements. Security is always at odds with performance. You want fast, use NFS. –  Chris Dec 18 '11 at 22:51
    
Have you tried tuning the transport layer's performance first? (note: the text that looks like broken links [more][less] etc is not broken, they just don't know how to style links properly) –  darvids0n Dec 18 '11 at 23:37
    
Does Kerberos count as "modern security"? If so, Samba with SMB2 enabled might work. –  grawity Dec 19 '11 at 15:48
1  
Given how much faster CPUs are than long-haul wires, security doesn't need to be opposed to performance. –  Jon Watte Dec 20 '11 at 6:27

2 Answers 2

up vote 4 down vote accepted

What is wrong with NFS 4?

Modern security -- ideally, uses public/private keys like SSH. Tunneling over TLS would be great.

NFS 4 offers modern authentication. Data security should be done via a VPN.

High performance -- scanning a file system or reading bulk data should run at the speed that the server and network can support.

NFS 4 is reasonable high performance for the general case. Lots of caching, aggregation of commands, etc.

Native clients for Windows and Linux -- others would be gravy but not interesting to me.

I am not aware of anything here. Any reasonable wide-spread solution comes from one of the corners and is only supported as secondary citizien in the other world. That said: I think it is fair to say that e.g. SMB is better supported in the Unix world (via Samba) than e.g. NFS is in the Windows world.

File locking compatible with Linux and Windows.

NFS gives you this.

Change notification compatible with Linux and Windows.

Good question. I don't think that NFS supports this in a good way. At least I am not aware of any command that e.g. would call a callback if something changes on the (meta data) server.

To your 2 MB/s with SMB? There should be something misconfigured. SMB is certainly capable of more than that. 100 MB/s (GB wirespeed) should be no problem in a reasonable configuration.

share|improve this answer
    
The 2 MB/s is probably because of iTunes, rather than the inherent set-up. That being said -- the answer to my question seems to be "no." Perhaps enough extensions can be added to NFS 4 to support change notifications and perhaps a Windows client could be written. But -- why hasn't anyone done this already? I want it all, right now, for free :-) –  Jon Watte Dec 20 '11 at 6:46

The fastest file sharing protocol in the world isn't going to be able to make up for the fact that some apps aren't written with remote filesystems in mind, so they dispatch lots of small reads or writes, and wait for one operation to complete before starting the next one. This is fine on the relatively low latency of a local disk, but it's painful over a network, where the latency is much higher.

If you want scanning of a remote media library to go fast, you need to have code running on the server do the scanning and report the results back to the client process in one big lump.

share|improve this answer
    
I agree that iTunes acts like a P.O.S. It's version 10.5, and it's still locking up the UI to scan a folder! That being said: you can solve this problem with judicious caching and pre-fetching. Which a high-performance remote file system would do. –  Jon Watte Dec 20 '11 at 6:44
    
I agree that iTunes acts like a P.O.S. It's version 10.5, and it's still locking up the UI to scan a folder! –  Jon Watte Dec 20 '11 at 6:46

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.