0

Setup Below*

So I have a direct connection between my PC and my server which hosts (10 GbE network card to 10 GbE network card) and the only time I use this connection is when I transfer files between the two. Right now my speeds hover between 40 MB/s to 120 MB/s depending on what I am transferring. My synthetic benchmarks give me ~250Mbps/~2Gbps upload/download performance. This is compared to my Cat5e LAN which gives me ~500Mbps/~1Gbps.

1st) why is my 10GB network card slower uploading than my 1 GbE onboard chip. I am thinking I have them set up wrong. 2nd) 2Gbps for download isn't even half of 10Gbps let alone (though I know 10Gbps is only the theoretical limit). So I think that I am bottle-necking somewhere. I think its either the storage devices' access speed (see below), the overhead of moving the files, or maybe even the file systems themselves. Is there anyway to squeak out some more speed or have I just hit a practical limit?

*Here is my setup:

  • 1x win 10 desktop w/ 16GB RAM & 4 core skylake i5 w/ 2x 1TB WD blue @ 7200rpm in RAID 0 using nfts.
  • 1x Ubuntu 15.04 (virtual hosted on win 10) w/ 2 logical cpu core (from an i3-3220) and 24GB of RAM with 2x WD RED 3TB @5400rpm in RAID 1 using ext4 in lvm.

Both are connected via Chelsio s310e-cr 10 GbE network cards using 850nm wavelength dual channel fiber optics.

iPerf Results -Updated

  1. Cat 5e regular LAN: Win 10 --> Ubuntu : ** 200 to 770 Mbits/sec** | Ubuntu --> Win 10: 400 to 920 Mbits/sec (these values fluctuate b/c others are using the network)

  2. Chelsio Network card (direct connection): Win 10 --> Ubuntu : 472 Mbits/sec | Ubuntu --> Win 10: 445Mbits/sec

so it looks like I am getting comparable speed either way :/ You think I have something set up wrong? Here is a link to my Chelsio S310E-CR network card

UPDATE

I shutdown my Linux VM and gave the host Win 10 full access to the netowrk card. Doing the iPerf tests gave me way better results: ~2 Gbits/sec from desktop to server host and from server host to desktop I get ~3Gbits/sec. So now I am thinking it is either Virtualbox itself or my guest Ubuntu vm.

  • You have two issues intertwined: network speed and disk read/write speeds. You should decouple them. Iperf, iperf.fr/iperf-download.php is an excellent instrument to test your network speed, for all OSes. You should try that, and report the results. Then we shall know whether you issues concerns cables or disks. – MariusMatutiae Mar 16 '16 at 10:38
  • I did what you said and updated my original post with the results – SilverEnsign99 Mar 16 '16 at 16:49
0

One way to ensure whether you're hitting maximum storage access speeds is to use another medium, such as a usb 3.0 drive, or transferring between drives on the machine. Since your Windows 10 device only has the "one" drive RAID 0, internal transfer may not be an option hard drive wise, but if you have speedy external storage that you can plug in to, that may help you determine whether your drives are operating slowly. RAID 0 should easily allow you faster than 40-120mb/s, even if they're slow, though. The double Western Digital Reds though...

From Tech Radar's review of the drive:

Crystal Disk Mark reported sequential read speeds of 173 MB/s and write speeds of 165 MB/s. This is slightly lower than the claimed 175 MB/s speeds, but equivalent to HGST's He6 6TB disk and less than Seagate's Enterprise Capacity 6TB (223 MB/s).

This may be your issue. Since they're in RAID 1, together, you should be expecting speeds much faster than the ~150-200 when reading since RAID 1 acts like RAID 0 when reading, but when writing it will write slowly. If you want to test it, I would try hooking up external storage to your server also, and see if speed improves significantly. Red (NAS) drives are generally pretty slow because they're built to be reliable more than anything else. They are likely the cause of the slowdown that you're seeing.

As for actually answering your question of speeding it up: http://www.dell.com/downloads/global/power/ps3q07-20070368-Olivarez-OE.pdf has some good tips but you'll want to look at ensuring that raid controllers are updated or looked at in case you can find better, more optimized, drivers, or consider switching to RAID 5 if you have the income, as that could help your performance as well. That would result in purchasing 1 or 2 additional drives though. I hope this helps.

  • Playing with disks while doing diagnostics is just toying round the edges - you should use a performance benchmark tool like iPerf or a RAMDISK. – davidgo Mar 16 '16 at 5:21
  • I am not surprised the reds are slow,since they are only 5400rpm. But shouldn't Linux cache the transfer into RAM anyways thereby "hiding" the bottleneck. I feel like I should be able to push a file from the SSD inside my PC (forgot to mention that) to my Ubuntu server lightning fast. Or is that not how caching a transfer in RAM works? – SilverEnsign99 Mar 16 '16 at 7:41
  • Oops totally didn't catch that you had 24Gigs of ram on there. Well the caching would depend on if you have the ram in use for one. If your resources are devoted it may not. The reason I mention checking with other resources in a direct transfer is to double check whether that's the issue or not. If you are running a transfer with an external drive and it's still getting the same speeds then it might not be caching. If the file's large enough it might not cache. If you see speeds increase, then you know that there's possibly something up with one of your chips? – Zang Mar 16 '16 at 10:37
  • Also you mentioned that it's virtual hosted on Windows 10? I'm assuming you mean that you're running a virtual machine server on a windows box?. It might not be able to cache after the resources taken by Windows 10. You may need to ensure that the proper resources are alloted to the virtual machine. Silly question: Are you using Cat5e or Cat6 on your 10Gbps cards? – Zang Mar 16 '16 at 10:40
  • yeah its a virutal box machine hosted by another windows 10 desktop I have. I have the network set up as bridged running a paravirtualized adapter so Windows 10 doesn't even see the data passing through. My network card is only used by my VM so I am not fighting for bandwidth there. Also the VM has 24 GB of RAM reserved to it and the VM itself doesn't do anything expect serve as a NAS so it shouldn't be in use unless actively transferring files. And both network cards are connected via 850nm fiber optics: the are old server grade cards. – SilverEnsign99 Mar 16 '16 at 14:03
0

So the fix was really simple: Drop Virtual Box. Instead I am using VMWare and literally everything was solved. I have no idea why Virtual box didn't like my network card but it didn't. Here are my new synthetic benchmarks using iPerf 3:

Cat5e LAN

  • Desktop --> Server : 30 Mbits/sec
  • Server --> Desktop: 767 Mbits/sec

10GBe Fiber LAN

  • Desktop --> Server : 1.24 Gbits/sec
  • Server --> Desktop: 1.96 Mbits/sec

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.