Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I just upgraded my LAN to gigabit. This is what netperf has to say about things.

Before:

marcus@lt:~$ netperf -H 192.168.1.1
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.1.1 (192.168.1.1) 
port 0 AF_INET : demo
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

 87380  16384  16384    10.02      94.13   

After:

marcus@lt:~$ netperf -H 192.168.1.1
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.1.1 (192.168.1.1) port 0 AF_INET : demo
Recv   Send    Send                          
Socket Socket  Message  Elapsed              
Size   Size    Size     Time     Throughput  
bytes  bytes   bytes    secs.    10^6bits/sec  

 87380  16384  16384    10.01     339.15   

Only 340 Mbps? What's up with that?

Background info: I'm connecting through a gigabit switch to a sheevaplug. I have Cat5e wiring in the walls and the run is maybe 30 feet. If you're not familiar with netperf, it has a tendency to give very stable results and never lie.

share|improve this question
    
Is this necessairly a Linux question? Have you tried this under a different OS (preferably a BSD) to establish that it is an OS specific issue? Otherwise, I reccomend removing the tag - it sounds like hardware to me. –  new123456 Jan 31 '11 at 0:14
    
Good point, will do. –  marcusw Jan 31 '11 at 0:17
    
What are the speed/duplex settings on the switch and the pc? –  Chris Jan 31 '11 at 0:19
    
The switch shows 1000Mb on both ports and both boxes' dmesg output agrees with this. –  marcusw Jan 31 '11 at 0:26
    
Yeah, it's a little slow, but not unexpected. I've seen max 500Mbps on gigabit, and that was a lucky break. I normally expect anything from 250 - 450Mbps. –  user3463 Jan 31 '11 at 0:54

3 Answers 3

up vote 2 down vote accepted

Check out this thread. One of the contributors (Frennzy) outlines this very nicely. I'll quote:

The "real" speed of gigabit ethernet is...

1Gbps.

That is to say, it will transfer bits at the rate of 1 billion per second.

How much data throughput you get is related to various and sundry factors:

NIC connection to system (PCI vs PCIe vs Northbridge, etc).

HDD throughput.

Bus contention.

Layer 3/4 protocol and associated overhead.

Application efficiency (FTP vs. SMB/CIFS, etc)

Frame size.

Packet size distribution (as relates to total throughput efficiency)

Compression (hardware and software).

Buffer contention, windowing, etc.

Network infrastructure capacity and architecture (number of ports, backplane capacity, contention, etc)

In short, you won't really know, until you test it. NetCPS is a good tool for this, as are many others.

And this, later in the thread (my highlighting):

Stop thinking like this. Stop it now. All of you.

As much as you would like to figure out kilo-or mega BYTE per second transfer, the fact is that it is variable, even when network speed remains constant. Network "speed" (bits per second) is absolute. Network throughput (actual payload data per second) is not.

To the OP: will you, in general, see faster data transfers when switching from 100Mbps to 1000Mbps? Almost definitely. Will it be anywhere close to the theoretical maximum? No. Will it be worth it? That's for you to decide.

If you want to talk about network speeds, talk about network speeds. If you want to talk about data throughput, talk about data throughput. The two are not tied together in a 1-1 fashion.

share|improve this answer
    
I read this before posting and am familiar with the issues involved. netperf measures bits/second based on throughput and you will note that I included results from 100Mbps for comparison. My ethernet card is PCIe on one end at least. I'm not sure about the sheevaplug but would assume that, being primarily a network device, it is capable of taking full advantage of the networking hardware inside it. What I'm really asking here is whether the numbers I get can be improved significantly by setting something up differently. –  marcusw Jan 31 '11 at 0:58
    
I doubt it. Aside from the SheevaPlug, I don't see anything you can do differently. As I said in my comment on your question, it seems a little slow but falls in the range I'd expect on gigabit. FWIW, I want a SheevaPlug. –  user3463 Jan 31 '11 at 0:59
1  
Your comment regarding the speeds you get really answers the whole thing. And if you're looking for a linux router, the sheevaplug plus a USB NIC for the internet interface is certainly the cheapest (and coolest) way to go. Add a USB hub and an external hard drive and you've got a full server. That's what I did, stuck debian lenny on there, and it's amazing. Just stay away from the OS it ships with...it's badly broken in many ways (even DNS queries won't work). But I ramble. Thanks! –  marcusw Jan 31 '11 at 3:36
    
Thanks for the tip. I like Lenny so I think it's a good option. –  user3463 Jan 31 '11 at 4:01
    
You will probably find a significant improvement if you enable Jumbo frames everywhere, and your switches support it. The typical 1500 byte ethernet frame has quite a high overhead (as a proportion), and this becomes more noticable when on gigabit (more packets, more latency, more overheads for ACKs, etc etc). See if jumbo frames with a size of about 6-7K are possible. You are likely to see an improvement of >10% if you do this. (On Windows, I see an improvement of >50% doing this.) –  quickly_now Jan 31 '11 at 23:19

The term 'theoretical maximum' is thrown around, but it does have a practical application with Ethernet technologies. On a CSMA/CD system like Ethernet, you can only send about half the bandwidth of traffic as the wire holds, often a bit less. The reason is because once you try to get beyond that 'maximum', then transceivers will start to detect collisions more than they are transmitting packets. Then exponential back-off comes into play and packet transmission degrades even further. Token ring got around this, but it had a lot of its own issues and isn't really used much anymore, I believe. Ethernet/IP became the de facto standard.

Uplink technologies, like T3, use asynchronous pairs which allow for the full throughput on each wire, but it is also not an Ethernet-based protocol.

While you are using basic, standard Ethernet devices, there will always be the 'theoretical maximum'.

share|improve this answer
1  
Which is why modern switched Ethernet networks don't use CSMA/CD anymore. –  grawity Jan 31 '11 at 13:31
    
A single link into a switch does not. But what goes on inside the switch? –  quickly_now Jan 31 '11 at 23:20

Talking about CSMA/CD in the context of GbE is entirely bogus. Gigabit Ethernet, or any "full-duplex" Ethernet, does not use CSMA/CD. And while GbE did still maintain the theoretical possibility for half-duplex operation I'm not at all sure there was ever any actual production GbE kit that did half-duplex.

As for why the OP only acheived 300-odd Mbit/s across a 1000 Gbit/s link, I would suggest gathering netstat statistics for TCP from before and after each netperf run, and include the -c and -C global command-line options to see what the CPU utilization is on either end. Perhaps something is dropping packets, or perhaps the CPU on one side or the other is becoming saturated. If the systems on either end are multicore, definitely check the per-core utilizations either with an external tool, or by wading through netperf debug output.

Other netperf questions probably best left to the netperf-talk at netperf.org mailing list.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.