I am working on two embeddded high performance network routers and I have connected them point-to-point via a 10Gbit optical fibre cable.
I ran iperf benchmarks on them and when I connect through an optical fibre point to point, I get slightly more than 2.5Gbit.
I know that this is a 10Gbps card, why is this card being under-utilized?
I also varied the MSS (maximum segment size) and the TCP window size but only with a marginal improvement in performance.
Thanks in advance.
As @Daniel B hints, you appear to have a GigaByte / Gigabit issue, since there is no such thing as 10GByte ethernet. 10Mb, 100Mb, 1Gb, 10Gb, 40Gb, 100Gb all exist (to some extent) but 10GB does not.
On 1 Gbit fiber links, raw data moves at 1.25Gb, but it's encoded 8->10 before transmission and 10->8 after transmission, so only the worst sort of salesdroids call it 1.25Gb, useable bit traffic is 1Gb, and traffic in BYTES is 125MByte/s
10Gbit fiber links use 64->66 bit encoding, for less total overhead, and work at 10.3125Gb/s on the physical layer so that they can get the stated amount of useful data through per second, in the same spirit as the 1Gb/s links (except when sleazy salesmen get involved.)
So, if you are getting 2.5 GByte/sec through a 10 Gbit link, you've managed about double the data rate, or you are getting a report that includes data in both directions (1.25GByte each way, since the link is full-duplex.) If you are only getting 2.5 Gbit/sec, you probably have hit limitations of the devices attached to the fiber to move data onto the link, or you have terrible error rates (dirty connections or some other physical layer problem).
I have iperf 2.0.5 and have noticed switch placement is importanet these work for me:
commands needed TESTING with TCP iperf -B 10.50.1.1 -s iperf -B 10.50.0.1 -c 10.60.1.1 -t 60 -i 10 -d
as above UDP TESTING UDP iperf -u -B 10.50.1.1 -s iperf -u -c 10.60.1.1 -w 16M -B 10.50.0.1 -b 10000m -l 19744 -i 10 -t 60
