Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a custom AVR microcontroller board that has a TCP/IP module (chip) for communication with PCs.

My PC has a 4-port Gigabit Ethernet card. Each port is connected to an Ethernet switch HP 1410-16G Switch(J9560A). Each Ethernet switch is connected to 10 ~ 15 of my custom controller boards.

Update I added 4-port Ethernet card. My PC now has 8 GigE port total. Each port is connected to a network switch. I tried to distribute the load.

Windows IP Configuration     Host Name . . . . . . . . . . . . : rdex4r0al    
Primary Dns Suffix  . . . . . . . :     
Node Type . . . . . . . . . . . . : Unknown    
IP Routing Enabled. . . . . . . . : No    
WINS Proxy Enabled. . . . . . . . : No  

Ethernet adapter HUB#2-4 (192.168.13.2):     
Connection-specific DNS Suffix  . :     
Description . . . . . . . . . . . : Intel(R) 82576 Gigabit Dual Port Network Connection #6    
Physical Address. . . . . . . . . : 00-25-90-98-FF-47    
DHCP Enabled. . . . . . . . . . . : No    
IP Address. . . . . . . . . . . . : 192.168.13.2    
Subnet Mask . . . . . . . . . . . : 255.255.255.0    
Default Gateway . . . . . . . . . :   

Ethernet adapter HUB#2-3 (192.168.12.2):     
Connection-specific DNS Suffix  . :     
Description . . . . . . . . . . . : Intel(R) 82576 Gigabit Dual Port Network Connection #5    
Physical Address. . . . . . . . . : 00-25-90-98-FF-46    
DHCP Enabled. . . . . . . . . . . : No    
IP Address. . . . . . . . . . . . : 192.168.12.2    
Subnet Mask . . . . . . . . . . . : 255.255.255.0    
Default Gateway . . . . . . . . . :   

Ethernet adapter HUB#2-2 (192.168.11.2):     
Connection-specific DNS Suffix  . :     
Description . . . . . . . . . . . : Intel(R) 82576 Gigabit Dual Port Network Connection #4    
Physical Address. . . . . . . . . : 00-25-90-98-FF-45    
DHCP Enabled. . . . . . . . . . . : No    
IP Address. . . . . . . . . . . . : 192.168.11.2    
Subnet Mask . . . . . . . . . . . : 255.255.255.0    
Default Gateway . . . . . . . . . :   

Ethernet adapter HUB#2-1 (192.168.10.2):     
Connection-specific DNS Suffix  . :     
Description . . . . . . . . . . . : Intel(R) 82576 Gigabit Dual Port Network Connection #3    
Physical Address. . . . . . . . . : 00-25-90-98-FF-44    
DHCP Enabled. . . . . . . . . . . : No    
IP Address. . . . . . . . . . . . : 192.168.10.2    
Subnet Mask . . . . . . . . . . . : 255.255.255.0    
Default Gateway . . . . . . . . . :   

Ethernet adapter Service (192.168.15.18):     
Media State . . . . . . . . . . . : Media disconnected    
Description . . . . . . . . . . . : Intel(R) 82576 Gigabit Dual Port Network Connection #2    
Physical Address. . . . . . . . . : 00-25-90-82-45-7F  

Ethernet adapter ToMain (192.168.2.50):     
Connection-specific DNS Suffix  . :     
Description . . . . . . . . . . . : Intel(R) 82576 Gigabit Dual Port Network Connection    
Physical Address. . . . . . . . . : 00-25-90-82-45-7E    
DHCP Enabled. . . . . . . . . . . : No    
IP Address. . . . . . . . . . . . : 192.168.2.50    
Subnet Mask . . . . . . . . . . . : 255.255.255.0    
Default Gateway . . . . . . . . . :   

Ethernet adapter Local Area Connection:     
Media State . . . . . . . . . . . : Media disconnected    
Description . . . . . . . . . . . : Intel(R) 82576 Gigabit Dual Port Network Connection #7    
Physical Address. . . . . . . . . : 00-25-90-98-7B-9E  

Ethernet adapter HUB#1-2(192.168.16.2):     
Connection-specific DNS Suffix  . :     
Description . . . . . . . . . . . : Intel(R) 82576 Gigabit Dual Port Network Connection #8    
Physical Address. . . . . . . . . : 00-25-90-98-7B-9F    
DHCP Enabled. . . . . . . . . . . : No    
IP Address. . . . . . . . . . . . : 192.168.16.2    
Subnet Mask . . . . . . . . . . . : 255.255.255.0    
Default Gateway . . . . . . . . . :   

Ethernet adapter HUB#1-3 (192.168.14.2):     
Connection-specific DNS Suffix  . :     
Description . . . . . . . . . . . : Intel(R) 82576 Gigabit Dual Port Network Connection #9    
Physical Address. . . . . . . . . : 00-25-90-98-7B-9C    
DHCP Enabled. . . . . . . . . . . : No    
IP Address. . . . . . . . . . . . : 192.168.14.2    
Subnet Mask . . . . . . . . . . . : 255.255.255.0    
Default Gateway . . . . . . . . . :   

Ethernet adapter HUB#1-4 (192.168.15.2):     
Connection-specific DNS Suffix  . :     
Description . . . . . . . . . . . : Intel(R) 82576 Gigabit Dual Port Network Connection #10    
Physical Address. . . . . . . . . : 00-25-90-98-7B-9D    
DHCP Enabled. . . . . . . . . . . : No    
IP Address. . . . . . . . . . . . : 192.168.15.2    
Subnet Mask . . . . . . . . . . . : 255.255.255.0    
Default Gateway . . . . . . . . . :  

My PC connects to almost 50 controllers in total. Sometimes I get a timeout error which means no ACK message. If I get several timeouts, the connection is lost and I can't connect to that controller again. It has to be power cycled before it will connect again.

If I connect to a controller directly without a switch, I don't get the timeout errors.

I'm trying to find the root cause of these lost packets. How can I monitor where the packet was lost? On the PC, the Ethernet switch, the customized controller...?

Would Wireshark be helpful to find the root cause?

Update

If one of the root causes is overloading, what is the solution? Do I need to add another 4-port network card and network switches in order to divide the connection? *May I ask more detail configuration or recommended product ? *

Update Question

As I mentioned about the configuration of the network switch, I heard if DNS is very slow, then every connection is slow. Many people like to use Google DNS at 8.8.8.8. It is much faster than others. Do I need to set that as the DNS server for my PC? Is there any relationship between DNS and speed? How can I know if the DNS speed is slow or not? In our application, what DNS is optimal? Any help is appreciated.

Further Questions Both sides should be using the same duplex setting. Should I enable or disable auto-negotiation on both sides? Do I understand correctly?

Additional Information

I'm focusing on three items.

  1. duplex mismatch : PC(Auto-Neg), Network Switch(Auto-Neg), Third-party controller(Auto-Neg option is enable, this controller has 100T-full duplex, 100T-half duplex, 10T-full duplex, 10T-half duplex))
  2. low quality cables : STP-CAT5e, Cable length within 5 m
  3. low signal environment : industrial factory lab

For this third-party controller, I don't know which mode is set or fixed. We can just guess that auto-negotiation is enabled. Do you have any idea how to keep good signal quality?

Update code

using System;
using System.Net;
using System.Net.Sockets;
using System.Reflection;
using System.Threading;
using LogManager;

namespace CoreUnitPlatform
{
    public class SocketCommCoreUnit
    {
        #region property
        private volatile bool _shouldStop;
        private LogWriter log = LogWriter.Instance;

        private bool m_bSocketConnected = false;

        private Socket m_clientSocket = null;

        private SocketCommType m_connectedSockType;

        private EventHandlerDataReceived m_evtHandlerDataReceived;

        private EventHandlerSocketConnected m_evtHandlerSocketConnected;

        private EventHandlerSocketConnectedFailed m_evtHandlerSocketConnectedFailed;

        private EventHandlerSocketDisconnected m_evtHandlerSocketDisconnected;

        private IPAddress m_IPAddress;

        private IPEndPoint m_IPEndPoint;

        private int m_portNo;

        private Socket m_serverSocket = null;

        private Thread m_threadConnectSocket = null;

        private string Name = string.Empty;

        #endregion

        #region constructor
        public SocketCommCoreUnit()
        {
            this.Name = "SocketCommCoreUnit";
            Instance();
        }
        #endregion

        #region delegatge
        public delegate void EventHandlerDataReceived(string msg);

        public delegate void EventHandlerSocketConnected();

        public delegate void EventHandlerSocketConnectedFailed();

        public delegate void EventHandlerSocketDisconnected();
        public enum SocketCommType { SERVER, CLIENT };

        public bool SocketConnected
        {
            get { lock (this) { return m_bSocketConnected; } }
            set { lock (this) { m_bSocketConnected = value; } }
        }
        #endregion

        #region public
        public void ConnectSocketProc()
        {
            while (!_shouldStop)
            {
                try
                {
                    if (SocketConnected == false)
                    {
                        if (m_connectedSockType == SocketCommType.SERVER)
                        {
                            m_clientSocket = m_serverSocket.Accept();   // If a client is connected, wait for data from client

                            m_evtHandlerSocketConnected();
                            SocketConnected = true;
                        }
                        else
                        {
                            m_clientSocket.Connect(m_IPAddress, m_portNo);
                            if (m_clientSocket.Connected == true)
                            {
                                m_evtHandlerSocketConnected();
                                SocketConnected = true;
                            }
                        }
                    }
                    else
                    {
                        try
                        {
                            byte[] buffer = new byte[1024];
                            int readBytes = this.m_clientSocket.Receive(buffer);
                            if (readBytes == 0)
                            {
                                this.reConnect();
                            }
                            else
                            {
                                string received = System.Text.Encoding.ASCII.GetString(buffer);
                                m_evtHandlerDataReceived(received);
                            }
                        }
                        catch (SocketException sex)
                        {
                            if (sex.NativeErrorCode.Equals(10054))
                            {
                                log.AddErrorLog(this.Name, MethodBase.GetCurrentMethod().Name, string.Format("Error Occured [{0}]: MESASGE[{1}]\r\nSOURCE[{2}]\r\nTRACE[{3}]", sex.NativeErrorCode, sex.Message, sex.Source, sex.StackTrace));

                                this.reConnect();
                            }
                        }
                    }
                }
                catch
                {
                    m_evtHandlerSocketConnectedFailed();
                }

                Thread.Sleep(100);
            }
        }

        public void Initialize(string IP, int port, SocketCommType sockType, EventHandlerDataReceived evtHandlerDataReceived, EventHandlerSocketConnected evtHandlerDataConnected, EventHandlerSocketDisconnected evtHandlerSocketDisconnected, EventHandlerSocketConnectedFailed evtHandlerSocketConnectedFailed)
        {
            m_connectedSockType = sockType;
            m_evtHandlerDataReceived = evtHandlerDataReceived;
            m_evtHandlerSocketDisconnected = evtHandlerSocketDisconnected;
            m_evtHandlerSocketConnected = evtHandlerDataConnected;
            m_evtHandlerSocketConnectedFailed = evtHandlerSocketConnectedFailed;

            m_portNo = port;
            m_IPAddress = IPAddress.Parse(IP);
            m_IPEndPoint = new IPEndPoint(m_IPAddress, m_portNo);

            if (sockType == SocketCommType.SERVER)
            {
                OpenServer();
            }
            else
            {
                OpenClient();
            }
        }

        public void Instance()
        {
        }

        public void OpenClient()
        {
            try
            {
#if _NO_USE_SOCKET
#else
                RunClientSocket();
#endif
            }
            catch (System.Exception ex)
            {
                log.AddErrorLog(this.Name, MethodBase.GetCurrentMethod().Name, string.Format("Error Occured: MESASGE[{0}]\r\nSOURCE[{1}]\r\nTRACE[{2}]", ex.Message, ex.Source, ex.StackTrace));
            }
        }

        public void OpenServer()
        {
            try
            {
#if _NO_USE_SOCKET
#else
                RunServerSocket();
#endif
            }
            catch (System.Exception ex)
            {
                log.AddErrorLog(this.Name, MethodBase.GetCurrentMethod().Name, string.Format("Error Occured: MESASGE[{0}]\r\nSOURCE[{1}]\r\nTRACE[{2}]", ex.Message, ex.Source, ex.StackTrace));
            }
        }

        public void Release()
        {
            try
            {
                if (this.m_clientSocket != null && this.m_clientSocket.Connected)
                {
                    SocketConnected = false;
                    m_evtHandlerSocketDisconnected();

                    this.m_clientSocket.Shutdown(SocketShutdown.Both);
                    this.m_clientSocket.Close();
                }

                if (m_serverSocket != null)
                {
                    m_serverSocket.Close();
                }

                if ((m_threadConnectSocket != null) && (m_threadConnectSocket.IsAlive == true))
                {
                    Thread.Sleep(1);
                    RequestStop();

                    SocketConnected = false;
                    m_threadConnectSocket.Abort();
                    m_threadConnectSocket.Join();
                }
            }
            catch (System.Exception ex)
            {
                log.AddErrorLog(this.Name, MethodBase.GetCurrentMethod().Name, string.Format("Error Occured: MESASGE[{0}]\r\nSOURCE[{1}]\r\nTRACE[{2}]", ex.Message, ex.Source, ex.StackTrace));
            }
        }

        public void RequestStop()
        {
            _shouldStop = true;
        }

        public void RunClientSocket()
        {
            m_clientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
            ConfigureTcpSocket(m_clientSocket, SocketCommType.CLIENT);

            m_threadConnectSocket = new Thread(new ThreadStart(ConnectSocketProc));
            m_threadConnectSocket.Start();
        }

        public void RunServerSocket()
        {
            m_serverSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
            m_serverSocket.Bind(m_IPEndPoint);
            m_serverSocket.Blocking = true;          // The server socket is working in blocking mode

            ConfigureTcpSocket(m_serverSocket, SocketCommType.SERVER);

            m_serverSocket.Listen(1);

            m_threadConnectSocket = new Thread(new ThreadStart(ConnectSocketProc));
            m_threadConnectSocket.Start();
        }

        public void Send(byte[] msg)
        {
#if _NO_USE_SOCKET
#else
            if (SocketConnected == false)
            {
                throw new Exception("SOCKET_NOT_CONNECT_BEFORE_SEND_DATA;");
            }

            try
            {
                m_clientSocket.Send(msg);
            }
            catch (System.Exception ex)
            {
                SocketConnected = false;
                m_evtHandlerSocketDisconnected();
                log.AddErrorLog(this.Name, MethodBase.GetCurrentMethod().Name, string.Format("Error Occured: MESASGE[{0}]\r\nSOURCE[{1}]\r\nTRACE[{2}]", ex.Message, ex.Source, ex.StackTrace));
            }
#endif
        }

        #endregion

        #region private

        private void ConfigureTcpSocket(Socket tcpSocket, SocketCommType socketCommType)
        {
            //// Don't allow another socket to bind to this port.
            //tcpSocket.ExclusiveAddressUse = true;

            //// The socket will linger for 10 seconds after
            //// Socket.Close is called.
            //tcpSocket.LingerState = new LingerOption(true, 10);

            // Disable the Nagle Algorithm for this tcp socket.
            tcpSocket.NoDelay = true;

            //if (socketCommType == SocketCommType.CLIENT)
            //{
            //    tcpSocket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.DontLinger, false);
            //    tcpSocket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
            //    //tcpSocket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveTimeout, 3000);
            //    //tcpSocket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.SendTimeout, 3000);

            //    // Set the receive buffer size to 8k
            //    tcpSocket.ReceiveBufferSize = 2048;

            //    // Set the send buffer size to 8k.
            //    tcpSocket.SendBufferSize = 2048;
            //}

            //// Set the receive buffer size to 8k
            //tcpSocket.ReceiveBufferSize = 1024;

            // Set the timeout for synchronous receive methods to
            // 1 second (1000 milliseconds.)
            //tcpSocket.ReceiveTimeout = 1000;

            //// Set the send buffer size to 8k.
            //tcpSocket.SendBufferSize = 1024;

            // Set the timeout for synchronous send methods
            // to 1 second (1000 milliseconds.)
            //tcpSocket.SendTimeout = 1000;

            //// Set the Time To Live (TTL) to 42 router hops.
            //tcpSocket.Ttl = 42;
        }

        private void ConfigureTcpSocket(Socket tcpSocket)
        {
            //// Don't allow another socket to bind to this port.
            //tcpSocket.ExclusiveAddressUse = true;

            //// The socket will linger for 10 seconds after
            //// Socket.Close is called.
            //tcpSocket.LingerState = new LingerOption(true, 10);

            // Disable the Nagle Algorithm for this tcp socket.
            tcpSocket.NoDelay = true;

            //// Set the receive buffer size to 8k
            //tcpSocket.ReceiveBufferSize = 8192;

            // Set the timeout for synchronous receive methods to
            // 1 second (1000 milliseconds.)
            //tcpSocket.ReceiveTimeout = 1000;

            //// Set the send buffer size to 8k.
            //tcpSocket.SendBufferSize = 8192;

            // Set the timeout for synchronous send methods
            // to 1 second (1000 milliseconds.)
            //tcpSocket.SendTimeout = 1000;

            //// Set the Time To Live (TTL) to 42 router hops.
            //tcpSocket.Ttl = 42;
        }


        private void reConnect()
        {
            ////if (this.m_clientSocket != null && this.m_clientSocket.Connected)
            {
                try
                {
                    SocketConnected = false;
                    m_evtHandlerSocketDisconnected();
                    m_clientSocket.Disconnect(true);

                    log.AddSystemLog(this.Name, MethodBase.GetCurrentMethod().Name, string.Format("Try Re-Connection..."));

                    if (m_connectedSockType == SocketCommType.SERVER)
                    {
                    }
                    else
                    {
                        m_clientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                    }
                }
                catch (System.Exception exc)
                {
                    log.AddErrorLog(this.Name, MethodBase.GetCurrentMethod().Name, string.Format("Error Occured: MESASGE[{0}]\r\nSOURCE[{1}]\r\nTRACE[{2}]", exc.Message, exc.Source, exc.StackTrace));
                }
            }
        }
        #endregion
    }
}
share|improve this question
1  
can the hub handle all this? to my understanding a hub is a layer 1 device where it simply takes in the communication at one port, and blasts it out to every other port. Can what you are trying to accomplish be done with a layer 2 switch instead, which can learn the MAC addresses of the other devices and therefore transmit it to that specific device instead of just blasting it out every port like a hub does? Also with a hub isnt that one big collision domain with CDMA/CD where it will have to issue a stop and retransmit everytime there is a collision. –  j_bombay Aug 23 '13 at 20:48
    
@j_bombayI checked the detail product which is HP 1410-16G Switch(J9560A). I think it is switch, not hub. To alleviates the overloading, do you think I use VLAN switch or router in order to divide the network ? –  Changju.rhee Aug 24 '13 at 1:11
    
This kind of problems tends to be either a duplex mismatch, (usually due to auto-negotiation being disabled on one side and enabled on the other) or poor quality cabling. –  David Schwartz Sep 7 '13 at 6:38
    
@DavidSchwartz Do you have any idea how to know duplex mismatch ? Third-party controller maker do not say about the configuration. –  Changju.rhee Sep 16 '13 at 1:50
    
@Changju.rhee: The most likely cause of a duplex mismatch is if you set the duplex to full on one side and leave it set to autonegotiate on the other. –  David Schwartz Sep 16 '13 at 8:35

1 Answer 1

Start simple. Connect directly, then one hub, then more controllers, etc. Add back one node at a time until it breaks, and then you'll either know the device causing the issue, or if it's a limit on the number of nodes.

share|improve this answer
    
thanks, but loosing message, it's randomly happening. –  Changju.rhee Aug 23 '13 at 20:31
1  
@Changju.rhee This advice is worth trying. Your present large setup is very likely overloading the host PC and/or the controllers. Scale it down to see if that alleviates the issues. Or replace the hubs with switches. Wireshark is just going to give a lot of data, and likely no answers unless you know what to look for. –  sawdust Aug 23 '13 at 21:06
    
@sawdust I checked the detail product which is HP 1410-16G Switch(J9560A). I think it is switch, not hub. To alleviates the overloading, do you think I use VLAN switch or router in order to divide the network ? –  Changju.rhee Aug 24 '13 at 1:09
    
@Changju.rhee if you know for certain it's overloading, you can use a switch instead of a router if they do not have to get to a different subnet. It doesn't have to be a VLAN capable switch, a regular switch will suffice. This may or may not help, if overloading the host PC is the problem. –  BigHomie Aug 26 '13 at 17:34
    
@MDMoore313 How to figure the host PC is problem or not. Do you have any idea ? I have added one more network card to distribute the clients. However, I have still time out error randomnly. –  Changju.rhee Sep 6 '13 at 7:43

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.