I have manually installed CentOS on all three hadoop nodes
There are 6 Ethernet interfaces in total for each box using lspci -v:
4 ports :from Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
Subsystem: Hewlett-Packard Company NC382i Integrated Multi-port PCI Express Gigabit Server Adapter
2 ports :from Ethernet controller: Mellanox Technologies MT26448 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
Subsystem: Mellanox Technologies Device 0018
I assume the first 4 ports come with the HP server along with driver installed. And those 2 ports from MT26448 are fancy 10G NIC card team added later to guarantee network performance, that is also the reason that why NIC driver need to be installed after installing the OS.
I am assuming behind the server, only those 2 10G ports have cable plugged in and I can see those two ports are within different IP blocks(both internal though): For example, this is what ifconfig shows on one of them:
eth4 Link encap:Ethernet HWaddr 00:02: ... :7E inet addr:10.0.188.xx Bcast:10.0.xx.xx Mask:255.255.252.0 UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1 RX packets:12940140032 errors:111345 dropped:111345 overruns:111345 frame:111345 TX packets:9162902211 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:9966465152889 (9.0 TiB) TX bytes:1911437036084 (1.7 TiB) eth5 Link encap:Ethernet HWaddr 00:02: ... :7F inet addr:172.16.32.xxx Bcast:172.16.xx.xx Mask:255.255.255.128 UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1 RX packets:110139 errors:0 dropped:0 overruns:0 frame:0 TX packets:129344 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:56380735 (53.7 MiB) TX bytes:604323107 (576.3 MiB)
I am assuming the box is using eth4 only who has been assigned the internal IP 10.0.188.xx. it is pretty obvious the assigned IP is in a much smaller subnet than 10.xxx, only 7 bits (128 addresses).. I guess is the network on the rack..
- I am not sure what is eth5 (172.16.32.xxx) of those boxes used for (backup)?
- If the computers want to talk to each other, will using 10.xxxx direct the data to the "big switch" of the whole company somewhere else instead of going to the box in the same rack?