Set up bonding in RHEL6

Jephe Wu - http://linuxtechres.blogspot.com

Environment: CentOS 6 x86_64 on HP Blade Proliant BL460c G6, 2 builtin broadcom NIC and a NC364m Quad Port 1Gb NIC for c-class BladeSystem
Objective: after using builtin NIC to install OS , then configure bonding for 2 of quad port NIC.

Steps:

1. OS installation
You can download http://mirror.optus.net/centos/6/isos/x86_64/CentOS-6.0-x86_64-netinstall.iso and install OS through http URL from Internet.

2. bonding configuration

Firstly, disable network manager if NM is running.
service NetworkManager stop
chkconfig NetworkManager off


a. /etc/udev/rules.d/70-persistent-net.ruels

make sure it contains all interfaces you have in the system and don't configure bond0 inside.

b. /etc/sysconfig/network-scripts/

configuring ifcfg-bond0 as follows: (assuming ip address is 192.168.0.1)
       
[root@db03 network-scripts]# more ifcfg-bond0
DEVICE="bond0"
BROADCAST="192.168.0.255"
GATEWAY="192.168.0.254"
IPADDR="192.168.0.1"
NETMASK="255.255.255.0"
ONBOOT="yes"
USERCTL=no
BONDING_OPTS="mode=0 miimon=100 downdelay=300 updelay=300 max_bonds=4"


note: you must use capital letter for left side variables, especially BONDING_OPTS, otherwise, you might not be able to see slave devices from command output of 'ifconfig'.
Do not configure BONDING_OPTS in /etc/modprobe.d/bonding.conf as suggested by Redhat.

c. /etc/sysconfig/network-scripts/ifcfg-ethX

[root@db03 network-scripts]# more ifcfg-eth*
::::::::::::::
ifcfg-eth0
::::::::::::::
DEVICE=eth0
BOOTPROTO=none
MASTER=bond0
HWADDR=00:01:02:03:04:00
ONBOOT=yes
SLAVE=yes
USERCTL=no
::::::::::::::
ifcfg-eth1
::::::::::::::
DEVICE=eth1
BOOTPROTO=none
HWADDR=00:01:02:03:04:01
ONBOOT=no
USERCTL=no
::::::::::::::
ifcfg-eth2
::::::::::::::
DEVICE=eth2
BOOTPROTO=none
MASTER=bond0
HWADDR=00:01:02:03:04:02
ONBOOT=yes
SLAVE=yes
USERCTL=no
::::::::::::::
ifcfg-eth3
::::::::::::::
DEVICE=eth3
BOOTPROTO=none
HWADDR=00:01:02:03:04:03
ONBOOT=no
USERCTL=no
::::::::::::::
ifcfg-eth4
::::::::::::::
DEVICE="eth4"
HWADDR="00:00:00:00:00:04"
NM_CONTROLLED="no"
ONBOOT="no"
::::::::::::::
ifcfg-eth5
::::::::::::::
DEVICE="eth5"
HWADDR="00:00:00:00:00:05"
NM_CONTROLLED="no"
ONBOOT="no"

note: we connected network cables for eth0 and eth2 only, so only use these 2 ports for bonding.

d. /etc/modprobe/bonding.conf
[root@db03 network-scripts]# more /etc/modprobe.d/bonding.conf
alias eth0 e1000e
alias eth1 e1000e
alias eth2 e1000e
alias eth3 e1000e
alias eth4 bnx2x
alias eth5 bnx2x
alias scsi_hostadapter cciss
alias scsi_hostadapter1 lpfc
alias bond0 bonding


3. Testing and debug
a. more /proc/net/bonding/bond0
b. ethtool eth0



4. References
a. RHEL6 deployment guide
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s2-networkscripts-interfaces-chan.html

b. Redhat knowledge base DOC-48159 for 'How do I configure the bonding device on Red Hat Enteprise Linux 6" or search bonding in access.redhat.com knowledge base.

1 comment:

  1. Had to move cables around on a server, which meant I had to change the bonding. Now getting double entries with and MAC addresses that wont bond to correct interfaces.





    # service network restart
    Shutting down interface bond0: bonding: bond0: Removing slave eth0
    bonding: bond0: Warning: the permanent HWaddr of eth0 - 00:21:28:c0:bd:32 - is still in use by bond0. Set the HWaddr of eth0 to a different address to avoid conflicts.
    bonding: bond0: releasing backup interface eth0
    bonding: bond0: Removing slave eth4
    bonding: bond0: releasing backup interface eth4
    [ OK ]
    Shutting down interface bond1: bonding: bond1: Removing slave eth2
    bonding: bond1: Warning: the permanent HWaddr of eth2 - 00:21:28:c0:bd:34 - is still in use by bond1. Set the HWaddr of eth2 to a different address to avoid conflicts.
    bonding: bond1: releasing active interface eth2
    bonding: bond1: making interface eth5 the new active one.
    bonding: bond1: Removing slave eth5
    bonding: bond1: releasing active interface eth5
    bonding: bond1: Removing slave eth6
    bonding: bond1: releasing backup interface eth6
    [ OK ]
    Shutting down interface bond2: bonding: bond2: Removing slave eth1
    bonding: bond2: releasing backup interface eth1
    bonding: unable to remove non-existent slave eth5 for bond bond2.
    bonding: unable to remove non-existent slave eth6 for bond bond2.
    [ OK ]
    Shutting down loopback interface: [ OK ]
    Bringing up loopback interface: lo: Disabled Privacy Extensions
    [ OK ]
    Bringing up interface bond0: bonding: bond0: setting mode to active-backup (1).
    bonding: bond0: Setting MII monitoring interval to 500.
    ADDRCONF(NETDEV_UP): bond0: link is not ready
    bonding: bond0: Adding slave eth0.
    bonding: bond0: enslaving eth0 as a backup interface with a down link.
    bonding: bond0: Adding slave eth4.
    bonding: bond0: enslaving eth4 as a backup interface with a down link.
    RTNETLINK answers: File exists
    [ OK ]
    Bringing up interfacebonding: bond1: setting mode to active-backup (1).
    bond1: bonding: bond1: Setting MII monitoring interval to 500.
    ADDRCONF(NETDEV_UP): bond1: link is not ready
    bonding: bond1: Adding slave eth2.
    bonding: bond1: enslaving eth2 as a backup interface with a down link.
    bonding: bond1: Adding slave eth5.
    bonding: bond1: enslaving eth5 as a backup interface with a down link.
    bonding: bond1: Adding slave eth6.
    bonding: bond1: enslaving eth6 as a backup interface with a down link.
    igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
    bonding: bond1: link status definitely up for interface eth2.
    bonding: bond1: making interface eth2 the new active one.
    bonding: bond1: first active interface up!
    ADDRCONF(NETDEV_CHANGE): bond1: link becomes ready
    e1000e: eth5 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
    bonding: bond1: link status definitely up for interface eth5.
    [ OK ]
    Bringing up interface bond2: bonding: bond2: setting mode to active-backup (1).
    bonding: bond2: Setting MII monitoring interval to 500.
    ADDRCONF(NETDEV_UP): bond2: link is not ready
    bonding: bond2: Adding slave eth1.
    bonding: bond2: enslaving eth1 as a backup interface with a down link.
    Device eth5 has different MAC address than expected, ignoring.
    Device eth6 has different MAC address than expected, ignoring.
    RTNETLINK answers: File exists
    [ OK ]
    Bringing up interface eth3: RTNETLINK answers: File exists
    RTNETLINK answers: File exists
    RTNETLINK answers: File exists
    [ OK ]

    ReplyDelete