Keep both Zabbix 1.8 and 2.2 running for real world testing

Jephe Wu - http://linuxtechres.blogspot.com

Objective: keep both Zabbix 1.8 and 2.2 system running for same monitored hosts for real world testing.
Challenge: make zabbix 2.2 proxies to be able to monitor same set of agent hosts without adding new proxies into /etc/zabbix/zabbix_agentd.conf
Environment: CentOS 6.4 64bit, Zabbix 1.8(old system) and Zabbix 2.2(new system), Percona server 5.5(both old and new), Zabbix proxy 1.8(old system) and Zabbix proxy 2.2(new system)

Zabbix monitoring network is at 192.168.0.0/24, monitored hosts are at different network, we use about 10 zabbix proxies for monitoring. All agent hosts are configured to use these 10 zabbix proxies for monitoring.


Network diagram


Steps:
1.  Preparing Zabbix 2.2 system server/VM first 

We can prepare Zabbix 2.2 web server VM, zabbix 2.2 server VM and all zabbix 2.2 Proxies first.

2. Online clone Percona server 5.5 db01 to db02
refer to http://linuxtechres.blogspot.com.au/2014/02/preparing-mysql-slave-database-by.html

3.  Enable ip_forward and iptables NAT on existing proxies
For proxy servers prox01,prox02,....prox10, run:

echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -p tcp -o eth0 -j SNAT --to 192.168.0.100

Note: this will enable ip forward for any traffic received from eth0, then the destination is not for prox01 itself

4.  on new proxies, enable default gateway to old proxies
on each proxy server 2.2, change default gateway to coresponding old 1.8 proxy server IP address 

5.  on each new proxy server, configure hostname as old proxy name
e.g. still use hostname=prox01 on prox01new

This way, new proxy will get data through old proxy server iptables IP Masquerading.

6. go to new Zabbix 2.2 web GUI to modify SMTP server setting to avoid sending double alert together with old 1.8 Zabbix system.

How to do migration to 2.2 after testing


1.  shutdown each proxy, bring up new coresponding proxy with same IP address
2.  remove default gateway, use normal default gateway instead.