Jephe Wu - http://linuxtechres.blogspot.com
Environment: CentOS 5 64bit, zabbix server with very large items.
Objective: make 'latest data' with group and host set to 'all' page loading faster.
Steps:
1. make sure deflate_module is enabled in apache configuration
LoadModule deflate_module modules/mod_deflate.so
2. filtered by type
for example:
<Location />
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/x-js text/css
</Location>
Note: you can use Lynx or similar to do a headers dump on the file and it will tell you want the mime type is,
e.g.:
$ lynx -head -dump http://www.example.com/js/jquery-1.2.6.min.js
HTTP/1.1 200 OK
Date: Wed, 25 Jun 2008 03:25:53 GMT
Server: Apache/2.2.3 (CentOS)
Last-Modified: Wed, 25 Jun 2008 03:24:47 GMT
ETag: "38016-d9de-37f45dc0"
Accept-Ranges: bytes
Content-Length: 55774
Connection: close
Content-Type: application/x-javascript
<Location />
AddOutputFilterByType DEFLATE application/x-javascript
</Location>
--------------------
3. example settings
root@zabbixserver01:/var/log/httpd/ # more /etc/httpd/conf.d/deflate.conf
<IfModule mod_deflate.c>
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/x-javascript
DeflateCompressionLevel 9
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
# Don't compress images
SetEnvIfNoCase Request_URI \
\.(?:gif|jpe?g|png)$ no-gzip dont-vary
# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary
DeflateFilterNote Input instream
DeflateFilterNote Output outstream
DeflateFilterNote Ratio ratio
LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' deflate
CustomLog logs/deflate_log deflate
</IfModule>
4. References
a. Apache mod_deflate example:
http://httpd.apache.org/docs/2.0/mod/mod_deflate.html
b. http://www.linuxjournal.com/article/6802?page=0,1
c. use php zlib compress by modifying php.ini
PHP with the --with-zlib configure option and then reconfiguring the php.ini file.
Below is what the output buffer method looks like:
output_buffering = On
output_handler = ob_gzhandler
zlib.output_compression = Off
The ZLIB method uses:
output_buffering = Off
output_handler =
zlib.output_compression = On
How to setup compression for high volume zabbix server
How dbbix works with Zabbix
Jephe Wu - http://linuxtechres.blogspot.com
Environment: dbforbix running on separated CentOS 5 box, Zabbix server or zabbix proxy is on another server
Objective: use separated server running dbbix to serve database performance check and show graph on zabbix web gui
Concept:
dbforbix collects data from separated server, then send it to zabbix server(destination server was configured without proxy) or zabbix proxy(server was configured with proxy)
Steps:
1. on dbforbix, configure it as follows:
ZabbixServerList=zabbixserver01,zabbixproxy01
zabbixserver01.Address=172.16.4.1
zabbixserver01.Port=10051
zabbixproxy01.Address=172.16.4.2
zabbixproxy01.Port=10051
DatabaseList=db01,db02
db01.Url=jdbc:mysql://db01:3306/mysqldb01.User=zabbix
db01.Password=zabbix
db01.DatabaseType=mysql
db01.QueryListFile=./conf/mysql/mysqlquery.props
oradb-01.Url=jdbc:oracle:thin:@oradb-01-vip:1521:LIVEDB1
oradb-01.User=zabbix
oradb-01.Password=zabbix
oradb-01.QueryListFile=./conf/oracle/query.props
dbent.Url=jdbc:jtds:sqlserver://172.16.7.2:1987/master
dbent.User=zabbix
dbent.Password=zabbix
dbent.DatabaseType=mssql
dbent.QueryListFile=./conf/mssql/mssqlquery.props
host1.Url=jdbc:postgresql://host1:5432/databasename
host1.User=zabbix
host1.Password=zabbix
host1.DatabaseType=pgsql
host1.QueryListFile=./conf/pgsql/pgsqlquery.props
.....
2. debug if the gui is not updating data
make sure dbforbix server can telnet zabbix server or proxy at port 10051, if not, make sure proxy or server is not running iptables.
otherwise, you will get !X (man traceroute, it's administratively prohibitted indication))
use traceroute or tcptraceroute to troubleshoot.
3. installation guide:
refer to http://www.smartmarmot.com/wiki/index.php/DBforBIX
How to use kickstart to install CentOS 6.1 on non-DHCP network
Jephe Wu - http://linuxtechres.blogspot.com
Environment: HP BladeSystem Proliant BL460c G6(ip: 192.168.0.0/24, DHCP/tftp/kickstart/PXE/HTTP server is on different network: 192.168.1.0/24)
Objective: install CentOS 6.1 into bare-metal blades from
Difficulties: We are not able to boot from PXE as it's in different network.
Concepts:
We can use CentOS netinstall ISO CD to boot up blades and specify dhcp/tftp/http server as installation source and use kickstart to install it automatically.
We can further modify CentOS netinstall iso CD to add kickstart location to make it automatically installed through kickstart
Steps:
1. make customized netinstall CentOS 6.1 CD
download iso from http://mirror.optus.net/centos/6.1/isos/x86_64/CentOS-6.1-x86_64-netinstall.iso and put it /root
cd /root; mkdir c
mount CentOS-6.1-x86_64-netinstall.iso c -o loop
cp -va c c.1
cd c.1
rm -fr images
cd isolinux
vi isolinux.cfg
-------------
default linux
prompt 0
timeout 1
display boot.msg
menu background splash.jpg
menu title Welcome to CentOS 6.2!
menu color border 0 #ffffffff #00000000
menu color sel 7 #ffffffff #ff000000
menu color title 0 #ffffffff #00000000
menu color tabmsg 0 #ffffffff #00000000
menu color unsel 0 #ffffffff #00000000
menu color hotsel 0 #ff000000 #ffffffff
menu color hotkey 7 #ffffffff #ff000000
menu color scrollbar 0 #ffffffff #00000000
label linux
menu label ^Install or upgrade an existing system
menu default
kernel vmlinuz
append initrd=initrd.img ks=http://jephe.server/ks_centos_6_x86_64 text ksdevice=eth0 gateway=192.168.0.254 ip=192.168.0.1 dns=192.168.1.1 netmask=255.255.255.0 --onboot=on
label vesa
menu label Install system with ^basic video driver
kernel vmlinuz
append initrd=initrd.img xdriver=vesa nomodeset
label rescue
menu label ^Rescue installed system
kernel vmlinuz
append initrd=initrd.img rescue
label local
menu label Boot from ^local drive
localboot 0xffff
label memtest86
menu label ^Memory test
kernel memtest
append -
-------------------
cat > /root/c.1/mkiso.sh << END
mkisofs -o ../CentOS-6.1_x86_64-Kickstart-192.168.0.1.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -R -J -v -T .
END
-------------------
then, cd /root/c.1, run
./mkiso.sh
it will create the iso file under /root
2. prepare kickstart file http://jephe.server/ks_centos_6_x86_64
Here is the KS file I used:
-------------
install
url --url=http://jephe.server/packages/centos_6_x86_64
text
lang en_US
keyboard us
rootpw password
firewall --disabled
services --disabled=NetworkManager,abrtd,acpid,certmonger,cgconfig,cgred,corosync,cpuspeed,cups,dovecot,ip6tables,iptables,kdump,modclusterd,multipathd,mysql-mmm-agent,mysql-proxy,netconsole,nfs,nscd,nslcd,ntpdate,oddjobd,psacct,quota_nld,rdisc,restorecond,rpcbind,rpcgssd,rpcidmapd,rpcsvcgssd,saslauthd,smartd,snmptrapd,spamassassin,sssd,vncserver,ypbind
firstboot --disable
selinux --disabled
authconfig --enableshadow --passalgo=sha512 --enablefingerprint
timezone Australia/Sydney
bootloader --location=mbr --driveorder=sda --append="rhgb crashkernel=auto rhgb quiet"
zerombr yes
clearpart --all --initlabel
part /boot --fstype=ext3 --size=256
part swap --asprimary --size=8192
part / --fstype=ext4 --grow --asprimary --size=200
%packages --ignoremissing
@base
@client-mgmt-tools
@compat-libraries
@console-internet
@core
@debugging
@directory-client
@mail-server
@hardware-monitoring
@java-platform
@large-systems
@mysql-client
@mysql
@network-file-system-client
@performance
@perl-runtime
@server-platform
@server-policy
@storage-client-multipath
@system-admin-tools
pax
oddjob
sgpio
certmonger
pam_krb5
krb5-workstation
nscd
pam_ldap
nss-pam-ldapd
perl-DBD-MySQL
perl-DBD-SQLite
screen
vlock
lsscsi
sendmail
sendmail-cf
lrzsz
dos2unix
unix2dos
tigervnc
%pre --log=/tmp/pre.log
ETH=eth0
IP=`ifconfig $ETH | grep "inet " | cut -d : -f 2 | cut -d " " -f 1`
MAC=`ifconfig $ETH | grep HWaddr | awk -F "HWaddr " '{print $2}'`
if [ $MAC == "00:00:B3:B0:44:00" ];then
NAME=db01.domain
SHORTNAME=db01
fi
if [ $MAC == "00:00:B3:B0:43:18" ];then
NAME=db02.domain
SHORTNAME=db02
fi
echo "NETWORKING=yes" > /tmp/network.txt
echo "HOSTNAME=${NAME}" >> /tmp/network.txt
cat > /tmp/hosts.txt <<'EOF'
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
EOF
echo "$IP $NAME $SHORTNAME" >> /tmp/hosts.txt
%post --nochroot --log=/tmp/post.log
cp /tmp/network.txt /mnt/sysimage/etc/sysconfig/network
. /mnt/sysimage/etc/sysconfig/network
/mnt/sysimage/bin/hostname $HOSTNAME
cp /tmp/hosts.txt /mnt/sysimage/etc/hosts
/usr/bin/cp /tmp/pre.log /mnt/sysimage/root/pre.log
/usr/bin/cp /tmp/post.log /mnt/sysimage/root/post.log
# if you use rollout for server configuration deployment. ( search 'rollout google' in google)
echo "Kickstarted `/usr/bin/date`" > /mnt/sysimage/root/kickstart_report
wget -O /mnt/sysimage/usr/local/sbin/rollout http://server.domain/rollout
sed -i -e 's#http://server.domain#http://rollout.domain#' /mnt/sysimage/usr/local/sbin/rollout
chmod +x /mnt/sysimage/usr/local/sbin/rollout
echo "Kickstart complete" >> /mnt/sysimage/root/kickstart_report
-------------
3. use HP ILO to boot from above-made CentOS netinstall ISO CD.
it will use kernel and initrd files from customized netinstall iso CD to boot up machine, and install packages from approved specified kickstart file
http://jephe.server/packages/centos_6_x86_64
For configuration files under pxelinux.cfg directory, you can use something like this:
append initrd=centos_6_x86_64/initrd.img ramdisk_size=8192 ks=http://kickstart.jephe/ks_centos_6_x86_64 ip=dhcp ksdevice=eth0
4. configuring bonding device
[root@db01 network-scripts]# more /etc/modprobe.d/bonding.conf
alias eth0 e1000e
alias eth1 e1000e
alias eth2 e1000e
alias eth3 e1000e
#alias eth4 bnx2x
#alias eth5 bnx2x
alias scsi_hostadapter lpfc
alias bond0 bonding
ifcfg-bond0
::::::::::::::
DEVICE=bond0
GATEWAY=192.168.0.254
IPADDR=192.168.0.1
IPV6INIT=no
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
BONDING_OPTS="mode=1 miimon=100"
::::::::::::::
ifcfg-eth0
::::::::::::::
DEVICE=eth0
IPV6INIT=no
MASTER=bond0
MTU=1500
ONBOOT=no
PEERDNS=yes
SLAVE=yes
TYPE=Ethernet
USERCTL=no
::::::::::::::
ifcfg-eth1
::::::::::::::
# Intel Corporation 82571EB Quad Port Gigabit Mezzanine Adapter
DEVICE=eth1
HWADDR=00:00:00:B0:44:01
ONBOOT=no
HOTPLUG=no
::::::::::::::
ifcfg-eth2
::::::::::::::
DEVICE=eth2
IPV6INIT=no
MASTER=bond0
MTU=1500
ONBOOT=no
PEERDNS=yes
SLAVE=yes
TYPE=Ethernet
USERCTL=no
::::::::::::::
ifcfg-eth3
::::::::::::::
# Intel Corporation 82571EB Quad Port Gigabit Mezzanine Adapter
DEVICE=eth3
HWADDR=00:00:00:B0:44:03
ONBOOT=no
HOTPLUG=no
::::::::::::::
::::::::::::::
ifcfg-eth4
::::::::::::::
# Broadcom Corporation NetXtreme II BCM57711E 10-Gigabit PCIe
DEVICE=eth4
HWADDR=F4:CE:46:7E:00:00
ONBOOT=no
HOTPLUG=no
::::::::::::::
ifcfg-eth5
::::::::::::::
# Broadcom Corporation NetXtreme II BCM57711E 10-Gigabit PCIe
DEVICE=eth5
HWADDR=F4:CE:46:7E:00:01
ONBOOT=no
HOTPLUG=no
::::::::::::::
ifcfg-lo
::::::::::::::
DEVICE=lo
IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
# If you're having problems with gated making 127.0.0.0/8 a martian,
# you can change this to something else (255.255.255.255, for example)
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback
Note: prior to CentOS 6, you may add bonding options as follows in /etc/modprobe.d/bonding.conf:
options bonding mode=1 miimon=100
How to setup MMM control under CentOS 6.1
Jephe Wu - http://linuxtechres.blogspot.com
Environment: CentOs 6.1, MMM 2.2.1, MySQL 5.1.52
Objective: setup MMM among the databases to gain HA and load balance for read access
Steps:
1. install required packages for mmm agent on database servers
yum -y install perl-Config-IniFiles
yum -y install mysql-mmm mysql-mmm-agent mysql-mmm-tools mysql-proxy
2. install required packages for mmm monitor on monitoring host
yum -y install mysql-mmm-monitor
chkconfig mysql-mmm-agent on
chkconfig mysql-mmm-monitor on
3. preparing ips for databases
db1: 172.17.7.1 master
db2: 172.17.7.2 master
db3: 172.17.7.3 slave
master exclusive ip: 172.17.7.4
slave balanced ip addresses: 172.17.7.5, 172.17.7.6, 172.17.7.7
4. setup /etc/mysql-mmm/mmm_agent.conf to change 'this' line to db1/db2 or db3
5. setup /etc/mysql-mmm/mmm_common.conf for all monitor and agents
# more /etc/mysql-mmm/mmm_common.conf
active_master_role writer
<host default>
cluster_interface bond0
pid_path /var/run/mysql-mmm/mmm_agentd.pid
bin_path /usr/libexec/mysql-mmm/
replication_user replication
replication_password replication_password
agent_user mmm_agent
agent_password agent_password
</host>
<host db1>
ip 172.17.7.1
mode master
peer db2
</host>
<host db2>
ip 172.17.7.2
mode master
peer db1
</host>
<host db3>
ip 172.17.7.3
mode slave
</host>
<role writer>
hosts db1, db2
ips 172.17.7.4
mode exclusive
</role>
<role reader>
hosts db1, db2, db3
ips 172.17.7.5, 172.17.7.6, 172.17.7.7
mode balanced
</role>
6. start up agents on database servers first
on db1,db2 then db3, to start up mmm agent: /etc/init.d/mysql-mmm-agent start
7. start up monitor process on monitoring host
on mon01, to start up: /etc/init.d/mysql-mmm-monitor start
8. bring up them online from mon01
mon01# mmm_control set_online db1
mon01# mmm_control set_online db2
mon01# mmm_control set_online db3
root@mon01:/etc/mysql-proxy/ # mmm_control show
db1(172.17.7.1) slave/REPLICATION_DELAY. Roles:
db2(172.17.7.2) master/ONLINE. Roles: reader(172.17.7.5), reader(172.17.7.6)
db3(172.17.7.3) master/ONLINE. Roles: reader(172.17.7.7), writer(172.17.7.4)
9. FAQ
a. read_only (on/off) from show variables comand in mysql
if db2 and db3 are writer, when writer role goes to db3, mmm will set read_only flag for db2 to prevent any writing process, but only for normal user, not root.
Posts
