Jephe Wu - http://linuxtechres.blogspot.com
Objective: understanding snmp, snmptrap under zabbix
Note:For zabbix, –with-net-snmp zabbix compiling option is not needed if you only want to receive traps. if you also want to poll snmp, then it is required.
part I - snmp monitoring - snmpget, snmpwalk
1. snmp items with zabbix
refer to http://www.zabbix.com/wiki/howto/monitor/snmp/snmp?s[]=snmp&s[]=item
2. get snmp item name
# snmpwalk snmpserver -v1 -c public
# snmpwalk -c public -v 2c ipaddress
root@snmp:~/ # snmpget -c public -v 1 172.16.62.3 .1.3.6.1.4.1.9.9.171.1.2.1.1.0
SNMPv2-SMI::enterprises.9.9.171.1.2.1.1.0 = Gauge32: 9
Note: 1.3.6.1.4.1.9.9.171.1.2.1.1 is a cisco ipsec mib at http://www.oidview.com/mibs/9/CISCO-IPSEC-FLOW-MONITOR-MIB.html, refer to cikeGlobalActiveTunnels
Refer to free snmp MIB library at http://www.oidview.com/mibs/detail.html
and http://www.iana.org/assignments/enterprise-numbers
Part II - snmptrap
1. snmptrapd.conf
root@snmptrap:/etc/snmp/ # chkconfig --list snmptrapd
snmptrapd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
root@snmptrap:/etc/snmp/ # /etc/init.d/snmptrapd status
snmptrapd (pid 27227) is running...
root@snmptrap:/etc/snmp/ # more snmptrapd.conf
authCommunity execute public
authCommunity execute PUBLIC
disableAuthorization yes
#log syslog
traphandle default /usr/bin/zabbix_snmptraphandler.pl
traphandle default /usr/bin/perl /usr/bin/traptoemail -s jephe.mailserver.com -f snmptrap@snmptrap.domain jwu@domain.com
Note: if community string is not public, please change the first/second line.
refer to http://www.zabbix.com/wiki/howto/monitor/snmp/snmp_traps_-_a_new_solution for traphandle perl script
2. test it from snmptrap server itself, you should receive email.
# snmptrap -Ci -v 2c -c public localhost "" "NET-SNMP-MIB::netSnmpExperimental" NET-SNMP-MIB:netSnmpExperimental s "test"
root@snmptrap:/etc/snmp/ # which snmptrap
/usr/bin/snmptrap
root@snmptrap:/etc/snmp/ # rpm -qf /usr/bin/snmptrap
net-snmp-utils-5.3.2.2-14.el5_7.1
Note: you might want to look at SNMPTT at http://snmptt.sourceforge.net/
3. sample zabbix templates for snmp devices
http://www.zabbix.com/wiki/templates/cisco_2960
Part III - References
http://www.zabbix.com/wiki/howto/monitor/snmp/a_simple_snmp_trap_handler
http://www.oidview.com/mibs/detail.html
http://www.iana.org/assignments/enterprise-numbers
http://ireasoning.com/mibbrowser.shtml (better to use for snmp walk)
http://www.manageengine.com/products/mibbrowser-free-tool/download.html (absolutely free, for snmpwalk, put .iso.org.dod.internet.mgmt in the Object ID column, then click 'snmpwalk' button)
http://www.youtube.com/watch?v=TiBrB0qxQao
http://www.youtube.com/watch?v=J0fPuXZySXE&feature=related
http://www.youtube.com/watch?v=biH-UAPxo_0&feature=related
Setup snmp and snmptrap monitoring under Zabbix
Disable the excessive logging of snmpd for RHEL 5
Jephe Wu - http://linuxtechres.blogspot.com
Problem: on RHEL 5 server, /var/log/messages logs too many logs such as the following:
snmpd[1901]: Received SNMP packet(s) from UDP: [127.0.0.1]:50736
snmpd[1901]: Connection from UDP: [127.0.0.1]:50736
Objective: suppress above logs.
Environment: RHEL 5.1
Concept: in order to suppress above logs, you need to remove -a (in /etc/sysconfig/snmpd.options) for snmpd sysconfig options (to suppress 'Received SNMP packets' line) and use dontLogTCPWrappersConnects option (only available since net-snmp 5.3.2.2 which in RHEL 5.3 update) in /etc/snmpd.conf (to suppress 'Connection from UDP' line)
You can run 'man snmpd' to search dontLogTCPWrappersConnects, you can only find in net-snmp 5.3.2.2 version.
Steps:
1. Firstly, you need to upgrade net-snmp rpm to the 5.3.2.2 which is included in RHEL 5.3 according to the RHEL 5.3 release notes at http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/pdf/Release_Notes.pdf (search 'snmp trap' or 'dontLogTCPWrappersConnects', you will find it says:
Net-SNMP has been re-based to upstream version 5.3.2.2. This update adds Stream Control Transmission Protocol (SCTP) support and introduces two new configuration options ( to be used in /etc/snmpd.conf)
- dontLogTCPWrappersConnects - suppresses logging of connection attempts
- v1trapaddress - enables administrators to set an agent's IP address inside outgoing SNMP traps
yum install net-snmp (assuming you registered redhat subscription for yum update, it will update 3 packages: net-snmp, net-snmp-utils, net-snmp-libs)
2. After upgrading net-snmp, make sure /etc/sysconfig/snmpd.options contains the following options:
OPTIONS="-Lsd -Lf /dev/null -p /var/run/snmpd.pid"
-a means "Log the source addresses of incoming requests" according to snmpd man page
3. edit /etc/snmp/snmpd.conf to include this line, check man page of snmpd for the meaning:
dontLogTCPWrappersConnects 1
4. restart snmpd daemon
service snmpd restart
5. check again
ps ax | grep snmpd
you should see something like this:
/usr/sbin/snmpd -Lsd -Lf /dev/null -p /var/run/snmpd.pid
6. check /var/log/messages to confirm the reduced log output
References:
a. access.redhat.com knowledge base search 'DOC-3466' and 'DOC-25144' for
How to disable the excessive logging of snmpd in Red Hat Enterprise Linux System?
and
How do I reduce the verbosity of SNMP logging?
Labels: snmp
Installing hardware monitoring system for HP Proliant servers
Jephe Wu - http://linuxtechres.blogspot.com
Environment: HP Proliant server with CentOS 5.3 x86_64 OS
Objective: Enable hardware monitoring for the server
Steps summary only for sending out SNMP trap
1. configure sendmail to be able to send out email through mailrelay host
2. configure ILO as below:
| SNMP/Insight Manager Settings |
| Configure and Test SNMP Alerts |
| SNMP Alert Destination(s): | |
| iLO 2 SNMP Alerts: | Enabled Disabled |
| Forward Insight Manager Agent SNMP Alerts: | Enabled Disabled |
| SNMP Pass-thru: | Enabled Disabled |
give snmp trap server for alert destination. and enable all of above options. Then you don't have to configure snmp trap setting in /etc/snmp/snmp.conf
Part I:
Steps: after using NFS method to install OS through ILO virtual media, then vi /etc/hosts to put the server real IP address as hostname as follows:
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
10.0.0.1 logserver.domain.com logserver
1. download the Proliant Support Package file from HP website for your server model such as Proliant DL360G6 x86_64
2. untar the file
3. change /etc/redhat-release file content from 'CentOS release 5.3 (Final)' to 'Red Hat Enterprise Linux Server release 5.3'
4. run ./install.sh to install it
5. point your browser to http://serverip:2301/, use your root user password to login
6. vi /opt/hp/hp-snmp-agents/cma.conf trapemail line
trapemail /bin/mail -s 'logserver HP Insight Management Agents Trap Alarm' root
note: this is for sending out email notification for any snmp trap, but if I reboot server, it won't send out any email according to my test. Also, please remember to configure /etc/aliases to enable root user email aliases
7. add the following to /etc/snmp/snmpd.conf to enable insight management agents to talk to snmp trap server. If configured this, the point 6 is basically not needed actually.
trapcommunity public
trapsink snmptrapserveripaddressorhostname
Part II: setup another server as SNMP trap server
1. setup snmp trap server, make /etc/snmp/snmptrapd.conf as follows:
[root@jephe snmp]# more snmptrapd.conf
traphandle default /usr/bin/perl /usr/bin/traptoemail -s localhost -f hpsnmptrap@domain.com jephe.wu@domain1.com
traphandle default /usr/bin/perl /usr/bin/traptoemail -s localhost -f hpsnmptrap@domain.com anotheruser@domain1.com
2. start up snmptrapd services and make it automatic for next restart, that's it.
note: You can configure to send out email as well from snmp trap server after generating snmp trap, just use above Part I step 6 to configure it.
note:
1. to start the HP Array Configuration Utility, go to /opt/compaq/cpqacuxe/bld/, run cpqacuxe , then you can see the utility option in system management homepape at https://serverip:2381/ , but you can only use the https://127.0.0.1:2381 to use the utility for security reason. So you might need to ssh into the server with -X option, then run firefox to open browser locally.
2. for configuring 'Accept SNMP packets from these hosts', the host list should be separated with semi-column ;
FAQ
1. when using PSP 8.0.0, the cma.log filling up, you will see a lot of messages like this:
netsnmp_assert 1 == _access_interface_init failed if-mib/data_access/interface.c:151 netsnmp_access_interface_container_load()
netsnmp_assert 1 == _access_interface_init failed if-mib/data_access/interface.c:199 netsnmp_access_interface_index_find()
netsnmp_assert 1 == _access_interface_init failed if-mib/data_access/interface.c:199 netsnmp_access_interface_index_find()
netsnmp_assert 1 == _access_interface_init failed if-mib/data_access/interface.c:279 netsnmp_access_interface_entry_create()
Solution=>
1)/opt/compaq/nic/etc/cmanicd stop
2)Add 'cmanicd' to the exclude line in the /opt/compaq/cma.conf so that it won't run after next reboot/shutdown.
how to setup hardware montoring for HP Proliant server under RHEL 5
note: this article is obsolete, please refer to http://linuxtechres.blogspot.com/2009/09/installing-hardware-monitoring-system.html
Objective: After installing RHEl 5 on HP Proliant server, whenever there's hardware failure, especially hard disk failure and server operating system restart, we will be notified by email.
Steps:
- install RHEL 5.1 remotely and disklessly, pls refer to another article - http://linuxtechres.blogspot.com/2009/01/how-to-use-hp-ilo-to-do-nfs-diskless.html
- download HP Proliant system management RPM packages HPSMH(system management homepage) and HPASM(HP system health application and insight management agents) from HP website
- rpm -ivh hpasm* and hpsmh*
- hpasm activate
- setup snmp trap server, make /etc/snmp/snmptrapd.conf as follows:
traphandle default /usr/bin/perl /usr/bin/traptoemail -s localhost -f hpsnmptrap@domain.com jephe.wu@domain1.com
traphandle default /usr/bin/perl /usr/bin/traptoemail -s localhost -f hpsnmptrap@domain.com jephe.wu@domain2.com
- start up snmptrapd services and make it automatic for next restart
You can configure to send out email as well after generating snmp trap:
- go to /opt/compaq to edit cma.conf trapemail line
- that's it.