Showing posts with label snmp. Show all posts
Showing posts with label snmp. Show all posts

Setup snmp and snmptrap monitoring under Zabbix


Jephe Wu - http://linuxtechres.blogspot.com

Objective: understanding snmp, snmptrap under zabbix

Note:For zabbix, –with-net-snmp zabbix compiling option is not needed if you only want to receive traps. if you also want to poll snmp, then it is required.


part I - snmp monitoring - snmpget, snmpwalk

1. snmp items with zabbix
refer to http://www.zabbix.com/wiki/howto/monitor/snmp/snmp?s[]=snmp&s[]=item


2. get snmp item name
# snmpwalk snmpserver -v1 -c public
# snmpwalk -c public -v 2c ipaddress


root@snmp:~/ # snmpget -c public -v 1 172.16.62.3 .1.3.6.1.4.1.9.9.171.1.2.1.1.0
SNMPv2-SMI::enterprises.9.9.171.1.2.1.1.0 = Gauge32: 9

Note: 1.3.6.1.4.1.9.9.171.1.2.1.1 is a cisco ipsec mib at http://www.oidview.com/mibs/9/CISCO-IPSEC-FLOW-MONITOR-MIB.html, refer to cikeGlobalActiveTunnels


Refer to free snmp MIB library at http://www.oidview.com/mibs/detail.html
and http://www.iana.org/assignments/enterprise-numbers

Part II - snmptrap


1. snmptrapd.conf 
root@snmptrap:/etc/snmp/ # chkconfig --list snmptrapd
snmptrapd       0:off 1:off 2:on 3:on 4:on 5:on 6:off


root@snmptrap:/etc/snmp/ # /etc/init.d/snmptrapd status
snmptrapd (pid  27227) is running...


root@snmptrap:/etc/snmp/ # more snmptrapd.conf 
authCommunity execute public
authCommunity execute PUBLIC
disableAuthorization yes
#log syslog
traphandle default /usr/bin/zabbix_snmptraphandler.pl
traphandle default /usr/bin/perl /usr/bin/traptoemail -s jephe.mailserver.com -f snmptrap@snmptrap.domain jwu@domain.com


Note: if community string is not public, please change the first/second line.
refer to http://www.zabbix.com/wiki/howto/monitor/snmp/snmp_traps_-_a_new_solution for traphandle perl script




2. test it from snmptrap server itself, you should receive email.
# snmptrap -Ci -v 2c -c public localhost "" "NET-SNMP-MIB::netSnmpExperimental" NET-SNMP-MIB:netSnmpExperimental s "test"


root@snmptrap:/etc/snmp/ # which snmptrap
/usr/bin/snmptrap
root@snmptrap:/etc/snmp/ # rpm -qf /usr/bin/snmptrap
net-snmp-utils-5.3.2.2-14.el5_7.1


Note: you might want to look at SNMPTT at http://snmptt.sourceforge.net/ 




3. sample zabbix templates for snmp devices


http://www.zabbix.com/wiki/templates/cisco_2960



Part III - References
http://www.zabbix.com/wiki/howto/monitor/snmp/a_simple_snmp_trap_handler
http://www.oidview.com/mibs/detail.html
http://www.iana.org/assignments/enterprise-numbers

http://ireasoning.com/mibbrowser.shtml  (better to use for snmp walk)
http://www.manageengine.com/products/mibbrowser-free-tool/download.html  (absolutely free, for snmpwalk, put .iso.org.dod.internet.mgmt in the Object ID column, then click 'snmpwalk' button)

http://www.youtube.com/watch?v=TiBrB0qxQao
http://www.youtube.com/watch?v=J0fPuXZySXE&feature=related
http://www.youtube.com/watch?v=biH-UAPxo_0&feature=related


Disable the excessive logging of snmpd for RHEL 5

Jephe Wu - http://linuxtechres.blogspot.com

Problem: on RHEL 5 server, /var/log/messages logs too many logs such as the following:

snmpd[1901]: Received SNMP packet(s) from UDP: [127.0.0.1]:50736
snmpd[1901]: Connection from UDP: [127.0.0.1]:50736



Objective: suppress above logs.
Environment: RHEL 5.1

Concept: in order to suppress above logs, you need to remove -a (in /etc/sysconfig/snmpd.options) for snmpd sysconfig options (to suppress 'Received SNMP packets' line) and use dontLogTCPWrappersConnects option (only available since net-snmp 5.3.2.2 which in RHEL 5.3 update) in /etc/snmpd.conf (to suppress 'Connection from UDP' line)

You can run 'man snmpd' to search dontLogTCPWrappersConnects, you can only find in net-snmp 5.3.2.2 version.


Steps:
1. Firstly, you need to upgrade net-snmp rpm to the 5.3.2.2 which is included in RHEL 5.3 according to the RHEL 5.3 release notes at http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/pdf/Release_Notes.pdf  (search 'snmp trap' or 'dontLogTCPWrappersConnects', you will find it says:

Net-SNMP has been re-based to upstream version 5.3.2.2. This update adds Stream Control Transmission Protocol (SCTP) support and introduces two new configuration options ( to be used in /etc/snmpd.conf)

  • dontLogTCPWrappersConnects - suppresses logging of connection attempts
  • v1trapaddress - enables administrators to set an agent's IP address inside outgoing SNMP traps
 In order to upgrade to RHEL 5.3 update so that you will have version 5.3.2.2 of net-snmp, you can do:
yum install net-snmp   (assuming you registered redhat subscription for yum update, it will update 3 packages: net-snmp, net-snmp-utils, net-snmp-libs)


2. After upgrading net-snmp, make sure /etc/sysconfig/snmpd.options contains the following options:

OPTIONS="-Lsd -Lf /dev/null -p /var/run/snmpd.pid"

-a means "Log the source addresses of incoming requests" according to snmpd man page

3. edit /etc/snmp/snmpd.conf to include this line, check man page of snmpd for the meaning:
dontLogTCPWrappersConnects 1

4. restart snmpd daemon
service snmpd restart

5. check again
ps ax | grep snmpd 

you should see something like this:
/usr/sbin/snmpd -Lsd -Lf /dev/null -p /var/run/snmpd.pid 

6. check /var/log/messages to confirm the reduced log output

References:

a. access.redhat.com knowledge base search 'DOC-3466' and 'DOC-25144' for
How to disable the excessive logging of snmpd in Red Hat Enterprise Linux System?
and
How do I reduce the verbosity of SNMP logging?

Installing hardware monitoring system for HP Proliant servers

Jephe Wu - http://linuxtechres.blogspot.com

Environment: HP Proliant server with CentOS 5.3 x86_64 OS
Objective: Enable hardware monitoring for the server


Steps summary only for sending out SNMP trap
1. configure sendmail to be able to send out email through mailrelay host
2. configure ILO as below:

SNMP/Insight Manager Settings
Configure and Test SNMP Alerts

SNMP Alert Destination(s): 
iLO 2 SNMP Alerts:  Enabled Disabled
Forward Insight Manager Agent SNMP Alerts:  Enabled Disabled
SNMP Pass-thru:  Enabled Disabled

give snmp trap server for alert destination. and enable all of above options. Then you don't have to configure snmp trap setting in /etc/snmp/snmp.conf

Part I:
Steps: after using NFS method to install OS through ILO virtual media, then vi /etc/hosts to put the server real IP address as hostname as follows:

127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
10.0.0.1 logserver.domain.com logserver

1. download the Proliant Support Package file from HP website for your server model such as Proliant DL360G6 x86_64
2. untar the file
3. change /etc/redhat-release file content from 'CentOS release 5.3 (Final)' to 'Red Hat Enterprise Linux Server release 5.3'
4. run ./install.sh to install it
5. point your browser to http://serverip:2301/, use your root user password to login
6. vi /opt/hp/hp-snmp-agents/cma.conf trapemail line
trapemail /bin/mail -s 'logserver HP Insight Management Agents Trap Alarm' root


note: this is for sending out email notification for any snmp trap, but if I reboot server, it won't send out any email according to my test. Also, please remember to configure /etc/aliases to enable root user email aliases
7. add the following to /etc/snmp/snmpd.conf to enable insight management agents to talk to snmp trap server. If configured this, the point 6 is basically not needed actually.

trapcommunity public
trapsink snmptrapserveripaddressorhostname



Part II: setup another server as SNMP trap server
1. setup snmp trap server, make /etc/snmp/snmptrapd.conf as follows:

[root@jephe snmp]# more snmptrapd.conf
traphandle default /usr/bin/perl /usr/bin/traptoemail -s localhost -f hpsnmptrap@domain.com jephe.wu@domain1.com
traphandle default /usr/bin/perl /usr/bin/traptoemail -s localhost -f hpsnmptrap@domain.com anotheruser@domain1.com

2. start up snmptrapd services and make it automatic for next restart, that's it.

note: You can configure to send out email as well from snmp trap server after generating snmp trap, just use above Part I step 6 to configure it.

note:
1. to start the HP Array Configuration Utility, go to /opt/compaq/cpqacuxe/bld/, run cpqacuxe , then you can see the utility option in system management homepape at https://serverip:2381/ , but you can only use the https://127.0.0.1:2381 to use the utility for security reason. So you might need to ssh into the server with -X option, then run firefox to open browser locally.
 2. for configuring 'Accept SNMP packets from these hosts', the host list should be separated with semi-column ;

FAQ
1. when using PSP 8.0.0, the cma.log filling up, you will see a lot of messages like this:

netsnmp_assert 1 == _access_interface_init failed if-mib/data_access/interface.c:151 netsnmp_access_interface_container_load()
netsnmp_assert 1 == _access_interface_init failed if-mib/data_access/interface.c:199 netsnmp_access_interface_index_find()
netsnmp_assert 1 == _access_interface_init failed if-mib/data_access/interface.c:199 netsnmp_access_interface_index_find()
netsnmp_assert 1 == _access_interface_init failed if-mib/data_access/interface.c:279 netsnmp_access_interface_entry_create()


Solution=> 
1)/opt/compaq/nic/etc/cmanicd stop
2)Add 'cmanicd' to the exclude line in the /opt/compaq/cma.conf so that it won't run after next reboot/shutdown.






how to setup hardware montoring for HP Proliant server under RHEL 5

note: this article is obsolete, please refer to http://linuxtechres.blogspot.com/2009/09/installing-hardware-monitoring-system.html

Objective: After installing RHEl 5 on HP Proliant server, whenever there's hardware failure, especially hard disk failure and server operating system restart, we will be notified by email.


Steps:

to follow the prompt from screen, and give the default answer except for SNMP trap server IP address and community string

  • setup snmp trap server, make /etc/snmp/snmptrapd.conf as follows:
[root@jephe snmp]# more snmptrapd.conf
traphandle default /usr/bin/perl /usr/bin/traptoemail -s localhost -f hpsnmptrap@domain.com jephe.wu@domain1.com
traphandle default /usr/bin/perl /usr/bin/traptoemail -s localhost -f hpsnmptrap@domain.com jephe.wu@domain2.com

  • start up snmptrapd services and make it automatic for next restart

You can configure to send out email as well after generating snmp trap:
  • go to /opt/compaq to edit cma.conf trapemail line
trapemail /bin/mail -s 'server1 HP Insight Management Agents Trap Alarm' root
  • that's it.