How to build Oracle VM Manager 3.1.1


Jephe Wu - http://linuxtechres.blogspot.com

Environment: Dell R610 server, Oracle Linux 6.3, Oracle VM Manager 3.1.1 with patch 399, Oracle database enterprise edition 11.2.0.3 with the latest patchset, Ksplice
Objective: install a fully patched Oracle VM Manager 3.1.1 production host


Steps:
1. BIOS config
Enable logical processor(hyper trending), virtualization, Turbo mode, disable C1E, C states, node interleaving(enable NUMA)

 C-state
In Oracle VM server 3.1.1, to check if C-state is enabled or not, use the following command to check if the total C-state is not more than 2.

# xenpm get-cpuidle-states | grep total | uniq
total C-states : 2

or in Linux, check it :
C-state: cat /proc/acpi/processor/CPU*/power
P-state: cat /sys/devices/system/cpu/cpu*/cpufreq/stats/total_tran
         cat /sys/devices/system/cpu/cpu*/cpufreq/stats/time_in_state

NUMA
# numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
node 0 size: 49139 MB
node 0 free: 33832 MB
node distances:
node 0
0: 10

How to disable NUMA in Linux OS and Oracle:
pass numa=off to kernel boot parameter in grub.conf, you can check the current kernel parameter in /proc/cmdline.
According to MOS: Enable Oracle NUMA support with Oracle Server Version 11.2.0.1 [ID 864633.1]:  Starting with 11.2.0.1, Oracle NUMA support is disabled by default.
example output: NUMA is disabled.

[root@util01 ~]# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                24
On-line CPU(s) list:   0-23
Thread(s) per core:    2
Core(s) per socket:    6
CPU socket(s):         2
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 44
Stepping:              2
CPU MHz:               2660.346
BogoMIPS:              5319.79
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              12288K
NUMA node0 CPU(s):     0-23

[root@util01 ~]# numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
node 0 size: 49139 MB
node 0 free: 40698 MB
node distances:
node   0 
  0:  10 

power mode - use maximum performance mode
References:
http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.0.pdf  check general bios settings and power management settings part.

2.  Configuring iDRAC ip address, netmask and gateway

After you have done this part, you are able to change it through DRAC and dell openmanage for Linux later on. The default DRAC username and password is root/calvin.

How to reset system by sshing into DRAC?
# racadm serveraction hardreset


3.  OS installation through kickstart by foreman

Preparing tftp/dhcp/[dns] server first by PXELinux, DNSmasq or cobber for foreman, we use traditional PXELinux here.
service tftp
{
    socket_type     = dgram
    protocol        = udp
    wait            = yes
    user            = root
    server          = /usr/sbin/in.tftpd
    server_args     = -vvv -s /tftpboot
    disable         = no
    per_source      = 11
    cps             = 100 2
    flags           = IPv4
}


under /tftpboot/pxelinux.cfg, put the following kernel files:

-rw-r--r-- 1 root root 30481341 Aug 20 15:27 OL-6.3-x86_64-initrd.img
-rw-r--r-- 1 root root  3986992 Aug 20 15:27 OL-6.3-x86_64-vmlinuz

# more /tftpboot/pxelinux.cfg/default
default linux
label linux
localboot 0  (important, boot into local hard disk if tftp failed, although DHCP is okay)

Preparing foreman
# more /etc/foreman/settings.yaml
:modulepath: /etc/puppet/modules/
:tftppath: /tftpboot/
:unattended: true

Preparing foreman-proxy

# cat /etc/foreman-proxy/settings.yml| grep -v ^# | grep -v ^$
:daemon: true
:daemon_pid: /var/run/foreman-proxy/foreman-proxy.pid
:port: 8443
:tftp: true
:tftproot: /tftpboot
:tftp_servername: 192.168.0.2
:dns: false
:dhcp: true
:dhcp_vendor: isc
:dhcp_config: /etc/dhcp/dhcpd.conf
:dhcp_leases: /var/lib/dhcpd/dhcpd.leases
:puppetca: true
:puppet: true
:log_file: /tmp/proxy.log
:log_level: DEBUG

Preparing DHCP server

#  more /etc/dhcp/dhcpd.conf
allow booting;
allow bootp;
omapi-port 7911;
deny unknown-clients;


subnet 192.168.0.0 netmask 255.255.255.0 {
  option broadcast-address 192.168.0.255;
  option routers 192.168.0.1;
  option domain-name "jephe";
    option domain-name-servers 192.168.0.254;
  next-server 192.168.0.2;
  filename "pxelinux.0";

}

After adding util01.jephe by foreman, it appears in /var/lib/dhcpd/dhcpd.leases as follows:

server-duid "xxxxxx";

host util01.jephe {
  dynamic;
  hardware ethernet d4:be:d9:ef:aa:bb;
  fixed-address 192.168.0.78;
        supersede server.filename = "pxelinux.0";
        supersede server.next-server = aa:bb:cc:dd; #actually, it's hex code of tftp server ip addr in /etc/foreman-proxy/settings.yml
        supersede host-name = "util01.jephe";
}


access DeLL iDRAC at https://dracipaddress , system, properties, EMBEDDED NIC MAC ADDRESSES to get onboard nic1 MAC address
so that you can configure foreman.

4. foreman configuration in GUI interface
kickstart config for Oracle Linux 6.3:
install
lang en_US
keyboard us
skipx
text
reboot
url --url http://repo.jephe/OL/6.3/os/x86_64/
rootpw --iscrypted $6$ruyiAJ82a/9Bazd7$zspMbTz4e7omVlb12uTa.bjuvc8SuxcGoVHgJv3Y3r1m/4.7tONDUzPVpJpYWA85uXaaaaaaaaaaaaa
firewall --disabled
selinux --disabled
authconfig --enableshadow --enablemd5
timezone --utc Australia/Sydney
bootloader --location=mbr
<%= @host.diskLayout %>
%packages
@base
-iptables
ntp
lrzsz
telnet
hwloc
numactl
# Oracle recommended tigervnc instead of its own one
tigervnc
oracle-rdbms-server-11gR2-preinstall

%post --log=/root/kickstart-post.log
echo "Syncing system date and update bios date"
ntpdate puppet.jephe
clock --systohc

# Disable most things. Puppet will activate these if required.
echo "Disabling various system services..."
<% %w{autofs gpm atd abrt-ccpp abrt-oops abrtd acpid cpuspeed rpcbind rpcgssd rpcidmapd sendmail cups iptables ip6tables auditd arptables_jf xfs pcmcia isdn rawdevices hpoj bluetooth

openibd avahi-daemon avahi-dnsconfd hidd hplip pcscd restorecond mcstrans rhnsd yum-updatesd}. each do |service| -%>
  /sbin/chkconfig --level 345 <%= service %> off 2>/dev/null
<% end -%>
     
echo "Running puppet..."
#/usr/sbin/puppetd --debug --no-daemonize --verbose --onetime
sync

# Force the right hosts file
echo "generating correct /etc/hosts"
IP=`ifconfig em1 | grep "inet " | cut -d : -f 2 | cut -d " " -f 1`
cat > /etc/hosts <<EOF
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1     localhost.localdomain localhost
::1           localhost6.localdomain6 localhost6
`echo -n $IP`         $(hostname)  $(hostname -s)
EOF

# generate dell openmanage script
cat > /root/dell_openmanage.sh <<END
#!/bin/sh
export http_proxy=http://puppet.jephe:8080
wget -q -O - http://linux.dell.com/repo/hardware/latest/bootstrap.cgi | bash
yum -y -e 0 install srvadmin-all
yum -y -e 0 install dell_ft_install
yum -y -e 0 install \$(bootstrap_firmware)
echo "now you can run  inventory_firmware then update_firmware"
END

chmod +x /root/dell_openmanage.sh

echo "Informing Foreman that we are built.."
wget -q -O /dev/null --no-check-certificate http://puppet.jephe:3000/unattended/built

After install dell openmanager, you can access it through https://serveripaddress:1311, login with root password

a. refresh cdrom installation anaconda config for VM server 3.1.1 (use eth4 as ovsagent interface and add to vlan 1001)

install
eula Accepted
cdrom
lang en_US.UTF-8
keyboard us
network --device eth4 --bootproto static --ip 192.168.0.2 --netmask 255.255.255.0 --gateway 192.168.0.254 --nameserver 192.168.0.3,8.8.8.8 --hostname util01.jephe
ovsagent --iscrypted X12345TkAjLZk+TLABCDEFGHJIg=
ovsmgmntif eth4.1001
rootpw --iscrypted $1$abcde$O5ngOGU.0eXsXIfk6QB2B.
firewall --enabled --port=21:tcp --port=22:tcp --port=53:udp --port=53:tcp --port=80:tcp --port=2049:tcp --port=5900-7999:tcp --port=8002:tcp --port=8003:tcp --port=8899:tcp --port=7777:tcp
authconfig --enableshadow --enablemd5
selinux --disabled
timezone --utc Australia/Sydney
bootloader --location=mbr --dom0_mem=5800 --driveorder=sda
# The following is the partition information you requested
# Note that any partitions you deleted are not expressed
# here so unless you clear all partitions first, this is
# not guaranteed to work
#clearpart --all --drives=sda
#part /boot --fstype ext3 --size=100 --ondisk=sda
#part / --fstype ext3 --size=3072 --ondisk=sda
#part swap --size=1024 --ondisk=sda

%packages
@base
@core
@ovs-virtualization

Note:
[root@util01 init.d]# openssl passwd -1 -salt abcde "password"
$1$abcde$O5ngOGU.0eXsXIfk6QB2B.

According to Oracle support, you can use the following in %post to display on screen
%post --log=/root/kickstart-post.log
#actually, if you specify tee /root/ks-post_install.log, you can ignore this --log as it won't generate anything inside

exec < /dev/tty3 > /dev/tty3
chvt 3
(
.....

) 2>&1 | /usr/bin/tee /root/ks-post_install.log
echo
chvt 1


b. without static ip address, use generate kickstart config file for everything
eula Accepted
reboot
install
lang en_US.UTF-8
keyboard us
skipx
timezone Australia/Sydney
ovsagent --iscrypted XM3Ve5TkAjLZAAAAAAAAAAAAAAA
ovsmgmntif eth0
url --url http://repo.jephe/xen/3.1.1/os/x86_64/
rootpw --iscrypted $1$/NL6hNln$mVk8AAAAAAAAAAAAAA.
firewall --disabled
selinux --disabled
authconfig --enableshadow --enablemd5
timezone  Australia/Sydney
bootloader --location=mbr
<%= @host.diskLayout %>
%packages
@core
@base
@ovs-virtualization

%post --log=/root/kickstart-post.log
echo "Informing Foreman that we are built.."
wget -q -O /dev/null --no-check-certificate http://puppet.jephe:3000/unattended/built

And diskLayout is as follows:
clearpart --all --drives=sda
part /boot --fstype ext3 --size=100 --ondisk=sda
part / --fstype ext3 --size=3072 --ondisk=sda
part swap --size=1024 --ondisk=sda

c. /tftpboot/pxelinux.cfg/01-MACaddress


[root@puppet pxelinux.cfg]# more 01-d4-ae-52-aa-bb-cc
default linux
label linux
kernel boot/OVS-OVS3.1-vmlinuz
append initrd=boot/OVS-3.1-OVS3.1-initrd.img ks=http://puppet.jephe:3000/unattended/provision ksdevice=bootif network kssendmac
ipappend 2

or use the following for specific ip address
append initrd=boot/OVS-3.1-OVS3.1-initrd.img ks=http://puppet.jephe:3000/unattended/provision ksdevice=bootif
network --device eth5 --bootproto static --ip 192.168.0.2 --netmask 255.255.255.0 --gateway 192.168.0.254 --nameserver 192.168.0.1,8.8.8.8 --hostname xens01.jephe kssendmac

or you can use general pxelinux.cfg, but in foreman provision script, use if-then to dynamic generate ipaddress for different hostname as folows:
echo "configuring correct IP address for bond1.2091 for `hostname`"
if [ `hostname` == 'xens01.jephe' ]
then
  cat >> ifcfg-bond1.91 <<EOF
IPADDR=192.168.0.2
GATEWAY=192.168.0.254
NETMASK=255.255.255.0
EOF
elif [ `hostname` == 'xens02.jephe' ]
then
   cat >> ifcfg-bond1.91 <<EOF
IPADDR=192.168.0.3
GATEWAY=192.168.0.254
NETMASK=255.255.255.0
EOF
fi

5. ksplice and ULN
quickstart guide: https://oss.oracle.com/ksplice/docs/ksplice-quickstart.pdf
firstly, get access key for ksplice,
then run the following command

wget -N https://www.ksplice.com/uptrack/install-uptrack
sh install-uptrack 123445555kklkkddkdkakdk
uptrack-upgrade -y

configuring https_proxy in /etc/uptrack/uptrack.conf


Uninstall of Uptrack is very easy. You should consider removing the updates with
uptrack-remove --all followed by #yum remove uptrack.


[root@util01 uptrack]# yum remove uptrack
Loaded plugins: dellsysid, rhnplugin, security
Setting up Remove Process
Resolving Dependencies
--> Running transaction check
---> Package uptrack.noarch 0:1.2.2-0.el6 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

==================================================================================================================================================================================================================
 Package                                        Arch                                          Version                                               Repository                                               Size
==================================================================================================================================================================================================================
Removing:
 uptrack                                        noarch                                        1.2.2-0.el6                                           @ksplice-uptrack                                        1.1 M

Transaction Summary
==================================================================================================================================================================================================================
Remove        1 Package(s)

Installed size: 1.1 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Erasing    : uptrack-1.2.2-0.el6.noarch                                                                                                                                                                     1/1
warning: /etc/uptrack/uptrack.conf saved as /etc/uptrack/uptrack.conf.rpmsave
  Verifying  : uptrack-1.2.2-0.el6.noarch                                                                                                                                                                     1/1

Removed:
  uptrack.noarch 0:1.2.2-0.el6                                                                                                                                                                                  

Complete!

Note: this will clear /var/cache/uptrack directory .


ULN:
go to linux.oracle.com to register which binds your SSO to ULN, then run uln_register --proxy=1.2.3.4:8080 to register Oracle Linux to ULN, proxy is optional, during text gui interface, it will ask you SSO, CSI etc information for register.

run yum repolist to check if the following are listed.


ol6_x86_64_UEK_latest                                                                                          Latest Unbreakable Enterprise Kernel for Oracle Linux 6 (x86_64)                                                                                             71
ol6_x86_64_latest                                                                                              Oracle Linux 6 Latest (x86_64)  


then run yum update to update system.

6.  install Dell openmanage after os install
refer to http://www.fatmin.com/2010/08/installing-dell-openmanage-server-administrator-on-linux.html

Install OpenManage Server Administrator

Step 1: wget -q -O - http://linux.dell.com/repo/hardware/latest/bootstrap.cgi | bash
Step 2: yum -y install srvadmin-all
Step 3: Start  Systems Management Data Engine - /etc/init.d/dataeng start
Step 4: Start webinterface - /etc/init.d/dsm_om_connsvc start

Your system homepage is now availible via https://hostname:1311, where hostname is the hostname of your server.

Install Firmware/Bios Management Tools

Step 1: yum install dell_ft_install
Step 2: yum install $(bootstrap_firmware)


Now run either inventory_firmware or inventory_firmware_gui to view your firmware versions, and perform firmware upgrades. Also, you can access server ip dell openmanage at https://serveripaddress:1311/


7. Dell Nagios plugin installation for hardware monitoring

Go to http://fedoraproject.org/wiki/EPEL/FAQ to download epel

rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-7.noarch.rpm
then download perl-Config-Tiny rpm from public yum repository:

then install Dell openmanage nagios plugin by running

yum install nagios-plugins-openmanage

[root@util02 yum.repos.d]# /usr/lib64/nagios/plugins/check_openmanage
OK - System: 'PowerEdge R610 II', SN: '5HZXXX', 48 GB ram (6 dimms), 1 logical drives, 2 physical drives

8. Silent install database

unzip downloaded Oracle database 11.2.0.3 64bit, unzip it, find *.rsp file

[root@util01 database]# cat inst.rsp | grep -v ^$ | grep -v ^# | grep -v =$
oracle.install.responseFileVersion=/oracle/install/rspfmt_dbinstall_response_schema_v11_2_0
oracle.install.option=INSTALL_DB_AND_CONFIG
ORACLE_HOSTNAME=util01
UNIX_GROUP_NAME=dba
INVENTORY_LOCATION=/u01/app/oraInventory
SELECTED_LANGUAGES=en
ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
ORACLE_BASE=/u01/app/oracle
oracle.install.db.InstallEdition=EE
oracle.install.db.EEOptionsSelection=false
oracle.install.db.optionalComponents=oracle.rdbms.partitioning:11.2.0.3.0,oracle.oraolap:11.2.0.3.0,oracle.rdbms.dm:11.2.0.3.0,oracle.rdbms.dv:11.2.0.3.0,oracle.rdbms.lbac:11.2.0.3.0,oracle.rdbms.rat:11.2.0.3.0
oracle.install.db.DBA_GROUP=dba
oracle.install.db.OPER_GROUP=dba
oracle.install.db.config.starterdb.type=GENERAL_PURPOSE
oracle.install.db.config.starterdb.globalDBName=orcl
oracle.install.db.config.starterdb.SID=orcl
oracle.install.db.config.starterdb.characterSet=AL32UTF8
oracle.install.db.config.starterdb.memoryOption=true
oracle.install.db.config.starterdb.memoryLimit=4096
oracle.install.db.config.starterdb.installExampleSchemas=false
oracle.install.db.config.starterdb.enableSecuritySettings=true
oracle.install.db.config.starterdb.password.ALL=password
oracle.install.db.config.starterdb.control=DB_CONTROL
oracle.install.db.config.starterdb.automatedBackup.enable=false
oracle.install.db.config.starterdb.storageType=FILE_SYSTEM_STORAGE
oracle.install.db.config.starterdb.fileSystemStorage.dataLocation=/u01/oradata
oracle.install.db.config.starterdb.fileSystemStorage.recoveryLocation=/u01/oraarch
DECLINE_SECURITY_UPDATES=true
oracle.installer.autoupdates.option=SKIP_UPDATES

[root@outil01 database]# mkdir -p /u01/app/oracle/
[root@util01 database]# chown oracle:oinstall /u01/ -R
[root@util01 database]# su - oracle
[oracle@util01 database]$ pwd
/home/oracle/database
[oracle@util01 database]$ ./runInstaller -ignoreSysPrereqs -force -silent -responseFile /home/oracle/database/inst.rsp

9. configure database environment

create /usr/local/bin/oraset
[oracle@util01 bin]$ ls -l oraset
-rw-r--r-- 1 oracle oinstall 149 Aug 27 16:54 oraset
[oracle@util01 bin]$ more oraset
export PATH=$PATH:/home/oracle/bin
export NLS_DATE_FORMAT='DD-MON-YYYY HH24:MI:SS'
export ORACLE_SID=orcl

ORAENV_ASK=NO
. oraenv
unset ORAENV_ASK

[oracle@util01 bin]$ ls -l ~/.bashrc
-rw-r--r-- 1 oracle oinstall 149 Aug 27 16:54 /home/oracle/.bashrc

[oracle@util01 bin]$ more ~/.bashrc
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

# User specific aliases and functions

. /usr/local/bin/oraset >/dev/null 2>&1

Note: if you don't put >/dev/null 2>&1, the scp file to this host from other server may fail.

[root@util01 init.d]# more oracle
#!/bin/bash 
# chkconfig: 35 98 10
# description: Starts and stops Oracle processes 

# startup/shutdown sequence,  the lower the number is , the high priority it gets, for shutdown, 5 shutdown first, 10 shutdown later

LOCKFILE=/var/lock/subsys/oracle

case "$1" in 
  'start') 
    su - oracle -c "dbstart;lsnrctl start;emctl start dbconsole"
  touch $LOCKFILE
   ;; 
  'stop') 
    su - oracle -c "emctl stop dbconsole;lsnrctl stop;dbshut"
    rm -f $LOCKFILE
   ;; 
esac 

Note: those red color part is very important, otherwise, auto shutdown won't work although auto start will be working as Redhat-alike system doesn't think oracle is running when rebooting if it cannot see oracle in /var/lock/subsys/, so ignore it.

# chkconfig --add oracle
# chkconfig oracle --level 35 on

You also need to modify dbstart and dbshut file as follows:

[oracle@util01 bin]$ grep -B 1 ORACLE_HOME_LISTNER dbstart
ORACLE_HOME_LISTNER=${1:-$ORACLE_HOME}

[oracle@util01 bin]$ grep -B 1 ORACLE_HOME_LISTNER dbshut
ORACLE_HOME_LISTNER=${1:-$ORACLE_HOME}

[root@util01 init.d]# ls -l oracle
-rwxr-xr-x 1 root root 354 Aug 30 14:31 oracle

a more complete auto stop/start file is as follows: Please refer to http://tldp.org/HOWTO/Oracle-9i-Fedora-3-Install-HOWTO/sect_08.html
----------------

#!/bin/bash
#
# Run-level Startup script for the Oracle Instance and Listener
#
# chkconfig: 345 91 19
# description: Startup/Shutdown Oracle listener and instance

ORA_HOME="/u01/app/oracle/product/9.2.0.1.0"
ORA_OWNR="oracle"

# if the executables do not exist -- display error

if [ ! -f $ORA_HOME/bin/dbstart -o ! -d $ORA_HOME ]
then
        echo "Oracle startup: cannot start"
        exit 1
fi

# depending on parameter -- startup, shutdown, restart 
# of the instance and listener or usage display 

case "$1" in
    start)
        # Oracle listener and instance startup
        echo -n "Starting Oracle: "
        su - $ORA_OWNR -c "$ORA_HOME/bin/lsnrctl start"
        su - $ORA_OWNR -c $ORA_HOME/bin/dbstart
        touch /var/lock/subsys/oracle
        echo "OK"
        ;;
    stop)
  # Oracle listener and instance shutdown
        echo -n "Shutdown Oracle: "
        su - $ORA_OWNR -c "$ORA_HOME/bin/lsnrctl stop"
        su - $ORA_OWNR -c $ORA_HOME/bin/dbshut
        rm -f /var/lock/subsys/oracle
        echo "OK"
        ;;
    reload|restart)
        $0 stop
        $0 start
        ;;
    *)
        echo "Usage: $0 start|stop|restart|reload"
        exit 1
esac
exit 0


10. patch 11.2.0.3 to the latest patchset
download OPATH latest version first:
How To Download And Install The Latest OPatch Version [ID 274526.1]

install opatch

oracle@util01 ~]$ cd $ORACLE_HOME
[oracle@util01 db_1]$ mv OPatch/ OPatch.orig
[oracle@util01 db_1]$ unzip ~/p6880880_112000_Linux-x86-64.zip

11. Patch Oracle database to the latest pathset

sqlplus / as sysdba
sql> shutdown immediate;
sql> exit
lsnrctl stop
emctl stop dbconsole

[oracle@util01 13923374]$ export PATH=$PATH:/u01/app/oracle/product/11.2.0/db_1/OPatch
[oracle@util01 13923374]$ opatch apply

Check the readme.html file from the patch, run below for database upgrade.
cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
sql> connect / as sysdba
sql> startup
sql> @catbundle_PSU_ORCL_ROLLBACK.sql
sql> quit

check logfile for any errors: $ORACLE_HOME/cfgtoollogs/catbundle/catbundle_PSU_ORCL_ROLLBACK_timestamp.log

12. install Oracle VM Manager
Download OVMM original and patch 399 ISO
https://updates.oracle.com/ARULink/PatchDetails/process_form?aru=15337157

Preparing config.yml for silent OVMM installation

[root@util01 oracle]# more config.yml
db:
install : No
host : localhost
sys_password : oracle
port : 1521
sid : orcl
mgmt_port : 8080
user : ovs
password : oracle

webLogic:
install : Yes
user : weblogic
password : oracle

java:
install : Yes

ADF:
install : Yes

OVMCore:
install : Yes
user : admin
password : oracle

OVMConsole:
install : Yes

OVMShell:
install : Yes

onFailure:
cleanup : Yes

OVMUpgrade:
install : Yes

13  Preparing Oracle environment

[root@util01 ovmm]# mount Oracle_VM_Manager_3.1.1_64bit.iso ovmm -o loop
[root@util01 ovmm]# ./createOracle.sh
Adding group 'oinstall' with gid '54323' ...
groupadd: group 'oinstall' already exists
Adding group 'dba'
groupadd: group 'dba' already exists
Adding user 'oracle' with user id '54322', initial login group 'dba', supplementary group 'oinstall' and  home directory '/home/oracle' ...
User 'oracle' already exists ...
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba)
Creating user 'oracle' succeeded ...
Verifying user 'oracle' OS prerequisites for Oracle VM Manager ...
oracle    soft        nofile                    8192
oracle    hard       nofile                    8192
oracle    soft        nproc                    4096
oracle    hard       nproc                   4096
oracle    soft        core                       unlimited
oracle    hard       core                       unlimited
Setting  user 'oracle' OS limits for Oracle VM Manager ...
Altered file /etc/security/limits.conf
Original file backed up at /etc/security/limits.conf.orabackup
Verifying & setting of user limits succeeded ...
Modifying iptables for OVM
Adding rules to enable access to:
     7001  : Oracle VM Manager http
     7002  : Oracle VM Manager https
     15901 : Oracle VM Manager VM console proxy
     54321 : Oracle VM Manager core
     54322 : Oracle VM Manager core via SSL
       123 : NTP
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]
Rules added.

---
when OVMS and OVMM are not on the same network, port 7001 is needed ,
OVMM events shows BindingMismatchEvent critical python bindings mismatch .

14. actual installation of OVMM 3.1.1
/runInstaller.sh --config=/home/oracle/config.yml --installtype=production –assumeyes

15.  Upgrade Oracle VM manager to 399
mount iso file as -o loop, then
#./runUpgrader.sh as root


access OVM at  http://ip:7001/ovm/console

16. database archive log mode and rman
oracle@util01 u01]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.3.0 Production on Wed Aug 22 15:41:19 2012
Copyright (c) 1982, 2011, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> alter system set log_archive_dest_1='location=/u01/oraarch' scope=both;
SQL> alter system set log_archive_format='%t_%s_%r.arc' scope=spfile;
SQL> shutdown immediate;
SQL> startup mount;
SQL> alter database archivelog;
SQL> alter database open;
SQL>exit;

$ rman target /
Rman> show all;
Rman> configure controlfile autobackup on;
Rman> backup database plus archivelog format ‘/u01/orarman/%U’

17. after renaming database host, re-generate em repository
modify the following files first.

./app/oracle/product/11.2.0/db_1/network/admin/listener.ora
/etc/hosts
/etc/sysconfig/network

[oracle@util01 bin]$ ./emca -deconfig dbcontrol db -repos drop
[oracle@util01 bin]$ ./emca -config dbcontrol db -repos create

18. known issue
If you configured vlan for em1 or bond0(em1 and em2), you might not able to start ovmm service with https port 7002.
The solutions is to disable ipv6 as follows:

a. put the following in /etc/modprobe/ipv6.conf
options ipv6 disable=1

b. append the following into /etc/sysconfig/network
NETWORKING_IPV6=off

19. references:

kickstart - http://en.wikipedia.org/wiki/Kickstart_(Linux)
http://fedoraproject.org/wiki/Anaconda/Kickstart
http://www.fatmin.com/2010/08/installing-dell-openmanage-server-administrator-on-linux.html
http://linux.dell.com/wiki/index.php/Repository/OMSA#Supported_Operating_Systems
http://www.fatmin.com/2010/08/installing-dell-openmanage-server-administrator-on-linux.html  4 steps to update dell firmware from Linux
http://folk.uio.no/trondham/software/check_openmanage.html#download  Nagios plugin to check dell openmanage