How to install ansible2.11 with python36 on RHEL7

                                  by Jephe Wu - http://linuxtechres.blogspot.com

Install python36 packages from subscription or RHEL7.9 ISO

[root@test ]# more /etc/yum.repos.d/local.repo
[localrepo]
name=Unixmen Repository
baseurl=file:///cdrom
gpgcheck=0
enabled=1

yum localinstall python3 python3-pip python3-rpm-generators python3-rpm-macros python3 python3-setuptools

    

Use EPEL

[root@test ]# grep proxy /etc/yum.conf
proxy=http://proxy.domain.com:8080

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

 

use pip3 to install ansible

pip3 install --upgrade pip

pip3 cache purge

export LANG=en_AU.UTF8

pip3 install ansible   ( or python3 -m pip install ansible )

pip3 install pywinrm

 

Check if win_ modules are there

run ansible-doc win_reboot

find /user/local/  -name "win_*"

 

 

[root@test unixadm]# pip3 install pywinrm
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Collecting pywinrm
Downloading pywinrm-0.4.2-py2.py3-none-any.whl (44 kB)
|████████████████████████████████| 44 kB 920 kB/s
Collecting requests>=2.9.1
Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
|████████████████████████████████| 62 kB 1.2 MB/s
Collecting six
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting xmltodict
Downloading xmltodict-0.12.0-py2.py3-none-any.whl (9.2 kB)
Collecting requests-ntlm>=0.3.0
Downloading requests_ntlm-1.1.0-py2.py3-none-any.whl (5.7 kB)
Collecting charset-normalizer~=2.0.0
Downloading charset_normalizer-2.0.9-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
|████████████████████████████████| 149 kB 28.9 MB/s
Collecting urllib3<1.27,>=1.21.1
Downloading urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
|████████████████████████████████| 138 kB 37.7 MB/s
Collecting idna<4,>=2.5
Downloading idna-3.3-py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 11.2 MB/s
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib64/python3.6/site-packages (from requests-ntlm>=0.3.0->pywinrm) (36.0.0)
Collecting ntlm-auth>=1.0.2
Downloading ntlm_auth-1.5.0-py2.py3-none-any.whl (29 kB)
Requirement already satisfied: cffi>=1.12 in /usr/local/lib64/python3.6/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm) (1.15.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.6/site-packages (from cffi>=1.12->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm) (2.21)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests, ntlm-auth, xmltodict, six, requests-ntlm, pywinrm
WARNING: The script normalizer is installed in '/usr/local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed certifi-2021.10.8 charset-normalizer-2.0.9 idna-3.3 ntlm-auth-1.5.0 pywinrm-0.4.2 requests-2.26.0 requests-ntlm-1.1.0 six-1.16.0 urllib3-1.26.7 xmltodict-0.12.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

 

Note: pip3 will download packages from site below

e.g.  Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.35.tar.gz

 

 How to find out which process is using swap space

                                     Jephe  Wu  - http://linuxtechres.blogspot.com

 

https://dbsysupgrade.com/how-to-display-processes-using-swap-space/

Use command below to find out pid and percentage of swap usage.

# find /proc -maxdepth 2 -path "/proc/[0-9]*/status" -readable -exec awk -v FS=":" -v TOTSWP="$(cat /proc/meminfo | sed -n -e "s/^SwapTotal:[ ]*\([0-9]*\) kB/\1/p")" '{process[$1]=$2;sub(/^[ \t]+/,"",process[$1]);} END {if(process["VmSwap"] && process["VmSwap"] != "0 kB") {used_swap=process["VmSwap"];sub(/[ a-zA-Z]+/,"",used_swap);percent=(used_swap/TOTSWP*100); printf "%10s %-30s %20s %6.2f%\n",process["Pid"],process["Name"],process["VmSwap"],percent} }' '{}' \; | awk '{print $(NF-2),$0}' | sort -hr | head | cut -d " " -f2-

Installing Ansible with winrm module for Windows server automation

                    by Jephe Wu - http://linuxtechres.blogspot.com

Steps

set proxy for /etc/yum.conf

proxy= http://proxy.domain.com:8080

Disable redhat registration

[root@ansible01 pluginconf.d]# grep enabled /etc/yum/pluginconf.d/subscription-manager.conf

enabled=0

Install EPEL release for yum repo

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Download and install ansible packages from EPEL together with local RHEL7 ISO mount

yum localinstall *  => install all below packages

Testing playbook

[root@ansible01 ~]# more hosts.ini
[win:vars]
ansible_user='DOMAIN\jephe'
ansible_password=xxxxxxxxxxx
ansible_connection=winrm
ansible_winrm_server_cert_validation=ignore
#ansible_winrm_scheme=http
ansible_port=5985
#ansible_winrm_transport=kerberos
#ansible_winrm_transport=credssp
ansible_winrm_transport=ntlm

[win]
192.168.1.10

root@ansible01 ~]# ansible -i hosts.ini win  -m win_ping

192.168.1.10 | SUCCESS => {
    "changed": false,
    "ping": "pong"

}

[root@ansible01 ~]# more win_play.yml
---

- name: test
  hosts: all
  become_method: runas

  tasks:

    - name: whoami

      win_shell: mkdir c:\tmp\jewu2

[root@ansible01 ~]# ansible-playbook -i hosts.ini win_play.yml

PLAY [test] ****************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************
ok: [192.168.1.10]

TASK [whoami] **************************************************************************************************************************************
changed: [192.168.1.10]

PLAY RECAP *****************************************************************************************************************************************

192.168.1.10              : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
===============

 How to manually configure ntpd and chronyd on RHEL 6/7/8

                              by Jephe Wu   - http://linuxtechres.blogspot.com

Add below lines into /etc/chrony.conf or /etc/ntp.conf

server x.x.x.x iburst

server y.y.y.y iburst 

Manually update time once

systemctl stop ntpd

ntpdate x.x.x.x

 

systemctl stop chronyd

chronyd -q 'server x.x.x.x iburst'

[root@server1 ]#  chronyd -q 'server 192.168.1.100 iburst'
2021-11-25T05:08:34Z chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG)
2021-11-25T05:08:34Z Initial frequency -11.868 ppm
2021-11-25T05:08:38Z System clock wrong by 0.000087 seconds (step)
2021-11-25T05:08:38Z chronyd exiting

Show current ntp stauts

ntpq -pn

chronyc [-n] sources

How to setup an ideal rsync task between two Linux servers

                                            by Jephe Wu - http://linuxtechres.blogspot.com

 

Summary

There's some issue for syncing server a(source) to server b (dest.) which caused server b partition full, in order to solve the problem, we need to adjust some rsync parameters for cronjob task.

Original rsync task under root cronjob

*/30 * * * *  /usr/bin/rsync --numeric-ids -avr --exclude=.snapshot 192.168.1.10::share/* /share

Note: this has caused server b /share partition full due to it won't delete files on destination site.

In order to make a full sync working again, we will need to manually run task below which will delete first to free up space, then sync files again.

/usr/bin/rsync --numeric-ids -avr --delete --delete-before --ignore-errors --exclude=.snapshot 192.168.1.10::share/* /share

The rsync daemon config on server a is below:

[root@servera share]# more /etc/rsyncd.conf

reverse lookup = false

[share]
  path = /share
  numeric ids = yes
  max connections = 10
  list = true


After full sync completed, we will change cronjob to below:

*/30 * * * *  /usr/bin/rsync --numeric-ids -avr --delete --delete-before --ignore-errors --exclude=.snapshot 192.168.1.10::share/* /share >> /var/log/rsync-cron.log

Note: sometimes, you will encounter error message 'IO error encountered -- skipping file deletion' for rsync command, you can use '--ignore-errors' option as above to avoid this issue.

 How to check if a remote server port is open or not

                                      by Jephe Wu - http://linuxtechres.blogspot.com

Objective: test from a Linux client to decide if remote server port is open or not

Method 1:  telnet

e.g. telnet server1 389  - check if tcp/389 is open on server1 to client

Method 2: nc 

e.g. nc -vz server1 389  - check if tcp/389 is open on server1 to client

or nc -w 5 -vz server1 389 to specify timeout value

Method 3: openssl

Sometimes, there's no telnet or nc available/installed on client, we can try openssl

e.g.  openssl s_client -connect server1:389

 

Method 4: curl 

server 1 has ip 1.2.3.4

[root@server1 unixadm]# curl -v  telnet://1.2.3.4:10002
* About to connect() to 1.2.3.4 port 10002 (#0)
*   Trying 1.2.3.4...
* Connected to 1.2.3.4 (1.2.3.4) port 10002 (#0)

Method 5: from Windows Powershell

PS c:\users\jwu> Test-NetConnection -computername xxxx -port 443

Method 6: Linux bash shell

(timeout 1 bash -c "</dev/tcp/x.x.x.x/8080" && echo -n PORT OPEN || echo -n PORT CLOSED)

 How to configure manual entries in /etc/resolv.conf instead of DNS1/DNS2 in ifcfg-ens192

                                  by Jephe Wu  - http://linuxtechres.blogspot.com

Summary

There are two things in RHEL7 which will use DNS1/DNS2 in /etc/sysconfig/network-scripts/ifcfg-ens192 instead of manual configuration for /etc/resolv.conf for nameserver lines.

1)  use NetworkManager in ifcfg-ens192 and also update dns in /etc/NetworkManager/NetworkManager.conf under main section by default

2) PEERDNS in ifcfg-ens192 will overwrite /etc/resolv.conf nameserver with DNS1/DNS2 anyway although IP is static not dhcp

Steps

You need to disable networkmanager and also PEERDNS

1) disable networkmanager

[root@server1 network-scripts]# grep -i nm_controlled ifcfg-ens192
NM_CONTROLLED=no

2) or disable dns in NetworkManager.conf

[root@server2 network-scripts]# grep dns /etc/NetworkManager/NetworkManager.conf
dns=none

3) also you need to disable PEERDNS

[root@server1 network-scripts]# grep -i peerdns /etc/sysconfig/network-scripts/ifcfg-ens192
PEERDNS=no

4) reload NetworkManager and restart network to make sure /etc/resolv.conf is not being emptied first before reboot

systemctl reload NetworkManager

systemctl restart network

5) reboot

 How to resize VM partition on Vmware 

                                          by Jephe Wu - http://linuxtechres.blogspot.com

Extend a LVM partition

1. from Vcenter, to edit configure, to add a new hard disk with required size
   
2.   echo "- - -" > /sys/class/scsi_host/host#/scan  for each host*  (replace # with actual numbers)
3. dmesg and cat /proc/partition to confirm new disk is added.

      4. fdisk /dev/sdb to create a new partition and use t  to change it to 8e LVM

      5 . pvcreate /dev/sdb1

     6. vgextend vg_root /dev/sdb1

     7. lvextend -l +100%FREE /dev/vg_root/lv_root

or  8. lvextend -L +20G /dev/vg_root/lv_root (if only increase 20G)

     9.  resize2fs /dev/mapper/vg_root-lv_root 

or xfs_growfs mountpoint (if it's xfs partition)

Modify existing sdb LVM PV disk size and use it in vg

1. echo "- - -" > /sys/class/scsi_host/host#/scan  for each host*  (replace # with actual numbers)

or

echo 1 > /sys/block/sdb/device/rescan

 

2.  use pvscan or  fdisk -l | grep -i sdb to confirm new disk size

3. blockdev --rereadpt /dev/sdb  to resize LVM volume size

4. pvresize /dev/sdb  to resize PV

Resize Procedure for /dev/sdb1 PV

https://access.redhat.com/solutions/57183

1. Confirm the actual storage size with fdisk -ul /dev/sdb. Observe the increased disk size. Depending on how the storage is presented, a system reboot may be necessary for this to appear.

2. Resize the partition on the disk. To achieve this, observe the starting sector in fdisk -ul /dev/sdb, then remove the partition with fdisk and re-create it with the same starting sector but the (default) last sector of the drive as the ending sector. Then write the partition table and confirm the change (and the correct starting sector) with fdisk -ul /dev/sdb.

3. Run pvresize /dev/sdb1 to grow (resize) the PV onto the rest of the expanded partition. This will create free extents within the Volume Group which then can be used to grow a Logical Volume. Running lvresize command with -r as lvresize -r will grow the filesystem within the Logical Volume as well.

Another solution would be creating a new partition under the same device starting where the first partition ended and using the rest of the cylinders to create the same, then put this new partition under LVM with the pvcreate command, extend the current VG (vgextend vg-name pv-name), then extend the current LV (lvextend) and finally resize the current filesystem (resize2fs).

Create a new LVM partition with new disk

pvcreate /dev/sdx1 

vgcreate vg01 /dev/sdx1

lvcreate -l +100%FREE -n apps vg01
mkfs -t xfs /dev/mapper/vg01_apps

For rollback

1) Unmount the mountpoint

umount -v /var/www

2) Check for filesystem error

e2fsck -f /dev/vg-repo/lv-repo

3) Reduce the logical volume

lvreduce --resize2fs -L -100G (-l 5119)  /dev/vg-repo/lv-repo

Note: use vgdisplay to record down existing used logic extent number, it's 5119 in this case before increase

4) Check for filesystem error

e2fsck -f /dev/vg-repo/lv-repo

5) Mount the filesystem back to the same mountpoint

mount /dev/vg-repo/lv-repo /var/www

6) Reduce the VG in order to release the PV /dev/sde1

vgreduce vg-repo /dev/sde1

7) Remove the PV

pvremove /dev/sde1

8) Remove the new created disk from vcenter in the backup plan?