Setting up virtual machines with CentOS, KVM, Socat and RealVNC

Jephe Wu - http://linuxtechres.blogspot.com

Objective: use open source softwares to set up a virutal Windows XP machine inside Linux with excellent remote access performance
Environment: Dell OptiPlex 780 as host server running CentOS 5.5 64bit with 8G RAM, both KVM and Xen virtualzation installed, socat, realvnc, Windows XP virtual machine with bridged network br0 and Qumranet paravirtualized ethernet adapter, as well as RedHat VirtIO SCSI Disk Device driver.


Concept:
1. use CentOS 5.5 with KVM instead of Xen as KVM has the future. 
You don't have to change anything after inital KVM setup to enable bridged network, For NATed Windows machine, you can use 'socat' for tcp port forwarder on the VNC ports to make inside Windows XP virtual machine accessable to outside world, but vnc response speed is quite slow.

To gain better network, hard disk and graphic drawing speed, you can:
2. use realvnc server on virtual Windows XP SP3 machine instead of KVM builtin vnc server for better response speed, so called VNC mirror driver
3. use the latest virtio-win ISO package to install qumranet paravirtualized ethernet adapter driver instead of the default realtek network adapter for better network performance
4. use bridged network interface instead of the default NAT for Windows virtual machine for even better network performance and easier configuration, this requires some configuration after initial setup.
5. use RedHat VirtIO SCSI Disk Device driver instead of the default IDE driver for better block device performance.

Steps
1. Installation of CentOS 5.5
Download CentOS 5.5 DVD and burn it for local installation, or use NFS network installation if you are using DVD iso file.

Choose KVM and Gnome during installation.

If you have only CentOS 5 lower version, after install, run 'yum update' to upgrade to the latest.

2. KVM configuration
If you have installed both KVM and Xen, server will boot from Xen kernel, you will need to vi /etc/grub.conf to change it to boot from the plain Linux kernel which has KVM support.

By default, when you configure KVM virtual machine, it use NATed network only because it doesn't have bridged network for you to use. After that, the localhost will be listening at port 5900 for vnc connection, but it cannot be connected remotely since it's only listening at localhost.

You can use 'socat' (http://www.dest-unreach.org/socat/) as proxy to forward the vnc port by following the steps below:

# Install DAG rpmforge package
# rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS//rpmforge-release0.3.6-1.el5.rf.x86_64.rpm

# yum install socat

# add local2 to /etc/syslog.conf
[root@jephe ~]# grep socat /etc/syslog.conf
local2.*                        /var/log/socat.log

# socat -d -d -lmlocal2 tcp4-listen:5900,bind=10.0.0.1,su=nobody,reuseaddr,fork tcp4:127.0.0.1:5900 &
note:
This will enable inside KVM virtual machine to accept connection from network.
a. log will be recorded in local2 which is /var/log/socat.log according to /etc/syslog.conf
b. listening on 10.0.0.1:5900 (host server tcp socket instead of the KVM builtin one which is localhost:5900)
c. su as nobody
d. receiving vnc connection from network then forward to inside Windows XP SP3 virtual machine at 192.168.100.230:5900

You can put above socat command into /etc/rc.d/rc.local.

Issues:
With above configuration, the VNC response speed is very slow. You can improve it by using realvnc or tightvnc on virtual machine itself and change network adapter to bridged mode as well as using Redhat VirtIO ethernet adapter and RedHat VirtIO SCSI Disk Device driver.

3. Installing Windows KVM guest
a. When installing Windows 7, use 'Windows vista' option. When you choose 'local CDROM install' option, the installation path part might become grey, you can check if you enabled 'messagebus' daemon and 'haldeamon' daemon:

service autofs start
service messagebus start
service haldaemon start

So that the auto mount will work under gnome desktop, check 'ls -l /dev/cdrom' which should be symbolic linked to /dev/scd0.

4. Improving performance  - RealVNC with mirror driver
installing realvnc or tightvnc latest version (both are supporting so-called mirror driver)
realvnc server free edition can enable bi-directional cut and paste but there's no password protection for configuration change once login
tightvnc has password protection for console admin but has no support for pasting out text from virtual machine, only one way pasting.

After installing realvnc or tightvnc, you can use socat like this:

# socat -d -d -lmlocal2 tcp4-listen:5900,bind=10.0.0.1,su=nobody,reuseaddr,fork tcp4:192.168.100.230:5900 &

5. Improving performance - Bridged network


Refer to http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Virtualization/sect-Virtualization-Network_Configuration-Bridged_networking_with_libvirt.html

Here are my steps according to above RHEL doc website:

vi /etc/xen/xend-config.sxp and changing the line:

 (network-script network-bridge)

To:

 (network-script /bin/true)

# chkconfig NetworkManager off
# service NetworkManager stop

# cd /etc/sysconfig/network-scripts
Add the following line to ifcfg-eth0 which will be used for bridge.
BRIDGE=br0
MTU=9000

add the following lines to ifcfg-br0
[root@jephe network-scripts]# more ifcfg-br0
DEVICE=br0
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
DELAY=0
BROADCAST=10.0.0.255
IPADDR=10.0.0.1
NETMASK=255.255.255.0
NETWORK=10.0.0.0

After configuring, restart networking or reboot.

# service network restart

Configure iptables to allow all traffic to be forwarded across the bridge.

# iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
# service iptables save
# service iptables restart

 Restart the libvirt daemon.

# service libvirtd reload

Exit virt-manger, then start it again. Delete storage device hda , then add again, choose 'virtio' type. You should now have a "shared physical device", which guests can be attached and have full LAN access. Verify your new bridge:

# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.000000000000       yes
br0             8000.000e0cb30550       no              eth0

Note, the bridge is completely independent of the virbr0 bridge. Do not attempt to attach a physical device to virbr0. The virbr0 bridge is only for Network Address Translation (NAT) connectivity.

6. Improving performance - Virtio ethernet adapter driver
download the latest Windows guest network drivers from http://www.linux-kvm.com/  (left side of web page, under 'Windows Guest Drivers' - download the driver CD.

mounted as local cdrom image in virt-manager before rebooting Windows virtual machine.
After booting up, it found the new ethernet adapter, then I used cdrom to install driver.

After that, shutdown virtual machine, delete the existing network card in virt-manager, then add one more network card with virtio type, now you should be able to see bridged network br0 option

7. Improving performance - Windows block driver (RedHat VirtIO SCSI Disk Device driver)
In my case, I use Windows XP SP3 32bit as virtual machine, so I have done the following:
Visiting website http://www.linux-kvm.com/ , at the left side to download the latest Windows guest drivers CD or floppy (during installation)

and check
http://www.linux-kvm.com/content/block-driver-updates-install-drivers-during-windows-installation

Important: You cannot just delete hda then create vda, then startup guest, you need to add a small vda storage first without deleting the main hda, after that, boot into guest to install Redhat virtio scsi disk driver, after that, shutdown, delete vda and hda, then create vda to use the existing image.
otherwise, it will not be able to boot Windows. Here are the steps:


Shutdown virtual machine, use virt-manager to add one more storage device ( use dd if=/dev/zero of=/root/testfile bs=1k count=1000 to create a small file first as vd0 just for installing virtio disk driver)


After booting up virtual machine again, Windows XP detected a new scsi hard disk, then use above driver inside the cdrom to install it.


Shutdown again, use virt-manager to delete vd0 and hd0, after that ,create another storage device, use the existing hard disk image file, boot up again, this time, hard disk will become RedHat VirtIO SCSI Disk Device driver

Now, use vnc client to connect to Windows virtual machine and enjoy the fast speed.

Let's say you want to use floppy disk version after initial installation of virtual machine, you can use dd if=viostor-floppy.img of=/dev/fd0 to write to floppy disk. You also can mount the floppy img file as loop device as follows:

mkdir /mnt/viostor
cd /mnt/viostor
mount viostor-31-03-2010-floppy.img viostor/


8. Using virtio driver to improve network and disk performance when installing RHEL 5.4 guest

How can I improve network and disk performance of Red Hat Enterprise Linux KVM guests?- https://access.redhat.com/kb/docs/DOC-18756 

and

How do I set up virtio on a kvm guest to get better network and I/O performance on RHEL4 and RHEL5? - https://access.redhat.com/kb/docs/DOC-25441

If using the virt-manager graphical tool to install a new guest, select following options on the OS type selection page to enable virtio network and block device drivers for this new guest:

OS type: Linux
Version: Generic 2.6.25 or later kernel with virtio

9. High Resolution Graphic on KVM Windows guest 
For Windows 7 pro guest in KVM, you can use advanced mode option to choose monitor model up to resolution 1280x1024.



http://www.linux-kvm.com/content/using-high-resolution-graphics
http://www.linux-kvm.com/content/using-vmware-vga-kvm-windows-guests




10. How to clone the existing KVM guest
cd /var/lib/libvirt/images
virt-clone --original guest1 --name newguest1 --file newguest1.img

11. See Also
a. http://www.linux-kvm.org/page/Main_Page
b. http://virtualization.info/en/
c. http://www.linux-kvm.com/

d. KVM virtio network drivers

http://www.linux-kvm.com/content/tip-how-setup-windows-guest-paravirtual-network-drivers
http://www.linux-kvm.com/content/latest-release-windows-virtio-network-drivers
http://sourceforge.net/projects/kvm/files/kvm-driver-disc/

e. KVM Windows block drivers

http://www.linux-kvm.com/content/redhat-54-windows-virtio-drivers-part-2-block-drivers
http://www.linux-kvm.com/content/block-driver-updates-install-drivers-during-windows-installation

Use socat as tcp forwarder on Windows and restrict a few hosts to connect to it

Jephe Wu - http://linuxtechres.blogspot.com

Objective: on the company LAN, only one Windows server is able to connect to smtp server, some other servers need to connect to smtp server as well
Environment: Windows 2000 server A with one NIC, another Windows 2000 server B will be connecting to A at port 25 to reach company smtp server smtp.domain.com.


Steps:

1. download and setup socat for Windows on server A
download 2 files below from http://www.gentilkiwi.com/telechargements-s43-t-socat.htm#englishversion

socat-1.7.1.3.zip and cyg-dep.zip

Create c:\socat directory and extract both zip files under it

How to test it?
c:
cd socat
socat
2010/10/06 10:59:02 socat[15822] E exactly 2 addresses required (there are 0); use option "-h" for help

Note: once you see above output, that means socat is installed successfully.

2. run socat command with options to do tcp port forwarder
on Windows server A:
cd socat
socat -d -d -lf socat.txt tcp4-listen:25,reuseaddr,fork,tcpwrap=socat,allow-table=socat-allow,deny-table=socat-deny tcp4:smtp.jephe.com:25

the following are the content inside socat-allow and socat-deny files:
For socat-allow:
socat:1.2.3.4,5.6.7.8

For socat-deny:
socat:all

on Windows server B:
You can use server A as smtp server which will be forwarded by socat to smtp.domain.com

note: you can use range option to restrict the source ip such as 10.0.0.0/8 if that's better for your situation.

Appendix:
a. You can use plugdaemon (http://www.taronga.com/plugdaemon/) on Linux or iptables(snat/dnat, even for single NIC with kernel parameter net.ipv4.ip_forward enabled) to achive port forwarding function also.
b. socat website is at http://www.dest-unreach.org/socat/

Common Linux tools usage

Jephe Wu - http://linuxtechres.blogspot.com/

• bash  - if

http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html

• vi

30G - go to 30th  line
30| - go to 30th column
change a file to remove the ending new line
vi file , :%s#$\n# #g

• sed (delete one line from the file itself)
  

for i in B*;do sed -i /A443/d $i;done 
JEPHE=A1234; for i in B*;do sed -i "/$JEPHE/d" $i;done

• Tar

Backup files and exclude a list of files from a file:
tar --exclude-from=/path/to/excludedfilelist -cvpzf  file.tar.gz *

note: inside excludedfilelist, give file or directory name line by line, don't put / for directories behind.

delete a file from a tar archive:
tar --delete --file=file.tar tobedeletedfile

transfer sparse file on the network:

tar cvzSpf - *|ssh jephe@remoteserver '(cd /path/to; tar xzSpf -)'

• rsync

use rsync to transfer specified files under some directories.
rsync -av -r  --include-from=include.txt  /cygdrive/e/ root@10.0.0.1:/data/backup/

$ cat include.txt
+ a
+ a/b/
+ b
+ b/c/
+ a/b/*.dat
+ b/c/*.exe
- *

/usr/bin/rsync --timeout=600 -v --progress --include=*.gpg --exclude=* -a -e ssh --delete /data/db 10.0.0.1:/data/db/ > /tmp/dbbackup

Note: refer to http://samba.anu.edu.au/ftp/rsync/rsync.html


transfer-root directory referes to the source directory on the source server.

Here are some examples of exclude/include matching:

o --exclude "*.o" would exclude all filenames matching *.o
o --exclude "/foo" would exclude a file called foo in the transfer-root directory
o --exclude "foo/" would exclude any directory called foo
o --exclude "/foo/*/bar" would exclude any file called bar two levels below a directory called foo in the transfer-root directory
o --exclude "/foo/**/bar" would exclude any file called bar two or more levels below a directory called foo in the transfer-root directory
o --include "*/" --include "*.c" --exclude "*" would include all directories and C source files
o --include "foo/" --include "foo/bar.c" --exclude "*" would include only foo/bar.c (the foo/ directory must be
explicitly included or it would be excluded by the "*")

o --exclude "*/foo/" would exclude any directory called foo which is one level below the transfer root directory
o --exclude "**/foo/" would exclude any directory called foo which is one or more levels below the transfer-root directory

•  curl & wget

download a iso file  -  curl -CO complete_download_url and wget -c complete_download_url
check web response header  - curl -I www.domain.com

• nc

nc -v -z -s 10.0.0.1 10.0.10.1 25
Connection to 10.0.10.1 25 port [tcp/smtp] succeeded!
for udp, use:
nc -vuz destination_ip_addr 53  (udp might always report it's successful when using -z option)

• socat  (linux or cygwin)

$ socat -d -d tcp4-listen:25,bind=192.168.100.20,fork,reuseaddr tcp4:smtp.gmail.com:25,bind=192.168.100.20 &

• awk

ls -l *.3.gz.gpg  | awk '{total += $5} END { print total}'
2298937968

db2 list application | awk '($3==495)'  => print the third column which equals 495

• netstat

netstat -tunelp  --list listening tcp and udp port numbers
netstat -natup

• ssh port forwarding 

Local port forwarding
$ ssh jephe@server1 -L 1234:10.0.0.1:22 [-g]  (-g means allows remote hosts to connect to local forwarded ports)
$ ssh jephe@localhost -p 1234

Remote port forwarding
$ ssh jephe@office_server -R 1234:10.0.0.2:22  (on office server, the user can connect to localhost at port 1234 to access the local ssh server at port 22)

or


$ ssh jephe@office_server -R 2222:server_on_internet:22 (when user ssh to port 2222 to office server, it actually goes to internet server ssh port)

If you want to anyone from your office network to access office server at port 2222 which will be forwarded to server_on_internet ssh server, uncomment GatewayPorts line as
GatewayPorts yes


or
$ ssh jephe@office_server -R 80:your_home_web_server:80 (your home web server is not necessarily same as your home ssh client pc, can be another server)


Reference: http://souptonuts.sourceforge.net/sshtips.htm

You can use Windows cygwin ssh server plus ssh remote port forwarding to achieve something. Assume you have lease line connected to remote office, you can only ssh into remote office, now you need to install Linux in one of machine, you can use this method to install from local office http server.

Use above local and remote port forwarding + cygwin + openssh + putty + proxytunnel at http://proxytunnel.sourceforge.net/ , you can do a lot of things you might think it's impossible before.

If your company only allows to use proxy to access Internet and you control one of ssh server on Internet, then you got the power to access office network from home.

• rpm

rpm -Uvh --root=/tmp/ --nodeps /mnt/iso/CentOS/glibc-2.5-34.x86_64.rpm 
or

1. Make a temporary directory to extract the rpm in and copy the rpm into the directory: 
   

   
    mkdir tempdir
    cp bash.rpm tempdir
    

   
   
2. Execute rpm2cpio in the temporary directory: 
   

   
    cd tempdir
    rpm2cpio bash.rpm | cpio -idmv
    

3. show architecture

 rpm -qf /bin/ls --qf '%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}\n'

• stty and setterm/xset

stty erase ^H  (ctrl V H at the same time) 

setterm -blank nn will tell the console driver to blank the screen after nn minutes of inactivity. (With nn = 0, screensaving is turned off. In some old kernels this first took effect after the next keyboard interrupt.)

The s option of xset(1) will set the X screensaving parameters: xset s off turns off the screensaver, xset s 10 blanks the screen after 10 minutes.

• ssh 

ssh user@host -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null

ssh-keygen -F 10.0.3.1  - find this host from .ssh/known_hosts file

ssh-keygen -R 10.0.3.1  - remove this host from .ssh/known_hosts file

$ sshfs jephe@server:/path/to /mnt/sshfs/ 

Working in a ssh shell that used forwarding:

Supported escape sequences:

~. – terminate connection
~B – send a BREAK to the remote system
~C – open a command line
~R – Request rekey (SSH protocol 2 only)
~^Z – suspend ssh
~# – list forwarded connections
~& – background ssh (when waiting for connections to terminate)
~? – this message
~~ – send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)

~. and ~# are particularly useful.

• ftp (windows)

literal pasv (to change to passive mode under Windows ftp CLI) 

• sftp

$ sftp -b /tmp/1 -o port=2222 jephe@1.2.3.4

$ more 1
put /etc/hosts

$ more 1
rm hosts

• grep/cat

grep -v "^#" /etc/httpd/conf/httpd.conf | cat -s | less

• tcpdump 

  Collecting a TCP dump from the server using a command like the following:

  tcpdump -s0 -w /tmp/tcpdump.pcap -i any host <client ip> and port 80
  
  
  and generating traffic (HTTP or LDAP) to the server captures evidence of the server not responding to the TCP SYN packets.  The output file can be analyzed with a command like
  tcpdump -r /<path>/<to>/tcpdump.pcap

For monitoring openbsd firewall PF rules, you can use 'tcpdump -n -e -ttt -i pflog0' to see which pf rules is matching the traffic, pass or block. use 'pfctl -sr' to get output of in-memory rules, the first block or pass rule will be rule 0, followed by rule 1, rule 2 and so on.

• Tshark

     tshark -i eth0 -f 'host 10.0.2.1' -w /tmp/upstream.cap -S

• screen

 screen -S main (manually)

/home/jephe/.bash_profile contact 'screen -D -R main'. this way, if I login server, it will disconnect then reconnect my screen session 'main' automatically every time. So we keep one session only.