How to tune Apache performance and harden it


  • understand How Apache works first
the main process is responsible to accept the new incoming connections and pass it to "worker" process, once the worker finishs its job which is reading user's request and send back response. it tells the main process it has done the job and waiting for the new connection

  • Apache and keepalive issue - reduce keepalive value to 2 seconds to allow Apache to terminate the connection soon enough, also
Imagine there're 1000 users connecting to your website at the same moment, how do you handle it? by default, Apache has setting for keepalive 15 seconds and timeout 300 seconds(5 minutes).

If there's no keepalive, each connection will only serve one file which needs to be downloaded for browser display, this is not efficient method since normally a website page contains a few files in order to be properly displayed.

You can reduce the KeepAlive timeout to 2 seconds (do not turn if off). 2 seconds is enough to let the client has the enouchg time to request all the files needed for a page display, also without having to open more than one connection, also let Apache to terminate the connection soon enough so that it can handle more clients then normal.
  • use a worker threaded MPM
use a work thread MPM will use less memory and increase the connection it will be able to handle.

  • Some other tips
Disable ExtendedStatus
HostnameLookups off
Setting Options -Indexes FollowSymLinks (so Apache doesn't have to check if the file is symbolic link or not, it just follow)
reasonable connection timeout value than 300 (use 30 seconds or less to avoid those modem user hog the connection)
  • use thttpd for static files
You can farm all your static files like images to thttpd , make sure it supports keep alive.

  • Apache is the only service on the web server
You can tune StartServers, MinSpareServers, MaxSpareServers, MaxClients and MaxRequestsPerChild parameters accordingly if you are running 3-tier archtecture such as Apache, Java-enabled middle server and database backend.

You can have a big StartServers value and MinSpareServer(0) and MaxSpareServers value the same as StartServers value.

If you trust your application, you can set MaxRequestPerClient a very big value.

  • hardening
user apache 2.2 builtin chroot (chrootdir /path)
use 'TraceEnable Off' to disable trace/track in main httpd.conf
use ServerTokens Prod 





Understanding SSL certificate


  •  Commonly used public key algorithms include RSA(for web), El Gamal(for gpg) and Diffie-Hellman (for ssh).
  • Generate a 1024 bit RSA private key
    Execute command: “openssl genrsa -out private_key.pem 1024”
    e.g.


    $ openssl genrsa -out private_key.pem 1024
    Generating RSA private key, 1024 bit long modulus
    .............................++++++
    ................................................................++++++
    e is 65537 (0x10001)

    Generating a public key from a private key

    Execute command: "openssl rsa -pubout -in private_key.pem -out public_key.pem"
    e.g.



    $ openssl rsa -pubout -in private_key.pem -out public_key.pem
    writing RSA key
    A new file is created, public_key.pem, with the public key.


    Viewing the key elements

    Execute command: "openssl rsa -text -in private_key.pem"


    For security purposes, the integers p and q should be chosen uniformly at random and should be of similar bit-length

    Compute n = pq.


    • n is used as the modulus for both the public and private keys

    All parts of private_key.pem are printed to the screen. This includes the modulus (also referred to as public key and n), public exponent (also referred to as e and exponent; default value is 0x010001 - 65537), private exponent, and primes used to create keys (prime1, also called p, and prime2, also called q), as well as a few other variables used to perform RSA operations faster and the Base64 PEM encoded version of the key.

    The "public key" actually represents a pair of parameters (numbers): a Modulus and a public exponent E. The public exponent is usually chosen to be relatively small (often 3 bytes). The size of the Modulus in bits is referred to as the "key size". A Modulus of size 128 bytes represents a "1024 bit RSA key".
    The "private key" is usually described as a number pair consisting of the same key Modulus and a private exponent D. D is usually chosen to be about the same size as the modulus (~128 bytes). Random selection of Modulus, E and D starts by random selection of two large prime numbers.

  • openssl genrsa command generates a pair of private key and public key actually, not only private key.
  • how to verify a ssl certificate, nowadays, CA use SHA1withRSAencryption to sign the public key as certificate.
To validate the certificate, one needs the certificate that matches
the Issuer (Thawte Server CA) of the first certificate. First one
verifies that the second certificate is of a CA kind; that is, that it
can be used to issue other certificates. This is done by inspecting a
value of the CA attribute in the X509v3 extension
section. Then the RSA public key from the CA certificate is used to
decode the signature on the first certificate to obtain a MD5 hash,
which must match an actual MD5 hash computed over the rest of the
certificate
  • how to verify CA root certificate itself
This is an example of a self-signed certificate, as the issuer and subject are the same. There's no way to verify this certificate except by checking it against itself; instead, these top-level certificates are manually stored by web browsers. Thawte is one of the root certificate authorities recognized by both Microsoft and Netscape. This certificate comes with the web browser and is trusted by default. As a long-lived, globally trusted certificate that can sign anything (as there are no constraints in the X509v3 Basic Constraints section), its matching private key has to be closely guarded.

  • how to show SSL connection information from the browser
  1. Internet Explorer
Version 6.0 - from file menu, choose properties, you will see something like this:
SSL 3.0, RC4 with 128 bit encryption (High); RSA with 1024 bit exchange
or right click on page blank area, choose properties.

Version 7.0 - firstly, show file menu by choosing tools/menu bar, then use the same method as above.

2. Firefox
right click on the blank area of SSL website homepage, choose 'view page info'. You will see something like this:
Connection encrypted: high-grade encryption, AES-256 256bit


  • Useful OpenSSL commands
1. generate a pair of RSA private and public key (will be triple-DES encrypted and PEM format which has begin certificate and end certificate)
$ openssl genrsa -des3 -out server.key 1024
or
$ openssl genrsa -out server.key 1024

note: the most browser only supports RSA 1024bit key. Not either DSA or 2048bit key.

2. View RSA private key details
$ openssl rsa -noout -text -in server.key

3. Create a decrypted PEM version of rsa private/public key pair
$ openssl rsa -des3 -in server.key -out server.key.new
$ mv server.key.new server.key
or
$ openssl rsa -in server.key -out server.key.unsecure

4. create CSR file from private/public key pair file, will be in PEM format
$ openssl req -new -key server.key -out server.csr

5. view CSR file details
$ openssl req -noout -text -in server.csr

6. view CRT file detail
$ openssl x509 -noout -text -in server.crt


  • Creating a certificate authority and certificates with openssl
(refer to http://www.octaldream.com/~scottm/talks/ssl/opensslca.html)

The short answer is to use the CA.sh or CA.pl script provided by OpenSSL (/usr/share/ssl/misc/CA)

The private key contains a series of numbers. Two of those numbers form the "public key", the others are part of your "private key". The "public key" bits are also embedded in your Certificate (we get them from your CSR). To check that the public key in your cert matches the public portion of your private key, you need to view the cert and the key and compare the numbers. To view the Certificate and the key run the commands:
$ openssl x509 -noout -text -in server.crt
$ openssl rsa -noout -text -in server.key
The `modulus' and the `public exponent' portions in the key and the Certificate must match. But since the public exponent is usually 65537 and it's bothering comparing long modulus you can use the following approach:
$ openssl x509 -noout -modulus -in server.crt | openssl md5
$ openssl rsa -noout -modulus -in server.key | openssl md5
And then compare these really shorter numbers. With overwhelming probability they will differ if the keys are different. BTW, if I want to check to which key or certificate a particular CSR belongs you can compute
$ openssl req -noout -modulus -in server.csr | openssl md5
  • convert PEM to DER format
The default certificate format for SSLeay/OpenSSL is PEM, which actually is Base64 encoded DER with header and footer lines. For some applications (e.g. Microsoft Internet Explorer) you need the certificate in plain DER format. You can convert a PEM file cert.pem into the corresponding DER file cert.der with the following command:
$ openssl x509 -in cert.pem -out cert.der -outform DER


  • how to generate .pem/.csr/.crt/ etc certificate files.
cd /usr/share/ssl/certs
make server.pem
make server.crt
etc
  • How to apply new SSL certificate in Apache without restarting service
ps -efH to find out the pidnum of parent httpd
then run kill -USR1 pidnum to make sure it generates new log file
Please refer to http://httpd.apache.org/docs/1.3/stopping.html

  • How to test to make sure the server has been restarted with new certificate?
use openssl s_client command to check if it get retrieve the new certificate:

openssl s_client help to get the the help manual
openssl s_client -connect 10.0.201.104:443 -tls1  to retrieve the certificate content.

or use curl directly
curl -v https://10.0.0.1 will display the certificate, even decrypted for PEM certificate, you can see the start date and end date directly on the screen.
  • Useful URLs
  1. http://www.modssl.org/docs/2.8/ssl_faq.html#ToC27
  2. http://www.herongyang.com/crypto/openssl_rsa.html

tomcat keystore and cacerts

Jephe Wu - http://linuxtechres.blogspot.com

Objective: create SSL certificate for tomcat
Enrironment: Linux server, tomcat, keytool, keystore, cacerts


Concept:
.keystore file which resides /usr/local/tomcat is the default keystore file for tomcat. It's SSL certificate container for server itself, the CA certs file is under /usr/local/jdk/jre/lib/security/.

The self signed SSL certificate generated by keytool or openssl has to be imported to ca certs file in order to avoid SSL warning when the https URL is being accessed from one server to another, without using browser.




Steps:

1. list the existing keys, the default file is .keystore under /usr/local/tomcat folder
# cd /usr/local/tomcat
# keytool -list -v -storepass changeit

2. delete the existing keys( key alias :tomcat)
# cd /usr/local/tomcat
# keytool -delete -alias tomcat -storepass changeit

3. generate self-signed key
# keytool -h for usage
# keytool -genkey -alias tomcat -keysize 1024 -validity 3650 -keypass changeit -storepass changeit
What is your first and last name?
[Unknown]: jephe
What is the name of your organizational unit?
[Unknown]: IS
What is the name of your organization?
[Unknown]: Jephe
What is the name of your City or Locality?
[Unknown]: Singapore
What is the name of your State or Province?
[Unknown]: Singapore
What is the two-letter country code for this unit?
[Unknown]: SG
Is CN=jephe, OU=IS, O=somename, L=Singapore, ST=Singapore, C=SG correct?
[no]: yes


for above self-generated key to work without SSL warning, you need to import to ca certs file

4. list the existing CA certificates from /usr/local/jdk/jre/lib/security/cacerts
# cd /usr/local/jdk/jre/lib/security
# keytool -list -v -keystore cacerts

5. in order to add self-signed key to cacerts, export it first from .keystore file
# keytool -export -alias jephe -keypass changeit -storepass changeit -file /tmp/jephe.der

6. then import to cacerts file under /usr/local/jdk/jre/lib/security/cacerts
# cd /usr/local/jdk/jre/lib/security
# keytool -import -alias jephe -trustcacerts -keystore cacerts -file /tmp/jephe.der -storepass changeit

note: add this key to trusted cacerts and give alias as jephe
you can add one more, but have to give the different alias name


7. you can delete the existing cacert key:
# cd /usr/local/jdk/jre/lib/security
#keytool -delete -keystore cacerts -alias jephe

8. import a openssl generated self signed pem format certificate from openldap server into ca certs file on tomcat server (for ldaps connection from tomcat server to openldap server)

# cd /usr/local/jdk/jre/lib/security
# keytool -import -alias jephe -trustcacerts -keystore cacerts -file /tmp/jephe.pem -storepass changei


References:
1. convert pem to der format

openssl x509 -in cacert.pem -inform PEM -out cacert.der -outform DER