OpenSSL

From HerzbubeWiki
Jump to: navigation, search

This page is concerned with the software OpenSSL, which is an implementation of the SSL/TLS protocols.

For background information, also see the Cryptography page on this wiki.


References

OpenSSL website 
The project page.
Transport Layer Security 
Wikipedia article on TLS
OpenSSL 
Wikipedia article on OpenSSL
man pages 
One of the more important information sources when it comes to working with OpenSSL, are the various man pages of the project. The main page (man openssl) provides the summary of the available commands. Each command also has its own dedicated man page, e.g. man x509 has all the details when it comes to working with X.509 certificates.


Terms & Glossary

CA 
Certificate Authority
CRL 
Certificate Revocation List
CSR 
Certificate Signing Request
GnuTLS 
An alternative implementation of SSL/TLS which was created because the OpenSSL license is not compatible to the GPL. GnuTLS is licensed under the LGPL.
SSL 
Secure Sockets Layer. A cryptographic protocol. The predecessor of TLS.
TLS 
Transport Layer Security. A cryptographic protocol. The successor of SSL.
Wildcard certificate 
A certificate whose common name uses a wildcard (e.g. "*.foobar.com") and that can be used for all subdomains of a given domain.


Key Management

RSA keys (openssl genrsa/rsa)

Create a new private RSA key. The key is formatted for PEM and is protected by a passphrase that you must specify interactively. The encryption algorithm/cipher used is Triple-DES.

openssl genrsa -des3 -out foobar.key 1024

Generate an un-encrypted, PEM formatted version of an existing private RSA key. This is useful if a server process needs to read the key without user interaction (i.e. the user entering the passphrase) during a system reboot. Note: The passphrase is removed because there is no cipher option such as -des3 after the rsa command.

openssl rsa -in foobar.key -out foobar.key.unsecure

Change the passphrase (requires that you enter the old and the new passphrase). This can also be used to encrypt an unprotected, unencrypted key.

openssl rsa -des3 -in foobar.key -out foobar.key.new

Display details about an RSA key:

openssl rsa -noout -text -in foobar.key


X.509 Certificate Management

Setting up a CA infrastructure

Many X.509 management tasks, such as generating a CSR, are solved by more or less trivial one-liner OpenSSL commands. Signing a CSR, however, is definitely non-trivial, even for testing purposes, because OpenSSL requires setting up a certain infrastructure for operating a CA that is capable of signing CSRs. This section explains how to set up such an infrastructure for testing purposes.

Important: The instructions in this section are not to be used to set up a production environment! I have not understood or even researched many aspects of X.509, especially security-related stuff, and I am blatantly ignoring these things.

The first thing to do is to create a base directory where all the files that are required by OpenSSL can be kept:

mkdir ca-base-directory

Next we need a configuration file that defines certain aspects of the signing process. OpenSSL has a default configuration file that we could use (on my Debian machine this is /usr/lib/ssl/openssl.cnf), but it's better to set up your own config file instead of fiddling with any defaults. I suggest to start out by making a copy of the default file:

cd /path/to/ca-base-directory
cp /usr/lib/ssl/openssl.cnf .

Now remove everything from the configuration file except for the following sections:

  • [ca]
  • [CA_default]
  • [policy_match]
  • [policy_anything]
  • [usr_cert]

Here is an excerpt of the most important options in the configuration file. Except for the dir option, all values in this excerpt are default values taken straight from the default OpenSSL configuration file. By specifying an absolute path for the dir option, it becomes possible to run openssl ca from anywhere in the system, i.e. the current directory when running openssl ca is not relevant. For this to work, other directory options must be based on the dir option, but fortunately this is already the case with the default values (they use the variable $dir as can be seen in the excerpt).

dir           = /path/to/ca-base-directory     # Where everything is kept
database      = $dir/index.txt                 # Database index file
new_certs_dir = $dir/newcerts                  # The directory where issued certificates are placed
certificate   = $dir/cacert.pem                # The CA certificate
serial        = $dir/serial                    # Text file with the next serial number to use in hex
private_key   = $dir/private/cakey.pem         # The CA private key

A few files and folders need to be created to match the configuration file:

cd /path/to/ca-base-directory
mkdir newcerts
touch index.txt
echo "01" >serial
mkdir private

Finally, the CA requires a private key and a self-signed certificate to sign CSRs. The following commands place their output into the locations defined by the configuration file.

cd /path/to/ca-base-directory
openssl genrsa -des3 -out private/cakey.pem 4096
openssl req -new -x509 -days 10950 -key private/cakey.pem -out cacert.pem


CA Management (openssl ca)

Sign a certificate signing request (CSR):

openssl ca -config /path/to/ca-base-directory/openssl.cnf -in foobar.csr -out foobar.crt -policy policy_anything

Notes:

  • A configuration file is required, it is simply not possible to specify all the required options on the command line. The default configuration file (/usr/lib/ssl/openssl.cnf on my Debian machine) is almost certainly not set up correctly for operating a CA, so a custom configuration file is needed. Read the previous section for details how to set up the infrastructure for operating a CA.
  • The -policy option defines which policy from the configuration file should be used. A policy defines which DN (distinguished name) fields must be present in the CSR, and which DN fields will appear in the final certificate.
    • A value of "match" means the DN field must be present in the CSR and have the same value as the corresponding DN field in the CA certificate. The DN field will be present in the certificate. This might be used by a corporate CA to ensure that the certificates it issues have certain DN field values (e.g. "organizationName" or "countryName"). The policy_match policy in the default OpenSSL configuration file shows how to use "match".
    • A value of "supplied" means the DN field must be present in the CSR and will also be present in the certificate.
    • A value of "optional" means the DN field can be present in the CSR but is not required; if it is present in the CSR it will also be present in the certificate.
    • A DN field not mentioned in a policy is ignored if it is present in the CSR and will not be present in the certificate (it can be said that the field is "deleted").
The policy_anything specified in the above command refers to a policy from the default OpenSSL configuration file. The policy_anything policy enforces the "commonName" DN field to be present (= supplied) but defines all other DN fields to be optional.


TODO: More sophisticated examples, e.g. how to override parameters in the CSR, such as the number of days that the certificate will be valid for.


X.509 Certificate Data Management (openssl x509)

Display details about a certificate:

openssl x509 -noout -text -in foobar.crt


CSR Management (openssl req)

Create a CSR that is formatted for PEM. This is typically used to request a server certificate.

openssl req -new -key foobar.key -out foobar.csr

Notes:

  • You must interactively specify several attributes that describe the requested certificate. These attributes together form the "distinguished name" (DN) of the certificate.
  • It is absolutely vital to set the "common name" attribute to the FQDN (fully qualified domain name) under which the server will operate! If this is omitted, or specified wrongly, the information in the certificate will not match the information that the server announces to its clients. If a client is picky about this (and it is very likely to be these days), the client will refuse communication with the server!
  • Specifying the common name *.foobar.com will create a wildcard certificate that is valid for all sub-domains of foobar.com


Display details about a CSR:

openssl req -noout -text -in foobar.csr


Create a new self-signed certificate with the following attributes:

  • The certificate has the X.509 structure
  • The certificate is signed with the CA's private RSA key
  • The certificate is formatted for PEM
  • The certificate is valid 30 years (= 10'950 days)
  • The certificate is capable of issuing other certificates (i.e. it is a CA certificate)
openssl req -new -x509 -days 10950 -key ca.herzbube.ch.key -out ca.herzbube.ch.crt


CRL Management (openssl crl)

TODO


Verification

Basics

The goal of verification is to determine whether a given certificate is valid before it is used for any purpose. Verification of a certificate consists of several steps:

  1. Building of a certificate chain
    • The chain starts with the supplied certificate, and ends with the first certificate that is its own issuer (i.e. a self-signed certificate). This is called the root certificate.
    • Verification fails if the certificate chain cannot be built
    • A certificate is looked up first from a list of untrusted certificates supplied to the verification operation.
    • If an untrusted certificate cannot be found, the lookup continues in a list of trusted certificates supplied to the verification operation.
    • The root certificate must always come from the list of trusted certificates.
    • The exact algorithm of how a certificate is looked up is too complicated to list here. Refer to the man page man verify for details.
  2. Check untrusted certificates's extensions against specified purpose
    • The extensions of all untrusted certificates are examined for consistency with the purpose supplied to the verification operation.
    • This step is skipped if no purpose is supplied to the verification operation.
    • Accepted purposes at the time of writing: sslclient, sslserver, nssslserver, smimesign, smimeencrypt
    • Refer to the man page man x509 for details.
  3. Check trust of the root certificate of the certificate chain
    • The root certificate must be trusted for the purpose supplied to the verification operation.
  4. Check validity of certificate chain
    • The validity period of all certificates in the certificate chain is checked against the current system time.
    • Cryptographic signatures must also be valid

A certificate is valid only if all of these steps are successful.


Sources for trusted certificates

A verification operation can be supplied with two sources for trusted certificates:

  • A file that contains a concatenated list of trusted certificates in PEM format. This is sometimes also known as the "CA file".
  • A folder that contains files with trusted certificates, one certificate per file. This is sometimes also known as the "CA directory".

Note that the file names in the CA directory must use the form "hash.n", where "hash" is the hashed subject name of the certificate, and "n" is a numerical suffix (starting with 0) that distinguishes same-subject certificates from each other. Lookups in the CA directory are made via the subject name hash, starting with suffix 0.


Certificate subject name hash

As explained above, the CA directory must contain files whose names contain a hash of the subject name of the certificate. Use the following command to generate such a hash:

openssl x509 -subject_hash -in foo.pem

Also see man x509.


OpenSSL provides a utility (a Perl script) that processes a folder that contains many certificate files. The utility generates a symbolic link whose name conforms to the "hash.n" scheme for each certificate file. Basic usage of this utility is

c_rehash /path/to/certificates/folder

The symbolic links are placed in the same folder alongside the certificate file. Also see man c_rehash.


Long-running server processes

The API function of the OpenSSL library that allows to specify trusted certificates sources is this:

SSL_CTX_load_verify_locations

Cf. the OpenSSL website.

The API functions lets the programmer specify both the "CA file" and "CA directory" location. The locations are processed like this:

  • The CA file is loaded into memory when the function is called, and remains there for use by future verification operations.
  • The CA directory is not processed immediately. Future verification operations perform lookups when needed.


The impact on long-running server processes (e.g. Courier IMAP server, Apache HTTP server / PHP) is clear: The CA file is static, and if the file content changes any server processes that use it must be restarted. The content of the CA directory, on the other hand, can change dynamically without any server restarts.


Verification on the command line (openssl verify)

The following command verifies the certificate in certificate.pem against the content of a CA file:

openssl verify -CAfile /etc/ssl/certs/ca-certificates.crt certificate.pem

The following command performs the same verification using a CA directory:

openssl verify -CApath /etc/ssl/certs certificate.pem

Notes:

  • The verify program continues even after some error occurred, whereas normally the verify operation would halt on the first error. This allows to diagnose verification problems.
  • The examples above use CA file/directory locations that match the system-wide trust stores of a Debian system.
  • Presumable (but untested), if neither CA file nor CA directory are specified, the verify program will consult the default locations of the system's trust store (i.e. whatever paths were compiled into OpenSSL)


Encoding and Hashes

Encoding with ciphers (openssl enc)

TODO


Generation of hashed passwords (openssl passwd)

TODO


Message Digest commands

TODO


Encoding commands

TODO


Cipher commands (openssl cipher)

To view the ciphers offered by OpenSSL by default:

openssl ciphers


The same command can be supplied with an additional cipher list. OpenSSL resolves this cipher list down to its individual ciphers. Examples:

openssl ciphers DEFAULT       # same as just "openssl ciphers"
openss ciphers HIGH           # encryption ciphers with high security, i.e. with usually >128-bit keys
openssl ciphers DH            # cipher suites that use Diffie-Hellman
openssl ciphers SSLv2         # SSL v2.0 cipher suites
openssl ciphers DEFAULT:!SSLv2:!SSLv3   # default cipher suites except SSL v2.0 and SSL v3.0 cipher suites


PKCS#12 (.p12) File Management

Overview

PKCS#12 files store private keys and accompanying certificates. The file contents are protected with a passphrase. The OpenSSL command pkcs12 can be used to convert various components into a single PKCS#12 file, or to extract components from a PKCS#12 file and write them to separate files.

References:


Inspect a .p12 file

The best way to see what is inside a .p12 file is to convert it to a different format. The -info option unfortunately provides only minimal information:

openssl pkcs12 -in foo.p12 -info -noout

The following example output says that the .p12 file contains 1 key, but no certificates:

MAC Iteration 2048
MAC verified OK
PKCS7 Data
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048

Example for a .p12 file with 1 certificate, but no keys:

MAC Iteration 2048
MAC verified OK
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
Certificate bag

Example for a .p12 file with 1 key and 2 certificates:

MAC Iteration 2048
MAC verified OK
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
Certificate bag
Certificate bag
PKCS7 Data
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048

Example for a .p12 file with 2 keys and 2 certificates:

TODO


Convert from .p12 to .pem

IMPORTANT: You must specify a passphrase of at least 4 characters to encrypt the key(s) in the output PEM file. If you do not specify a passphrase you will not see an error, but the key will NOT be exported. This behaviour can be disabled with the -nodes (read "no DES") option.


Extract components and write them to an output file using the PEM format:

openssl pkcs12 -in foo.p12 -out foo.pem

Extract only keys but no certificates:

openssl pkcs12 -nocerts -in foo.p12 -out foo.pem

Ditto, but don't encrypt the key(s) in the output PEM file:

openssl pkcs12 -nodes -nocerts -in foo.p12 -out foo.pem

Extract only certificates but no keys:

openssl pkcs12 -nokeys -in foo.p12 -out foo.pem


Convert from .pem to .p12

Gather certificates and an RSA key from different files, all of which must be in PEM format, and store them together in a single PKCS#12 file:

openssl pkcs12 -export -in foo.pem -inkey foo.rsa -out foo.p12

Store a key but no certificates

openssl pkcs12 -export -nocerts -inkey foo.rsa -out foo.p12

Note: So far I have been unable to store more than 1 key in a .p12 file.


Networking

Generic SSL/TLS client (openssl s_client)

The s_client command can be used to connect to a remote host using SSL/TLS. The command's documentation is available via man s_client, or on the openssl.org website.

Basic usage:

openssl s_client -connect foo.com:443

Send STARTTLS command for the IMAP or SMTP protocols. A few more protocols are supported, see man s_client for a complete list.

openssl s_client -connect mail.herzbube.ch:143 -starttls imap
openssl s_client -connect smtp.herzbube.ch:25 -starttls smtp

Show certificates sent by the server:

openssl s_client -connect foo.com:12345 -showcerts


Generic SSL/TLS server (openssl s_server)

TODO


Other commands

View content of DH group file

This command prints out information about a DH group file, specifically its size in bit:

openssl dhparam -text -in foo.pem


Create a DH group file

Create a new DH group file with size 2048 bits (see the Guide to Deploying Diffie-Hellman for TLS for details why this is required; keyword: Logjam; also see this security.stackexchange question):

openssl dhparam -out dhparams-2048-bit.pem 2048

I usually place such a file in /etc/ssl/private, alongside with my private keys, and set permissions of the file so that multiple services can use the file:

cd /etc/ssl/private
chmod 400 dhparams-2048-bit.pem
chown daemon:daemon dhparams-2048-bit.pem


OpenSSL vulnerability in Debian

A brief check using the SSL certificate checker on the Heise web site

http://www.heise.de/netze/tools/chksslkey

reveals that all the certificates I created on my system are vulnerable to the infamous weakness in the Debian OpenSSL package. This is bad news, but since I cannot ignore the problem I will try to make the best of it and improve my security infrastructure in the following way:

  • Revoke all certificates signed so far by CAcert
  • Issue a new certificate for ca.herzbube.ch through CAcert, just in case I want to do my own certification business
  • Re-issue all the other server certificates through CAcert
  • Using CAcert to issue certificates has the benefit that certificates are not valid such a long time (i.e. 30 years) and can be managed through a CRL
  • Check out how to manage my own CRL