OpenSSL

From HerzbubeWiki
Jump to navigation Jump to search

This page is concerned with the software OpenSSL, which is an implementation of the SSL/TLS protocols.

For background information, also see the Cryptography page on this wiki.


References

OpenSSL website
The project page.
Transport Layer Security
Wikipedia article on TLS
OpenSSL
Wikipedia article on OpenSSL
man pages
One of the more important information sources when it comes to working with OpenSSL, are the various man pages of the project. The main page (man openssl) provides the summary of the available commands. Each command also has its own dedicated man page, e.g. man x509 has all the details when it comes to working with X.509 certificates.


Terms & Glossary

CA
Certificate Authority
CRL
Certificate Revocation List
CSR
Certificate Signing Request
GnuTLS
An alternative implementation of SSL/TLS which was created because the OpenSSL license is not compatible to the GPL. GnuTLS is licensed under the LGPL.
SSL
Secure Sockets Layer. A cryptographic protocol. The predecessor of TLS.
TLS
Transport Layer Security. A cryptographic protocol. The successor of SSL.
Wildcard certificate
A certificate whose common name uses a wildcard (e.g. "*.foobar.com") and that can be used for all subdomains of a given domain.


Key Management

RSA keys (openssl genrsa/rsa)

Create a new private RSA key. The key is formatted for PEM and is protected by a passphrase that you must specify interactively. The encryption algorithm/cipher used is Triple-DES. 

openssl genrsa -des3 -out foobar.key 4096

Generate an un-encrypted, PEM formatted version of an existing private RSA key. This is useful if a server process needs to read the key without user interaction (i.e. the user entering the passphrase) during a system reboot. Note: The passphrase is removed because there is no cipher option such as -des3 after the rsa command. 

openssl rsa -in foobar.key -out foobar.key.unsecure

Change the passphrase (requires that you enter the old and the new passphrase). This can also be used to encrypt an unprotected, unencrypted key. 

openssl rsa -des3 -in foobar.key -out foobar.key.new

Display details about an RSA key: 

openssl rsa -noout -text -in foobar.key


X.509 Certificate Management

Creating a self-signed certificate for a CA

The basis of a CA is a private key and a self-signed certificate to sign CSRs. Once you have the private key, the self-signed certificate is created with this command: 

openssl req -new -x509 -days 10950 -key cakey.pem -out cacert.pem

Notes:

  • The certificate has the X.509 structure.
  • The certificate is signed with the CA's private RSA key.
  • The certificate is formatted for PEM.
  • The certificate is valid 30 years (= 10'950 days).
  • The certificate is capable of issuing other certificates (i.e. it is a CA certificate).
  • This is a special usage of the openssl req command which is normally used to generate CSRs (see next section).


OpenSSL will interactively ask for various information that describes the CA. The information is then used both for the subject and for the issuer of the certificate (which are identical because the certificate is self-signed). To make the command non-interactive, the information can be specified using the parameter -subj "/attribute1=value1/attribute2=value2[...]".

To submit values for all the interactively requested information pieces 

openssl req -new -x509 -days 10950 -key cakey.pem -out cacert.pem -subj "/c=xx/st=state/l=city/o=organization name/ou=organizational unit/cn=common name/emailAddress=foo@bar.baz"

Notes:

  • Quoting is necessary because the values contain space characters.
  • You cannot specify arbitrary attribute names - OpenSSL validates them against valid names according to a schema.
  • The schema also contains some limits regarding the lengths of the values. For instance, the "c" attribute requires a two character country code.
  • Read the man page to find out how to form more complex -subj values.


CSR Management (openssl req)

Create a CSR that is formatted for PEM. This is typically used to request a server certificate. 

openssl req -new -key foobar.key -out foobar.csr

Notes:

  • You must interactively specify several attributes that describe the subject of the requested certificate. These attributes together form the "distinguished name" (DN) of the certificate.
  • The -subj parameter can be used to specify the needed values on the command line and thus make the command non-interactive. See the [#Creating_a_self-signed_certificate_for_a_CA previous section] for details.
  • It is vital to set the "common name" attribute to the FQDN (fully qualified domain name) under which the server will operate! If this is omitted, or specified wrongly, the information in the certificate will not match the information that the server announces to its clients. If a client is picky about this (and it is very likely to be these days), the client will refuse communication with the server!
  • Specifying the common name *.foobar.com will create a wildcard certificate that is valid for all sub-domains of foobar.com.


Display details about a CSR: 

openssl req -noout -text -in foobar.csr


Signing a CSR and generating a certificate - simple case (openssl x509)

The following command signs a CSR and generates a certificate: 

openssl x509 -req -in foobar.csr -days 10950 -CA cacert.pem -CAkey cakey.pem -out foobar.crt

Notes:

  • The certificate is valid 30 years (= 10'950 days).
  • A random number is generated for the serial number. The options -CAserial and -CAcreateserial can be used to gain control over which serial number is used. Read the man page for more details.
  • As no extensions of any kind are used, the generated certificate is a v1 (version 1) certificate.
  • See the section Setting up a CA infrastructure for details about how to sign CSRs and generate certificates with a proper CA setup.


A certificate can also be generated with extensions, resulting in a v3 (version 3) certificate. Examples are to restrict the certificate to certain usages, or - even more importantly - to indicate which domain name(s) and/or IP address(es) the certificate is bound to. The extensions to be used need to be specified in an "extensions file", like in the following example: 

cat << EOF >openssl.cnf
basicConstraints=CA:FALSE
keyUsage=digitalSignature,nonRepudiation,keyEncipherment
subjectAltName=DNS:www.example.com,IP:1.2.3.4
EOF
openssl x509 -req -in foobar.csr -days 10950 -CA cacert.pem -CAkey cakey.pem -extfile openssl.cnf -out foobar.crt

Notes:

  • The --extfile option is used to specify the file that contains the extensions. See man x509 and man x509v3_config for details about the structure of the extensions file and the meaning of each extension.
  • If bash is used, then a mechanism called "process substitution" can be employed to specify the extensions directly without taking the detour of a real file. The -extfile option from the example above would then look like this:
    -extfile <(printf 'basicConstraints=CA:FALSE\nkeyUsage=digitalSignature,nonRepudiation,keyEncipherment\nsubjectAltName=DNS:www.example.com,IP:1.2.3.4')
  • The extension "subjectAltName" (often abbreviated to SAN) is important because modern browser versions (e.g. Firefox since version 48, Chrome since version 58) no longer match the domain or IP address used in a request to the "common name" part of the certificate's subject, because the "common name" format is ambiguous. Instead browsers require the certificate to specify the domain name(s) and/or IP address(es) it is bound to in the "subjectAltName extension.


X.509 Certificate Data Management (openssl x509)

Display details about a certificate: 

openssl x509 -noout -text -in foobar.crt


CRL Management (openssl crl)

TODO


Verification

Basics

The goal of verification is to determine whether a given certificate is valid before it is used for any purpose. Verification of a certificate consists of several steps:

  1. Building of a certificate chain
    • The chain starts with the supplied certificate, and ends with the first certificate that is its own issuer (i.e. a self-signed certificate). This is called the root certificate.
    • Verification fails if the certificate chain cannot be built
    • A certificate is looked up first from a list of untrusted certificates supplied to the verification operation.
    • If an untrusted certificate cannot be found, the lookup continues in a list of trusted certificates supplied to the verification operation.
    • The root certificate must always come from the list of trusted certificates.
    • The exact algorithm of how a certificate is looked up is too complicated to list here. Refer to the man page man verify for details.
  2. Check untrusted certificates's extensions against specified purpose
    • The extensions of all untrusted certificates are examined for consistency with the purpose supplied to the verification operation.
    • This step is skipped if no purpose is supplied to the verification operation.
    • Accepted purposes at the time of writing: sslclient, sslserver, nssslserver, smimesign, smimeencrypt
    • Refer to the man page man x509 for details.
  3. Check trust of the root certificate of the certificate chain
    • The root certificate must be trusted for the purpose supplied to the verification operation.
  4. Check validity of certificate chain
    • The validity period of all certificates in the certificate chain is checked against the current system time.
    • Cryptographic signatures must also be valid

A certificate is valid only if all of these steps are successful.


Sources for trusted certificates

A verification operation can be supplied with two sources for trusted certificates:

  • A file that contains a concatenated list of trusted certificates in PEM format. This is sometimes also known as the "CA file".
  • A folder that contains files with trusted certificates, one certificate per file. This is sometimes also known as the "CA directory".

Note that the file names in the CA directory must use the form "hash.n", where "hash" is the hashed subject name of the certificate, and "n" is a numerical suffix (starting with 0) that distinguishes same-subject certificates from each other. Lookups in the CA directory are made via the subject name hash, starting with suffix 0.


Certificate subject name hash

As explained above, the CA directory must contain files whose names contain a hash of the subject name of the certificate. Use the following command to generate such a hash: 

openssl x509 -subject_hash -in foo.pem

Also see man x509.


OpenSSL provides a utility (a Perl script) that processes a folder that contains many certificate files. The utility generates a symbolic link whose name conforms to the "hash.n" scheme for each certificate file. Basic usage of this utility is 

c_rehash /path/to/certificates/folder

The symbolic links are placed in the same folder alongside the certificate file. Also see man c_rehash.


Long-running server processes

The API function of the OpenSSL library that allows to specify trusted certificates sources is this: 

SSL_CTX_load_verify_locations

Cf. the OpenSSL website.

The API functions lets the programmer specify both the "CA file" and "CA directory" location. The locations are processed like this:

  • The CA file is loaded into memory when the function is called, and remains there for use by future verification operations.
  • The CA directory is not processed immediately. Future verification operations perform lookups when needed.


The impact on long-running server processes (e.g. Courier IMAP server, Apache HTTP server / PHP) is clear: The CA file is static, and if the file content changes any server processes that use it must be restarted. The content of the CA directory, on the other hand, can change dynamically without any server restarts.


Verification on the command line (openssl verify)

The following command verifies the certificate in certificate.pem against the content of a CA file: 

openssl verify -CAfile /etc/ssl/certs/ca-certificates.crt certificate.pem

The following command performs the same verification using a CA directory: 

openssl verify -CApath /etc/ssl/certs certificate.pem

Notes:

  • The verify program continues even after some error occurred, whereas normally the verify operation would halt on the first error. This allows to diagnose verification problems.
  • The examples above use CA file/directory locations that match the system-wide trust stores of a Debian system.
  • Presumable (but untested), if neither CA file nor CA directory are specified, the verify program will consult the default locations of the system's trust store (i.e. whatever paths were compiled into OpenSSL)


Setting up a CA infrastructure

Many X.509 management tasks, such as generating a CSR, are solved by more or less trivial one-liner OpenSSL commands. Operating a proper CA that is capable of signing CSRs and generating certificates, however, is definitely non-trivial, because OpenSSL requires setting up a certain infrastructure. This section explains how to set up such an infrastructure for testing purposes.


Important: The instructions in this section are not to be used to set up a production environment! I have not understood or even researched many aspects of X.509, especially security-related stuff, and I am blatantly ignoring these things.


The first thing to do is to create a base directory where all the files that are required by OpenSSL can be kept: 

mkdir ca-base-directory


Next we need a configuration file that defines certain aspects of the signing process. OpenSSL has a default configuration file that we could use (on my Debian machine this is /usr/lib/ssl/openssl.cnf), but it's better to set up your own config file instead of fiddling with any defaults. I suggest to start out by making a copy of the default file: 

cd /path/to/ca-base-directory
cp /usr/lib/ssl/openssl.cnf .


Now remove everything from the configuration file except for the following sections:

  • [ca]
  • [CA_default]
  • [policy_match]
  • [policy_anything]
  • [usr_cert]


Here is an excerpt of the most important options in the configuration file. Except for the dir option, all values in this excerpt are default values taken straight from the default OpenSSL configuration file. By specifying an absolute path for the dir option, it becomes possible to run openssl ca from anywhere in the system, i.e. the current directory when running openssl ca is not relevant. For this to work, other directory options must be based on the dir option, but fortunately this is already the case with the default values (they use the variable $dir as can be seen in the excerpt). 

dir           = /path/to/ca-base-directory     # Where everything is kept
database      = $dir/index.txt                 # Database index file
new_certs_dir = $dir/newcerts                  # The directory where issued certificates are placed
certificate   = $dir/cacert.pem                # The CA certificate
serial        = $dir/serial                    # Text file with the next serial number to use in hex
private_key   = $dir/private/cakey.pem         # The CA private key


A few files and folders need to be created to match the configuration file: 

cd /path/to/ca-base-directory
mkdir newcerts
touch index.txt
echo "01" >serial
mkdir private


Finally, the CA requires a private key and a self-signed certificate to sign CSRs. The following commands place their output into the locations defined by the configuration file. 

cd /path/to/ca-base-directory
openssl genrsa -des3 -out private/cakey.pem 4096
openssl req -new -x509 -days 10950 -key private/cakey.pem -out cacert.pem


With this infrastructure in place, the following command can be used to sign a CSR and generates a certificate: 

openssl ca -config /path/to/ca-base-directory/openssl.cnf -in foobar.csr -out foobar.crt -policy policy_anything

Notes:

  • The CA's configuration file needs to be specified.
  • The -policy option defines which policy from the configuration file should be used. A policy defines which DN (distinguished name) fields must be present in the CSR, and which DN fields will appear in the final certificate.
    • A value of "match" means the DN field must be present in the CSR and have the same value as the corresponding DN field in the CA certificate. The DN field will be present in the certificate. This might be used by a corporate CA to ensure that the certificates it issues have certain DN field values (e.g. "organizationName" or "countryName"). The policy_match policy in the default OpenSSL configuration file shows how to use "match".
    • A value of "supplied" means the DN field must be present in the CSR and will also be present in the certificate.
    • A value of "optional" means the DN field can be present in the CSR but is not required; if it is present in the CSR it will also be present in the certificate.
    • A DN field not mentioned in a policy is ignored if it is present in the CSR and will not be present in the certificate (it can be said that the field is "deleted").
The policy_anything specified in the above command refers to a policy from the default OpenSSL configuration file. The policy_anything policy enforces the "commonName" DN field to be present (= supplied) but defines all other DN fields to be optional.


TODO: More sophisticated examples, e.g. how to override parameters in the CSR, such as the number of days that the certificate will be valid for.


Encoding and Hashes

Encoding with ciphers (openssl enc)

TODO


Generation of hashed passwords (openssl passwd)

TODO


Message Digest commands

TODO


Encoding commands

TODO


Cipher commands (openssl cipher)

To view the ciphers offered by OpenSSL by default: 

openssl ciphers


The same command can be supplied with an additional cipher list. OpenSSL resolves this cipher list down to its individual ciphers. Examples: 

openssl ciphers DEFAULT       # same as just "openssl ciphers"
openss ciphers HIGH           # encryption ciphers with high security, i.e. with usually >128-bit keys
openssl ciphers DH            # cipher suites that use Diffie-Hellman
openssl ciphers SSLv2         # SSL v2.0 cipher suites
openssl ciphers DEFAULT:!SSLv2:!SSLv3   # default cipher suites except SSL v2.0 and SSL v3.0 cipher suites


PKCS#12 (.p12) File Management

Overview

PKCS#12 files store private keys and accompanying certificates. The file contents are usually encrypted with a passphrase - further down there is an example how an unencrypted PKCS#12 file can be created. The OpenSSL command pkcs12 can be used to convert various components into a single PKCS#12 file, or to extract components from a PKCS#12 file and write them to separate files.

References:


Inspect a PKCS#12 file

The best way to see what is inside a PKCS#12 file is to convert it to a different format. The -info option unfortunately provides only minimal information: 

openssl pkcs12 -in foo.p12 -info -noout

The following example output says that the PKCS#12 file contains 1 key, but no certificates: 

MAC Iteration 2048
MAC verified OK
PKCS7 Data
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048

Example for a PKCS#12 file with 1 certificate, but no keys: 

MAC Iteration 2048
MAC verified OK
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
Certificate bag

Example for a PKCS#12 file with 1 key and 2 certificates: 

MAC Iteration 2048
MAC verified OK
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
Certificate bag
Certificate bag
PKCS7 Data
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048

Example for a PKCS#12 file with 2 keys and 2 certificates: 

TODO


Convert from PKCS#12 to PEM

IMPORTANT: You must specify a passphrase of at least 4 characters to encrypt the key(s) in the output PEM file. If you do not specify a passphrase you will not see an error, but the key will NOT be exported. This behaviour can be disabled with the -nodes (read "no DES") option.


Extract components and write them to an output file using the PEM format: 

openssl pkcs12 -in foo.p12 -out foo.pem

Extract only keys but no certificates: 

openssl pkcs12 -nocerts -in foo.p12 -out foo.pem

Ditto, but don't encrypt the key(s) in the output PEM file: 

openssl pkcs12 -nodes -nocerts -in foo.p12 -out foo.pem

Extract only certificates but no keys: 

openssl pkcs12 -nokeys -in foo.p12 -out foo.pem


Convert from PEM to PKCS#12

Gather certificates and an RSA key from different files, all of which must be in PEM format, and store them together in a single PKCS#12 file: 

openssl pkcs12 -export -in foo.pem -inkey foo.rsa -out foo.p12

Store a key but no certificates 

openssl pkcs12 -export -nocerts -inkey foo.rsa -out foo.p12

Note: So far I have been unable to store more than 1 key in a .p12 file.

Create an unencrypted PKCS#12 file: 

openssl pkcs12 -export -in foo.pem -inkey foo.rsa -out foo.p12 -keypbe NONE -certpbe NONE -passout pass:

Notes:

  • The -keypbe and -certpbe options define which algorithms should be used to encrypt the keys and certificates in the PKCS#12 file. Specifying the algorithm "NONE" is what causes OpenSSL to use no encryption.
  • Even with the algorithm "NONE", OpenSSL will still interactively ask you for a password, because that workflow is baked into the openssl pkcs12 command. The option -passout can be used to tell OpenSSL to take the password from another source - either from a file supplied as the option value, or from the command line if the special "pass:" option value is supplied. When -passout pass: is given, then OpenSSL uses an empty string password.
  • Because the algorithm "NONE" was specified, the empty string password will simply be not used. If -keypbe and -certpbe were not specified, then the PKCS#12 file would be encrypted with an empty string password.
  • Kudos: The solution for this comes from this StackOverflow post.


Networking

Generic SSL/TLS client (openssl s_client)

The s_client command can be used to connect to a remote host using SSL/TLS. The command's documentation is available via man s_client, or on the openssl.org website.

Basic usage: 

openssl s_client -connect foo.com:443

Send STARTTLS command for the IMAP or SMTP protocols. A few more protocols are supported, see man s_client for a complete list. 

openssl s_client -connect mail.herzbube.ch:143 -starttls imap
openssl s_client -connect smtp.herzbube.ch:25 -starttls smtp

Show certificates sent by the server: 

openssl s_client -connect foo.com:12345 -showcerts


Generic SSL/TLS server (openssl s_server)

TODO


Other commands

View content of DH group file

This command prints out information about a DH group file, specifically its size in bit: 

openssl dhparam -text -in foo.pem


Create a DH group file

Create a new DH group file with size 2048 bits (see the Guide to Deploying Diffie-Hellman for TLS for details why this is required; keyword: Logjam; also see this security.stackexchange question): 

openssl dhparam -out dhparams-2048-bit.pem 2048

I usually place such a file in /etc/ssl/private, alongside with my private keys, and set permissions of the file so that multiple services can use the file: 

cd /etc/ssl/private
chmod 400 dhparams-2048-bit.pem
chown daemon:daemon dhparams-2048-bit.pem


OpenSSL vulnerability in Debian

A brief check using the SSL certificate checker on the Heise web site 

http://www.heise.de/netze/tools/chksslkey

reveals that all the certificates I created on my system are vulnerable to the infamous weakness in the Debian OpenSSL package. This is bad news, but since I cannot ignore the problem I will try to make the best of it and improve my security infrastructure in the following way:

  • Revoke all certificates signed so far by CAcert
  • Issue a new certificate for ca.herzbube.ch through CAcert, just in case I want to do my own certification business
  • Re-issue all the other server certificates through CAcert
  • Using CAcert to issue certificates has the benefit that certificates are not valid such a long time (i.e. 30 years) and can be managed through a CRL
  • Check out how to manage my own CRL
Retrieved from "https://wiki.herzbube.ch/index.php?title=OpenSSL&oldid=7049"