OpenSSL
This page is concerned with the software OpenSSL, which is an implementation of the SSL/TLS protocols.
For background information, also see the Cryptography page on this wiki.
References
- OpenSSL website
- The project page.
- Transport Layer Security
- Wikipedia article on TLS
- OpenSSL
- Wikipedia article on OpenSSL
- man pages
- One of the more important information sources when it comes to working with OpenSSL, are the various man pages of the project. The main page (
man openssl
) provides the summary of the available commands. Each command also has its own dedicated man page, e.g.man x509
has all the details when it comes to working with X.509 certificates.
Terms & Glossary
- CA
- Certificate Authority
- CRL
- Certificate Revocation List
- CSR
- Certificate Signing Request
- GnuTLS
- An alternative implementation of SSL/TLS which was created because the OpenSSL license is not compatible to the GPL. GnuTLS is licensed under the LGPL.
- SSL
- Secure Sockets Layer. A cryptographic protocol. The predecessor of TLS.
- TLS
- Transport Layer Security. A cryptographic protocol. The successor of SSL.
- Wildcard certificate
- A certificate whose common name uses a wildcard (e.g. "*.foobar.com") and that can be used for all subdomains of a given domain.
Key Management
RSA keys (openssl genrsa/rsa)
Create a new private RSA key. The key is formatted for PEM and is protected by a passphrase that you must specify interactively. The encryption algorithm/cipher used is Triple-DES.
openssl genrsa -des3 -out foobar.key 4096
Generate an un-encrypted, PEM formatted version of an existing private RSA key. This is useful if a server process needs to read the key without user interaction (i.e. the user entering the passphrase) during a system reboot. Note: The passphrase is removed because there is no cipher option such as -des3
after the rsa
command.
openssl rsa -in foobar.key -out foobar.key.unsecure
Change the passphrase (requires that you enter the old and the new passphrase). This can also be used to encrypt an unprotected, unencrypted key.
openssl rsa -des3 -in foobar.key -out foobar.key.new
Display details about an RSA key:
openssl rsa -noout -text -in foobar.key
X.509 Certificate Management
Creating a self-signed certificate for a CA
The basis of a CA is a private key and a self-signed certificate to sign CSRs. Once you have the private key, the self-signed certificate is created with this command:
openssl req -new -x509 -days 10950 -key cakey.pem -out cacert.pem
Notes:
- The certificate has the X.509 structure.
- The certificate is signed with the CA's private RSA key.
- The certificate is formatted for PEM.
- The certificate is valid 30 years (= 10'950 days).
- The certificate is capable of issuing other certificates (i.e. it is a CA certificate).
- This is a special usage of the
openssl req
command which is normally used to generate CSRs (see next section).
OpenSSL will interactively ask for various information that describes the CA. The information is then used both for the subject and for the issuer of the certificate (which are identical because the certificate is self-signed). To make the command non-interactive, the information can be specified using the parameter -subj "/attribute1=value1/attribute2=value2[...]"
.
To submit values for all the interactively requested information pieces
openssl req -new -x509 -days 10950 -key cakey.pem -out cacert.pem -subj "/c=xx/st=state/l=city/o=organization name/ou=organizational unit/cn=common name/emailAddress=foo@bar.baz"
Notes:
- Quoting is necessary because the values contain space characters.
- You cannot specify arbitrary attribute names - OpenSSL validates them against valid names according to a schema.
- The schema also contains some limits regarding the lengths of the values. For instance, the "c" attribute requires a two character country code.
- Read the man page to find out how to form more complex
-subj
values.
CSR Management (openssl req)
Create a CSR that is formatted for PEM. This is typically used to request a server certificate.
openssl req -new -key foobar.key -out foobar.csr
Notes:
- You must interactively specify several attributes that describe the subject of the requested certificate. These attributes together form the "distinguished name" (DN) of the certificate.
- The
-subj
parameter can be used to specify the needed values on the command line and thus make the command non-interactive. See the [#Creating_a_self-signed_certificate_for_a_CA previous section] for details. - It is vital to set the "common name" attribute to the FQDN (fully qualified domain name) under which the server will operate! If this is omitted, or specified wrongly, the information in the certificate will not match the information that the server announces to its clients. If a client is picky about this (and it is very likely to be these days), the client will refuse communication with the server!
- Specifying the common name *.foobar.com will create a wildcard certificate that is valid for all sub-domains of foobar.com.
Display details about a CSR:
openssl req -noout -text -in foobar.csr
Signing a CSR and generating a certificate - simple case (openssl x509)
The following command signs a CSR and generates a certificate:
openssl x509 -req -in foobar.csr -days 10950 -CA cacert.pem -CAkey cakey.pem -out foobar.crt
Notes:
- The certificate is valid 30 years (= 10'950 days).
- A random number is generated for the serial number. The options
-CAserial
and-CAcreateserial
can be used to gain control over which serial number is used. Read the man page for more details. - As no extensions of any kind are used, the generated certificate is a v1 (version 1) certificate.
- See the section Setting up a CA infrastructure for details about how to sign CSRs and generate certificates with a proper CA setup.
A certificate can also be generated with extensions, resulting in a v3 (version 3) certificate. Examples are to restrict the certificate to certain usages, or - even more importantly - to indicate which domain name(s) and/or IP address(es) the certificate is bound to. The extensions to be used need to be specified in an "extensions file", like in the following example:
cat << EOF >openssl.cnf basicConstraints=CA:FALSE keyUsage=digitalSignature,nonRepudiation,keyEncipherment subjectAltName=DNS:www.example.com,IP:1.2.3.4 EOF openssl x509 -req -in foobar.csr -days 10950 -CA cacert.pem -CAkey cakey.pem -extfile openssl.cnf -out foobar.crt
Notes:
- The
--extfile
option is used to specify the file that contains the extensions. Seeman x509
andman x509v3_config
for details about the structure of the extensions file and the meaning of each extension. - If bash is used, then a mechanism called "process substitution" can be employed to specify the extensions directly without taking the detour of a real file. The
-extfile
option from the example above would then look like this:-extfile <(printf 'basicConstraints=CA:FALSE\nkeyUsage=digitalSignature,nonRepudiation,keyEncipherment\nsubjectAltName=DNS:www.example.com,IP:1.2.3.4')
- The extension "subjectAltName" (often abbreviated to SAN) is important because modern browser versions (e.g. Firefox since version 48, Chrome since version 58) no longer match the domain or IP address used in a request to the "common name" part of the certificate's subject, because the "common name" format is ambiguous. Instead browsers require the certificate to specify the domain name(s) and/or IP address(es) it is bound to in the "subjectAltName extension.
X.509 Certificate Data Management (openssl x509)
Display details about a certificate:
openssl x509 -noout -text -in foobar.crt
CRL Management (openssl crl)
TODO
Verification
Basics
The goal of verification is to determine whether a given certificate is valid before it is used for any purpose. Verification of a certificate consists of several steps:
- Building of a certificate chain
- The chain starts with the supplied certificate, and ends with the first certificate that is its own issuer (i.e. a self-signed certificate). This is called the root certificate.
- Verification fails if the certificate chain cannot be built
- A certificate is looked up first from a list of untrusted certificates supplied to the verification operation.
- If an untrusted certificate cannot be found, the lookup continues in a list of trusted certificates supplied to the verification operation.
- The root certificate must always come from the list of trusted certificates.
- The exact algorithm of how a certificate is looked up is too complicated to list here. Refer to the man page
man verify
for details.
- Check untrusted certificates's extensions against specified purpose
- The extensions of all untrusted certificates are examined for consistency with the purpose supplied to the verification operation.
- This step is skipped if no purpose is supplied to the verification operation.
- Accepted purposes at the time of writing: sslclient, sslserver, nssslserver, smimesign, smimeencrypt
- Refer to the man page
man x509
for details.
- Check trust of the root certificate of the certificate chain
- The root certificate must be trusted for the purpose supplied to the verification operation.
- Check validity of certificate chain
- The validity period of all certificates in the certificate chain is checked against the current system time.
- Cryptographic signatures must also be valid
A certificate is valid only if all of these steps are successful.
Sources for trusted certificates
A verification operation can be supplied with two sources for trusted certificates:
- A file that contains a concatenated list of trusted certificates in PEM format. This is sometimes also known as the "CA file".
- A folder that contains files with trusted certificates, one certificate per file. This is sometimes also known as the "CA directory".
Note that the file names in the CA directory must use the form "hash.n", where "hash" is the hashed subject name of the certificate, and "n" is a numerical suffix (starting with 0) that distinguishes same-subject certificates from each other. Lookups in the CA directory are made via the subject name hash, starting with suffix 0.
Certificate subject name hash
As explained above, the CA directory must contain files whose names contain a hash of the subject name of the certificate. Use the following command to generate such a hash:
openssl x509 -subject_hash -in foo.pem
Also see man x509
.
OpenSSL provides a utility (a Perl script) that processes a folder that contains many certificate files. The utility generates a symbolic link whose name conforms to the "hash.n" scheme for each certificate file. Basic usage of this utility is
c_rehash /path/to/certificates/folder
The symbolic links are placed in the same folder alongside the certificate file. Also see man c_rehash
.
Long-running server processes
The API function of the OpenSSL library that allows to specify trusted certificates sources is this:
SSL_CTX_load_verify_locations
Cf. the OpenSSL website.
The API functions lets the programmer specify both the "CA file" and "CA directory" location. The locations are processed like this:
- The CA file is loaded into memory when the function is called, and remains there for use by future verification operations.
- The CA directory is not processed immediately. Future verification operations perform lookups when needed.
The impact on long-running server processes (e.g. Courier IMAP server, Apache HTTP server / PHP) is clear: The CA file is static, and if the file content changes any server processes that use it must be restarted. The content of the CA directory, on the other hand, can change dynamically without any server restarts.
Verification on the command line (openssl verify)
The following command verifies the certificate in certificate.pem
against the content of a CA file:
openssl verify -CAfile /etc/ssl/certs/ca-certificates.crt certificate.pem
The following command performs the same verification using a CA directory:
openssl verify -CApath /etc/ssl/certs certificate.pem
Notes:
- The
verify
program continues even after some error occurred, whereas normally the verify operation would halt on the first error. This allows to diagnose verification problems. - The examples above use CA file/directory locations that match the system-wide trust stores of a Debian system.
- Presumable (but untested), if neither CA file nor CA directory are specified, the
verify
program will consult the default locations of the system's trust store (i.e. whatever paths were compiled into OpenSSL)
Setting up a CA infrastructure
Many X.509 management tasks, such as generating a CSR, are solved by more or less trivial one-liner OpenSSL commands. Operating a proper CA that is capable of signing CSRs and generating certificates, however, is definitely non-trivial, because OpenSSL requires setting up a certain infrastructure. This section explains how to set up such an infrastructure for testing purposes.
Important: The instructions in this section are not to be used to set up a production environment! I have not understood or even researched many aspects of X.509, especially security-related stuff, and I am blatantly ignoring these things.
The first thing to do is to create a base directory where all the files that are required by OpenSSL can be kept:
mkdir ca-base-directory
Next we need a configuration file that defines certain aspects of the signing process. OpenSSL has a default configuration file that we could use (on my Debian machine this is /usr/lib/ssl/openssl.cnf
), but it's better to set up your own config file instead of fiddling with any defaults. I suggest to start out by making a copy of the default file:
cd /path/to/ca-base-directory cp /usr/lib/ssl/openssl.cnf .
Now remove everything from the configuration file except for the following sections:
- [ca]
- [CA_default]
- [policy_match]
- [policy_anything]
- [usr_cert]
Here is an excerpt of the most important options in the configuration file. Except for the dir
option, all values in this excerpt are default values taken straight from the default OpenSSL configuration file. By specifying an absolute path for the dir
option, it becomes possible to run openssl ca
from anywhere in the system, i.e. the current directory when running openssl ca
is not relevant. For this to work, other directory options must be based on the dir
option, but fortunately this is already the case with the default values (they use the variable $dir
as can be seen in the excerpt).
dir = /path/to/ca-base-directory # Where everything is kept database = $dir/index.txt # Database index file new_certs_dir = $dir/newcerts # The directory where issued certificates are placed certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # Text file with the next serial number to use in hex private_key = $dir/private/cakey.pem # The CA private key
A few files and folders need to be created to match the configuration file:
cd /path/to/ca-base-directory mkdir newcerts touch index.txt echo "01" >serial mkdir private
Finally, the CA requires a private key and a self-signed certificate to sign CSRs. The following commands place their output into the locations defined by the configuration file.
cd /path/to/ca-base-directory openssl genrsa -des3 -out private/cakey.pem 4096 openssl req -new -x509 -days 10950 -key private/cakey.pem -out cacert.pem
With this infrastructure in place, the following command can be used to sign a CSR and generates a certificate:
openssl ca -config /path/to/ca-base-directory/openssl.cnf -in foobar.csr -out foobar.crt -policy policy_anything
Notes:
- The CA's configuration file needs to be specified.
- The
-policy
option defines which policy from the configuration file should be used. A policy defines which DN (distinguished name) fields must be present in the CSR, and which DN fields will appear in the final certificate.- A value of "match" means the DN field must be present in the CSR and have the same value as the corresponding DN field in the CA certificate. The DN field will be present in the certificate. This might be used by a corporate CA to ensure that the certificates it issues have certain DN field values (e.g. "organizationName" or "countryName"). The
policy_match
policy in the default OpenSSL configuration file shows how to use "match". - A value of "supplied" means the DN field must be present in the CSR and will also be present in the certificate.
- A value of "optional" means the DN field can be present in the CSR but is not required; if it is present in the CSR it will also be present in the certificate.
- A DN field not mentioned in a policy is ignored if it is present in the CSR and will not be present in the certificate (it can be said that the field is "deleted").
- A value of "match" means the DN field must be present in the CSR and have the same value as the corresponding DN field in the CA certificate. The DN field will be present in the certificate. This might be used by a corporate CA to ensure that the certificates it issues have certain DN field values (e.g. "organizationName" or "countryName"). The
- The
policy_anything
specified in the above command refers to a policy from the default OpenSSL configuration file. Thepolicy_anything
policy enforces the "commonName" DN field to be present (= supplied) but defines all other DN fields to be optional.
TODO: More sophisticated examples, e.g. how to override parameters in the CSR, such as the number of days that the certificate will be valid for.
Encoding and Hashes
Encoding with ciphers (openssl enc)
TODO
Generation of hashed passwords (openssl passwd)
TODO
Message Digest commands
TODO
Encoding commands
TODO
Cipher commands (openssl cipher)
To view the ciphers offered by OpenSSL by default:
openssl ciphers
The same command can be supplied with an additional cipher list. OpenSSL resolves this cipher list down to its individual ciphers. Examples:
openssl ciphers DEFAULT # same as just "openssl ciphers" openss ciphers HIGH # encryption ciphers with high security, i.e. with usually >128-bit keys openssl ciphers DH # cipher suites that use Diffie-Hellman openssl ciphers SSLv2 # SSL v2.0 cipher suites openssl ciphers DEFAULT:!SSLv2:!SSLv3 # default cipher suites except SSL v2.0 and SSL v3.0 cipher suites
PKCS#12 (.p12) File Management
Overview
PKCS#12 files store private keys and accompanying certificates. The file contents are usually encrypted with a passphrase - further down there is an example how an unencrypted PKCS#12 file can be created. The OpenSSL command pkcs12
can be used to convert various components into a single PKCS#12 file, or to extract components from a PKCS#12 file and write them to separate files.
References:
man pkcs12
- OpenSSL PKCS#12 FAQ
Inspect a PKCS#12 file
The best way to see what is inside a PKCS#12 file is to convert it to a different format. The -info
option unfortunately provides only minimal information:
openssl pkcs12 -in foo.p12 -info -noout
The following example output says that the PKCS#12 file contains 1 key, but no certificates:
MAC Iteration 2048 MAC verified OK PKCS7 Data Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048
Example for a PKCS#12 file with 1 certificate, but no keys:
MAC Iteration 2048 MAC verified OK PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048 Certificate bag
Example for a PKCS#12 file with 1 key and 2 certificates:
MAC Iteration 2048 MAC verified OK PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048 Certificate bag Certificate bag PKCS7 Data Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048
Example for a PKCS#12 file with 2 keys and 2 certificates:
TODO
Convert from PKCS#12 to PEM
IMPORTANT: You must specify a passphrase of at least 4 characters to encrypt the key(s) in the output PEM file. If you do not specify a passphrase you will not see an error, but the key will NOT be exported. This behaviour can be disabled with the -nodes
(read "no DES") option.
Extract components and write them to an output file using the PEM format:
openssl pkcs12 -in foo.p12 -out foo.pem
Extract only keys but no certificates:
openssl pkcs12 -nocerts -in foo.p12 -out foo.pem
Ditto, but don't encrypt the key(s) in the output PEM file:
openssl pkcs12 -nodes -nocerts -in foo.p12 -out foo.pem
Extract only certificates but no keys:
openssl pkcs12 -nokeys -in foo.p12 -out foo.pem
Convert from PEM to PKCS#12
Gather certificates and an RSA key from different files, all of which must be in PEM format, and store them together in a single PKCS#12 file:
openssl pkcs12 -export -in foo.pem -inkey foo.rsa -out foo.p12
Store a key but no certificates
openssl pkcs12 -export -nocerts -inkey foo.rsa -out foo.p12
Note: So far I have been unable to store more than 1 key in a .p12 file.
Create an unencrypted PKCS#12 file:
openssl pkcs12 -export -in foo.pem -inkey foo.rsa -out foo.p12 -keypbe NONE -certpbe NONE -passout pass:
Notes:
- The
-keypbe
and-certpbe
options define which algorithms should be used to encrypt the keys and certificates in the PKCS#12 file. Specifying the algorithm "NONE" is what causes OpenSSL to use no encryption. - Even with the algorithm "NONE", OpenSSL will still interactively ask you for a password, because that workflow is baked into the
openssl pkcs12
command. The option-passout
can be used to tell OpenSSL to take the password from another source - either from a file supplied as the option value, or from the command line if the special "pass:" option value is supplied. When-passout pass:
is given, then OpenSSL uses an empty string password. - Because the algorithm "NONE" was specified, the empty string password will simply be not used. If
-keypbe
and-certpbe
were not specified, then the PKCS#12 file would be encrypted with an empty string password. - Kudos: The solution for this comes from this StackOverflow post.
Networking
Generic SSL/TLS client (openssl s_client)
The s_client
command can be used to connect to a remote host using SSL/TLS. The command's documentation is available via man s_client
, or on the openssl.org website.
Basic usage:
openssl s_client -connect foo.com:443
Send STARTTLS command for the IMAP or SMTP protocols. A few more protocols are supported, see man s_client
for a complete list.
openssl s_client -connect mail.herzbube.ch:143 -starttls imap openssl s_client -connect smtp.herzbube.ch:25 -starttls smtp
Show certificates sent by the server:
openssl s_client -connect foo.com:12345 -showcerts
Generic SSL/TLS server (openssl s_server)
TODO
Other commands
View content of DH group file
This command prints out information about a DH group file, specifically its size in bit:
openssl dhparam -text -in foo.pem
Create a DH group file
Create a new DH group file with size 2048 bits (see the Guide to Deploying Diffie-Hellman for TLS for details why this is required; keyword: Logjam; also see this security.stackexchange question):
openssl dhparam -out dhparams-2048-bit.pem 2048
I usually place such a file in /etc/ssl/private
, alongside with my private keys, and set permissions of the file so that multiple services can use the file:
cd /etc/ssl/private chmod 400 dhparams-2048-bit.pem chown daemon:daemon dhparams-2048-bit.pem
OpenSSL vulnerability in Debian
A brief check using the SSL certificate checker on the Heise web site
http://www.heise.de/netze/tools/chksslkey
reveals that all the certificates I created on my system are vulnerable to the infamous weakness in the Debian OpenSSL package. This is bad news, but since I cannot ignore the problem I will try to make the best of it and improve my security infrastructure in the following way:
- Revoke all certificates signed so far by CAcert
- Issue a new certificate for ca.herzbube.ch through CAcert, just in case I want to do my own certification business
- Re-issue all the other server certificates through CAcert
- Using CAcert to issue certificates has the benefit that certificates are not valid such a long time (i.e. 30 years) and can be managed through a CRL
- Check out how to manage my own CRL