ServiceEncryptionWithSSL

I wrote this page when I first became interested in providing HTTPS access to my Linux box. I first had to learn a lot of stuff about SSL/TLS, and about X.509 certificate management. This page therefore has some introductory material on these topics, but its main concern is to show how to either

• Manage server certificates with CAcert.org as the Certificate Authority (simple), or
• Set up your own Certificate Authory and issue your own certificates (more complicated)

Important: In mid-2018 I switched from CAcert.org to the Let's Encrypt certificate authority. Although the concepts and processes described on this page remain valid, most of the manual management tasks described on this page are fully automated by Let's Encrypt. I have therefore decided to place all information on how to use Let's Encrypt on its own dedicated wiki page. Consequently, as I continue to work with Let's Encrypt, this page probably will not get many more updates.

References

Provides a good overview over SSL/TLS, and also gives some details how to create your own SSL certificates:

http://httpd.apache.org/docs/current/ssl/ssl_faq.html

Another interesting article/FAQ that is OpenLDAP-centric:

http://www.openldap.org/faq/data/cache/185.html

And, of course, the helpful Wikipedia article:

http://en.wikipedia.org/wiki/Transport_Layer_Security

A Debian article that describes how to become your own root CA and sign your own certificates (note that the article was written in 2005 and is probably outdated at least in some parts):

http://www.debian-administration.org/articles/284

Basics on SSL/TLS

What is SSL/TLS? 

A protocol to encrypt communication over a computer network. The protocol runs on top of the TCP and UDP protocols, but below any application protocols such as HTTP or FTP. In other words, it runs between the transport and the application network layers.

What is the difference between SSL and TLS? 

TLS is the successor to SSL. The SSL 3.0 specification was the last version of SSL to be published in 1996 by Netscape. The first version of TLS was TLS 1.0, published in 1999 in RFC 2246. The latest version of TLS as of writing this document is TLS 1.3, published in 2018 in RFC8446.

Implementations? 

Probably most popular is the OpenSSL library, which is basically free but has a license that is not compatible with the GPL (a process to re-license OpenSSL under the Apache License 2.0 is underway since 2017). An alternative was therefore created by the GNU project in the form of the GnuTLS library, which uses the LGPL.

Basics on how communication encryption works

SSL/TLS is based on X.509 digital certificates, which in turn relies on the concept of public key cryptography. As users, we don't need to understand the mathematics that is underlying public key cryptography, so we can simply treat it as some kind of black magic that "just works". What we do need to know, though, are the working principles of public/private keys and X.509.

We begin at the beginning: Network communication occurs between 2 hosts. One of these hosts is the client, the other host is the server. The server has a pair of keys: The first of these is a public key which can be shared with the outside world, the second one is a private key that must remain secret to the server. When the client contacts the server, the server hands out its public key in the form of a digital certificate. A digital certificate is like a package that contains not only the public key, but also additional information that the client uses to make sure that the server really is whom it claims to be (more on that later). Assuming that the client accepts the certificate, it can now start to use the public key that was part of the certificate to encrypt the communication that it sends to the server. The server, being in the possession of the private key, is of course capable of decrypting the communication it receives from the client.

Before the client sends its first encrypted communication, it generates its own public/private key pair. It then encrypts the public key and sends it to the server. This enables the server to encrypt the communication it sends back to the client. The client, being in possession of the appropriate private key, is capable of decrypting the communication it receives from the server.

This is how a basic 2-way encrypted communication channel is established between a client and a server. Once the channel exists, client and server can use it to exchange information that further improves the security of the channel. Details on how this is done is beyond the scope of this wiki, though.

Basics on certificates

Having an encrypted communication channel is all good and fine, but there are more things to consider, chief among them how the client knows that the server it is communicating with is really the one that it thinks it is. When a client starts to communicate with www.herzbube.ch, what happens behind the scenes is that the DNS system resolves www.herzbube.ch to some IP address, then the client establishes a TCP connection with that IP address. So how does the client know that the server at the IP address really is www.herzbube.ch? Enter certificates...

How does the client know whom it is talking to, i.e. the server's identity? 

The server's identity is part of the certificate that it sends to the client. For instance, a certificate may claim that the server's identity is www.herzbube.ch.

How does the client know that the server's identity is genuine? 

The certificate that contains the server's identity must be digitally signed by a third party, a so-called Certificate Authority (CA). By issuing a signature, the CA vouches that the server's identity is genuine. An example for a commercial CA is VeriSign, an example for a non-profit CA is Let's Encrypt, and an example for a community CA is CAcert.org. A short discourse on digital signatures:

• Digital signatures also work with the black magic of public key cryptography
• The CA has a public/private key pair
• The CA creates the signature using its private key
• The CA hands out the public key - in the form of a digital certificate - that others can then use to verify that the signature is genuine

It is the server's responsibility to hand out to the client the CA certificate together with its own signed server certificate. Using the public key that is part of the CA certificate, the client is now capable of verifying that the server certificate's digital signature is genuine.

How does the client know that the CA certificate is genuine? 

Well, therein lies the rub: The client can't really know. Although it is possible to insert many levels of indirection, and to construct long chains of certificate signing, at some point the chain must come to an end. At the end of the chain is a so-called root CA - a certificate authority that nobody else is vouching for. The certificate of a root CA is self-signed. When the client encounters a self-signed CA certificate it can either decide to accept that certificate (and implicitly the entire certificate chain) as genuine, or it can decide to not accept the certificate. An SSL/TLS client bases its decision on whether it finds the self-signed CA certificate in a so-called trust store. The trust store is a list of CA certificates that are automatically trusted to be genuine. Which trust store a client consults, and how that trust store is managed, is an entirely different subject. Operating system usually provide a system-wide trust store that clients can consult either via a library or an OS-specific API. The decision whether or not to consult the system trust store is made by the client, though: For instance, most web browsers use the OS trust store (Chrome, Safari, Edge, Internet Explorer) but Firefox is a notable example that ships with its own embedded trust store.

To sum this up, here's an overview of the certificates that are involved:

• The client needs the server certificate so that it can encrypt messages it sends to the server. The client gets the server certificate from the server.
• If the server certificate is signed, the client needs the CA certificate so that it can verify the signature. If there is a chain of signatures, the client needs the entire certificate chain. The client gets the certificate chain from the server.
• If the server certificate is signed, the client needs a trust store of CA certificates that it ultimately trusts. The trust store is managed locally on the client side.

Generic procedure for creating and signing a certificate

• The private RSA key is generated by the operator of the server
• To let the CA sign the certificate, the server operator must contact the CA
  • First the operator creates a Certificate Signing Request (CSR)
    • The CSR contains various information about the applicant (in our case a server). The most important information for a server certificate is the attribute "common name" (sometimes also called "distinguished name"). This must be set to the FQDN under which the server is going to operate. If the FQDN contains a wildcard character (e.g. "*.herzbube.ch") the resulting certificate is called a "wildcard certificate".
    • The CSR is signed with the private key of the server operator
    • The CSR is accompanied with the public key of the server operator so that the CA can verify the signature when it receives the CSR
  • The operator sends his or her CSR to the CA
• In response to the CSR, the CA generates the signed certificate
  • The CA first verifies the CSR signature against the server operator's public key which must accompany the CSR
  • The CA then generates the certificate using the information that the CSR contains about the applicant. The CA may decide not to use all information in the CSR. For instance, CAcert.org does not issue personalized server certificates, so any personal information in the CSR is simply ignored and will not be part of the certificate.
  • Finally, the CA signs the certificate with its private key
• The server operator receives the signed certificate from the CA and may now use it
• The server operator must make sure that the server hands out the signed certificate together with the CA certificate(s) that the client requires to verify the signature

The Certificate Authority (CA)

Let's Encrypt

See the Let's Encrypt wiki page.

CAcert.org

Get root certificates

The public root certificates of CAcert.org can be obtained from this page (use the PEM variants):

https://www.cacert.org/index.php?id=3

At the time of writing there are 2 root certificates available:

• a class 1 root certificate
• a class 3 root certificate; this one is signed by the class 1 root certificate

Debian used to distribute the CAcert.org certificates in the Debian package ca-certificates. Unfortunately they stopped distributing the certificates in March 2014, so now we must install them manually. Following the conventions of the ca-certificates package, we install the certificate files in /usr/local/share/ca-certificates and create symlinks to those files in /etc/ssl/certs

root@pelargir:~# ls -l /usr/local/share/ca-certificates
total 8
-rw-r--r-- 1 root staff 2610 Oct 31 18:35 cacert.org_class3.crt
-rw-r--r-- 1 root staff 2569 Oct 31 18:35 cacert.org_root.crt

root@pelargir:~# ls -l /etc/ssl/certs | grep cacert.org
lrwxrwxrwx 1 root root     54 Oct 31 18:43 cacert.org_class3.pem -> /usr/local/share/ca-certificates/cacert.org_class3.crt
lrwxrwxrwx 1 root root     52 Oct 31 18:43 cacert.org_root.pem -> /usr/local/share/ca-certificates/cacert.org_root.crt

Certificate chain file

Create a so-called certificate chain file in the following way:

cd /etc/ssl/certs
rm cacert.org.certchain
cat class3.crt >>cacert.org.certchain
cat root.crt >>cacert.org.certchain

The Apache web server is one of the applications that requires this chain file (it is specified by the directive SSLCertificateChainFile): When a client requests a server certificate such as for www.herzbube.ch (which is signed by the class 3 CAcert.org root certificate), the server not only hands out the server certificate, but also provides the root certificates in the chain file. If the client trusts one of the CAcert.org root certificates, it will thereby automatically trust the server certificate, too.

Note: The certificates in the chain file should appear in the order of the certificate chain order, i.e. the root certificate that was used to sign the server certificate should appear first. I have not tested what happens if this is not the case, but it may affect browsers on the client side.

Manually add root certificates to browser

As per Wikipedia:

• Class 1 root certificate is for email
• Class 3 root certificate is for server certificates and software signing

If the browser supports it (e.g. Firefox does), also add the CRLs (certificate revocation list) for both root certificates.

Manually add root certificates to system (Mac OS X)

The following procedure works for me on Mac OS X 10.5.4:

• Download the class 1 and class 3 certificates to the desktop (use the files in the DER format)
• Import the class 1 certificate into the system keychain
  • Double-click on root.crt; this will open "Keychain Access.app" ("Schlüsselbundverwaltung" in German)
  • Select the keychain "System" (refers to /Library/Keychains/System.keychain)
  • Authenticate by specifying your administrator user's name and password
  • Confirm that you want to trust the certificate
• Repeat the procedure by double-clicking on class3.crt (the class 3 certificate)
  • The certificate is automatically trusted because the root cert was already imported

Creating your own Certificate Authority (CA)

In order to act as your own CA, two things are needed

• a private key
• a certificate

In theory, the certificate can be obtained in 2 ways:

• either create a self-signed public certificate
• or create a CSR and convince a third party to issue the certificate for you

In practice, you will only use a self-signed certificate, because no other CA will issue a certificate to you that is capable of doing CA things. The reason is obvious: The other CA would lose control over the certificate issuing process, which is exactly what must never happen to a CA.

Private key

The following command creates a new private key with the following attributes:

• the key is an RSA key
• the key is protected by a passphrase that you must specify; the encryption algorithm/cipher used is Triple-DES
• the key is formatted for PEM

openssl genrsa -des3 -out ca.herzbube.ch.key 1024

Example key file content:

osgiliath:~/ssltest# cat ca.herzbube.ch.key 
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,D238054F2ED5C62B

hRy60/LS02UiWCtxJ5exSCX4nc1/bjSTfkR02PQhyrieAZjWK05YC5cYl17nobho
2evuG8QXCfEn4u6F/ckG9KcVD4LSpEPNOFijd/7lZjE7g07aP7Z11Bhqed03Orzh
UTUwCPUXm9lxE+zypUmfY4F5r0MVSS3v5ULA4ckZMfkuCX4Ed0WNZjmqfyjLbwOr
FA8TMDj7pb/HLiUt74TQyw58nENdNRG2NP/IAapMZvrj0+izxPD9LSmdFAAexAbY
IcACNR4Ou5FypJE5d05PUVAPtBKYJQWdWW3lbOGovpQOdgLW7M13q/7doHYS0IHl
sETcibGidhA6Nt4BI3XQ/LJgGV8v7ravmi7wj7ZvQ0l8INR5208E8rBQ6+QtoBsf
A2nzZrd3L9k3GE05Tll8xp+vjkTkr38Kn31AGDz/W47bQfErYlbUutq4TvGIyFHl
o6+MwqnsLHw88u9CneGprd6UNo49dLbcOrEX7CCnWSfRJR6oTb17yBbtCi2+yfBi
1Q+an7t3jTwDews3OAraSqCt/W5I8IHT4VLsDd+4T7TDXMWXYkqGv8rs+HlsMvGV
YjOpwhFMk0ITJy8CkqP72TrsqRDBlhI7Rif2nT3kPOfJKwK6uc6F0S/Y0EuamhQu
BqzPym4Nn35NmnWp7kI0C8ro9eu2i3shDUyW1GJdNvbzgiO3NVkvwq55LZA7YRQT
tD0UIlEYmG8rD6BpQX3rwmU7utaE3jMuuW/G8BLHsRwo4F1IhbFQ0rmfXNKGBy+l
NNx2zhpA44Hh+KR5RYcyl13nKsgmkevdt365XvGeQNrvrS8xHlGzLA==
-----END RSA PRIVATE KEY-----

Self-signed certificate

The following command creates a new self-signed certificate with the following attributes:

• the certificate has the X.509 structure
• the certificate is signed with the CA's private RSA key
• the certificate is formatted for PEM
• the certificate is valid 30 years (= 10'950 days)
• the certificate is capable of issuing other certificates (i.e. it is a CA certificate)

openssl req -new -x509 -days 10950 -key ca.herzbube.ch.key -out ca.herzbube.ch.crt

Note: You must specify the passphrase in order to access the private RSA key.

OpenSSL will also ask for some information that will be incorporated into the certificate. The information is also called a "distinguished name" (DN) and consists of multiple fields or attributes. Attributes that you want to leave blank must be entered with a value "." It is advisable to enter at least the "common name" attribute:

Example DN:

• Country Name (2 letter code) = CH
• State or Province Name (full name) = .
• Locality Name (eg, city) = .
• Organization Name (eg, company) = herzbube.ch Certificate Authority
• Organizational Unit Name (eg, section) = .
• Common Name (eg, YOUR name) = ca.herzbube.ch
• Email Address = ca@herzbube.ch

Example certificate file content:

osgiliath:~/ssltest# cat ca.herzbube.ch.crt 
-----BEGIN CERTIFICATE-----
MIICsDCCAhmgAwIBAgIJAJS709dIawALMA0GCSqGSIb3DQEBBQUAMEUxCzAJBgNV
BAYTAkFVMRMwEQYDVQQIEwpTb21lLVN0YXRlMSEwHwYDVQQKExhJbnRlcm5ldCBX
aWRnaXRzIFB0eSBMdGQwHhcNMDcwMzI3MTkzMzIxWhcNMzcwMzE5MTkzMzIxWjBF
MQswCQYDVQQGEwJBVTETMBEGA1UECBMKU29tZS1TdGF0ZTEhMB8GA1UEChMYSW50
ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
gQC8TmwotmWgM4mXoBqUOD/BYgwfPQ3Z31uMZnpAE1mvmYuE//pzy2dzTX4Y8wFH
2j6VzNFnqXy0Y5dULLc5kURswCIUSiMVUYWqSc5b+ybv1d0P5CSrypgk4tC0uYAz
L43RAaAp+CJsBjKkesIKzQKfNODD5DoFf2z6SS6pwe52EwIDAQABo4GnMIGkMB0G
A1UdDgQWBBSpWd567nU+3gOlS6b9wquTl3hBljB1BgNVHSMEbjBsgBSpWd567nU+
3gOlS6b9wquTl3hBlqFJpEcwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgTClNvbWUt
U3RhdGUxITAfBgNVBAoTGEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZIIJAJS709dI
awALMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAWy/SVVDw1YvorgU3
6yRtqjXBxSDEqTZRfGfGOxGiYhhJ6QyMHjy4b+MHoJRBYKaaq/ZmJbvob4bYskat
X7vqrXO9/71JXSU10ZRRr7kczXNz0bZqeMtvtKi8ook0bdN/JwELp4pKHqWbI5od
7FuhwUSF17elipH/RuLLpalLfhk=
-----END CERTIFICATE-----

Storage

The private key and the certificate (either self-signed or signed by a third party) should be stored in a permanent location. For instance:

Store the private key in:

/etc/ssl/private

Store the certificate in

/etc/ssl/certs

To follow the guidelines of the ca-certificates package (see section below), we also need to create a hash symlink:

cd /etc/ssl/certs
ln -s ca.herzbube.ch.crt "$(openssl x509 -subject_hash -in ca.herzbube.ch.crt | head -1).0"

Last but not least, for those applications that are not able to follow a certificate chain (e.g. Apache, OpenLDAP), a single file with such a chain should be created:

cd /etc/ssl/certs
rm ca.herzbube.ch.certchain
cat ca.herzbube.ch.crt >>ca.herzbube.ch.certchain
cat class3.crt >>ca.herzbube.ch.certchain
cat root.crt >>ca.herzbube.ch.certchain

The server

Let's Encrypt

Let's Encrypt automatically creates and manages everything described in the following sections. See the Let's Encrypt wiki page for details. To understand what is happening behind the scenes, though, you might still be interested to read on.

Create a server key

The following command creates a new private key. It has the same attributes as the key that was created for the CA (see further up).

openssl genrsa -des3 -out foobar.herzbube.ch.key 1024

Create a Certificate Signing Request (CSR)

The following command creates a CSR with the following attributes:

• the CSR is formatted for PEM

openssl req -new -key foobar.herzbube.ch.key -out foobar.herzbube.ch.csr

Example DN:

• Country Name (2 letter code) = CH
• State or Province Name (full name) = .
• Locality Name (eg, city) = .
• Organization Name (eg, company) = .
• Organizational Unit Name (eg, section) = .
• Common Name (eg, YOUR name) = foobar.herzbube.ch
• Email Address = herzbube@herzbube.ch
• Challenge password: .
• Optional company name: .

Note 1: It is important to set the "common name" attribute = the FQDN (fully qualified domain name) under which the server will operate, otherwise the information in the certificate will not match the information that the server announces to its clients. If a client is picky about this, the client may even refuse to cooperate with the server!!!

Note 2: Specifying the common name *.herzbube.ch will create a wildcard certificate that is valid for all sub-domains of herzbube.ch.

Storage

Store the private key in:

/etc/ssl/private

Store the CSR in:

/etc/ssl/csr

Note: You may wish to store an un-encrypted version of the private key in /etc/ssl/private so that a server process can read the key without user interaction (i.e. the user entering the passphrase) during a system reboot. See the OpenSSL page for an example of how to do this.

The certificate

Let's Encrypt

Let's Encrypt automatically creates and manages everything described in the following sections. See the Let's Encrypt wiki page for details. To understand what is happening behind the scenes, though, you might still be interested to read on.

Sign the CSR through CAcert.org

The form to sign CSRs can be found here

https://www.cacert.org/account.php?id=10

Paste the CSR into the form, then sign it (by the class 3 root certificate, which is selected by default in the Web form) and thereby generate the certificate. The certificate can be copied and pasted into a certificate file. That file should be stored in

/etc/ssl/certs

Sign the CSR through your own CA

Note: I recently discovered /usr/lib/ssl/misc/CA.sh which is provided by the openssl Debian package. Maybe it would be worth to have a look at that.

The first CSR

If this is the first CSR to be signed, I recommend creating a working directory that can be used to sign other CSRs in the future:

mkdir /usr/local/ssl
cd /usr/local/ssl
cp /usr/share/doc/libapache-mod-ssl/examples/sign.sh .
ln -s /etc/ssl/certs/ca.herzbube.ch.crt
ln -s /etc/ssl/private/ca.herzbube.ch.key

Make the following modifications to the script sign.sh:

• set CA_PREFIX to use ca.herzbube.ch
• set default_days to a value that you find suitable; the attribute determines how long the signed certificate remains valid

Sign the CSR

The following command uses the script sign.sh discussed in the above chapter to sign the CSR:

cd /usr/local/ssl
./sign.sh foobar.herzbube.ch.csr

Store the file that contains the signed certificate in

/etc/ssl/certs