Courier

From HerzbubeWiki

Contents 1 Debian packages 2 SSL/TLS 2.1 Basics 2.2 Certificates 2.3 SNI 2.4 Concrete configuration 3 Ports 4 Configuration 4.1 Automatically purge trash folder 5 Mail folder 6 IMAP Authentication 7 Web administration 7.1 Installation 7.2 Configuration 7.3 Usage 8 Apple Mail 9 IMAP proxy 10 Manually manage Maildir content 10.1 Overview 10.2 Maildir folders 10.3 Error investigation 11 Diagnostics 11.1 Network traffic sniffing 11.2 Manual IMAP session 12 Troubleshooting 12.1 Permissions on Maildir folder

Debian packages

I use the courier daemon to provide the IMAP service on my machine. The following Debian packages need to be installed:

courier-imap
courier-imap-ssl
courier-authdaemon
courier-doc

SSL/TLS

Basics

IMAP over SSL (IMAPS) works through port 993, whereas TLS can be run over the normal port 143. Courier's SSL configuration happens in

/etc/courier/imapd-ssl

To enable IMAPS, say

IMAPDSSLSTART=YES

To enable TLS, say

IMAPDSTARTTLS=YES

Both capabilities can be run simultaneously. In addition, it is possible to restrict connections on the normal port 143 to TLS-only:

IMAP_TLS_REQUIRED=1

Certificates

IMAPS and TLS both require an X.509 certificate. When courier-imap-ssl is installed, two self-signed certificates are automatically generated and configured. The certificates are located here:

/etc/courier/imapd.pem
/etc/courier/pop3d.pem

If you want to use certificates that are not self-signed, but signed by a concrete Certificate Authority (e.g. CAcert, or you run your own CA), you will want to replace the certificates that were automatically generated by the Debian package. More information about creating certificates and maintaining your own CA can be found on the following page:

ServiceEncryptionWithSSL

In order to use your own certificate, you have to change a setting within /etc/courier/imapd-ssl:

TLS_CERTFILE=/etc/ssl/private/imap.herzbube.ch.key.unsecure

Important note: Unfortunately the TLS_CERTFILE file must contain both the certificate and the private key. At the moment there is no way to configure Courier so that it takes the two information sets from two different files.

If the certificate is signed by a CA (i.e. not self-signed), Courier needs to be able to find the entire certificate chain from the immediate CA who made the signature up to a root certificate. The following configuration snippet tells Courier to look for trusted certificates, either inside a file (file must contain all certificates) or inside a directory (directory must contain the certificates, one per file and hashed using OpenSSL's c_rehash script):

TLS_TRUSTCERTS=/etc/ssl/certs

Note: Courier logs any problems in this area to the syslog, so have a look in there if you suspect anything.

SNI

If the IMAP server is supposed to work for different domain names, the TLS extension SNI comes into play. The way how Courier implements this is:

• Set TLS_CERTFILE to a base path, e.g.

TLS_CERTFILE=/etc/ssl/private/imap.key.unsecure

• The concrete certificates must then be stored in files that are formed by appending the domain name to the base path, e.g.

/etc/ssl/private/imap.key.unsecure.imap.herzbube.ch

• Courier will look up the correct certificate based on the host name advertised during the TLS/SNI exchange

Concrete configuration

I want to configure Courier to only accept encrypted connections on its regular port 143. The reason behind this is that I want to make sure that no unencrypted passwords are transmitted when IMAP is accessed over the Internet. To make the TLS-only restriction:

IMAP_TLS_REQUIRED=1

In an ideal world, I would like to avoid duplication and too many open ports, therefore I would like to completely disable IMAPS on port 993 and accept only TLS connections on the regular port 143. Unfortunately this is not possible because SquirrelMail does not support the STARTTLS command. This means that SquirrelMail cannot connect on the regular port 143, because that port accepts TLS connection only. This in turn means that Courier must continue to offer IMAPS, otherwise I would not be able to use SquirrelMail.

Ports

The standard configuration of courier uses the the following ports:

• Port 143 for IMAP2
  • is used for connection by the squirrelmail package (see SquirrelMail)
  • may be used for TLS connections
• Port 993 for IMAPS
  • may be used as an alternative to TLS
• Port 1143 (or any other port) if imapproxy is used

Configuration

courier configuration options are stored in

/etc/courier/imapd
/etc/courier/imapd-ssl

Automatically purge trash folder

By default the trash folder of every user is, on login, purged of messages that have been moved there 7 days (or longer) ago. The setting that does this looks like this:

IMAP_EMPTYTRASH=Trash:7

I don't like this automatic behaviour, so I just disable the setting:

IMAP_EMPTYTRASH=

Mail folder

By default courier stores mail in the so-called Maildir format. This requires that every user has a folder in his or her home directory that corresponds to the Maildir conventions. This folder can be created with the following command:

maildirmake /home/<user>/Maildir

As seen in the above example, the name Maildir should be used, as this is the default name used by courier. The setting that defines the name of a user's mail folder is stored both in /etc/courier/imapd and /etc/courier-imapd-ssl:

MAILDIRPATH=Maildir

Maildir integration with the Debian MTA exim is described on the following page:

Exim

Note: Newer versions of courier require that the Maildir folder's owner/group is set to the user/user's main group. After upgrading to 0.61.0, courier refused connections for a user whose Maildir folder's group was set to mail.

IMAP Authentication

Clients such as SquirrelMail that authenticate through IMAP actually use Courier's authdaemon (provided by the courier-authdaemon Debian package). This daemon supports several schemes how to authenticate users (e.g. through MySQL or LDAP). The default is to delegate authentication to PAM. See this file for the daemon configuration:

/etc/courier/authdaemonrc

The courier-imap package installs the PAM file

/etc/pam.d/imap

Because the content of this file looks like this:

@include common-auth
@include common-account
@include common-password
@include common-session

the actual authentication method is dependent on the system-wide PAM default authentication method. If the common-* files are changed, for instance, to use pam_ldap instead of pam_unix, this not only sets the authentication method to LDAP for the system console or SSH login prompt, but also for all clients of Courier's authdaemon. Very neat!

See the PAM page for details.

Web administration

Installation

In order to administrate courier via a web browser install the Debian package

courier-webadmin

Documentation can be found here

http://osgiliath/doc/courier-doc/htmldoc/install.html#webadmin

DebConf asks the question whether it should copy /usr/lib/courier/courier/webmail/webadmin to /usr/lib/cgi-bin/courierwebadmin and set the suid bit. I answer this question with "no" and instead create the following symlink:

ln -s /usr/lib/courier/courier/webmail/webadmin /usr/lib/cgi-bin/courierwebadmin.cgi

Note: The extension .cgi is important in my setup because I have configured Apache to recognize only files with this ending as CGI scripts. See section "CGI scripts" on Apache for details.

Configuration

Configuration of the package is located in the directory

/etc/courier/webadmin

The password is stored in clear text in password, therefore this file has to be protected with minimal security:

chmod 400 /etc/courier/webadmin/password
chown daemon:daemon /etc/courier/webadmin/password

By default courier-webadmin allows only https connections because the password is transmitted in clear text and is then stored in a cookie. This can be circumvented (although it is not recommended!) by saying

touch /etc/courier/webadmin/unsecureok

Usage

Whenever a change is made the link "Install new configuration" in the main menu of courier-webadmin must be clicked. This writes the changes to the configuration files and restarts the courier daemons.

Apple Mail

Apple Mail needs to be configured like this:

• Create new folders for standard mailboxes Trash, Sent, Drafts
• For each new folder choose the menu item "Postfach -> Ausgewähltes Postfach verwenden für -> <mailbox>"
• When the account is configured, say everywhere that mail is stored on the server

IMAP proxy

Install the package

imapproxy

Configuration is located in

/etc/imapproxy.conf

Change the following option:

listen_port 1143

(or set any other port that suits you)

Update all clients (e.g. SquirrelMail, Apple Mail) that connect on the default IMAP port 143 so that they now connect to port 1143.

Manually manage Maildir content

Overview

Under certain circumstances, it may be convenient or even necessary to manually manage the contents of a Maildir. "Manually" in this context means:

• Without using an IMAP client that talks to the IMAP server through the IMAP protocol
• Possibly do the manipulations in the shell

Specifically, I have encountered the problem that some of my archive mail folders have become so large that they have become totally unmanageable through a normal IMAP client. For instance, in a folder that contains 30'000 spam messages it has become impossible to clean out, say, the oldest 10'000 messages. In another case where the folder contains 6000 mails, I cannot even move 100 messages to a different folder. In both cases, the operation on the IMAP server seems to encounter a timeout, but I have not been able to detect the actual source of the problem: Is it a bug in the IMAP server itself? Is it a file system limitation? I don't know.

The worst thing is that in some circumstances a part of the operation completes but another part does not. For instance, when I want to move some messages, sometimes the messages "arrive" in the destination folder but are left intact in the source folder, so I suddenly have two copies of the messages. When I want to delete the messages from the source folder, the delete operation does not work.

Most of the time, if I reduce my activities to affect only 2 or 3 messages, everything works fine, but of course this is no solution when you want to manage messages in the hundreds or even thousands. After some frustrating trial-and-error sessions, I have finally decided to find out whether I can do my work "behind the back" of the IMAP server. The next chapter documents my investigations in the inner workings of Maildir folders.

Maildir folders

When I create a folder in my mailbox, the Courier IMAP server creates a Maildir folder with the following layout:

osgiliath:/home/patrick/Maildir/.foo# l
total 32
drwx------  6 patrick patrick 4096 Dec 22 16:01 .
drwx------ 60 patrick mail    4096 Dec 22 16:03 ..
-rw-r--r--  1 patrick patrick   43 Dec 22 16:01 courierimapacl
drwx------  2 patrick patrick 4096 Dec 22 16:01 courierimapkeywords
-rw-r--r--  1 patrick patrick   86 Dec 22 16:03 courierimapuiddb
drwx------  2 patrick patrick 4096 Dec 22 16:03 cur
-rw-------  1 patrick patrick    0 Dec 22 16:01 maildirfolder
drwx------  2 patrick patrick 4096 Dec 22 16:01 new
drwx------  2 patrick patrick 4096 Dec 22 16:03 tmp

The Wikipedia article on the Maildir format (http://en.wikipedia.org/wiki/Maildir) gives hints about some of the files and directories:

• the tmp directory is used during message delivery only
• message delivery places the new message in the new directory
• the mail user agent process finds messages in the new directory it moves them to cur, appending an informational suffix (see below for details)

The maildirfolder file always seems to present and empty. Googling for "maildirfolder" seems to confirm this, although the information cannot be seen as conclusive. Additionally, googled information tells us that the file is specific to Courier IMAP and not generally required by IMAP or Maildir.

The courierimap* files/directories obviously belong to the Courier IMAP server. It is difficult to get information about the meaning of those files/directories, but they must certainly be kept in mind when manipulating a Maildir folder, since they may contain metadata about messages being moved around:

• courierimapacl seems to contain folder-specific authorization information and has nothing to do with specific messages
• courierimapkeywords refers to messages, but seems to be populated only under certain (as yet unknown) circumstances; for instance, in a newly created folder with one seen message the directory is completely empty
• courierimapuiddb refers to messages; the file seems to always contain one more line than there are messages in the Maildir folder
• this thread seems to indicate that it is not necessary to update courierimapkeywords and courierimapuiddb even if messages are deleted from the Maildir folder

Looking into the cur directory after it contains a read message reveals the following content:

osgiliath:/home/patrick/Maildir/.foo# l cur
total 12
drwx------ 2 patrick patrick 4096 Dec 22 16:03 .
drwx------ 6 patrick patrick 4096 Dec 22 16:01 ..
-rw-r--r-- 1 patrick patrick  900 Dec 22 16:03 1198335805.M980307P19715V0000000000000307I0007BCE6_0.osgiliath,S=900:2,S

Again, the Wikipedia article cited above helps with the file name format:

• the file name must, of course, be unique
• the algorithm to guarantee uniqueness combines the time, the host name, and a number of pseudo-random parameters
• the informational suffix consists of a colon (to separate the unique part of the filename from the actual information), a '2', a comma and various flags
• the '2' specifies the version of the information that follows the comma; '2' is the only currently officially specified version, '1' being an experimental version (probably used while the Maildir format was under development)
• specified flags are "P", "R", "S", "T", "D" and "F"

The Maildir specs (http://cr.yp.to/proto/maildir.html) list the purpose of the flags:

• "P" = passed: the user has resent/forwarded/bounced this message to someone else
• "R" = replied: the user has replied to this message
• "S" = seen: the user has viewed this message
• "T" = trashed: the user has moved this message to the trash; the trash will be emptied by a later user action
• "D" = draft: the user considers this message a draft; toggled at user discretion
• "F" = flagged: user-defined flag; toggled at user discretion
• flags must be stored in ASCII order: e.g., "2,FRS"

Error investigation

Sometimes, when I attempt to do something in SquirrelMail that affects a lot of messages, I get an error message that looks like this:

ERROR: Connection dropped by IMAP server.
Query: COPY 4181,5001:5135,5396,5434,5446,5491,5499,5506,5514,5522,5531,5539,5547,5555,5565,5633,5654,5756,5783,5791,5821,5830:5831,5859:5862,5864:5866,5919 "INBOX.Trash"

In the example above, I tried to delete the last 167 messages from a folder that contains 6167 messages. The end result of the operation was:

• the messages were placed into .Trash/cur,
• the messages were not deleted from .Junk.ham

The error message refers to a query named "COPY". This prompted me to have a look at RFC 3501 (http://tools.ietf.org/html/rfc3501), the IMAP4rev1 specification. I noticed that the specs do not contain a "MOVE" command. I conclude that in order to implement a move operation, an IMAP client such as SquirrelMail needs to translate the operation into 2 IMAP commands: first a COPY, second a DELETE command. The observed manner of failure of the operation supports this conclusion and suggests the following order of events:

• SquirrelMail initiates the COPY command
• the IMAP server process starts to process the command, but takes a long time
• SquirrelMail times out and displays the error message
• the IMAP server process continues to process the command until it finishes, but never receives the "DELETE" command from SquirrelMail
• the end result displayed by SquirrelMail are the copied messages in the Trash, but the originals still in place in the source folder

So the all-important question is: Why does the IMAP server process take so long to process the COPY command? Too much load on the system? Not enough disk space? Not enough memory?

Things that I have tried without success:

• at first I assumed that imapproxyd had something to do with the problem, but stopping that daemon does not resolve the issue
• stopping boinc to give imapd all the available processor power also does not help (which seems logical after top revealed that the imapd process does not use any processing power at all)
• checking the /home filesystem + optimizing the filesystem for directories with many files did not help anything, either

tune2fs -O dir_index /dev/hda7
e2fsck -D /dev/hda7

• run a manual IMAP session without SquirrelMail as the intermediate to check whether or not SquirrelMail somehow plays a role; except for the fact that it times out, SquirrelMail cannot be blamed, though: in both sessions copying 167 mails takes about the same time (3 minutes)

Let's watch the system while we repeat the operation from above:

• CPU activity
  • top shows that the operation spawns an imapd process running as user "patrick" (the user logged in to SquirrelMail)
  • after an initial burst, the imapd process no longer displays any noticeable CPU activity; the other Courier processes never show any activity at all
  • top -u patrick continuously displays the imapd process at 0.0% CPU activity, i.e. no CPU activity at all
  • the imapd process continues to run long after SquirrelMail reports its "Connection dropped by IMAP server" error; this supports the assumed order of events from further up
• file activity
  • monitoring .Trash/tmp by repeatedly running "ls -l" reveals that the imapd process continuously places files into this directory until it has created files for all the messages requested to be moved
  • lsof -r 2 -c imapd lists the open files of the imapd process every 2 seconds. The result shows that the imapd daemon reads a new file from the Junk.ham folder about every 2 seconds. The only variation between samples is the file being read. Sample output:

COMMAND  PID    USER   FD   TYPE     DEVICE    SIZE   NODE NAME
imapd   7504 patrick  cwd    DIR        3,7    4096 474209 /home/patrick/Maildir
imapd   7504 patrick  rtd    DIR        3,3    4096      2 /
imapd   7504 patrick  txt    REG        3,5 1297640  33099 /usr/bin/imapd
imapd   7504 patrick  mem    REG        0,0              0 [heap] (stat: No such file or directory)
imapd   7504 patrick  mem    REG        3,5  928628 359813 /usr/lib/libdb-4.3.so
imapd   7504 patrick  mem    REG        3,3   38412 211935 /lib/libnss_files-2.7.so
imapd   7504 patrick  mem    REG        3,5   16540 359921 /usr/lib/libnss_db-2.2.3.so
imapd   7504 patrick  mem    REG        3,3   41876 211863 /lib/libgcc_s.so.1
imapd   7504 patrick  mem    REG        3,3  149328 211893 /lib/libm-2.7.so
imapd   7504 patrick  mem    REG        3,5  937800 360104 /usr/lib/libstdc++.so.6.0.9
imapd   7504 patrick  mem    REG        3,3 1356012 211875 /lib/libc-2.7.so
imapd   7504 patrick  mem    REG        3,3   38300 211887 /lib/libcrypt-2.7.so
imapd   7504 patrick  mem    REG        3,5   37232 572333 /usr/lib/courier-authlib/libcourierauth.so.0.0.0
imapd   7504 patrick  mem    REG        3,5   19940 359983 /usr/lib/libgdbm.so.3.0.0
imapd   7504 patrick  mem    REG        3,5   30868 360024 /usr/lib/libfam.so.0.0.0
imapd   7504 patrick  mem    REG        3,3  117344 211806 /lib/ld-2.7.so
imapd   7504 patrick    0u  IPv6      20663            TCP osgiliath.herzbube.ch:imap2->osgiliath.herzbube.ch:35449 (ESTABLISHED)
imapd   7504 patrick    1u  IPv6      20663            TCP osgiliath.herzbube.ch:imap2->osgiliath.herzbube.ch:35449 (ESTABLISHED)
imapd   7504 patrick    2w  FIFO        0,5           1797 pipe
imapd   7504 patrick    3u  unix 0xd79e9b20          20664 socket
imapd   7504 patrick    4w  FIFO        0,5           1795 pipe
imapd   7504 patrick    5r   REG        3,7   10242 531752 /home/patrick/Maildir/.Junk.ham/cur/1167674801.M935872P1602V0000000000000307I00081D28_5012.osgiliath,S=10242:2,S
imapd   7504 patrick    6u  unix 0xd79e96a0          20680 socket

• log activity
  • no activities in any log files by any IMAP/courier process can be detected

Confirmation of some facts, but otherwise no real clues. Another attempt: Try to delete the last 314 messages from a different folder than the one I have been exercising so far. The new folder contains 6814 messages. Result: The operation completes blazingly fast in about 5 seconds! Conclusions:

• it cannot be the folder size that causes imapd to run so slow
• or at least it's not the folder size alone
• the destination folder (Trash) does not affect the outcome of the test

Let's look at the differences between the two folders and hope we get some clues:

osgiliath:/home/patrick/Maildir# l .Junk .Junk.ham
.Junk:
total 1160
drwx------  6 patrick patrick   4096 Oct 26 00:25 .
drwx------ 60 patrick mail      4096 Dec 22 16:03 ..
-rw-r--r--  1 patrick patrick     43 Oct 26 00:24 courierimapacl
drwx------  2 patrick patrick  32768 Dec 22 21:42 courierimapkeywords
-rw-r--r--  1 patrick patrick 343640 Dec 22 21:25 courierimapuiddb
drwx------  2 patrick patrick 774144 Dec 22 21:42 cur
-rw-------  1 patrick patrick      0 Oct 26 00:24 maildirfolder
drwx------  2 patrick patrick   4096 Dec 22 21:45 new
drwx------  2 patrick patrick   4096 Dec 22 21:45 tmp

.Junk.ham:
total 1024
drwx------  6 patrick patrick   4096 Oct 26 00:38 .
drwx------ 60 patrick mail      4096 Dec 22 16:03 ..
-rw-r--r--  1 patrick patrick     17 Jan  1  2007 courierimapacl
drwx------  2 patrick patrick   4096 Dec 22 14:54 courierimapkeywords
-rw-r--r--  1 patrick patrick 471777 Dec 22 15:19 courierimapuiddb
drwx------  2 patrick patrick 532480 Dec 22 15:19 cur
-rw-------  1 patrick patrick      0 Jan  1  2007 maildirfolder
drwx------  2 patrick patrick   4096 Jan  4  2007 new
drwx------  2 patrick patrick   4096 Dec 22 21:42 tmp

Hm, there is a difference in size between the courierimapacl files. Let's look at their content:

osgiliath:/home/patrick/Maildir# cat .Junk/courierimapacl                          
owner aceilrstwx
administrators aceilrstwx
osgiliath:/home/patrick/Maildir# cat .Junk.ham/courierimapacl 
owner aceilrstwx

No idea why one file contains an additional "administrators" line. Another attempt with the 2 files being the same does not help anything. Another observation is that the courierimapkeywords directories are not of the same size. Comparing their content:

osgiliath:/home/patrick/Maildir# l .Junk/courierimapkeywords .Junk.ham/courierimapkeywords/
.Junk.ham/courierimapkeywords/:
total 452
drwx------ 2 patrick patrick   4096 Dec 22 14:54 .
drwx------ 6 patrick patrick   4096 Dec 22 22:04 ..
-rw-r--r-- 1 patrick patrick 446431 Dec 22 15:19 :list

.Junk/courierimapkeywords:
total 40
drwx------ 2 patrick patrick 32768 Dec 22 21:42 .
drwx------ 6 patrick patrick  4096 Oct 26 00:25 ..

I have learned elsewhere that keywords are not really crucial, so I move away the :list file and make another attempt, and lo and behold! the damn thing suddenly works!!!

Diagnostics

In case of trouble, it is difficult to find any useful diagnostic information, since courier does not write any useful logfiles (a few things are written to syslog, but not nearly enough). This chapter provides information about diagnostics that I found useful in the past.

Network traffic sniffing

Use Wireshark to observe network traffic details:

tshark -i lo -w capture_file -S -f "src or dst port 143"

Manual IMAP session

Telnet to localhost port 143 and initiate your own IMAP commands. At the moment, I have no useful commands - maybe observing a SquirrelMail IMAP session with Wireshark can provide hints.

Troubleshooting

Permissions on Maildir folder

The Maildir folder in a user's home directory must have the following permissions, otherwise courier drops the connection immediately after a successful login:

osgiliath:~# ls -ld /home/patrick/Maildir
drwx------ 55 patrick patrick 4096 Aug 12 01:06 /home/patrick/Maildir