Courier

From HerzbubeWiki
Jump to navigation Jump to search

This page has information about how to use Courier as an IMAP server. Courier also is an MTA, but I'm preferring Exim for that because it's the default MTA on Debian.


Debian packages

This Debian package should pull in all the necessary dependencies 

courier-imap-ssl

Make sure that the following packages are on the dependency list: 

courier-imap
courier-authdaemon


Configuration

Files

courier configuration options are stored in 

/etc/courier/imapd
/etc/courier/imapd-ssl


Debconf configuration

Debconf asks only one question: "Create directories for web-based administration?"

courier-webadmin requires that configuration is not kept as a single configuration file, but instead split into directories. I answer "no" to the Debconf question because I don't want to use courier-webadmin. Also, "no" is already the default answer.


SSL/TLS

Basics

Courier's SSL configuration happens in 

/etc/courier/imapd-ssl


IMAP over SSL (IMAPS) works through port 993, whereas TLS can be run over the normal port 143. For years on end I had to keep IMAPS working because I was using the SquirrelMail webmail client, which does not support the STARTTLS command. Fortunately in 2012 I was able to switch to the more modern Roundcube webmail client, which does not suffer the same limitation. So these days I completely disable port 993 (IMAPS): 

IMAPDSSLSTART=NO

Instead, I enable TLS with this: 

IMAPDSTARTTLS=YES

Notes:

  • If necessary, both IMAPS and TLS can be run simultaneously.
  • With IMAPS disabled, it's no longer necessary to start/stop /etc/init.d/courier-imap-ssl when the configuration changes. In fact, since Debian stretch running this init script yields an error if IMAPS is disabled (in Debian jessie and before the script didn't error out, it simply didn't do anything).


Last but not least, I don't want clear-text communication on port 143, so I restrict connections to TLS-only: 

IMAP_TLS_REQUIRED=1


Certificates

IMAPS and TLS both require an X.509 certificate. When courier-imap-ssl is installed, two self-signed certificates are automatically generated and configured. The certificates are located here: 

/etc/courier/imapd.pem
/etc/courier/pop3d.pem


If you want to use certificates that are not self-signed, but signed by a concrete Certificate Authority (e.g. Let's Encrypt, CAcert, or you run your own CA), you will want to replace the certificates that were automatically generated by the Debian package. More information about creating certificates and maintaining your own CA can be found on this wiki page.


In order to use your own certificate, you have to change the TLS_CERTFILE option in /etc/courier/imapd-ssl. For my "Let's Encrypt" setup I use the following: 

TLS_CERTFILE=/etc/letsencrypt/live/herzbube.ch/courier.cert-and-key.unsecure


Important: Unfortunately the file referenced by TLS_CERTFILE must contain both the certificate and the private key (presumably in that order). At the moment there is no way to configure Courier so that it takes the two pieces of information from two different files. After obtaining a new certificate from "Let's Encrypt", it is therefore necessary to manually stitch together the referenced file with the following commands. During automated renewal these commands are run by a Certbot deploy hook - see the LetsEncrypt page for details. 

cd /etc/letsencrypt/live/herzbube.ch
cat fullchain.pem privkey.pem >courier.cert-and-key.unsecure


Also important: Because it contains a private key, the file referenced by TLS_CERTFILE must not be world-readable. The permissions on the file itself are not terribly important, more important is that the folder in which the file is located is not generally accessible. 

root@pelargir:~# ls -ld /etc/letsencrypt/live/
drwx--x--- 8 root ssl-cert 4096 Jun  8 01:18 /etc/letsencrypt/live/


If the certificate is signed by a CA (i.e. not self-signed), Courier needs to be able to find the entire certificate chain from the immediate CA who made the signature up to a root certificate. The following configuration snippet tells Courier to look for trusted certificates, either inside a file (file must contain all certificates) or inside a directory (directory must contain the certificates, one per file and hashed using OpenSSL's c_rehash script): 

TLS_TRUSTCERTS=/etc/ssl/certs


Note: Courier logs any problems in this area to the syslog, so have a look in there if you suspect anything.


SNI

If the IMAP server is supposed to work for different domain names, the TLS extension SNI comes into play. The way how Courier implements this is:

  • Set TLS_CERTFILE to a base path, e.g.
TLS_CERTFILE=/path/to/imap.key.unsecure
  • The concrete certificates must then be stored in files that are formed by appending the domain name to the base path, e.g.
/path/to/imap.key.unsecure.imap.herzbube.ch
  • Courier will look up the correct certificate based on the host name advertised during the TLS/SNI exchange


Note: I am not using this approach because I am offering IMAP only on one server, mail.herzbube.ch.


Custom DH group file

With the advent of the Logjam attack, it has become important to have a DH group file with increased bit size (see Guide to Deploying Diffie-Hellman for TLS). Unfortunately, the Courier IMAP default file is only 768 bits, which is so low that modern clients (notably the iOS mail client app in iOS 9) will refuse to connect to the IMAP server. I have therefore created a new DH group file with 2048 bit that I keep in a shared location so that other services (e.g. Apache) can also use it. See the OpenSSL wiki page for instructions how to create the new file. Here is how to configure Courier to use the file: 

TLS_DHPARAMS=/etc/ssl/private/dhparams-2048-bit.pem


Ports

The standard configuration of courier uses the the following ports:

  • Port 143 for IMAP2
    • Is used for connection by the squirrelmail package (see the SquirrelMail wiki page)
    • May also be used for TLS connections
  • Port 993 for IMAPS
    • May be used as an alternative to TLS
  • Port 1143 (or any other port) if imapproxy is used


Automatically purge trash folder

By default the trash folder of every user is, on login, purged of messages that have been moved there 7 days (or longer) ago. The setting that does this looks like this: 

IMAP_EMPTYTRASH=Trash:7

I don't like this automatic behaviour, so I just disable the setting: 

IMAP_EMPTYTRASH=


Mail folder

By default courier stores mail in the so-called Maildir format. This requires that every user has a folder in his or her home directory that corresponds to the Maildir conventions. This folder can be created with the following command: 

maildirmake /home/<user>/Maildir

As seen in the above example, the name Maildir should be used, as this is the default name used by courier. The setting that defines the name of a user's mail folder is stored both in /etc/courier/imapd and /etc/courier-imapd-ssl: 

MAILDIRPATH=Maildir

Maildir integration with the Debian MTA exim is described on the Exim wiki page.


Note: Newer versions of courier require that the Maildir folder's owner/group is set to the user/user's main group. After upgrading to 0.61.0, courier refused connections for a user whose Maildir folder's group was set to mail.


IMAP Authentication

Clients such as SquirrelMail that authenticate through IMAP actually use Courier's authdaemon (provided by the courier-authdaemon Debian package). This daemon supports several schemes how to authenticate users (e.g. through MySQL or LDAP). The default is to delegate authentication to PAM. See this file for the daemon configuration: 

/etc/courier/authdaemonrc

The courier-imap package installs the PAM file 

/etc/pam.d/imap

Because the content of this file looks like this: 

@include common-auth
@include common-account
@include common-password
@include common-session

The actual authentication method is dependent on the system-wide PAM default authentication method. If the common-* files are changed, for instance, to use pam_ldap instead of pam_unix, this not only sets the authentication method to LDAP for the system console or SSH login prompt, but also for all clients of Courier's authdaemon. Very neat!

See the PAM wiki page for details.


Web administration

Installation

In order to administrate courier via a web browser install the Debian package 

courier-webadmin

Documentation can be found here 

http://osgiliath/doc/courier-doc/htmldoc/install.html#webadmin

Debconf asks the question whether it should copy /usr/lib/courier/courier/webmail/webadmin to /usr/lib/cgi-bin/courierwebadmin and set the suid bit. I answer this question with "no" and instead create the following symlink: 

ln -s /usr/lib/courier/courier/webmail/webadmin /usr/lib/cgi-bin/courierwebadmin.cgi

Note: The extension .cgi is important in my setup because I have configured Apache to recognize only files with this ending as CGI scripts. See section "CGI scripts" on the Apache wiki page for details.


Configuration

Configuration of the package is located in the directory 

/etc/courier/webadmin

The password is stored in clear text in password, therefore this file has to be protected with minimal security: 

chmod 400 /etc/courier/webadmin/password
chown daemon:daemon /etc/courier/webadmin/password

By default courier-webadmin allows only https connections because the password is transmitted in clear text and is then stored in a cookie. This can be circumvented (although it is not recommended!) by saying 

touch /etc/courier/webadmin/unsecureok


Usage

Whenever a change is made the link "Install new configuration" in the main menu of courier-webadmin must be clicked. This writes the changes to the configuration files and restarts the courier daemons.


Apple Mail

Apple Mail needs to be configured like this:

  • Specify that the account uses SSL for authentication. Apple Mail now sets the port to the default IMAPS port 993. Override this and specify that the account uses port 143 to connect. This tells Apple Mail that it should use TLS to SSL-encrypt the session. I have not tested what happens if the IMAP daemon allows plain-text/non-TLS connections over port 143, but if the daemon requires TLS everything works out beautifully.
  • Specify for all special folders that mail is to be stored on the server
  • After you connect to the account for the first time, for each of the following special folders select the correct IMAP folder and choose the menu item "Postfach -> Dieses Postfach verwenden für -> <Special folder>". The goal here is to select the same folders that are also used by other IMAP clients, e.g. SquirrelMail or Roundcube.
    • Inbox
    • Trash
    • Sent
    • Drafts
  • Delete any IMAP folders that Apple Mail has created on its own that are now no longer needed. For instance, Apple Mail will probably have created an IMAP folder "Sent Messages" to act as the special folder "Sent". This is now no longer needed and can be deleted.


IMAP proxy

Install the package 

imapproxy

Configuration is located in 

/etc/imapproxy.conf

Change the following option: 

listen_port 1143

(or set any other port that suits you)


Update all clients (e.g. SquirrelMail, Apple Mail) that connect on the default IMAP port 143 so that they now connect to port 1143.


Managing Maildir folders

References


Content of a Maildir folder

When I use a mail client to create a new mailbox folder, the Courier IMAP server creates a corresponding Maildir filesystem folder with the following layout: 

osgiliath:/home/patrick/Maildir/.foo# ls -la
total 32
drwx------  6 patrick patrick 4096 Dec 22 16:01 .
drwx------ 60 patrick mail    4096 Dec 22 16:03 ..
-rw-r--r--  1 patrick patrick   43 Dec 22 16:01 courierimapacl
drwx------  2 patrick patrick 4096 Dec 22 16:01 courierimapkeywords
-rw-r--r--  1 patrick patrick   86 Dec 22 16:03 courierimapuiddb
drwx------  2 patrick patrick 4096 Dec 22 16:03 cur
-rw-------  1 patrick patrick    0 Dec 22 16:01 maildirfolder
drwx------  2 patrick patrick 4096 Dec 22 16:01 new
drwx------  2 patrick patrick 4096 Dec 22 16:03 tmp


The Wikipedia article on the Maildir format gives hints about some of the files and directories:

  • The tmp directory is used during message delivery only
    • Important: The Courier IMAP server requires that this directory is present. Courier refuses to open the mailbox if the directory is missing. Also see Missing tmp subfolders in the troubleshooting section further down.
  • Message delivery places the new message in the new directory
  • The mail user agent process finds messages in the new directory it moves them to cur, appending an informational suffix (see below for details)


The maildirfolder file always seems to present and empty. Googling for "maildirfolder" seems to confirm this, although the information cannot be seen as conclusive. Additionally, googled information tells us that the file is specific to Courier IMAP and not generally required by IMAP or the Maildir format.


The courierimap* files/directories obviously belong to the Courier IMAP server. It is difficult to get information about the meaning of those files/directories, but they must certainly be kept in mind when manipulating a Maildir folder, since they may contain metadata about messages being moved around:

  • courierimapacl seems to contain folder-specific authorization information and has nothing to do with specific messages
  • courierimapkeywords refers to messages, but seems to be populated only under certain (as yet unknown) circumstances; for instance, in a newly created folder with one seen message the directory is completely empty
  • courierimapuiddb refers to messages; the file seems to always contain one more line than there are messages in the Maildir folder
  • this thread seems to indicate that it is not necessary to update courierimapkeywords and courierimapuiddb even if messages are deleted from the Maildir folder


Looking into the cur directory after it contains a read message reveals the following content: 

osgiliath:/home/patrick/Maildir/.foo# l cur
total 12
drwx------ 2 patrick patrick 4096 Dec 22 16:03 .
drwx------ 6 patrick patrick 4096 Dec 22 16:01 ..
-rw-r--r-- 1 patrick patrick  900 Dec 22 16:03 1198335805.M980307P19715V0000000000000307I0007BCE6_0.osgiliath,S=900:2,S


Again, the Wikipedia article cited above helps with the file name format:

  • The file name must, of course, be unique
  • The algorithm to guarantee uniqueness combines the time, the host name, and a number of pseudo-random parameters
  • The informational suffix consists of a colon (to separate the unique part of the filename from the actual information), a '2', a comma and various flags
  • The '2' specifies the version of the information that follows the comma; '2' is the only currently officially specified version, '1' being an experimental version (probably used while the Maildir format was under development)
  • Specified flags are "P", "R", "S", "T", "D" and "F"


The Maildir specs (http://cr.yp.to/proto/maildir.html) list the purpose of the flags:

  • "P" = passed: the user has resent/forwarded/bounced this message to someone else
  • "R" = replied: the user has replied to this message
  • "S" = seen: the user has viewed this message
  • "T" = trashed: the user has moved this message to the trash; the trash will be emptied by a later user action
  • "D" = draft: the user considers this message a draft; toggled at user discretion
  • "F" = flagged: user-defined flag; toggled at user discretion
  • Flags must be stored in ASCII order: e.g., "2,FRS"


Adding messages to a Maildir from an external IMAP account

Summary

Goal: Download all mailboxes and messages from an external IMAP account, then add the data to a local Maildir folder that is accessed via Courier IMAP.

I selected the utility isync for the task.


isync solution

Install Debian package 

isync


The project/package name is "isync", but on the command line the tool to use is 

mbsync --config /path/to/file

mbsync will not run without a configuration file. If the --config option is not specified, the default config file location is ~/.mbsyncrc. The syntax of the configuration file is fully described in the mbsync man page, but it's difficult to understand without an example. The best way to start probably is by making a copy of this sample file: 

/usr/share/doc/isync/examples/mbsyncrc.sample


Concepts:

Store
A Store defines a collection of mailboxes; basically a folder, either local or remote.
Channel
A Channel connects two Stores, describing the way the two are synchronized.
Account
An Account describes the connection part of remote Stores, so a server connection can be shared between multiple Stores.
Group
A Group aggregates multiple Channels to save typing on the command line.


The following configuration file is suitable for fetching all mailboxes from a single IMAP account: 

# ------------------------------------------------------------
# Global configuration section
#   Values here are used as defaults for any following Channel section that
#   doesn't specify them.
# ------------------------------------------------------------

# Pull changes from master (remote) to slave (local),
# All changes are pulled because we don't specify
# any type flags (see man page)
Sync Pull

# Make sure that no messages are deleted
Expunge None

# Make sure that new mailboxes are automatically created,
# but only on the slave. Since we Pull changes only, the
# "slave-only" restriction is a bit pointless, but better
# safe than sorry.
Create Slave

# Save all sync states in the directory specified here.
# Because we specify a directory we must use a trailing slash (/)
SyncState ./mbsyncstate/


# ------------------------------------------------------------
# Stores and channel
# ------------------------------------------------------------
MaildirStore local-foo
# Because we specify a directory we must use a trailing slash (/)
Path ./foo/
# The location of the special mailbox INBOX. By default this
# is placed in ~/Maildir, even though we specified "Path"
Inbox ./foo/Maildir
Trash Trash

IMAPStore remote-foo
Host imap.mail-ch.ch
User someusername
Pass secret
UseIMAPS yes
CertificateFile /etc/ssl/certs/ca-certificates.crt

Channel foo
Master :remote-foo:
Slave :local-foo:
# Synchronize all mailboxes. Without this, only INBOX is
# synchronized.
Pattern *
CopyArrivalDate yes


The local folder that this creates must be converted into a format this is suitable for Courier IMAP. I am pretty sure this could be achieved by correctly configuring isync, however I didn't have the endurance to deal with cryptic man page explanations - or maybe I just don't understand enough IMAP to make sense of those explanations. Anyway, I ended up with a little shell script that performs the necessary conversion. It basically does two things:

  • Rename mailbox folders so that they start with a dot and have a common prefix
  • Remove the file .uidvalidity that is added by isync but which we don't need when the data is actually added to the final Maildir folder

Here is the script: 

#!/bin/bash

MYNAME="$(basename $0)"
MAILBOX_FOLDER_PREFIX=".switchplus"

if test $# -ne 1; then
  echo "Usage: $MYNAME /path/to/mbsync/folder"
  exit 1
fi

MBSYNC_FOLDER="$(readlink --canonicalize-missing --no-newline "$1")"
if test $? -ne 0; then
  echo "Error canonicalizing folder name $MBSYNC_FOLDER"
  exit 1
fi
if test ! -d "$MBSYNC_FOLDER"; then
  echo "Folder not found $MBSYNC_FOLDER"
  exit 1
fi

CONVERTED_FOLDER="$MBSYNC_FOLDER.converted"
if test -d "$CONVERTED_FOLDER"; then
  echo "Deleted converted folder $CONVERTED_FOLDER"
  rm -rf "$CONVERTED_FOLDER"
  if test $? -ne 0; then
    echo "Error deleting converted folder"
    exit 1
  fi
fi

mkdir -p "$CONVERTED_FOLDER"
if test $? -ne 0; then
  echo "Error creating converted folder $CONVERTED FOLDER"
  exit 1
fi

cd "$MBSYNC_FOLDER"
if test $? -ne 0; then
  echo "Cannot change working directory to $MBSYNC_FOLDER"
  exit 1
fi

for MAILBOX_FOLDER in *; do
  if test "$MAILBOX_FOLDER" = "Maildir"; then
    RENAMED_MAILBOX_FOLDER="$CONVERTED_FOLDER/$MAILBOX_FOLDER_PREFIX"
  else
    RENAMED_MAILBOX_FOLDER="$CONVERTED_FOLDER/$MAILBOX_FOLDER_PREFIX.$MAILBOX_FOLDER"
  fi

  cp -R "$MAILBOX_FOLDER" "$RENAMED_MAILBOX_FOLDER"
  if test $? -ne 0; then
    echo "Error copying mailbox folder $MAILBOX_FOLDER"
    exit 1
  fi

  if test -f "$RENAMED_MAILBOX_FOLDER/.uidvalidity"; then
    rm "$RENAMED_MAILBOX_FOLDER/.uidvalidity"
    if test $? -ne 0; then
      echo "Error deleting .uidvalidity from $RENAMED_MAILBOX_FOLDER"
      exit 1
    fi
  fi
done

echo "Conversion finished: $CONVERTED_FOLDER"


Here's a transcript of all commands that are required to download the data for one IMAP account: 

# Setup environment
mkdir mbsync
cd mbsync
vi mbsyncrc                # add the configuration snippet above
vi convert-to-courier.sh   # add the conversion script from above
chmod +x convert-to-courier.sh

# Download data
mkdir foo
mbsync --config ./mbsyncrc foo

# Massage data. Note: This creates a copy of the downloaded data,
# so it may use a lot of diskspace!
./convert-to-courier.sh foo

# Move massaged data to final Maildir
chown -R localusername:localusergroup foo
mv foo/.switchplus* /home/localusername/Maildir


Other IMAP utilities

Searching the package database provides a rather long list of utilities that have something to do with IMAP: 

apt-cache search imap


I manually culled the list down to the following interesting looking utilities:

  • imapcopy - IMAP backup, copy and migration tool
  • isync - IMAP and MailDir mailbox synchronizer
    • Website = http://isync.sourceforge.net/
    • Synchronizes two IMAP mailboxes. A mailbox can be a Maildir or an IMAP server.
    • Supports TLS and is fairly well maintained (at the time of writing the last release is from November 2015)
  • larch - tool to copy messages from one IMAP server to another
  • mailsync - Synchronize IMAP mailboxes
  • offlineimap - IMAP/Maildir synchronization and reader support
  • syncmaildir - Sync Mail Dir is a set of tools to synchronize Maildirs


A few more tools that looked interesting, but I didn't look at them any closer

  • archivemail - archive and compress or delete your old email
  • fdm - fetching, filtering and delivering emails
  • fetchmail - SSL enabled POP3, APOP, IMAP mail gatherer/forwarder
  • getmail4 - mail retriever with support for POP3, IMAP4 and SDPS
  • im - mail/news handling commands and Perl modules
  • mailutils - GNU mailutils utilities for handling mail


Diagnostics

In case of trouble, it is difficult to find any useful diagnostic information, because the Courier daemons do not write any useful logfiles (a few things are written to syslog, but not nearly enough). This chapter provides information about diagnostics that I found useful in the past.


Testing TLS connection

This command performs a basic test to see whether or not Courier is set up correctly for TLS: 

openssl s_client -connect mail.herzbube.ch:143 -starttls imap


Logging authentication problems

To diagnose authentication problems you can increase the amount of debug logging by changing the following setting in /etc/courier/authdaemonrc 

DEBUG_LOGIN=


Manual IMAP session

Things that I have been able to find out about the IMAP dialog:

  • IMAP commands are case insensitive. I'm not sure whether this is also true for mailbox names.
  • IMAP commands begin with an arbitrary label. You can't just type "select inbox", you must prefix the command like this "mylabel select inbox". Apparently it is customary to use labels of the form "A<nnnn>", where nnnn is the number of the command within the IMAP session.


Authentication

  • The configuration I use for Courier does not allow the "login" command, instead I must use the command "authenticate plain"
  • This initiates the SASL PLAIN authentication mechanism, which is described in RFC 4616
  • First you have to Base64-encode both username and password like this:
echo -en "\0username\0secret" | base64
  • The IMAP session snippet then looks like this:
a0001 authenticate plain
+ 
AHVzZXJuYW1lAHNlY3JldA==
a0001 OK LOGIN Ok.


Connect to the IMAP server

  • Without TLS:
telnet mail.herzbube.ch 143
  • With TLS
openssl s_client -connect mail.herzbube.ch:143 -starttls imap


Once connected to the server, here is a simple list of commands that you can send to the server. Server responses are not shown. The session uses SASL PLAIN for authentication (see above for details), selects the main mailbox, retrieves message 12 and then logs out. 

a0001 authenticate plain
+ 
AHVzZXJuYW1lAHNlY3JldA==
a0002 select inbox
a0003 fetch 12 full
a0005 logout


This Courier tutorial page has a few additional IMAP commands that you can try out.


Network traffic sniffing

As a last resort, you can use Wireshark to observe network traffic details. This only works if you don't use TLS. 

tshark -i lo -w capture_file -S -f "src or dst port 143"


Troubleshooting

Permissions on Maildir folder

The Maildir folder in a user's home directory must have the following permissions, otherwise courier drops the connection immediately after a successful login: 

osgiliath:~# ls -ld /home/patrick/Maildir
drwx------ 55 patrick patrick 4096 Aug 12 01:06 /home/patrick/Maildir


Missing tmp subfolders

Every mailbox must have a tmp folder. Courier refuses to access a mailbox if that folder is missing. The error message Courier returns varies with the IMAP command you use. For instance: 

[...]
6 examine INBOX.Sent
6 NO Unable to open this mailbox.
7 status INBOX.Sent (messages unseen)
7 NO [ALERT] STATUS failed
8 select INBOX.Sent
8 NO Unable to open this mailbox.

Here is a listing of a correctly set up mailbox folder 

drwx------  6 patrick patrick  4096 Jul 12 03:33 .
drwx------ 73 patrick patrick  4096 Jul 12 01:10 ..
-rw-r--r--  1 patrick patrick    17 Jul 10 20:21 courierimapacl
drwx------  2 patrick patrick  4096 Jul 12 03:31 courierimapkeywords
-rw-r--r--  1 patrick patrick 47290 Jul 10 20:21 courierimapuiddb
drwx------  2 patrick patrick 77824 Jul 10 20:21 cur
-rw-r--r--  1 patrick patrick     0 Jul 10 20:21 maildirfolder
drwx------  2 patrick patrick  4096 Jul 12 03:31 new
drwx------  2 patrick patrick  4096 Jul 12 03:31 tmp

Here is a nice shell script (adapted from here) that fixes all mailboxes inside a single Maildir: 

root@pelargir:~# cat /tmp/fix-maildir.sh 
#!/bin/bash

if test $# -ne 1; then
  echo "Usage: $(basename $0) /path/to/Maildir"
  exit 1
fi

M="$1"
if test ! -d "$M"; then
  echo "Maildir does not exist: $M"
  exit 1
fi

echo checking $M
for d in cur new tmp ; do
    [ -d "$M/$d" ] || { echo "* Creating $M/$d"; mkdir "$M/$d"; }
    chown -c --reference "$M" "$M/$d"
    chmod -c 700 "$M/$d"
done
find $M -type f -name maildirfolder | while read ; do
    f=`dirname "$REPLY"`
    echo "  checking $f"
    for d in cur new tmp ; do
        [ -d "$f/$d" ] || { echo "  * Creating $f/$d"; mkdir "$f/$d"; }
        chown -c --reference "$f" "$f/$d"
        chmod -c 700 "$f/$d"
    done
done


IMAP operations timeout

Sometimes, when I attempt to do something in SquirrelMail that affects a lot of messages, I get an error message that looks like this: 

ERROR: Connection dropped by IMAP server.
Query: COPY 4181,5001:5135,5396,5434,5446,5491,5499,5506,5514,5522,5531,5539,5547,5555,5565,5633,5654,5756,5783,5791,5821,5830:5831,5859:5862,5864:5866,5919 "INBOX.Trash"

In the example above, I tried to delete the last 167 messages from a folder that contains 6167 messages. The end result of the operation was:

  • The messages were placed into .Trash/cur
  • The messages were not deleted from .Junk.ham


The error message refers to a query named "COPY". This prompted me to have a look at RFC 3501 (http://tools.ietf.org/html/rfc3501), the IMAP4rev1 specification. I noticed that the specs do not contain a "MOVE" command. I conclude that in order to implement a move operation, an IMAP client such as SquirrelMail needs to translate the operation into 2 IMAP commands: First a COPY, second a DELETE command. The observed manner of failure of the operation supports this conclusion and suggests the following order of events:

  • SquirrelMail initiates the COPY command
  • The IMAP server process starts to process the command, but takes a long time
  • SquirrelMail times out and displays the error message
  • The IMAP server process continues to process the command until it finishes, but never receives the "DELETE" command from SquirrelMail
  • The end result displayed by SquirrelMail are the copied messages in the Trash, but the originals still in place in the source folder


So the all-important question is: Why does the IMAP server process take so long to process the COPY command? Too much load on the system? Not enough disk space? Not enough memory?


Things that I have tried without success:

  • At first I assumed that imapproxyd had something to do with the problem, but stopping that daemon does not resolve the issue
  • Stopping boinc to give imapd all the available processor power also does not help (which seems logical after top revealed that the imapd process does not use any processing power at all)
  • Checking the /home filesystem + optimizing the filesystem for directories with many files did not help anything, either
tune2fs -O dir_index /dev/hda7
e2fsck -D /dev/hda7
  • Run a manual IMAP session without SquirrelMail as the intermediate to check whether or not SquirrelMail somehow plays a role. Except for the fact that it times out, SquirrelMail cannot be blamed, though: In both sessions copying 167 mails takes about the same time (3 minutes)


Let's watch the system while we repeat the operation from above:

  • CPU activity
    • top shows that the operation spawns an imapd process running as user "patrick" (the user logged in to SquirrelMail)
    • After an initial burst, the imapd process no longer displays any noticeable CPU activity; the other Courier processes never show any activity at all
    • top -u patrick continuously displays the imapd process at 0.0% CPU activity, i.e. no CPU activity at all
    • The imapd process continues to run long after SquirrelMail reports its "Connection dropped by IMAP server" error; this supports the assumed order of events from further up
  • File activity
    • Monitoring .Trash/tmp by repeatedly running "ls -l" reveals that the imapd process continuously places files into this directory until it has created files for all the messages requested to be moved
    • lsof -r 2 -c imapd lists the open files of the imapd process every 2 seconds. The result shows that the imapd daemon reads a new file from the Junk.ham folder about every 2 seconds. The only variation between samples is the file being read. Sample output:
COMMAND  PID    USER   FD   TYPE     DEVICE    SIZE   NODE NAME
imapd   7504 patrick  cwd    DIR        3,7    4096 474209 /home/patrick/Maildir
imapd   7504 patrick  rtd    DIR        3,3    4096      2 /
imapd   7504 patrick  txt    REG        3,5 1297640  33099 /usr/bin/imapd
imapd   7504 patrick  mem    REG        0,0              0 [heap] (stat: No such file or directory)
imapd   7504 patrick  mem    REG        3,5  928628 359813 /usr/lib/libdb-4.3.so
imapd   7504 patrick  mem    REG        3,3   38412 211935 /lib/libnss_files-2.7.so
imapd   7504 patrick  mem    REG        3,5   16540 359921 /usr/lib/libnss_db-2.2.3.so
imapd   7504 patrick  mem    REG        3,3   41876 211863 /lib/libgcc_s.so.1
imapd   7504 patrick  mem    REG        3,3  149328 211893 /lib/libm-2.7.so
imapd   7504 patrick  mem    REG        3,5  937800 360104 /usr/lib/libstdc++.so.6.0.9
imapd   7504 patrick  mem    REG        3,3 1356012 211875 /lib/libc-2.7.so
imapd   7504 patrick  mem    REG        3,3   38300 211887 /lib/libcrypt-2.7.so
imapd   7504 patrick  mem    REG        3,5   37232 572333 /usr/lib/courier-authlib/libcourierauth.so.0.0.0
imapd   7504 patrick  mem    REG        3,5   19940 359983 /usr/lib/libgdbm.so.3.0.0
imapd   7504 patrick  mem    REG        3,5   30868 360024 /usr/lib/libfam.so.0.0.0
imapd   7504 patrick  mem    REG        3,3  117344 211806 /lib/ld-2.7.so
imapd   7504 patrick    0u  IPv6      20663            TCP osgiliath.herzbube.ch:imap2->osgiliath.herzbube.ch:35449 (ESTABLISHED)
imapd   7504 patrick    1u  IPv6      20663            TCP osgiliath.herzbube.ch:imap2->osgiliath.herzbube.ch:35449 (ESTABLISHED)
imapd   7504 patrick    2w  FIFO        0,5           1797 pipe
imapd   7504 patrick    3u  unix 0xd79e9b20          20664 socket
imapd   7504 patrick    4w  FIFO        0,5           1795 pipe
imapd   7504 patrick    5r   REG        3,7   10242 531752 /home/patrick/Maildir/.Junk.ham/cur/1167674801.M935872P1602V0000000000000307I00081D28_5012.osgiliath,S=10242:2,S
imapd   7504 patrick    6u  unix 0xd79e96a0          20680 socket
  • Log activity
    • No activities in any log files by any IMAP/courier process can be detected


The monitoring activities above confirm some of the facts we already know, but otherwise they provide no real clues. Another attempt: Try to delete the last 314 messages from a different folder than the one I have been exercising so far. The new folder contains 6814 messages. Result: The operation completes blazingly fast in about 5 seconds! Conclusions:

  • It cannot be the folder size that causes imapd to run so slow
  • Or at least it's not the folder size alone
  • The destination folder (Trash) does not affect the outcome of the test


Let's look at the differences between the two folders and hope we get some clues: 

osgiliath:/home/patrick/Maildir# ls -la .Junk .Junk.ham
.Junk:
total 1160
drwx------  6 patrick patrick   4096 Oct 26 00:25 .
drwx------ 60 patrick mail      4096 Dec 22 16:03 ..
-rw-r--r--  1 patrick patrick     43 Oct 26 00:24 courierimapacl
drwx------  2 patrick patrick  32768 Dec 22 21:42 courierimapkeywords
-rw-r--r--  1 patrick patrick 343640 Dec 22 21:25 courierimapuiddb
drwx------  2 patrick patrick 774144 Dec 22 21:42 cur
-rw-------  1 patrick patrick      0 Oct 26 00:24 maildirfolder
drwx------  2 patrick patrick   4096 Dec 22 21:45 new
drwx------  2 patrick patrick   4096 Dec 22 21:45 tmp

.Junk.ham:
total 1024
drwx------  6 patrick patrick   4096 Oct 26 00:38 .
drwx------ 60 patrick mail      4096 Dec 22 16:03 ..
-rw-r--r--  1 patrick patrick     17 Jan  1  2007 courierimapacl
drwx------  2 patrick patrick   4096 Dec 22 14:54 courierimapkeywords
-rw-r--r--  1 patrick patrick 471777 Dec 22 15:19 courierimapuiddb
drwx------  2 patrick patrick 532480 Dec 22 15:19 cur
-rw-------  1 patrick patrick      0 Jan  1  2007 maildirfolder
drwx------  2 patrick patrick   4096 Jan  4  2007 new
drwx------  2 patrick patrick   4096 Dec 22 21:42 tmp


Hm, there is a difference in size between the courierimapacl files. Let's look at their content: 

osgiliath:/home/patrick/Maildir# cat .Junk/courierimapacl                          
owner aceilrstwx
administrators aceilrstwx
osgiliath:/home/patrick/Maildir# cat .Junk.ham/courierimapacl 
owner aceilrstwx


I have no idea why one file contains an additional "administrators" line. Another attempt with the 2 files being the same does not help anything. Another observation is that the courierimapkeywords directories are not of the same size. Comparing their content: 

osgiliath:/home/patrick/Maildir# ls -la .Junk/courierimapkeywords .Junk.ham/courierimapkeywords/
.Junk.ham/courierimapkeywords/:
total 452
drwx------ 2 patrick patrick   4096 Dec 22 14:54 .
drwx------ 6 patrick patrick   4096 Dec 22 22:04 ..
-rw-r--r-- 1 patrick patrick 446431 Dec 22 15:19 :list

.Junk/courierimapkeywords:
total 40
drwx------ 2 patrick patrick 32768 Dec 22 21:42 .
drwx------ 6 patrick patrick  4096 Oct 26 00:25 ..


I have learned elsewhere (can't remember the link to the resource) that keywords are not really crucial, so I boldly remove the :list file and make another attempt - Lo and behold! the damn thing suddenly works!!!

Retrieved from "https://wiki.herzbube.ch/index.php?title=Courier&oldid=6468"