DAViCal

From HerzbubeWiki
Jump to: navigation, search

This page provides details about setting up DAViCal, a CalDAV calendar and CardDAV contact sharing service, on Debian Linux. Most of the information on this page can also be found, in one form or another, on the DAViCal website and the DAViCal wiki.


Contents

References

External:


On this wiki:


Glossary

RSCDS 
"Really Simple CalDAV Store". The original project name under which DAViCal was started.
AWL 
Andrew's Web Libraries


Research

Even though this page is about the CalDAV store DAViCal, I include a bit of research that I have made before deciding to go with DAViCal.

  • Q: Why use a CalDAV server instead of, for instance, the PHP based project WebCalendar?
  • A: WebCalendar exports .ics files, but CalDAV is server oriented


Other CalDAV servers:


Other URLs:


Debian installation

Non-Debian APT source

Note: The information in this chapter is no longer accurate because DAViCal packages are now part of the official Debian distribution. I like to keep this chapter around for historical purposes, though.


DAViCal is available as a Debian package, but it must be installed from a non-Debian APT source. The first step is to tell APT to trust that source: 

apt-key advanced --keyserver subkeys.pgp.net --recv-keys F6E0FA5CF0307507BB23A512EAFCFEBF8FEB8EBF

The PGP key ID to use can be found somewhere on the DAViCal website, usually the installation page.


Configure the new APT source by adding the following line to /etc/apt/sources.list: 

deb http://debian.mcmillan.net.nz/debian lenny awm


Finally, run aptitude and update the update the list of available packages ("u")


Debian packages

Install the following Debian packages: 

davical
davical-doc
postgresql
postgresql-client

Notice that the postgresql server package needs to be explicitly installed. Also, postgresql-client should be installed as the davical package depends on PostgreSQL client packages with explicit version numbers (e.g. postgresql-client-8.4).


PostgreSQL configuration

New database users

To grant the DAViCal service access to its database, a new database user must be created. The following command achieves this: 

su postgres -c "createuser --no-superuser --no-createdb --no-createrole --pwprompt --encrypted davical_app"

Discussion:

  • su is necessary even if logged in as root, because the default configuration of PostgreSQL does not grant special privileges to root. The UNIX user postgres, however, is allowed to connect as the database super user (which "happens" to have the same name, postgres)
  • createuser is a wrapper around the SQL command "create role", which means that we should probably say that davical_app is the new database role (instead of the new database user)
  • --pwprompt prompts you for a password; --encrypted causes the password to be stored in the database in encrypted format. Use these two options only if you plan to use password authentication
  • --no-superuser etc.: The new role definitely is no super user. It only has minimal privileges, notably it does not have the right to create new databases, nor can it create new roles


In addition, DAViCal requires an administrative user for things such as database upgrades. This user is expected to be named davical_dba. 

su postgres -c "createuser  --no-superuser --no-createdb --no-createrole --pwprompt --encrypted davical_dba"


New database

To create the new database for the DAViCal service, run the following command: 

su postgres -c /usr/share/davical/dba/create-database.sh

Discussion:

  • As above, su is necessary
  • The create-database.sh script calls the AWL database scripts as part of itself and it expects them to be located in /usr/share/awl/dba. This path exists if the Debian package libawl-php is installed (which should be the case, it's a dependency of the davical package)
  • AWL = Andrew's Web Libraries
  • The new database created by the script is called davical
  • When the script has finished, it prints the password that was automatically generated for the DAViCal administrator user "admin". Make a note of this!


Database user authentication

DAViCal needs to authenticate as the database user that was previously created. This is configured by adding an entry to 

/etc/postgresql/8.4/main/pg_hba.conf

Information about this file is available here: http://www.postgresql.org/docs/8.4/interactive/auth-pg-hba-conf.html. Detail information about the various authentication methods is located here: http://www.postgresql.org/docs/8.4/interactive/auth-methods.html.


The DAViCal installation page suggests the following entry for a simple installation with no untrusted users on the system. The entry must be placed in front of the default access rule (which is listed here for clarity): 

# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD
local   davical     davical_app                       trust
local   all         all                               ident

Because of the "trust" authentication method, this entry allows anyone to connect to the davical database as the davical_app database user. Although I don't have a high-security environment, this is too insecure for my taste. After all, every MySQL database application on my system already requires a password, so I want to have at least the same security for PostgreSQL. The entry I use therefore looks like this: 

local   davical     davical_app                       md5

Note: The authentication method "md5" requires the client to supply an MD5-encrypted password for authentication. This is mainly of interest to address password sniffing if the client connects to the database over TCP/IP. Since in the current case DAViCal connects using Unix-domain sockets, it would probably be OK to use authentication method "password" (which allows the password to be transmitted in clear text).


Restart the database server to activate the new authentication rule: 

/etc/init.d/postgresql-8.4 restart


Of course, the DBA user also needs access, so we need to add another line: 

local   davical     davical_dba                       md5


DNS configuration

To allow DAViCal its own Apache virtual host (see next section), a new DNS name must be added. Add the following public CNAME: 

cal.herzbube.ch


Apache configuration

Although it appears to be possible to configure Apache so that DAViCal lives in a sub-location of an existing virtual host (e.g. under /davical), the best results (= those with the least trouble) are achieved when DAViCal has an entire virtual host for its own.


Therefore, edit the file 

/etc/apache2/sites-available/cal.herzbube.ch

and add the following Virtual Host configuration. Note that we only define an SSL host because we want to completely disable non-https access (http connections to cal.herzbube.ch will be answered by the default vhost which forbids all access). 

# --------------------------------------------------------------------------------
# cal.herzbube.ch
# --------------------------------------------------------------------------------

# --------------------------------------------------------------------------------
# SSL Host
# --------------------------------------------------------------------------------
<IfModule mod_ssl.c>
  <VirtualHost *:443>
    ServerName cal.herzbube.ch
    ServerAlias cal.lan.herzbube.ch cal.wifi.herzbube.ch
    ServerAdmin webmaster@herzbube.ch
    ErrorLog /var/log/apache2/cal.herzbube.ch/error.log
    CustomLog /var/log/apache2/cal.herzbube.ch/access.log combined

    DocumentRoot /usr/share/davical/htdocs
    Alias /robots.txt /var/www/cal.herzbube.ch/robots.txt

    DirectoryIndex index.php index.html
    Alias /images/ /usr/share/davical/htdocs/images/

    <Directory /usr/share/davical/htdocs/>
      AllowOverride None
      Order allow,deny
      Allow from all
      php_admin_flag engine on
    </Directory>

    <IfModule mod_php5.c>
      php_value include_path /usr/share/awl/inc
      php_value magic_quotes_gpc 0
      php_value register_globals 0
      php_value error_reporting "E_ALL & ~E_NOTICE"
      php_value default_charset "utf-8"
    </IfModule>

    <IfModule mod_rewrite.c>
      RewriteEngine On
      # Not if it's the root URL.  You might want to comment this out if you
      # want to use an explicit /index.php for getting to the admin pages.
      RewriteCond %{REQUEST_URI} !^/$
      # Not if it explicitly specifies a .php program, stylesheet or image
      RewriteCond %{REQUEST_URI} !\.(php|css|js|png|gif|jpg)
      # Everything else gets rewritten to /caldav.php/...
      RewriteRule ^(.*)$ /caldav.php$1  [NC,L]
    </IfModule>

    SSLEngine on
    SSLCertificateFile /etc/ssl/certs/herzbube.ch.crt
    SSLCertificateKeyFile /etc/ssl/private/herzbube.ch.key.unsecure
    SSLCertificateChainFile /etc/ssl/certs/cacert.org.certchain
    SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown
  </VirtualHost>
</IfModule>

To activate the new virtual host: 

mkdir /var/log/apache2/cal.herzbube.ch
a2ensite cal.herzbube.ch
apache2ctl restart


OpenLDAP configuration

To enable LDAP authentication for DAViCal, I make the following changes to the OpenLDAP configuration:

  • Add a new object class davicalAccount to /etc/ldap/schema/naef.schema (see OpenLDAP for details)
  • Augment those user accounts under ou=users,dc=herzbube,dc=ch with the new object class that should be allowed to login to DAViCal


DAViCal configuration

General configuration

The DAViCal configuration generally resides in 

/etc/davical/<domain>-conf.php

In my case the file is named 

/etc/davical/cal.herzbube.ch-conf.php

The configuration file is a regular PHP file which sets (or overrides) some specific variables. Various pages on the DAViCal wiki document different configuration aspects, but this is the main page.

The minimal configuration tells DAViCal how to connect to its database: 

<?php
  $c->pg_connect[] = 'dbname=davical port=5432 user=davical_app password=secret';

Notice that the PHP tag (<?php) is not closed (?>).


A more complete configuration (i.e. the one I use) is this: 

<?php
  $c->pg_connect[] = 'dbname=davical port=5432 user=davical_app password=secret';
  $c->system_name = "herzbube.ch CalDAV Server";
  $c->readonly_webdav_collections = true;
  $c->admin_email = "herzbube@herzbube.ch";

Discussion:

  • The LDAP authentication part is missing here and is discussed further down
  • readonly_webdav_collections: I'm not entirely sure what this does. The DAViCal wiki mentions that the default for this option is false for backwards compatibility, and that it may be desirable to set it to false during initial setup because it is supposed to make it easier for people to "convert their data". "After this it is recommended to turn it off so that clients which have been misconfigured are readily identifiable.". All this lets me assume that it's ok for me to set this option to true.


Securing the configuration

Because the configuration file contains one or more passwords, it should be secured so that it is not world-readable: 

chmod 640 /etc/davical/cal.herzbube.ch-conf.php
chgrp www-data /etc/davical/cal.herzbube.ch-conf.php


LDAP authentication

Documentation for setting up LDAP authentication can be found here:


By default DAViCal authenticates its users against its own internal user/password list, which is stored in a table in DAViCal's PostgreSQL database. This can be changed by setting the following two configuration options: 

$c->authenticate_hook['call'] = 'foo';
$c->authenticate_hook['config'] = array(
  [...]
);


For authentication against LDAP, I add the following snippet to /etc/davical/cal.herzbube.ch-conf.php: 

$c->authenticate_hook['call'] = 'LDAP_check';
$c->authenticate_hook['config'] = array(
  'host' => 'localhost',
  'port' => '389',

  // DN and password for initial bind
  // Note: Comment these out for anonymous initial bind
  'bindDN'=> 'cn=readonly-users,ou=users,dc=herzbube,dc=ch',
  'passDN'=> 'secret',

  // Version of LDAP protocol to use
  'protocolVersion' => '3',
  // Where to look for valid user
  'baseDNUsers'=> 'ou=users,dc=herzbube,dc=ch',
  // Only look at entries that match this filter
  'filterUsers' => 'objectClass=davicalAccount',
  // The next two options are not used at the moment and
  // can therefore remain commented out
//  'baseDNGroups' => 'ou=groups,dc=herzbube,dc=ch',
//  'filterGroups' => 'objectClass=groupOfUniqueNames',
  // Map DAViCal attributes (from database table "usr") to LDAP
  // attributes. Notes:
  // - "updated" must be set
  // - "modifyTimestamp" is an internal attribute of OpenLDAP
  'mapping_field' => array("username" => "uid",
                           "updated" => "modifyTimestamp",
                           "fullname" => "displayName",
                           "email" =>"davicalEmail"
                      ),
  // Set default values for all users
  // Note: Fields that are also defined in mapping_field will
  // override fields defined here
  'default_value' => array("date_format_type" => "E",
                           "locale" => "en_EN"),

  // Map parts of the OpenLDAP timestamp to parts of the
  // DAViCal timestamp in the SQL attribute "updated". 
  // Example for an OpenLDAP timestamp: 20070503162215Z
  'format_updated' => array('Y' => array(0,4),
                            'm' => array(4,2),
                            'd' => array(6,2),
                            'H' => array(8,2),
                            'M' => array(10,2),
                            'S' => array(12,2)),

  // Whether or not to use TLS.
  // Note: Comment this out to disable TLS (setting to "no"
  // does not work)
//  'startTLS' => 'yes',

  // Search scope to use, defaults to subtree.
  // Allowed values: base, onelevel, subtree.
  'scope' => 'subtree',
);

include('drivers_ldap.php');


Database upgrades

When DAViCal is upgraded, the database scheme often needs an upgrade as well. For this purpose we earlier created the database administrative user 

davical_dba

and gave it access to the davical database via an entry in pg_hba.conf. The general DB upgrade procedure works like this: 

cd /tmp   # cwd must not be /root because user postgres has no access there
sudo -u postgres bash
/usr/share/davical/dba/update-davical-database

The problem here is that the updating script is not very flexible. Although the DAViCal wiki pages talk about storing the password in a file named /etc/davical/administration.yml, this doesn't really work. If the update script needs to apply several patches to the database scheme, it will start to ask for the password for each patch after the first. The only way I have found to prevent the script from asking is to store the password in the PostgreSQL password file 

/var/lib/postgresql/.pgpass

File content: 

*:*:davical:davical_dba:secret


PostgreSQL upgrades

When PostgreSQL is upgraded to a new version, the upgrade process described on the PostgreSQL page on this wiki must be followed.

In addition, the DAViCal database needs to be moved from the old to the new PostgreSQL cluster: 

# create backup on old cluster
sudo -u postgres pg_dump --clean davical >/tmp/davical.sql
# recreate roles on new cluster
createuser --no-superuser --no-createdb --no-createrole --pwprompt --encrypted davical_app
createuser  --no-superuser --no-createdb --no-createrole --pwprompt --encrypted davical_dba
# recreate database
createdb -O davical_dba davical
# restore database content from backup
psql -d davical </tmp/davical.sql


Web browser access

Initial login

If everything has been set up correctly, it should now be possible to point your web browser at http://cal.herzbube.ch/. You should be able to log in as the administration user "admin", using the initial password that was printed out after the database creation script has run (see the corresponding section further above). If the password is unknown, it can be retrieved with this command: 

su postgres -c "psql davical -c 'select username, password from usr;'"

Note: Once you change the initial password (which is recommended) it won't be readable in this way - only the initial configuration leaves passwords readable like this.


New users

If LDAP authentication is used, users must not be created in DAViCal. Instead, they must be created in the LDAP directory. A subsequent login of that user to the DAViCal web interface will sync the user's data in DAViCal with the information obtained from LDAP.


Client configuration

iCal

References:


Open the preferences dialog and do the following

  • Select the "Accounts" tab
  • Click "+" button to add a new account
  • Account details:
  • After the account has been added, there are a couple of additional options that can now be configured, e.g. how frequently calendars should be synchronized

Note: It is also possible to create an account that does not use an SSL connection. Such an account uses a http (instead of an https) URL. When you add such an account, iCal warns you that username/password will be transmitted over an insecure connection. This warning can simply be confirmed/ignored, and the account will be created without problems.


iPhone

References:


Open the preferences app and do the following

  • Select the entry "Mail, Contacts, Calendars"
  • Select the entry "Add Account"
  • Select "Other"
  • Select "Add CalDAV account"
  • Account details:
    • Server = cal.herzbube.ch
    • Username = username (e.g. patrick, or calshare)
    • Password = <secret>
    • Description = Full User Name
  • Select "Continue" button
  • The iPhone now tells you that the connection could not be made over SSL (probably because it tries over port 8443 instead over the standard port 443), and asks whether it should try to add the account without SSL. Say "yes" to this question (actually select "continue" button)
  • The iPhone now tells you that checking the account has failed, however now it has become possible to change a couple of advanced settings. Select the entry "Advanced settings"
  • Advanced settings
  • After finishing the advanced settings, return to the previous configuration page
  • Select "Continue" button
  • The iPhone now tries again to check the account. This should now work and after a moment the account should be successfully created


iPad

Configuring the iPad works pretty much the same as with the iPhone, therefore refer to the section above.


Address Book

  • Launch the Address Book app
  • Open the preferences dialog
  • Switch to the "Accounts" tab
  • Click the "+" button to add a new account
  • Select the following properties for the new account
    • Account type = CardDAV
    • Username = addressbookshare
    • Password = secret
    • Server address = cal.herzbube.ch:443


Populating calendars

Calendar weeks

LDAP:

  • Create a new LDAP user named calshare


DAViCal:

  • Login with the calshare user via the DAViCal web interface
  • This creates the user and an initial calendar
  • Remove the initial calendar.


iCal:

  • Create an account that connects to the calshare user
  1. Create a new calendar, label it "Calendar weeks"
  2. Google to find someone who provides an up-to-date .ics file that contains pseudo-events which mark the calendar week
  3. Download the .ics file
  4. Double click the .ics file. This will open iCal where you can select the previously created calendar as the target of the import


Troubleshooting

PHP error reporting

PHP's error reporting has been severely cut down in the Apache virtual host configuration. The first step in diagnosing any problems should be to comment out the following line in /etc/apache2/sites-available/cal.herzbube.ch: 

php_value error_reporting "E_ALL & ~E_NOTICE"


Debugging configuration

DAViCal can also be told to write debug output to the Apache error log (in my case /var/log/apache2/davical/error.log). This is done by adding the following setting to the PHP configuration file: 

$c->dbg['ALL'] = 1;

This example is extreme because it logs everything! I have seen a few comments that suggest that such heavy logging might have side effects and lead to other problems. To get a more moderate amount of logging, change the "ALL" string to something else. This page on the DAViCal wiki has more details.


Testing + Upgrading

General upgrade procedure

These days DAViCal has official Debian package, so an upgrade should be not more complicated than upgrading the Debian package, and possibly run the DB upgrade script 

sudo -u postgres /usr/share/davical/dba/update-davical-database

(I am not sure if this is run as part of the Debian package upgrade process, but running it a second time does not hurt)


Since the Debian package Changelog does not contain any useful information, one mighth also check the release notes first. The notes for every release nicely state the software dependencies, and if there are changes to the database scheme.


0.9.7.6 -> 0.9.8

Tests

I did some testing for Andrew before he officially released 0.9.8. To achieve this, I manually installed a pre-release version (version 0.9.7.99-1) of the software like this:

  • Download the following files 
http://debian.mcmillan.net.nz/packages/davical/testing/libawl-php_0.39-0_all.deb
http://debian.mcmillan.net.nz/packages/davical/testing/davical_0.9.7.99-1_all.deb
  • Install them in this order 
dpkg -i libawl-php_0.39-0_all.deb
dpkg -i davical_0.9.7.99-1_all.deb
  • Upgrade the database 
sudo -u postgres /usr/share/davical/dba/update-davical-database


Upgrade

After doing the manual tests and running the pre-release version for maybe a month, I upgraded to the final release 0.9.8 like this (note that I upgraded even libawl, although the pre-release already installed the same version; I am not taking any unnecessary risks here):

  • Download the following files 
http://debian.mcmillan.net.nz/packages/awl/libawl-php_0.39-0_all.deb
http://debian.mcmillan.net.nz/packages/davical/davical_0.9.8-0_all.deb
http://debian.mcmillan.net.nz/packages/davical/davical-doc_0.9.8-0_all.deb
  • Install them in this order 
dpkg -i libawl-php_0.39-0_all.deb
dpkg -i davical_0.9.8-0_all.deb
dpkg -i davical-doc_0.9.8-0_all.deb
  • Upgrade the database 
sudo -u postgres /usr/share/davical/dba/update-davical-database


The release notes for version 0.9.8 can be found here: http://wiki.davical.org/w/Release_Notes/0.9.8


0.9.8 -> 0.9.8.4

After performing other system upgrades, DAViCal stopped working for unknown reasons. I then upgraded DAViCal to 0.9.8.4 via aptitude, but the problem persisted. Enabling more verbose warnings (i.e. commenting out php_value error_reporting "E_ALL & ~E_NOTICE") revealed two problems related to changes in the new PHP 5.3:

  • A problem with the PHP variable open_basedir. The mailing list suggests that the following line in the Apache configuration can be removed: 
php_value open_basedir 1
  • A problem with the PHP date functions. This Debian bug report explains why PHP now generates log warnings when no default timezone is hardcoded in php.ini. The bug report also promises a patch that will fix the problem, so at the moment I don't do anything about this issue

After fixing the first problem, everything again works as expected.


0.9.8.4 -> 0.9.9.4

This upgrade also was part of the upgrade from server osgiliath to pelargir. This also meant that the PostgreSQL version moved from 8.4 to 9.0.

  • Set up the new PostgreSQL server 9.0 for use with DAViCal
    • Create normal and administrative database users
    • Create database specifying davical_dba as the owner
    • Add access rights to pg_hba.conf of the 9.0 server
    • Restart the server
  • Dump the database in 8.4 (uses the non-default port 5433 on the new server pelargir!)
  • Shut down the 8.4 server to avoid confusion
  • Restore the database in 9.0
  • Since the 9.0 server uses the default port, the DAViCal configuration does not need to be updated
  • Upgrade the database in 9.0
    • For some reason, the DBA user davical_dba was not set up correctly in a number of ways
    • Password was not set, or unknown -> I specified a new, known password
    • Some tables/objects in the database were not owned by davical_dba; had to correct this first by running the upgrade script, then followed by another statement to explicitly set ownership of a sequence 
cd /tmp
sudo -u postgres bash
/usr/share/davical/dba/update-davical-database --nopatch --dbuser postgres --owner davical_dba
psql davical
alter sequence dav_id_seq owner to davical_dba;
    • Finally run the upgrade script 
sudo -u postgres /usr/share/davical/dba/update-davical-database

The release notes for version 0.9.9.4 can be found here: http://wiki.davical.org/w/Release_Notes/0.9.9.4


0.9.9.4 -> 0.9.9.7

  • The problem during this upgrade was that package dependencies forced a new PostgreSQL version 9.1 to be installed
  • I followed the PostgreSQL upgrade process detailed on the PostgreSQL page on this wiki
  • After PostgreSQL was upgraded and the DAViCal database moved to the new cluster, the actual DAViCal upgrade was simple: 
sudo -u postgres /usr/share/davical/dba/update-davical-database


0.9.9.7 -> 1.1.1-1

This upgrade went very smooth:

  • Upgrade the regular Debian package
  • Run the regular DB upgrade script (not sure if it has been necessary to run this manually, possibly this is already part of the Debian package upgrade process) 
sudo -u postgres /usr/share/davical/dba/update-davical-database
Retrieved from "http://wiki.herzbube.ch/index.php?title=DAViCal&oldid=5471"
Personal tools
Namespaces

Variants
Actions
Navigation
herzbube
francesca
Toolbox