Syslog

Overview

This page has information about the System Logging Daemon (syslogd) and the syslog service it provides. At the moment I use rsyslog because this has become the default since Debian "lenny".

Syslog Daemon packages

When I started using Debian, the standard syslogd package on Debian was sysklogd. This provides an advanced version of the standard Berkeley utility program. Its rules allow to redirect log messages based on the terms "facility" and "priority", which are both concepts originating in C preprocessor macros defined in the POSIX standard system header

/usr/include/syslog.h

sysklogd is therefore limited in its capabilities to values preconceived by those who designed the POSIX standard. This has long been sufficient for me, but at a certain point I was interested in getting more fine-grained levels of control, especially when I enabled my ADSL router and Wi-Fi access point to forward their log messages to my Linux server.

A few noteworthy alternatives to sysklogd are:

• dsyslog: More modular and expandable than the regular package
• syslog-ng: Improved configurability, also filtering based on message content
• rsyslog: Enterprise-class, may write to databases (e.g. MySQL), may be used to form relay chains over TCP and SSL/TLS

I eventually decided to start to use rsyslog as a replacement for sysklogd, mainly because this has become the standard syslogd on Debian since the release of Debian 5.0 (lenny). There are a variety of reasons why Debian has gone for rsyslog; some of them can be read up on this wiki page (e.g. sysklogd has become pretty much unmaintained over the years), but the main reason why not to go for syslog-ng is that this project is dual-licensed, i.e. not entirely GPL. Finally, read this blog article by the author of rsyslog for his (IMHO sound) reasons for creating yet another syslog solution.

sysklogd

References

man sysklogd
man syslog.conf

Remote logging

If external sources (e.g. ADSL router, Wi-Fi access point) should be able to log messages over the network, the syslog daemon needs to be run with the special option -r. This can be configured in the following file:

pelargir:/etc/init.d# cat /etc/default/syslogd 
SYSLOGD="-r"

When started with the -r option, sysklogd listens on UDP port 514.

Configuration

The configuration file is this

/etc/syslog.conf

If something in the configuration file has changed, the daemon can be notified so that it re-reads the file, in the same way as inetd:

kill -SIGHUP $(cat /var/run/syslogd.pid)

The configuration file consists of rules that specify what is logged where. Each rule consists of two fields:

• The selector field (defining which messages are logged)
• The action field (defining where messages are sent, often the path to a file)

The selector field itself again consists of two parts, which are separated by a period ("."):

• The facility (specifying the subsystem that produced the message)
• The priority (defining the severity of the message)

Both facility and priority names correspond to the similar LOG_ values in

/usr/include/syslog.h

An asterisk ("*") stands for "all" facilities or priorities.

rsyslog

Upgrade from sysklogd

The rsyslog package description says that "it is quite compatible to stock sysklogd and can be used as a drop-in replacement." Since I have not made any customizations to /etc/syslog.conf, the upgrade was very simple:

• Install rsyslog package
• This automatically causes sysklogd to be removed
• klogd is also automatically removed because its status of "automatically installed" due to a sysklogd dependency
• sysklogd and klogd simply need to be purged to remain all configuration file traces
• Finally, mark rsyslog as automatically installed

References

• man rsyslogd
• man rsyslog.conf
• /usr/share/doc/rsyslog-doc (if the rsyslog-doc package is installed)
• Filter conditions: http://www.rsyslog.com/doc-rsyslog_conf_filter.html
• Available properties: http://www.rsyslog.com/doc-property_replacer.html
• Actions: http://www.rsyslog.com/doc-rsyslog_conf_actions.html

Configuration

The main configuration file is

/etc/rsyslog.conf

The configuration can be extended by dropping files in

/etc/rsyslog.d

If something in the configuration files has changed, the daemon can be notified so that it re-reads the files, in the same way as inetd:

kill -SIGHUP $(cat /var/run/rsyslogd.pid)

For easy maintenance, I create the following file with all my local modifications

/etc/rsyslog.d/pelargir.conf

Note that the file must have the .conf extension to be recognized.

Remote logging

If external sources (e.g. ADSL router, Wi-Fi access point) should be able to log messages over the network, the following configuration snippet needs to be placed into /etc/rsyslog.d/pelargir.conf:

# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

rsyslogd now listens on UDP port 514.

Place messages in separate files depending on the name of the logging service

First create a directory that will receive the log files:

mkdir /var/log/remote

Then place the following configuration snippet into /etc/rsyslog.d/pelargir.conf:

# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# If you add services here, you must also edit the logrotate
# configuration.
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

# Template for service name-based log files
$template ServiceLogfile,"/var/log/%programname%/%programname%.log"

# Route messages from defined services into log files based on
# the name of the service. The parts of this rule are:
# - ":" indicates that this is a property-based filter (traditionally
#   the filter would be a severity/facility based selector)
# - "programname" names the property whose value should be examined
# - "," a simple separator
# - "ereregex" indicates the compare-operation, in this case that the
#   property value should be compared against an extended regular
#   expression (another compare-operation is "isequal")
# - "," a simple separator
# - "(programname1|programname2|...)" the regular expression to compare
#   against
# - "?" indicates that the action is a dynamic filename (as opposed
#   to static files that must be specified starting with a "/")
# - "ServiceLogfile" is the name of the template that must be
#   evaluated to get the actual filename
# - "&" on a new line indicates that for the same filter rule there
#   is another action coming up
# - "stop" prevents the message from being processed any further
:programname, ereregex, "(slapd|imapd|gitolite)" ?ServiceLogfile
& stop

Possible additional service names to add

• named
• dhcpd
• hddtemp
• collectd
• [...]

To support remote logging, add the following snippet:

# Template for hostname-based log files
$template RemoteHostLogfile,"/var/log/remote/system-%HOSTNAME%.log"

# Route messages from defined remote hosts into log files based on
# the name of the remote host. For a detailed discussion of the parts
# of this rule, see the rule above that handles message routing based
# on service names.
:hostname, ereregex, "(landroval|alcarondas)" ?RemoteHostLogfile
& stop

Note that "landroval" and "alcarondas" are just two examples for host names.

Log rotation

Rotation of default log files such as /var/log/syslog is triggered by

/etc/logrotate.d/rsyslog

Rotation of non-default log files must be managed by a custom logrotate config snippet. Details are available on this page.