BasicSystemConfiguration
tasksel
Debian package: tasksel
Let's the user choose from a task-oriented list what she wants to do with the system (examples: "desktop environment", "web server", "mail server", etc.). Each entry in the list corresponds to an entire collection of Debian packages that is installed when the user selects that entry.
It is probably not a very good idea to run tasksel after the system has been customized to some extent.
base-config
Debian package: base-config
From the man page:
base-config is the program that was run when you first rebooted into your newly installed debian system. It walks you through setting up the system and down-loading additional software, and so forth. The program can be run at any later date to walk you through essentially the same process again.
Locales
Having the proper locales is very important for working on the command line. An UTF-8 locale is especially important so that umlaut ("ä") and other non-ASCII characters are handled properly. Entering an umlaut without a locale simply does not work!
The locales present on a system also determine which languages are supported in gettext-based PHP applications. For instance, Gallery requires that one of the various "de_DE" locales is installed before the German language can be used in the web interface.
The package
locales
is responsible for managing locales. Add/remove locales and define a default locale using the following command:
dpkg-reconfigure locales
To see which locales currently are present on the system:
locale -a
Setting a default locale causes the environment variable LANG to be set to that locale upon login. The default locale is written to
/etc/default/locale
In a quick test I tried to manually change this file. This seemed to work: When I did my next login, the LANG envvar was properly set to the new value. However, I would still recommend setting the default locale by running dpkg-reconfigure - I simply don't know enough about the system,
Keyboard configuration
The keyboard configuration matters when you have physical access to the system and want to login at the console. When you administrate the system remotely via SSH the computer you're sitting at determines the keyboard layout.
Use loadkeys to load the kernel keymap for the console. Available keymaps can be found in /usr/share/keymaps.
At boot-time the script /etc/init.d/keymap.sh loads the keymap file /etc/console/boottime.kmap.gz.
It should be possible to change this setting by issuing the command dpkg-reconfigure console-common and answering the subsequent debconf questions. If that doesn't help you can manually copy the keymap to /etc/console.
Note: It is possible that the boot-time keymap file includes another file that is not yet available when booting reaches the keymap-loading stage, because the /usr filesystem has not yet been mounted. A solution for this is to create a specific keymap loader script in /etc/init.d that will be executed at a later time when /usr has become mounted.
/etc/motd
The content of /etc/motd is printed to the console or remote terminal session screen when a user logs in. The default in Debian is no fun, so change this to something more suitable.
Note: Around the release of Debian etch, the file to edit was /etc/motd.tail.
Time zone
The system's current time zone is stored in
/etc/timezone
Time zone data is maintained in the Debian package tzdata. Change the system's time zone with this command:
dpkg-reconfigure tzdata
Shell environment
/etc/profile.d
Files present in the folder
/etc/profile.d
are sourced by the system-wide /etc/profile. This happens for all users when they log in.
For instance, to enable the HTB (see the wiki page HTB for details) I have this:
root@pelargir:~# cat /etc/profile.d/pelargir_htb.sh HTB_BASE_DIR=/usr/local/htb . $HTB_BASE_DIR/etc/htb-profile.sh
/etc/bash.bashrc
This file is sourced by /etc/profile when the user logs in. It is also read by each new invocation of bash, as long as the new sub-shell is interactive (i.e. the file is not read when a script is run).
Currently I have added the following stuff:
# ------------------------------------------------------------ # BEGIN pelargir # ------------------------------------------------------------ HTB_BASE_DIR=/usr/local/htb . $HTB_BASE_DIR/etc/htb-bashrc.sh # ------------------------------------------------------------ # END pelargir # ------------------------------------------------------------
See the wiki page HTB for details.
Automatically display compressed files with less
The Debian package less provides two useful shell scripts /usr/bin/lesspipe (this is the one I prefer) and /usr/bin/lessfile. As advised in /usr/share/doc/less/LESSOPEN:
You just need to put the following in your .zlogin/.login/.bash_profile/whatever:
eval $(lesspipe) or eval $(lessfile)
and from now on you can directly open any compressed text file with less as if the program itself were able to read the compressed file. For instance:
less /usr/share/doc/less/changelog.Debian.gz
Note: I put the "eval" above command into
/etc/profile.d/pelargir_lesspipe.sh
Automatically display compressed files with vi
See VimStuff.
User/group management
Add/remove users and groups
The following are low-level tools to manage users and groups. They are present on any Linux system.
useradd userdel usermod groupadd groupdel groupmod
The following are Debian specific front-ends to the above low-level tools. They should be used on Debian systems because they conform to Debian policies.
adduser deluser addgroup delgroup
Note: Both the low-level tools and their Debian specific front ends manipulate system files (/etc/passwd, /etc/group, /etc/shadow). If the user database is located in an LDAP directory, they cannot be used anymore.
Disable an account
To disable an account so that the user can no longer log in:
passwd -l
This changes the password hash in /etc/shadow to a value that is not possible to generate by entering any password. In other words, the account still exists and the user can try to log in, but she will never succeed.
Log file rotation
Debian package
Log file rotation is done by the Debian package
logrotate
Configuration
The package's configuration file is /etc/logrotate.conf, which in turn includes all the package specific files located in
/etc/logrotate.d
The configuration file format is explained in
man logrotate
Periodically running logrotate
Traditionally logrotate is executed periodically by cron. The trigger script is this:
/etc/cron.daily/logrotate
and the time when the daily cron routines are run can be found like this:
# grep daily /etc/crontab 25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
In addition logrotate is periodically run by a systemd timer. The cron script, while still being triggered as usual, aborts early on if it finds that systemd is installed. The end result is that only the systemd invocation takes effect. The systemd timer configuration looks like this:
# cat /lib/systemd/system/logrotate.timer [Unit] Description=Daily rotation of log files Documentation=man:logrotate(8) man:logrotate.conf(5) [Timer] OnCalendar=daily AccuracySec=1h Persistent=true [Install] WantedBy=timers.target
The expression "daily" is equivalent to "every day at midnight". For more details on systemd timers see the systemd wiki page.
Manually running logrotate
logrotate can be manually executed like this
logrotate /etc/logrotate.conf
Possible options are:
--force: Force log rotation even iflogrotatedoesn't think it needs to actually rotate a file.--debug: Print debug output to stderr. No changes will be made to the logs or to the state file.--state /path/to/file: Use the status file specified here. The default status file is stored in/var/lib/logrotate/status.
To test a configuration you can use the following endlessly looping shell script snippet:
while test true; do echo "rotating..." echo "foo" >/path/to/file logrotate --force --state /tmp/logrotate-test.status /tmp/logrotate-test.conf sleep 1 done
Configuration on pelargir
Here is a nifty configuration to rotate backup files in several stages: Daily (7x), weekly (4x) and monthly (12x):
root@pelargir:~# cat /etc/logrotate.d/pelargir-backupninja
# ----------------------------------------
# This configuration rotates backup files in 3 stages:
# - Daily, for the last 7 days
# - Weekly, for the last 4 weeks
# - Monthly, for the last 12 months
#
# After one year backups are rotated out of existence.
# ----------------------------------------
# ----------------------------------------
# Daily rotation stage
# ----------------------------------------
/var/backups/debconfsel.txt
/var/backups/git/*.tar
/var/backups/hardware.txt
/var/backups/home-directories/*.tar
/var/backups/ldap/*.ldif
/var/backups/mysql/sqldump/*.sql
/var/backups/postgres/*.sql
/var/backups/sysreport.txt
/var/backups/usr-local/*.tar
/var/backups/var-www/*.tar {
daily
missingok
# Rotate 7+1 days
rotate 8
compress
# The most recent backup remains uncompressed so that it can
# be transferred to an off-site backup location, and further
# processed by a tool that supports de-duplication (e.g. bup).
# De-duplication works best on uncompressed data.
delaycompress
# These are backup files, not log files, so it wouldn't make
# sense to re-created the original file after it was rotated
nocreate
# Rename the oldest rotation so that it can enter the weekly
# rotation stage
postrotate
OLDEST_ROTATION_FILE="$1.8.gz"
NEXT_STAGE_FILE="$1.weekly.gz"
if test -f "$OLDEST_ROTATION_FILE"; then
mv "$OLDEST_ROTATION_FILE" "$NEXT_STAGE_FILE"
fi
endscript
}
# ----------------------------------------
# Weekly rotation stage
# ----------------------------------------
/var/backups/debconfsel.txt.weekly.gz
/var/backups/git/*.tar.weekly.gz
/var/backups/hardware.txt.weekly.gz
/var/backups/home-directories/*.tar.weekly.gz
/var/backups/ldap/*.ldif.weekly.gz
/var/backups/mysql/sqldump/*.sql.weekly.gz
/var/backups/postgres/*.sql.weekly.gz
/var/backups/sysreport.txt.weekly.gz
/var/backups/usr-local/*.tar.weekly.gz
/var/backups/var-www/*.tar.weekly.gz {
weekly
missingok
# Rotate 4+1 weeks
rotate 5
# The file is already compressed
nocompress
# Rotated files keep the compression extension .gz. This results
# in "foo.weekly.1.gz" instead of the default "foo.weekly.gz.1".
extension .gz
nocreate
# Rename the oldest rotation so that it can enter the monthly
# rotation stage
postrotate
# $1 = foo.weekly.gz
# Oldest = fook.weekly.5.gz
# Next = foo.monthly.gz
OLDEST_ROTATION_FILE="$(echo "$1" | sed -e 's/\.gz$/.5.gz/')"
NEXT_STAGE_FILE="$(echo "$1" | sed -e 's/\.weekly\.gz$/.monthly.gz/')"
if test -f "$OLDEST_ROTATION_FILE"; then
mv "$OLDEST_ROTATION_FILE" "$NEXT_STAGE_FILE"
fi
endscript
}
# ----------------------------------------
# Monthly rotation stage
# ----------------------------------------
/var/backups/debconfsel.txt.monthly.gz
/var/backups/git/*.tar.monthly.gz
/var/backups/hardware.txt.monthly.gz
/var/backups/home-directories/*.tar.monthly.gz
/var/backups/ldap/*.ldif.monthly.gz
/var/backups/mysql/sqldump/*.sql.monthly.gz
/var/backups/postgres/*.sql.monthly.gz
/var/backups/sysreport.txt.monthly.gz
/var/backups/usr-local/*.tar.monthly.gz
/var/backups/var-www/*.tar.monthly.gz {
monthly
missingok
# Rotate 12 months
rotate 12
nocompress
extension .gz
nocreate
# No further rotation stages
}
Here is the configuration to rotate the log files of all Apache virtual hosts:
root@pelargir:/etc/logrotate.d# cat pelargir-apache-vhost
# This configuration closely resembles the one that the apache
# Debian package provides in /etc/logrotate.d/apache2.
#
# A notable difference is that we don't need a prerotate
# directive here, we rely on the fact that such a directive
# already exists in /etc/logrotate.d/apache2
# (pre-rotation triggers awstats log file evaluation)
#
# Additional notes:
# - The permissions specified by the "create" directive
# (see below) normally would prevent the awstats update
# script from accessing the log files, because the update
# script runs as user www-data. To circumvent that problem
# user www-data has been added to group adm. See the
# awstats wiki page for a discussion of other solutions,
# and why this solution is probably the best.
/var/log/apache2/*/*.log {
weekly
missingok
rotate 52
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if /etc/init.d/apache2 status > /dev/null ; then \
/etc/init.d/apache2 reload > /dev/null; \
fi;
endscript
}
Here is the configuration to rotate the log files created by rsyslog for named services (see this wiki page for details):
root@pelargir:/etc/logrotate.d# cat pelargir-rsyslog-services
# This configuration closely resembles the one that the rsyslog
# Debian package provides in /etc/logrotate.d/rsyslog.
/var/log/gitolite/gitolite.log
/var/log/imapd/imapd.log
/var/log/slapd/slapd.log {
size 1000k
missingok
rotate 10
compress
delaycompress
notifempty
nocreate
postrotate
/usr/lib/rsyslog/rsyslog-rotate
endscript
}
Manage SysV services
An interactive front-end to enable/disable SysV services on the terminal is
sysv-rc-conf
A command line utility to do the same, but with more control over when exactly a service is started or stopped (i.e. the sequence number can be specified):
update-rc.d
The latter is especially useful if the rc script contains an LSB-style header. In this case, it should be sufficient to say
update-rc.d <myservice> defaults
Partitioning and file system management
Partitioning and formatting
There are two commonly used partitioning utilities:
fdisk: Is used if the partition table format is MBR. Does not support the partition table format GPT.parted: Is used if the partition table format is GPT (possibly MBR, too).
For the MacMini I had to use parted.
Both utilities work interactively. Important: Specify the device that represents the entire hard disk drive.
fdisk /dev/sda parted /dev/sda
Once partitions have been created, they need to be formatted. The following example formats the first partition on /dev/hda using the ext3 file system:
mke2fs -j /dev/hda1
File system management
Parameters of an existing ext[234] file system can be changed. The file system must be unmounted for this to work. Use the following command:
tune2fs
For instance, to optimize lookups in large directories with many files:
tune2fs -O dir_index /dev/hda7 e2fsck -D /dev/hda7
(the second command changes existing directories to use the hashed b-tree that were enabled by tune2fs)
File system layout
| Mount point | Size | In use | Remark |
|---|---|---|---|
| / | 2 GB | 250 MB | The current size seems to be grossly oversized, but on the other hand we really want to be on the safe side. |
| /tmp | 4 GB | 150 MB | Exists so that an out-of-control program does not crash the system by filling up the root file system. |
| /usr | 20 GB | 2.7 GB | Contains the main system installation, including /usr/local. Very stable, not much fluctuation.
|
| /home | 20 GB | 1.4 GB | Contains the user home directories (excluding the directory of the super user). At the moment does not contain much data except mail. |
| /var | 20 GB | 2.5 GB | Contains data and all kinds of stuff that is variable and volatile. Things that use up a lot of space have been "outsourced" to their own file systems. |
| /var/backups | 100 GB | 194 MB | Has been separated from /var so that a rampaging backup script does not crash the system by filling up /var.
|
| /var/cache | 50 GB | 14 GB | Mainly exists as a separate file system because BitTorrent files require a lot of space. |
| /var/samba/daten | 100 GB | 7.0 GB | File server #1, contains files that are not media. |
| /var/samba/media | 450 GB | 377 GB | File server #2, contains media files that require a lot of space (e.g. music, images, movies). |
| swap | 4 GB | - | Ditched the old rule "swap = double the RAM size" - same as RAM size should be sufficient. |
Mount file systems based on UUID
Traditionally file systems in /etc/fstab are referenced using a device node under /dev. Example:
/dev/sda5 /mnt/foo ext4 defaults 0 2
This becomes a problem if the file system is located on a removable medium such as an USB stick or an external USB hard disk drive. If the medium is disconnected, then reconnected at a later time, the kernel will not assign the same device node to the file system if in the meantime another device has been connected to the host.
The solution to this is to reference file systems using their UUID. Type the following command to get a list of all devices that are currently connected to the host, including their UUIDs:
blkid
Note that if blkid is run without root privileges it will merely print cached information. If blkid prints nothing at all the cache is still empty and you'll need to run the command once with root privileges (e.g. using sudo).
Once you have obtained a file system's UUID you can rewrite the file system's entry in /etc/fstab like this:
UUID=ca8f1d49-429f-4f12-a587-79e299cb4f67 /mnt/foo ext4 defaults 0 2
Important: A file system's UUID is randomly created when the file system is created, so if you reformat a partition for any reason the partition will get a new UUID and any /etc/fstab references to the old UUID will have to be updated.
Information on /etc/fstab
Lines in /etc/fstab have the following format:
/dev/sda5 /mnt/foo ext4 defaults 0 2
Discussion:
- Column 4 lists mount options. The string "defaults" says that the default mount options for the file system type apply.
- Column 5 indicates whether the
dumpcommand needs to dump the file system. Value 0 = don't dump. - Column 6 is a numerical value that indicates the order in which order file systems have to be checked when the system boots. Value 0 = don't check. Value 1 = check file system (used for root file system). Value 2 = check file system (used for non-root file systems); file systems on different drives are checked in parallel.
Mount options (cf. man mount):
- user = Non-root users can mount the file system. If specified this implies noexec, nosuid, and nodev (unless overridden by subsequent options).
- exec = Programs stored on that file system can be executed
- suid = Allow set-user-identifier or set-group-identifier bits to take effect
- rw = Mount file system read-write
- dev = Interpret character or block special devices on the filesystem
- async = I/O to the filesystem should be done asynchronously
- auto / noauto = File system is mounted / not mounted when
mount -ais used - defaults = rw,suid,dev,exec,auto,nouser,async
Working with btrfs
If it is not already loaded, load the btrfs kernel module
modprobe btrfs
Install the user-space utilities that allow us to create or otherwise handle a btrfs file system
btrfs-tools
Create the filesystem
mkfs.btrfs /dev/sda1
cron
The Debian package for running the cron service is, unsurprisingly,
cron
The cron daemon looks for things to execute in the following places
/var/spool/cron/crontabs- The traditional user-based crontab files which can be managed using the
crontabcommand line utility /etc/crontab- A single file that lists commands to execute by user. The format of the file is similar to the traditional crontab format, with the addition of a "user" field that indicates the username that each command should be executed as. The cron daemon monitors
/etc/crontabfor changes. /etc/cron.d- An entire directory that may contain many files, in the same format as
/etc/crontab. The cron daemon monitors/etc/cron.dfor changes. The man page discourages using this directory.
It gets interesting if we look at the default content of /etc/crontab:
SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin # m h dom mon dow user command 17 * * * * root cd / && run-parts --report /etc/cron.hourly 25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
This illustrates that there are a number of directories where we can drop scripts that are then run at specific times, either hourly, daily, weekly or monthly.
Init scripts
An init script is a script that should be executed during system boot, or when a switch to a different run level occurs. Init scripts must be placed into the folder
/etc/init.d
The classic way to enable an init script is to create a symlink in one of the runlevel specific folders (e.g. /etc/rc2.d). The number used in the symlink name determined the order in which scripts where executed. Debian used to create these symlinks with the command update-rc.d, however since Debian Squeeze the new command to enable a script is
insserv foobar
insserv parses the LSB headers of an init script and then calculates the proper start sequence by resolving the dependency list found in all init scripts. The symlinks in the runlevel specific folders are then (re)created as necessary. For this to work, init scripts must be LSB compliant and contain LSB headers so that they can specify their dependencies.
For details see
- This status page with details about the initiative to migrate Debian to dependency-based booting
- How to LSBize an Init Script
/etc/init.d/skeletonwhich might serve as a template for a completely new init script (the template contains useful Debian-specific stuff for starting/stopping a daemon process)
This is how the LSB header of my script /etc/init.d/pelargir looks like:
#! /bin/bash ### BEGIN INIT INFO # Provides: pelargir # Required-Start: $network # Required-Stop: $network # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Host specific small stuff # Description: Currently this script only manages a few (important!) # iptables rules. ### END INIT INFO # Notes # - See http://wiki.debian.org/LSBInitScripts for a primer on LSB compliance # - "Provides" is set to the script name, as recommended in the primer # - Currently this script only manages iptables rules, and iptables is in /sbin, # so there is no need to depend on either $local_fs or $remote_fs [...]