DebianPackageManagement

Configuring APT

Configure package sources

APT package sources are configured in two places. The main list is maintained in this file:

/etc/apt/sources.list

Additional sources.list entries can be placed in separate files that are located in the following folder. Note that file names may only consist of a relatively restricted subset of characters, and must end with .list. APT displays a warning if it ignores a file because of an illegal name.

/etc/apt/sources.list.d

Every line in sources.list (or one of the distributed files) contains a pointer to a package tree within a release. There are 2 different kinds of package trees:

Binary package tree. This kind of package tree is referenced by the prefix "deb"
Source package tree. This kind of package tree is referenced by the prefix "deb-src"

Package trees can be accessed differently:

On a CD-ROM. This location is referenced by the prefix "cdrom".
Via HTTP. This is referenced by the prefix "http://".
Via FTP. This is referenced by the prefix "ftp://".
See "man sources.list" for more types

To select a specific Debian release, it must be specified either using a symbolic name (e.g. "stable", "oldstable", "testing", "unstable"), or a concrete release name (e.g. "woody", "sarge", "etch", "lenny", "squeeze").

The release name is then followed by a list of "components":

Component is one of "main", "contrib", "non-free" or "non-us"
It may also be something like "non-us/main"

Examples:

deb cdrom:[Debian GNU/Linux 8.2.0 _Jessie_ - Official amd64 CD Binary-1 20150906-11:13]/ jessie main

deb http://ftp.ch.debian.org/debian/ jessie main
deb-src http://ftp.ch.debian.org/debian/ jessie main

deb http://security.debian.org/ jessie/updates main
deb-src http://security.debian.org/ jessie/updates main

# jessie-updates, previously known as 'volatile'
deb http://ftp.ch.debian.org/debian/ jessie-updates main
deb-src http://ftp.ch.debian.org/debian/ jessie-updates main

deb http://ftp.ch.debian.org/debian/ jessie-backports main

Important notes:

Changing from one distribution to another usually is a one-way process: You can go from "stable" to "testing" to "unstable", but the reverse direction is not easily possible. This means that when the distribution is changed, no new lines need to be added to /etc/apt/sources.list - instead the lines that refer to the old distribution need to be changed so that they now point to the new distribution
By using a concrete release name (e.g. "etch") you make sure that your system stays at a well-known release, whereas using a symbolic name (e.g. "stable") is a moving target which may be useful if you want your system to automatically upgrade when a new Debian release is made
Security updates are offered for "stable" and "testing", so if you are running "unstable" or "experimental", make sure that you remove the lines relating to security updates in /etc/apt/sources.list
The experimental distribution is not a complete distribution and must therefore be used in conjunction with one of the full releases (probably testing or unstable, as experimental packages will usually have dependencies that are not satisfied by stable)
The volatile distribution usually only makes sense together with the stable distribution; see http://www.debian.org/volatile/ for details

Modifying /etc/apt/sources.list

The file can be modified manually. For adding a CD-ROM, there is the command apt-cdrom add. There is no corresponding command to remove the CD-ROM - simply edit the file and remove the CD-ROM line.

After the file has been changed, the package lists need to be updated

Either by executing apt-get update
Or by selecting "u" from within aptitude
Or by selecting menu 2 from within dselect

If apt-get displays the error "Dynamic MMap ran out of room" after you have added a new source, the cache limit of APT must be set to a higher value. Modify the limit in the file /etc/apt/apt.conf like this (the example sets the value to 16MB):

APT::Cache-Limit 16777216;

Backports

The backports package tree can be enabled by adding a file to sources.list.d. Here's how I do it in Debian jessie:

root@pelargir:~# cat /etc/apt/sources.list.d/jessie-backports.list 
deb http://ftp.ch.debian.org/debian/ jessie-backports main

Backports are disabled by default. To allow installation of a package, use the --target-release command line option for apt-get or aptitude. Example:

apt-get -t jessie-backports install "package"
aptitude -t jessie-backports install "package"

To use aptitude in interactive mode:

aptitude -t jessie-backports install "package"

SecureApt

Imagine someone has managed to hack your favourite Debian mirror, and imagine further that the package you just installed via apt was an Easter egg placed on the mirror by Mr. Hacker. Not nice...!

Luckily the people from Debian have added stuff to APT that allows to check against nasty surprises such as the example above. The whole thing is called "SecureApt", or "apt-secure" respectively. You can read more about it at

http://wiki.debian.org/SecureApt

Basically, SecureApt is about using strong cryptography to construct a "chain of verification from the package that you are going to install all the way back to the provider of the package" (quote from the Debian Wiki page mentioned above).

If on your system apt-get and/or aptitude complain about a package being "from an untrusted source", you have already encountered SecureApt.

What you will have to do to enable SecureApt is this: For every apt repository listed in your /etc/apt/sources.list you have to add an up-to-date GPG key to a keyring that is stored in this file:

/etc/apt/trusted.gpg

The key is used by the APT tools to check if the central "Release" file of an APT repository comes from a trusted source. To add a key to the keyring, use one of the following commands:

apt-key add /path/to/key/file
apt-key advanced --keyserver subkeys.pgp.net --recv-keys <key ID>

You can download the current key file for official Debian archives from (replace year with current year)

http://ftp-master.debian.org/ziyi_key_2006.asc

Note: The official Debian archives key rotates every year in January, so you will have to update at least once every year.

For convenience, I recommend you install the package

debian-archive-keyring

which contains the current keys. If you update the package somewhen around January, the most up-to-date keys will automatically be imported.

Configure custom package sources

APT is not restricted to working with the official Debian distributions, it is also possible to configure APT to use custom package sources. Examples where this might be useful:

An organization creates and maintains .deb packages for software developed in-house
A software project makes an effort to provide its own .deb packages because no Debian developer has had the time (or the inclination) to make official packages

TODO: Write more about this. Also describe how PGP keys must be created and provided to satisfy SecureAPT.

APT::AutoRemove::SuggestsImportant

By default aptitude treats the "suggests" relationship as important. Example:

Package A and B are both installed
Package B is marked as "automatically installed"
Package A "suggests" package B
I remove package A
Package B is not auto-removed

This is not what I want, I want package B to be auto-removed. Luckily, aptitude has a configuration option for this:

root@pelargir:~ # cat /etc/apt/apt.conf.d/99local-autoremove-suggests 
# Allow aptitude to auto-remove packages that are
# - marked as automatically installed, and 
# - are only installed because of a "Suggests" dependency
APT::AutoRemove::SuggestsImportant "false";

dpkg

Useful commands

To install a package:

dpkg -i /path/to/package.deb

To list the contents of a .deb package file:

dpkg-deb --contents /path/to/package.deb

To list the contents of an already installed package:

dpkg --listfiles package-name
dpkg -L package-name

To extract (but not install) the contents of a package:

dpkg --extract /path/to/package.deb /path/to/target-directory

To find out which installed package provides a certain file on your system. Note that the actions taken by the usrmerge package in Debian 12 may cause incorrect results when you query for something in /usr/bin, /usr/sbin or /usr/lib.

dpkg -S /path/to/file

To finish the configuration of all packages or a single package which were left unfinished due to some installation/upgrade problem:

dpkg --configure --pending      # all packages
dpkg --configure package-name   # single package

To re-display the configuration questions that were asked when a package was installed for the first time:

dpkg-reconfigure package-name

stat override

If you want to remove a package but dpkg does not let you, and instead displays an error message that looks like this

An override for "/path/to/file" already exists, aborting

This means that dpkg somewhere in the depths of its package database has recorded a so-called "stat override" for the mentioned file.

A "stat override" for a file (or a directory, or for any other filesystem object) is the instruction to dpkg that it should apply a specific set of owner, group and permissions to the file when it is installed. This specific set overrides (hence the name) the owner, group and permissions that are stored for the file within it's .deb file.

To list all existing "stat overrides":

dpkg-statoverride --list

To remove a "stat override" (the solution for the problem above where you are prevented from removing a package):

dpkg-statoverride --remove /path/to/file

Note: I encountered the "stat override" problem when I tried to remove the midentd package.

apt-get

Information about APT can be found at http://www.debian.org/doc/manuals/apt-howto/

.deb Files that apt-get downloads are stored for later use in /var/cache/apt/archives. It is also possible to manually download a .deb file from packages.debian.org (use the i386 architecture), put it into /var/cache/apt/archives, and then install it via apt-get.

To install a package:

apt-get install package-name

To remove a package (but leave config files on the system):

apt-get remove package-name

To completely remove a package, including all config files:

apt-get --purge remove package-name

To upgrade all installed packages to the newest available version (without installing new packages):

apt-get update
apt-get --show-upgraded upgrade

Ditto, but allow new packages to be installed:

apt-get update
apt-get --show-upgraded dist-upgrade

DebConf

Introduction

DebConf is the "Debian configuration management system". It is not to be confused with DebConf, the Debian Conference.

The DebConf system is used to obtain and store information that is required to configure Debian packages. The process works like this:

A package declares which configuration information it requires. It does this via a .templates file that is set up by the package maintainer.
When the user installs the package, DebConf queries the user for the required information by asking questions from the package's .templates file.
DebConf will not ask questions whose importance falls below a certain threshold that the user has previously defined. For questions like these, DebConf will use default answers provided by the package's .templates file.
DebConf stores the answers that the user provides, or the default answers that the package provides, in the DebConf database.
The answers are then used in the "postinst" phase during package installation to set up configuration files in /etc, start services, or do whatever else is required to make the package functional.
The user can later re-configure the package. DebConf will again query the user, but it will use the answers that are already in the DebConf database as default values.

References

A simple wiki page with a description of the key elements of debconf: https://wiki.debian.org/debconf
DebConf protocol specification
man -s 7 debconf. You need to install the package debconf-doc for this.

Question attributes

DebConf questions have these attributes:

Priority level: There are currently four priority levels: low, medium, high and critical. The user can set the priority level threshold, i.e. which questions he doesn't want to see because they are too low-priority. DebConf will use default answers for those questions and only show questions with a higher priority level. The package maintainer is responsible for defining a question's priority level.
Seen: Whether or not the question has already been answered.

To change the priority level threshold, use this command

dpkg-reconfigure debconf

Front-ends

DebConf can use a variety of front-ends to interact with the user. For instance

A simple, text-based frontent
A more sophisticated ncurses-based frontend
Or even an X11-based frontend such as a GNOME control applet

The special front-end "noninteractive" is useful in a fully automated installation scenario.

To change the default front-end, use this command

dpkg-reconfigure debconf

Some utilities, notably dpkg-reconfigure, let you specify the front-end to use. This can be used to override the default front-end. See each utility's man page to find out if this is possible.

Re-configure a package

After a package has been installed, the user can later re-configure the package with

dpkg-reconfigure <package name>

The DebConf system will again query the user for the required information. Answers already present in the DebConf database are filled in as default answers.

List answers in the DebConf database

For any given package, the answers in the DebConf database can be listed with this command:

debconf-show <package name>

Note: Be aware that a package may ask questions that are defined in a different package. For instance, the package libpam-ldap asks a question that comes from the package libpam-runtime. So in this example when you run debconf-show libpam-ldap you won't see all the answers that the libpam-ldap package asked for when it was installed. If you don't know which package defines a given question, you can search the physical storage of the DebConf database for the question.

Purge answers from the DebConf database

When a package is uninstalled, the answers given when it was installed may remain in the DebConf database - even if the package is purged! Whether or not this happens depends on how careful the package maintainer has written the "postrm" step of his package.

In any case, if the answers remain in the DebConf dabase, a subsequent re-install of the package will reuse the old answers. Depending on the priority level of the package's questions, one may not even see any questions asked.

If this happens, the only way to remove the answers from the DebConf database is to use the low-level utility

debconf-communicate

This utility communicates with the DebConf system via the debconf protocol (the "Reference" section above has a link to the protocol specification). For instance:

echo get package-name/question-name | debconf-communicate

queries the DebConf database for the value of the named question in the named package. Example:

echo get debconf/frontend | debconf-communicate

The following command will purge all question templates and answers of a given package from the debconf database.

echo PURGE | debconf-communicate packagename

To remove only the answer for a given question, without removing the question template from the debconf database:

echo RESET question | debconf-communicate packagename

Credits: I did not find this solution in any of the DebConf man pages, it's an answer to this ServerFault question.

Pre-configure answers

It is possible to pre-populate the DebConf database with answers so that when a package is installed it won't need to ask any questions. This is the command to do so:

dpkg-preconfigure <package name>

dpkg-preconfigure still asks questions interactively. If one wants to get rid of all interactivity, one can use the frontent "noninteractive". This will cause DebConf to use the default answers, or the answers that are already in the DebConf database.

dpkg-preconfigure cannot be used to set arbitrary answer values - the values must always come from the user (interactive), or they must be default values (non-interactive). If you want to programmatically pre-populate the DebConf database with arbitrary values, you have to use this utility:

debconf-set-selections

Read the man page for details how to use this utility.

debconf-utils

The package debconf-utils may contain other useful things, although I have not investigated.

Storage

Storage of the DebConf database is configured in this file:

/etc/debconf.conf

This is described in detail in the man page "debconf.conf", which is available only if the pcakge "debconf-doc" is installed.

The DebConf database consists of three parts

Answers to questions that are not passwords
Answers to questions that are passwords
Question templates

Usually all parts of the DebConf database are stored in files, but it is also possible to store some, or even all of the database parts in other formats, such as an LDAP directory. Read the man page for details.

By default all parts are stored in files. The default locations are these:

/var/cache/debconf/config.dat
/var/cache/debconf/passwords.dat
/var/cache/debconf/templates.dat

Creating a package from source

From the man page of apt-get: "source packages are not tracked like binary packages, they exist only in the current directory and are similar to downloading source tar balls."

Usually, the first step is to create a folder in which you want to build your package. cd into that directory, then execute this command:

apt-get source package-name

Usually this will download 3 files: the original source tar ball (e.g. with extension .tar.gz), a file that contains patches (.diff.gz) and a control file (.dsc).

After the files are downloaded, apt-get automatically invokes dpkg-source, which performs a number of actions according to the control information it finds in the .dsc file. Usually this includes extracting the source tar ball and applying the patches.

If you had specified the option "-b" to apt-get, it would have automatically compiled and built your package. But now you have to do this yourself.

cd /path/to/directory-with-extracted-sources
dpkg-buildpackage -rfakeroot -uc -b

"-b" tells dpkg-buildpackage to only build the binary package (not the source package).
"-uc" tells dpkg-buildpackage not to sign the .changes File with a GPG key
"-rfakeroot" tells dpkg-buildpackage to try to gain root privileges through fakeroot; of course you can omit this if you are already root

Before launching dpkg-buildpackage you may wish to make changes that affect the outcome of the build. Go to the "debian" sub-directory inside the "extracted-sources" directory and update the necessary files; for instance, debian/rules contains the options that are passed to configure.

The product of dpkg-buildpackage is a .deb package file which you can install with "dpkg -i".

Re-packaging a binary package

If the installation process of a package is broken, it may be useful to fix the issue by yourself instead of waiting for the package maintainer to release a fixed package version. These are the commands to extract the content of a package, and to re-package the edited content into a custom package so that installation can proceed from there:

dpkg --extract /var/cache/apt/archives/bugzilla3_3.4.7.0-2_all.deb /tmp/bugzilla3_mine
dpkg --control /var/cache/apt/archives/bugzilla3_3.4.7.0-2_all.deb /tmp/bugzilla3_mine/DEBIAN
# edit some files
dpkg --build /tmp/bugzilla3_mine /tmp/bugzilla3_mine.deb
dpkg --install /tmp/bugzilla3_mine.deb

Also see Debian New Maintainer's Guide.

popularity-contest

The package popularity-contest collects statistics about which packages are installed on a system, how these packages are updated and used. The software "phones home" once in a while (weekly at the time I installed it) to submit the collected statistical data to the Debian servers. On the following site, global results of the survey can be viewed:

http://popcon.debian.org/

Even with an installed popcon package it is possible not to participate in the survey for some time:

dpkg-reconfigure popularity-contest

Removing unused packages

deborphan

From the Manual

From the man page of deborphan:

deborphan - Orphaned package finder

deborphan finds packages that have no packages depending on them. The default operation is to search only within the libs and oldlibs sections to hunt down unused libraries.

Overview

deborphan helps to find orphaned packages, i.e. packages that have no packages depending on them. deborphan operates on the dpkg/apt package database. Candidate packages need to be removed manually, which makes deborphan safe and simple to use. On the other hand, deborphan does not tell you anything about why a package is orphaned.

Usage

As the man page explains, simply running deborphan without any options searches for orphaned packages only within the libs and oldlibs sections. This is usually OK since libraries are mostly installed only as a dependency to other packages, hence an orphaned library is usually one of those accidental orphans that we actually want to find.

To include a couple more libraries (in section libdevel):

deborphan --libdevel

For a comprehensive search that includes really all packages (not only from all sections, but also with all priorities):

deborphan --all-packages --priority=1 --show-priority

This kind of search will produce a list that is much too large because it includes all those packages you have explicitly installed by intent. To exclude those packages:

deborphan --all-packages --priority=1 --show-priority --exclude=foo,bar[,...]

deborphan is capable of maintaining a so-called keep file; this is a file that permanently stores package names that you would otherwise specify with --exclude. To add/remove packages to/from the keep file, or to list the content of the keep file:

deborphan --add-keep foo bar
deborphan --del-keep foo bar
deborphan --list-keep

The default keep file is /var/lib/deborphan/keep. It will be taken into account whenever you run deborphan. If you want to maintain/use a different keep file, simply add the --keep-file=my.keep.file.

Aggressive usage

An aggressive search can be made by adding the --nice-mode option. A little bit counter-intuititve, this option turns nice-mode off. Nice-mode (which is turned on by default) means that a package is not considered an orphan if another package suggests or recommends it. If nice-mode is turned off, only hard dependencies will prevent a package from becoming an orphan.

Another method to treat more packages as orphans is the --force-hold option. This lets deborphan ignore the "hold" flag present on any packages. Due to a bug in aptitude (see BTS), "hold" flags set by aptitude seem to be ignored by deborphan anyway.

Unpurged packages

Another useful operation mode is

deborphan --find-config

This finds all packages that are no longer installed but which still have configuration files on the system. Purging a package might be especially useful to remove the last trace of the package's still lingering "ghost" from the package database :-) i.e. purging the package will make it disappear completely from the package database and the package lists seen by apt/aptitude.

Note: Unpurged packages can also be found by searching in aptitude for "~c" (tilde c).

debfoster

Don't Use It!

I have stopped evaluating debfoster when the second package that it queried me about was a package installed due to a "recommends" dependency. Since the debfoster default configuration is supposed to treat recommendations as real dependencies, I conclude that debfoster is seriously broken and cannot properly evaluate dependencies!

This harsh statement is somewhat supported by http://www.fruit.je/debfoster/ where it says

As of 2006-01-01, debfoster is officially deprecated: aptitude does the same stuff as debfoster but integrated into the apt system

OTOH I found an email thread that says that debfoster is being taken over by other people. What should I believe? The package changelog seems not too encouraging, listing exactly one update since July 2006. I conclude not to waste my time with something that I can do with deborphan and aptitude.

The following stuff is kept for the moment "just in case".

From the Manual

From the man page of debfoster:

debfoster - weed unnecessary Debian packages

debfoster maintains a list of installed packages that were explicitly requested rather than installed as a dependency. [...] debfoster can be invoked per se after each run of dpkg and/or apt-get. [...] If a new package is encountered or if debfoster notices that a package that used to be a dependency is now an orphan, it will ask you what to do with it. If you decide to keep it, debfoster will just take note and continue. If you decide that this package is not interesting enough it will be removed as soon as debfoster is done asking questions. If your choices cause other packages to become orphaned more questions will ensue.

Overview

debfoster examines orphaned packages more closely than deborphan. For instance, it tells you which other packages are kept installed by the orphan package, i.e. which packages would get uninstalled if you were to remove the orphan package. debfoster is usually run as an interactive program. To be useful it needs to maintain its own package database that is separate from the dpkg/apt database.

Usage

The database kept by debfoster is in this file:

/var/lib/debfoster/keepers

debfoster uses this configuration file (options in the configuration file can be overridden with the --option command line argument):

/etc/debfoster.conf

Running debfoster without options start an interactive session where the user is queried about each package that debfoster considers an orphan. The user must decide which packages should be kept and which packages can be removed.

Aggressive usage

debfoster --option MaxPriority=required
debfoster --option UseHold=no
debfoster --option UseEssential=no
debfoster --option UseRecommends=no
debfoster --option UsePreDepends=no

aptitude

An extremely useful feature of aptitude is to ask "why", such as:

osgiliath:~# aptitude why alien
i   nagios-text   Depends nagios-common (= 2:1.4-3.1)
i A nagios-common Depends lsb-core                   
i A lsb-core      Depends alien (>= 8.36)

apt-rdepends

A package P may not be orphaned in the strict sense, i.e. there may be other packages around that are still depending on P. Surprisingly, the search/list features of neither dpkg nor apt can give you a list of those package.

Enter the tool apt-rdepends. This tool is really sweet because it allows you the recursively crawl the package database, using a combination of filters for following links and package states.

This example lists all packages that are installed and that depend on package "ruby". The dependency can be either direct or indirect because apt-rdepends recursively follows dependency links.

root@pelargir:~# apt-rdepends --state-follow=Installed --state-show=Installed -r ruby
Reading package lists... Done
Building dependency tree       
Reading state information... Done
ruby
  Reverse Depends: ruby2.1 (>= 2.1.2-2)
  Reverse Depends: rubygems-integration (1.1)
ruby2.1
  Reverse Depends: libruby2.1 (= 2.1.2-2)
  Reverse Depends: ruby (1:2.1.0.1)
libruby2.1
  Reverse Depends: ruby2.1 (>= 2.1.2-2)
rubygems-integration
  Reverse Depends: ruby2.1 (2.1.2-2)

In this example it becomes apparent that the ruby packages are installed only because of a cyclic dependency between the packges "ruby" and "ruby2.1".

Upgrading to new OS version

Introduction

Upgrading Debian to a newer OS version is, at least in theory, not a terribly hard thing to do. It basically consists of the following steps:

Update the system to the newest packages available for the current OS version
Change package repositories to point to the new OS version
Perform the upgrade

Nevertheless, things can go wrong so it's always best to proceed with caution.

Read the release notes of the new OS version so that you know in advance what the most important changes are. This could be as important as the switch from MySQL to MariaDB, hence an incompatible binary format change.
The release notes should also contain upgrade instructions
If possible test the upgrade on a test system.
Make sure you create backups of whatever is relevant to you.

jessie (8.x) to stretch (9.x)

I used the following sources to prepare and perform the upgrade

The official Debian stretch release notes
linuxconfig.org article

Changes that are important for me:

The system upgrade will switch from MySQL 5.5 to MariaDB 10.1. The process should be transparent. In the background, database files will be altered to use the MariaDB binary format.
New "stretch" installations use a new network interface naming scheme. For instance, the first ethernet interface will no longer be named "eth0". The new naming scheme will not take effect when upgrading from jessie, the names will still be determined by /etc/udev/rules.d/70-persistent-net.rules. I currently have "eth0, lo" and "docker0".
The net-tools package is being deprecated in favor of iproute2. The package is marked as "manually installed" on my system, so it won't get uninstalled during system upgrade. Nevertheless, I should start to familiarize myself with the commands from the iproute2 package. These commands that I sometimes use will no longer be available: ifconfig, netstat, route, arp.
APT is now fetching files as unprivileged user _apt. This might cause problems when doing package updates.

Backups I made as recommended by the Debian release notes:

/etc
/var/lib/dpkg
/var/lib/apt/extended_states
/var/lib/aptitude/pkgstates. 

the output of dpkg --get-selections "*" (the quotes are important)

Modified /etc/apt/soures.list. Original content:

deb http://ftp.ch.debian.org/debian/ jessie main
deb-src http://ftp.ch.debian.org/debian/ jessie main

deb http://security.debian.org/ jessie/updates main
deb-src http://security.debian.org/ jessie/updates main

# jessie-updates, previously known as 'volatile'
deb http://ftp.ch.debian.org/debian/ jessie-updates main
deb-src http://ftp.ch.debian.org/debian/ jessie-updates main

New content, predictably replaces all occurrences of "jessie" with "stretch":

deb http://ftp.ch.debian.org/debian/ stretch main
deb-src http://ftp.ch.debian.org/debian/ stretch main

deb http://security.debian.org/ stretch/updates main
deb-src http://security.debian.org/ stretch/updates main

# stretch-updates, previously known as 'volatile'
deb http://ftp.ch.debian.org/debian/ stretch-updates main
deb-src http://ftp.ch.debian.org/debian/ stretch-updates main

After this, update the package list with this command:

apt-get update

This command lists all packages that can be updated. It's a long list, probably too long to review...

apt list --upgradable

As recommended by the Debian manual, first we perform a minimal system upgrade, then we perform the main upgrade:

apt-get upgrade
apt-get dist-upgrade

Basically this worked flawlessly, although there were a few things to research and take care of. Here's the rundown:

Upgrade of package grub-pc
- In the configuration phase I received this message: "The GRUB boot loader was previously installed to a disk that is no longer present, or whose unique identifier has changed for some reason."
- I then had to decide which devices I wanted GRUB to be installed on. Possible selections were /dev/sda, /dev/sdb, /dev/md0 and /dev/md2
- /dev/md0 is the RAID 1 device that consists of the boot partitions /dev/sda1 and /dev/sdb1
- /dev/md2 is the RAID 1 device that consists of the root filesystem partitions /dev/sda6 and /dev/sdb6
- I attempted to find out where GRUB was previously installed. I was unable to find this information in any of the configuration files known to me (/etc/grub.d, /boot/grub) - I didn't even find out the device UUID that the package thought GRUB was installed on.
- I ended up inspecting the raw device data, leading me to the conclusion that GRUB was installed on both the physical devices /dev/sda and /dev/sdb. This is the command I used to inspect the raw device: dd bs=512 count=1 if=/dev/sda 2>/dev/null | strings
- After this I was still no closer to a decision. After reading a lot of semi-relevant comments on many different forums and Q&A platforms, I decided to select /dev/sda and /dev/sdb because it simply seemed to be the most natural choice.
- Later on, there was a tight moment when the reboot took longer than expected, but in the end the system came up successfully so I knew I had made the right choice.
Upgrade of package sa-compile
- The package's setup phase failed with this error message: -su: 65: /usr/local/htb/etc/htb-bashrc.sh: Syntax error: "(" unexpected
- I attempted to run sa-compile manually, this worked flawlessly.
- After some more fumbling around, I got it to work by disabling the HTB ("herzbube's toolbox") includes in /etc/profile.d/pelargir_htb.sh and /etc/bash.bashrc
- The root problem was that sa-compile invokes a simple shell (sh) instead of bash, but the code in /usr/local/htb/etc/htb-bashrc.sh (which is invoked via /etc/profile.d/pelargir_htb.sh) contains bash-specific syntax. I fixed this after the upgrade.
Some packages that I use reported important changes in their changelogs.
- The courier package changelog reported that "SSL packages has been merged into it's non-SSL variants". This led me to purge the transitional courier-ssl package after the upgrade.
- The exim4 package changelog reported that "Starting with 4.87~RC1-1 exim will not accept or send out messages with physical lines longer than 998 characters by SMTP DATA. Delivery of such RFC-violating message might fail and subsequently cause routing errors and loss of legitimate mail.". After the upgrade I tested whether I can still send/receive messages with such long lines via Apple Mail. It worked, so assume this is merely a warning against non-standard-conforming mail clients.
- The fail2ban package changelog reported that "Although .local files are still supported, customizations are advised to be provided under corresponding .d/ directories." Research failed to turn up any documentation regarding this, so I assume that deprecation of customization via ".local" suffix is not yet official and I don't have to do anything.
- The systemd package changelog reported that "This version drops support for running /etc/rcS.d SysV init scripts. [...] If you have custom or third-party rcS scripts you need to convert them or change them to run in rc2.d/ - rc5.d/". Since I no longer have any custom RC scripts I don't have to do anything here.
Upgrade of package courier-base
- In the configuration phase I received this message: "The default user and group for Courier MTA has been changed to courier:courier. The package tries hard to make all files belong to correct user:group and the permissions on those files are correct, but if you have a non-default setup, you will have to make sure that 1) All file owners and file in /etc/courier and /var/lib/courier are correctly set. 2) MAILUSER and MAILGROUP settings in /etc/courier/esmtpd is set to correct user and group, both has to be set to `courier'."
- Checking /etc/courier and /var/lib/courier reveals that files and folders have inconsistent owner/group and permissions
- The folder /var/lib/courier was owned by courier:courier and had permissions 750. However, the only file inside the folder, couriersslcache, was owned by root:root and had permissions 600. I deleted the file and restarted the service to let the service auto-create the file - the result was the same.
- The folder /etc/courier was owned by root:courier and had permissions 755. All files inside the folder except two .pem files were at least readable by the courier user, so I didn't have to do anything. The .pem files are not relevant because I don't use the self-signed certificate and the DH group file that the package auto-creates on installation.
- The file /etc/courier/esmtpd does not exist on my system, so I can ignore that part of the attention-grabbing message.
- I noticed that with the upgrade to Debian stretch the init script /etc/init.d/courier-imap-ssl prints an error message when IMAPS is disabled. Previously in Debian jessie there was no error, I assume that the script simply didn't do anything.
dpkg was unable to remove a number of folders because they were not empty. I noted these down and researched each one after the upgrade was complete. Some of them I was able to delete, others had to stay because during the upgrade another package dropped new content into the folder. dpkg reports these reports like this: dpkg: warning: unable to delete old directory '/path/to/folder': Directory not empty.
I had to manually upgrade the configuration for the following packages:
- courier-imap
- roundcube
PostgreSQL
- A new version of PostgreSQL required me to perform the cluster upgrade from 9.4 to 9.6.
I successfully attempted to use pg_upgradecluster to perform the upgrade for the first time - it's much simpler than my old upgrade procedure.
phpMyAdmin
- The phpMyAdmin upgrade failed completely!
- The error message shown was "mysqldump: Couldn't execute 'SHOW FUNCTION STATUS WHERE Db = 'phpmyadmin: Cannot load from mysql.proc. The table is probably corrupted (1728) "
- I solved the problem by uninstalling then re-installing phpMyAdmin, but even that didn't go smoothly. The package purge resulted in the same error message, so in the end I had to manually run MySQL commands such as "drop database" and "drop user".
After the upgrade I purged a number of auto-removed packages
- This command (taken from the Debian upgrade instructions) lists all removed packages that can be purged: dpkg -l | awk '/^rc/ { print $2 }'
- I ended up with this package list: exim4-daemon-light, libcwidget3:amd64, librpmsign1, libsigc++-2.0-0c2a:amd64, mediawiki-extensions-base, mysql-server-5.5, perl-modules
- After doing these purges I was shocked to find out that the database service (MariaDB) was no longer running. I must assume that purging mysql-server-5.5 cause the service to be stopped.
After the upgrade I purged a number of obsolete packages that were still installed
- Purging php5 and dependent packages caused the Apache configuration to break. Although php7 had been installed by the upgrade, the module had not been enabled. I had to manually enable the module with a2enmod php7.
- Without php5 I also had to remove drush and - more importantly - phpldapadmin. Apparently phpldapadmin is not yet compatible with PHP 7, which is why it was removed from Debian testing in May 2016 (cf. bug 821585) and thus did not make it into the stretch release.
The only obsolete package left after cleanup is linux-image-3.16.0-6-amd64. I'm leaving this until the next Linux kernel upgrade

DebianPackageManagement

Configuring APT

Configure package sources

Modifying /etc/apt/sources.list

Backports

SecureApt

Configure custom package sources

APT::AutoRemove::SuggestsImportant

dpkg

Useful commands

stat override

apt-get

DebConf

Introduction

References

Question attributes

Front-ends

Re-configure a package

List answers in the DebConf database

Purge answers from the DebConf database

Pre-configure answers

debconf-utils

Storage

Creating a package from source

Re-packaging a binary package

popularity-contest

Removing unused packages

deborphan

From the Manual

Overview

Usage

Aggressive usage

Unpurged packages

debfoster

Don't Use It!

From the Manual

Overview

Usage

Aggressive usage

aptitude

apt-rdepends

Upgrading to new OS version

Introduction

jessie (8.x) to stretch (9.x)

Navigation menu

Search