Drupal

From HerzbubeWiki
Jump to: navigation, search

This is the main wiki page for setting up, configuring and maintaining the CMS named "Drupal".

This page was last updated when I migrated from Drupal 7 (D7) to Drupal 8 (D8), so it now contains information that is current for D8. I have started to use Drupal when it was still version 4.x, so if you consult the page history you may find setup/config stuff that pertains to older Drupal versions.


Version

The Drupal version I use is a moving target because releases are made so frequently. I have started using Drupal "in earnest" at 4.7.0-rc2 and, at the time of writing/creating this document, am upgrading to 5.1 6.14 7.14 7.50 8.5.6.


System installation

Why not the Debian package

For a short period of time I have used the Drupal package provided by Debian. Unfortunately, Debian packaging is always two or three versions behind. For instance, at the time of writing Drupal 5.1 has just been released, but the Debian package in testing is still at 4.5.3. So, if I had gone with Debian I would have completely missed versions 4.6 and 4.7.

It is a known fact that Debian is always "a little bit backward". Ordinarily, with packages for software that is rather stable and does not release as often as Drupal, this is no problem. I even have sympathy for the Debian package maintainer who attempts to keep up with, but gets overrun by, the active Drupal community.

Whatever the reasons, the current situation with the Debian package is not satisfying, therefore I have opted for a manual installation of Drupal.


References

The installation "manual" is found inside the Drupal tar ball, in the file

core/INSTALL.txt

The general procedure to follow when updating from one version of Drupal to the next can be found in

core/UPDATE.txt


Manual/shell installation

Note: The following information refers to a new installation. My upgrade procedure is described on a separate page DrupalMaintenance.


Tar ball

Unpack the tar ball downloaded from http://drupal.org/ into /var/www and create a symbolic link /var/www/drupal to the versioned directory. With time, after you have upgraded several times, /var/www will look something like this:

root@pelargir:/var/www# ls -l
total 104
lrwxrwxrwx  1 root     root       11 Jun 15 00:16 drupal -> drupal-7.14
drwxr-sr-x  9 root     root     4096 Oct 12  2009 drupal-5.1
drwxr-xr-x  9 root     root     4096 Jun 14 22:41 drupal-6.14
drwxr-xr-x  9 root     root     4096 Jun 15 00:16 drupal-6.26
drwxr-xr-x  9 root     root     4096 Oct 12  2009 drupal-6.8
drwxr-xr-x  9 root     root     4096 Jun 15 03:23 drupal-7.14

The Apache configuration can now point to the symlink. Whenever you upgrade to a new version of Drupal, you only have to switch the symlink to point to the new version of the code base, but the Apache configuration can remain the same.


Document root folders

It is perfectly possible to point the document root of an Apache virtual host (vhost) directly to /var/www/drupal. In fact, you can even point several document roots to /var/www/drupal and Drupal will figure out automatically which site configuration it should use. The different site configurations are placed in sub-folders of the site directory inside the Drupal main folder. For instance:

root@pelargir:/var/www/drupal/sites# ls -l
total 28
drwxr-xr-x 4 root root 4096 May  3 00:10 all
drwxr-xr-x 2 root root 4096 Jun 15 01:31 default
drwxr-xr-x 3 root root 4096 Jun 15 03:08 grunzwanzling.ch
drwxr-xr-x 3 root root 4096 Jun 15 03:08 herzbube.ch
drwxr-xr-x 3 root root 4096 Jun 15 03:08 kino.herzbube.ch

For details about Drupal's multi-site detection mechanism, refer to core/INSTALL.txt file inside the Drupal tar ball.


Instead of pointing document roots to /var/www/drupal, I prefer to create a dedicated document root folder which I populate with symlinks that point back to the top-level files and folders in /var/www/drupal. This is how it looks:

root@pelargir:/var/www/www.herzbube.ch# ls -l
total 732
lrwxrwxrwx  1 root   root        29 Jun 15 01:19 authorize.php -> /var/www/drupal/authorize.php
lrwxrwxrwx  1 root   root        24 Nov 14  2009 cron.php -> /var/www/drupal/cron.php
lrwxrwxrwx  1 root   root        25 Nov 14  2009 .htaccess -> /var/www/drupal/.htaccess
lrwxrwxrwx  1 root   root        24 Nov 14  2009 includes -> /var/www/drupal/includes
lrwxrwxrwx  1 root   root        25 Nov 14  2009 index.php -> /var/www/drupal/index.php
lrwxrwxrwx  1 root   root        27 Nov 14  2009 install.php -> /var/www/drupal/install.php
lrwxrwxrwx  1 root   root        20 Nov 14  2009 misc -> /var/www/drupal/misc
lrwxrwxrwx  1 root   root        23 Nov 14  2009 modules -> /var/www/drupal/modules
-rw-r--r--  1 root   root       448 Nov  4  2009 PGPHerzbubeAtHerzbubeDotCH.txt
-rw-r--r--  1 root   root     13462 Dec 14  2009 pgp-key-signing-policy.html
lrwxrwxrwx  1 root   root        24 Nov 14  2009 profiles -> /var/www/drupal/profiles
-rw-r--r--  1 root   root      1913 Jun 15 01:25 robots.txt
lrwxrwxrwx  1 root   root        23 Nov 14  2009 scripts -> /var/www/drupal/scripts
lrwxrwxrwx  1 root   root        21 Nov 14  2009 sites -> /var/www/drupal/sites
lrwxrwxrwx  1 root   root        22 Nov 14  2009 themes -> /var/www/drupal/themes
lrwxrwxrwx  1 root   root        26 Nov 14  2009 update.php -> /var/www/drupal/update.php
lrwxrwxrwx  1 root   root        26 Jun 15 01:21 web.config -> /var/www/drupal/web.config
lrwxrwxrwx  1 root   root        26 Nov 14  2009 xmlrpc.php -> /var/www/drupal/xmlrpc.php

This type of setup gives me a lot of flexibility because I can add files or folders to the top level of the document root that are not managed by Drupal. In the example above, I have added my PGP public key and my PGP key signing policy, and also a version of robots.txt that is tailored to www.herzbube.ch (Drupal provides a robots.txt, to use that I would simply create another symlink that points back to /var/www/drupal/robots.txt).


Because the symlinks refer to /var/www/drupal I can still upgrade to a new version of Drupal by exchanging the code base that "hides" behind /var/www/drupal.


Apache configuration

In /etc/apache2/conf-available/pelargir-drupalsite.conf:

# ============================================================
# The following directives need to be included in the context
# of <Directory ...>. The directory in question must be a
# DocumentRoot.
# ============================================================

# Generally allow access
Require all granted
# Symlinks must be followed because either the DocumentRoot
# is a symlink, or inside it are many symlinks that point to
# the Drupal resources.
Options FollowSymLinks Includes
# Enables Drupal's .htaccess which 1) protects various files
# and resources, and 2) uses rewrites to beautify URLs.
AllowOverride All
# Drupal is written in PHP, therefore PHP must obviously be
# turned on. The memory limit is set to a value that should
# satisfy even memory hungry modules such as Views.
php_admin_flag engine on
# Drupal 8 requires at least 64 MB
php_value memory_limit 64M


Don't use a2enconf to enable the above config file - the file is intended to be included. Usage example:

<Directory /var/www/www.herzbube.ch/>
  Include conf-available/pelargir-drupalsite.conf
</Directory>


mod_rewrite / Clean URLs

When you run the Drupal installation routine (install.php) it checks if the Apache module mod_rewrite is enabled. If yes then the installer will automatically enable a Drupal feature called "Clean URLs". If the feature is enabled, Drupal will "beautify" URLs like in this example:

It is highly recommended to enable mod_rewrite, and thus "Clean URLs".


Important: If you move the site to a different server (e.g. from test to production, or restore from backup) it is very important that the new server also has mod_rewrite enabled! If you forget to enable the module but "Clean URLs" is turned on in the Drupal configuration, you won't be able to login or do anything else that requires submitting a form!


"Clean URLs" can be manually enabled/disabled under the non-intuitive location Administer > Configuration > Search and metadata > Clean URLs. In Drupal 8 it is no longer possible to disable "Clean URLs". See Drupal docs.


Database configuration

Useful information is in the DBMS-specific text file INSTALL.pgsql.txt or INSTALL.mysql.txt.


First create one user (MySQL) or role (PostgreSQL) drupal. Optionally: create one user or role per site/database. For MySQL the usual procedure applies. For PostgreSQL, the Drupal folks recommend this command to create the role:

su postgres -c "createuser --pwprompt --encrypted --no-createrole --no-createdb drupal"


Next, create a database for every site. If you omit this step, the Drupal setup script install.php will attempt to create the database itself. This might work for MySQL, but it definitely fails for PostgreSQL because the Drupal database user has insufficient privileges to create databases.

sudo -u postgres createdb --encoding=UTF8 --owner=drupal drupal856_herzbube

Notes:

  • Add a version number to the database name so that the database can be cloned for site upgrades. The cloned database will have the version number of the Drupal code base that we are upgrading to.
  • For PostgreSQL, the command shows how to make the previously created role the owner of the databases. For MySQL you use the standard procedure to grant the user all privileges on the databases.


It used to be necessary to load the database schema manually. Nowadays the Drupal setup script install.php will do this automatically for you. Before you proceed to the setup, though, you need to prepare some stuff in the filesystem for the sites you are going to use. The procedure for this is described in the next section.


Drupal configuration

sites.php

New in Drupal 8 is the sites.php file. Create it by making a copy from the provided template:

cp sites/example.sites.php sites/sites.php

I'm not entirely sure whether the file is required.


Site configuration

In /var/www/drupal/sites, create one sub-directory for each of your sites. The rules how Drupal matches an URL against site sub-directories are listed in INSTALL.txt. On my system, the current setup looks like this:

root@pelargir:/var/www/drupal/sites# ls -l
total 28
drwxr-xr-x 4 root root 4096 May  3 00:10 all
drwxr-xr-x 2 root root 4096 Jun 15 01:31 default
drwxr-xr-x 3 root root 4096 Jun 15 03:08 grunzwanzling.ch
drwxr-xr-x 3 root root 4096 Jun 15 03:08 herzbube.ch
drwxr-xr-x 3 root root 4096 Jun 15 03:08 kino.herzbube.ch


Every site sub-directory needs its own file settings.php. Copy the template file from the "default" site:

cp sites/default/default.settings.php sites/<xxx>/settings.php


Add an entry like the following at the appropriate location. This entry is a security mechanism that defines which domain names a user can use to access the site. It protects against so-called "HTTP HOST header attacks". The problem is explained in detail in this drupal.org article.

$settings['trusted_host_patterns'] = [
  '^www\.herzbube\.ch$',
  '^old-site\.herzbube\.ch$',
  '^new-site\.herzbube\.ch$',
];


The settings.php file also stores the details for accessing the database. You could add these details manually, but it's easier to let the Drupal setup script write them into the file (together with some other stuff). Later, when the settings.php file is complete, you must protect it because it contains a password. These are the owner/privileges you want to set:

chown www-data:www-data settings.php
chmod 400 settings.php


Every site has its own file area. Create a site subfolder named files and make sure that the web server has access to it. Also keep in mind that the files folder needs to be specified when configuring the Drupal site in the web browser (see DrupalConfiguration). The entry can be found in the "Site settings" and needs to be changed to sites/<xxx>/files.

mkdir /var/www/drupal/sites/<xxx>/files
chown www-data:www-data /var/www/drupal/sites/<xxx>/files


Finally, every site can have its own modules and themes. However, I prefer to share modules and themes between sites by placing them into the folders /var/www/drupal/modules and /var/www/drupal/themes. For completeness' sake, these are the site-specific folders:

mkdir /var/www/drupal/sites/<xxx>/modules
mkdir /var/www/drupal/sites/<xxx>/themes


Web browser setup

Point the browser to your site URL, e.g. https://www.herzbube.ch/. Drupal automatically redirects to and launches the installer script install.php


Answers for the wizard

  • Language = English
  • Installation profile = Standard (when upgrading from a D7 or D6 site, use the "Minimal" profile)
  • Database type = PostgreSQL
  • Database name = drupal861_xxx (I'm using a version number in the database name for later upgrade scenarios, as outlined on the DrupalMaintenance page)
  • Database username = drupal
  • Database password = secret


The install script now performs the following operations:

  • Connects to the database server and populates the specified database with the DB schema
  • Configures the site's "files" folder (e.g. it places a .htaccess file inside
  • Writes the database configuration and some other things into the site's settings.php


It is important to secure settings.php because it now contains a password:

chown www-data:www-data settings.php
chmod 400 settings.php


After this step the setup wizard now offers a final step where you can perform some basic site configuration. The data you enter here all goes into the database. Details are on the separate wiki page DrupalConfiguration.


cron configuration

A "healthy" Drupal site should run periodic maintenance tasks. There are several ways how this can be done

  • A user can do this manually by visiting the page "Administration > Configuration > System > Cron" and clicking the link for running cron manually
  • Drupal Core includes a module "Automated Cron" (sometimes also called "poor man's cron"), which when enabled triggers the cron task whenever an end user visits the site. The frequency with which this happens can be set in the module's configuration.
  • An external, true cron job can be configured. This is my preferred solution. In this case it makes sense to uninstall the "Automated Cron" module (which might have been automatically installed during an installation that used the "Standard Profile").


My external cron job script is this:

/etc/cron.daily/pelargir-drupal

It has the following content:

#!/bin/sh
#
# Run Drupal's maintenance tasks. This should be run regularly (daily) by cron.
#
# The cron key required as an argument for the /cron page (or in fact the
# entire URL to visit) can be obtained from within Drupal in the admin section
# "Administration > Configuration > System > Cron".
#
# Note: Normally the /cron page does not output anything, therefore output is not
# redirected (e.g. to /dev/null). If something goes wrong, though, it can be
# expected that some output will be generated. If this script is run by cron,
# the output will be forwarded to the mail account of the user under which
# cron executes this script (usually root).

URLS="https://www.herzbube.ch/cron/secret"
for URL in $URLS; do
  wget -O - -q -t 1 "$URL"
done


LDAP

Overview

Since LDAP integration is not part of Drupal core, it must be achieved via a module. Over time there have been several LDAP integration projects, but the one that seems to be most promising and future-oriented is the aptly named LDAP project. The project has wisely split its functionality into several modules, notably:

  • LDAP Servers: Allows the definition of LDAP servers/directories, and where to find user account entries in each directory
  • LDAP Authentication: Allows to specify how the authentication process (i.e. login) should work
  • LDAP Authorization: Allows to specify how authorizations (i.e. roles) are assigned to users

I don't need the authorization functionality, as I intend to manage Drupal user roles entirely in Drupal.

As for authentication, I do NOT want to allow all users defined in my LDAP directory to be able to log in to my Drupal site, only those that have been marked in a certain way to be a Drupal user.


Schema customizations to support Drupal

The OpenLDAP page has the details about how to customize my personal LDAP schema. As an overview:

  • There is a new attribute type "drupalUid" that I use to store the Drupal user account ID
  • There is a second new attribute type "drupalMail" that I use to store the email address for the Drupal user
  • There is a new auxiliary object class "drupalAccount" that I add to those user account entries in my LDAP directory that should be marked as Drupal users
  • The object class specifies that "drupalUid" and "drupalMail" are mandatory attributes


Mark user accounts in LDAP directory as Drupal users

Simply add the new auxiliary object class "drupalAccount" to those user accounts that should be allowed to login to Drupal. Set the attribute "drupalUid" to the intended Drupal user account ID, and the attribute "drupalMail" to a value that looks like a legitimate email address.


Install Drupal modules

  • Download the proper tar ball version from the LDAP project page
    • At the time of my first installation, the project was still in its beta phase. As recommended on the project page, I used the 7.x-1.0-dev tar ball.
    • Later when I upgraded to Drupal 8, the module was, again, in beta stage. As recommended by the project maintainers, I used the 8.x-3.0-beta4 tar ball.
  • Also download the tar ball for the module "External Authentication" (project page). The "LDAP Authentication" and "LDAP Users" modules depend on this module (at least they did in LDAP release 8.x-3.0-beta4). It's purely a helper module which has no configuration of its own.
  • Extract the tar balls and place everything in /var/www/drupal/modules
  • Log in to Drupal and activate the following modules
    • Step 1: External Authentication
    • Step 2: LDAP Authentication. This automatically activates the dependency modules "LDAP Servers" and "LDAP Users"

IMPORTANT: For Drupal 7 I had to increase the PHP memory limit in the Apache web server configuration before I was able to activate "LDAP Authentication": 40 MB was not enough, I ended up giving Drupal 60 MB. The indicator that something went wrong was the "white screen of death" (WSOD) that I got after I had activated the module. I found out about the memory problem only later, when I looked at the Apache server error log. The evil thing (which cost me a lot of time) was that I was able to reload the page in my browser to make the WSOD go away, but afterwards almost my entire admin menu in Drupal became non-functional: "You do not have any administrative items." was the only message that I got out of Drupal. Apparently the activation process of the "LDAP Authentication" module was interrupted in mid-flight when Drupal hit the memory limit, leaving the site in its precarious state. Luckily it was always possible to disable the module to get back to a normal state...


Configure LDAP modules

Module "LDAP Servers" (Administration > Configuration > People > LDAP Servers > Servers):

  • Add a new server configuration with the following settings. Note: Here I list only those settings that need to be changed from the defaults, or which I find too important to omit.
    • Name = localhost
    • Machine-readable name = localhost
    • LDAP Server Type = OpenLDAP
    • Server address = ldapi://
    • The "ldapi" protocol causes the LDAP connection to run over a UNIX domain socket ("i" possibly stands for "IPC communication")
    • Use StartTLS = false (StartTLS is actually displayed as "Start-TLS")
    • Binding Method for searches = Service Account Bind
    • DN for non-anonymous search = cn=readonly-users,ou=users,dc=herzbube,dc=ch
    • Password for non-anonymous search = secret
    • Base DNs for LDAP users, groups, and other entries = ou=users,dc=herzbube,dc=ch
    • AuthName attribute = drupalUid
    • AccountName attribute = <leave empty> (this defaults to the value specified for AuthName)
    • Email attribute = drupalMail
      • Having this attribute was important in D7 due to the reason stated below. I didn't verify whether D8 and the D8 version of the module still exhibit the described behaviour, but on the grounds of "better safe than sorry" I assume that having this attribute is still important in D8.
      • If this attribute is missing the Drupal account's email field will remain empty, which in turn prevents Drupal users from modifying their user node later on because 1) Drupal enforces that the email field be non-empty, but at the same time 2) the LDAP module prevents the user from modifying the email field because it thinks that the value for the email field comes from the LDAP directory.
    • Testing Drupal Username = <an existing user name>
    • Groups are not relevant to this Drupal site = True
  • After saving the new configuration, click its "test" link to check if the configuration works correctly. Make sure to also test for an account that should NOT be able to log in!
    • A successful configuration test displays a page with a wealth of information about the LDAP entry retrieved and the corresponding Drupal account. This information can be used for the subsequent definition of mappings between LDAP entry attributes and Drupal user node fields, which are required for provisioning from LDAP to Drupal. See below for more details.


Module "LDAP Users" (Administration > Configuration > People > LDAP Servers > User mappings):

  • Under "LDAP Servers Providing Provisioning Data", enable the server configuration previously created. This makes a series of additional settings available.
  • Create or Sync to Drupal user on successful authentication with LDAP credentials = True
  • Under " Orphaned account cron job", select the option "Do not check for orphaned Drupal accounts". I don't need this because I don't allow visitors to create new accounts.
  • Under "Provisioning from LDAP to Drupal Mappings", specify the following mappings between LDAP entry attributes and Drupal user node fields. These mappings are used to create a new Drupal account when a user successfully authenticates against the LDAP directory but no Drupal account exists for that user. The mappings are also used to update an existing Drupal account with data retrieved from the LDAP entry on successful authentication. This creation/sync mechanism is called "Provisioning". Note that the information to define these mappings can be obtained from the page that is displayed when an LDAP server connection is tested (see above).
    • Mapping 1
      • Source LDAP token = [drupaluid]
      • Target Drupal attribute = "Property: Username"
      • This mapping should already be present, probably due to the "AuthName attribute" specification in the server configuration.
    • Mapping 4
      • Source LDAP token = [drupalmail]
      • Target Drupal attribute = "Property: Email"
      • This mapping should already be present, probably due to the "Email attribute" specification in the server configuration.
    • Mapping 2
      • Source LDAP token = TODO (Get from GECOS field. Is this really necessary? After all we alreay specified the "AccountName attribute" in the server configuration.)
      • Target Drupal attribute = "Field: Name"
    • Mapping 3
      • Source LDAP token = TODO (Is this really necessary? I don't really want the password to be duplicated in the Drupal database.)
      • Target Drupal attribute = "Field: Password"
    • For all mappings
      • Convert from binary = false
      • On Drupal user creation = true
      • On Sync to Drupal user = false
  • Under "LDAP Servers to Provision LDAP Entries on", make sure that "None" is selected. I don't want LDAP entries to be created due to activities on Drupal sites, the communication is one way only.


Module "LDAP Authentication" (Administration > Configuration > People > LDAP Servers > Authentication):

  • Allowable Authentications = Only LDAP Authentication is allowed except for user 1
  • Under "Authentication LDAP Server Configurations", enable the server configuration previously created


Web browser based configuration

See the separate page DrupalConfiguration.


Modules

See the separate page DrupalModules.


Themes

See the separate page DrupalThemes.


Sites

See the separate page DrupalSites.


Upgrades & Troubleshooting

See the separate page DrupalMaintenance.


Site installation and maintenance with Drush

This section contains some quick & dirty notes on how to install and maintain a site with Drush. The section is incomplete, I need to investigate this further.


To work with Drush you need to install Drush. Beginning with Drush 9, the newest Drush version and the one I want to work with, Drush is only available as part of a Drupal site that was set up with PHP Composer. References:


This new workflow is markedly different from what I am used to: It is developer-centric and uses dev tools such as Git, instead of sysadmin-centric. The only thing that I am immediately concernced about is: How do I split the site configuration so that the database access credentials are not committed to a Git repository?


Here's the command to initialize a folder with the Drupal codebase:

composer create-project drupal-composer/drupal-project:8.x-dev pelargir.herzbube.ch --stability dev --no-interaction

Notes:

  • The "8.x-dev" refers to the version of the Composer package "drupal-composer/drupal-project". There package versioning does not reflect the versions of Drupal releases. At the time of writing there were only two package versions available: 7.x-dev and 8.x-dev for a Drupal 7 or Drupal 8 installation, respectively.
  • The "--stability" parameter provides the value for the "minimum-stability" field in composer.json. At the time of writing, only "dev" can be used - if "stable" is used the initialization process fails because no stable version of package "drupal-composer/drupal-project" exists. Contrary to the documentation on getcomposer.org, if it is omitted the parameter defaults to "dev" - this is my conclusion because if I omit the parameter the initialization process succeeds.
  • The Drupal codebase that is installed depends on the dependencies that the Composer package "drupal-composer/drupal-project" has at the time of initialization. At the time of writing the package had the dependency "drupal/core": "^8.6.0", which is a dependency on Composer package "drupal/core" with a version in the range

>=8.6.0 and <9.0.0. At the time of writing this causes the Drupal codebase of release 8.6.3 to be installed because at the time of writing Drupal 8.6.3 is the latest release.

  • For me the most important takeway is that the Drupal codebase installed is from an officially released version, it's not the dev version of Drupal!


If the Drupal codebase that is installed is too new, an older version can be installed like this:

  • Change the entry for "drupal/core" so that it refers to the desired version. For instance, to install the specific version 8.6.1 you can write "drupal/core": "8.6.1".
  • Delete the vendor folder and the composer.lock file
  • Run composer install


After the initialization process is complete, we follow the recommendation to create a Git repo. Note that a .gitignore file with sensible content is already present. For instance, among other things it excludes the vendor subfolder and also any settings.php files. The latter is good news because it prevents the database access credentials and other sensitive information from accidentally entering version history.

git init
git add .
git commit -m "initial commit"


TODO: Find out how to leverage this approach for updating my site. In theory

  • The web subfolder is what I want to place into /var/www/drupal
  • So I would have to create a Git repo with the web subfolder as its content, then clone that repo in /var/www/drupal
  • After committing an update to the Git repo I could then fetch the updates into /var/www/drupal
  • /var/www/drupal would mix version controlled files with local files
  • This approach completely ignores Drush, though.