Self Hosting Project

This page explains the basics of my project of hosting my own Internet and Intranet services (aka my "Self Hosting" project).

Motivation & History

I started the project in July 2003 by buying some cheap PC hardware and a fixed IP address. My main motivation for doing this at that time was to learn new things, and to exercise my Unix skills that I had acquired in my first job. Nowadays the project is self-perpetuating because I have become accustomed to the freedom and flexibility that I gain from having my own server - I simply cannot imagine anymore how it is to live in slavery dependence of cloud and other service providers.

In 2011 I became fed up with the creaky old PC tower that was chugging along in a permanent semi-disemboweled state somewhere in a dusty corner. I switched to a small, neat and silent Mac Mini, which was a joy to install and maintain.

In 2016 one of the two hard disks in the Mac Mini gave out in a catastrophic hardware failure, and I realized what a fool I had been to neglect my backups. Although I eventually got my data back (for a price), I resolved to no longer keep the server hardware at home. This decision was not an easy one, because it meant I had to relinquish physical control over the hardware, but on the other hand I had become increasingly tired of dealing with hardware problems over the years and simply didn't want to have to deal with such things anymore. Other compelling advantages such as security against fire or burglars, or a faster backbone connection, finally convinced me to rent a dedicated server at a trustworthy Swiss provider.

Rationale

Even though I have been using Mac computers for many years now, it is important to me that the services provided by my server can be used from any platform, specifically

• Mac OS X
• Linux
• Windows

The central strategy to make this requirement a reality is to use open protocols and browser-based software. The benefit of this approach is not only platform independency, but also access to the services from anywhere in the world, be it from my workplace, or while travelling abroad.

Types of Services

I am classifying the services provided by my server into the following types:

• End user services: These are services that are recognizable as such by the end user, and that provide a real-world benefit to the end user. One example for such a service is Email. The purpose of all the other service types is to facilitate having end user services in the first place.
• Infrastructure services: These services work in the background and form the technical basis for the other service types. Two examples for such services are TCP/IP and DNS.
• Supporting services: These services are used by the system administrator (me) to keep the system running. One example for such a service is system monitoring.

Infrastructure Services

Infrastructure services can be broken down into the following main categories:

• Services that provide the basic networking capabilities
  • TCP/IP (both wire and WiFi)
  • DHCP
  • DNS
• All the other infrastructure services are layered on top of the network, i.e. they require a functional network
• One service needs to be highlighted: LDAP
  • To simplify the administration of various things such as users and computer hosts, there is a central LDAP directory
  • The keyword here is central: Without the LDAP directory pretty much all the end user services stop to work
  • The main reason is that the LDAP directory contains the user database and that most of the end user services are hooked up to LDAP to perform authentication and authorization
  • Two infrastructure services that work very close to LDAP are PAM and NSS

End User Services

Here is an incomplete list of end user services, sorted by importance:

• The most important end user service of the entire project is Email. It is extremely important because it provides the main means of communication with rest of the world. Because it is used not only for casual communication with friends, but also for real-world communication with companies, employers, government, etc., the Email service needs to be absolutely reliable.
  • A distinction must be made between the mail server (which sends and receives email) and the mail client (which allows the end user to access the email service)
  • Infrastructure services that are required for the email server: Central user database, spam recognition, greylisting
  • Infrastructure services that are required for a desktop email clients: IMAP server
  • Infrastructure services that are required for a web mail client: IMAP server, HTTP server, database server
  • I do not use POP because it causes mail to be stored on the client
• The second-most important service is the file server
  • Because of Windows the only practical choice here is to use Samba
• The third-most important service is the address and calendaring service
  • Infrastructure services required are: CalDAV server and CardDAV server (which in turn require HTTP server, database server)
  • For a long time I tried to store address data in LDAP. This approach failed, though, due to the lack of a standardized data structure, so that I was never able to access the address data except from a web LDAP client.
• The fourth-most important service is my personal website
• Other services include: Mailing list, photo gallery, WebDAV, version control

Supporting Services

The most important supporting service is this Wiki: I use it to document the entire project, and more.

Other important supporting services are:

• Remote access via SSH
• System monitoring