GitServer
This page has information on how to set up a Git server. This wiki page covers the client side.
Summary
Git is a distributed version control system, so there is per se no central server that stores "the" repository. However, it is possible to define a workflow where there is a "blessed repository" on a dedicated server machine.
This chapter in the Pro Git book explains various options that can be used to setup a git server:
- git-daemon (provided by the git-daemon-run Debian package) provides unauthenticated access over the dedicated TCP port 9418
- Smart HTTP provides read+write access over HTTP/HTTPS
- SSH also provides read+write access
- Dumb HTTP provides read-only access over HTTP
- last - and least! - read+write access is possible over regular file system operations; this is called the "local protocol"
The following is a short discussion of each option:
- File system access / local protocol: This is from the SourceSafe stone age - I am not even considering this option.
- git-daemon: This is out of the question because it does not provide authentication. In addition, using a specific TCP port has two drawbacks: 1) Another open port makes my server more vulnerable, and 2) it may be impossible to access the repository from a machine behind a proxy.
- Smart HTTP is attractive because it works over the well-known and widely available HTTP and HTTPS ports. I have no experience with this. Nowadays I might go for this one, but when I first researched Git, the only way to use Git over HTTP/HTTPS was via WebDAV, and that was considered not too good in terms of performance and reliability.
- SSH therefore seems to be the only remaining viable option for providing read AND write access. Note, however, that this solution does not provide anonymous read-only access (e.g. for public projects).
There are a number of options how to run a SSH-based Git server:
- gitolite: Hosts all repositories under a single dedicated system user. Access is granted to individual users via SSH public key, but those users do not have shell access to the server. Gitolite provides fine-grained access control to hosted repositories. This is the solution I am currently using.
- gitosis: Similar to gitolite, but no longer maintained. I am listing this because when I first started to run my own Git server I used gitosis.
- GitLab: A full-blown database-based solution that also allows web-based access. GitLab can be compared to GitHub, only that is fully open source. The Pro Git book explicitly mentions GitLab.
References
- gitolite GitHub page (has quick & dirty setup instructions]
- Full gitolite documentation
- Git on the Server, a section of the Pro Git book
git-over-SSH (read+write)
Client side: Create the "admin" identity
Before we start actually doing anything with gitolite, we need to create an identity that we can use in the future to perform administrative work such as adding and removing repositories and users.
Because gitolite uses SSH for user identification, we have to do the following to create the "admin" identy:
- Login to your local machine (NOT the git server!!!) with the user that is going to administrate git repositories and users in the future; you will do these admin tasks locally, NOT on the git server
- Generate a 2048 bit RSA key for use with SSH (unless you already have such a key that you can use for SSH authentication); obviously you should protect the key by entering a nice passphrase.
ssh-keygen -t rsa -f ~/.ssh/admin.id_rsa
As a result, you now have two files: one file that contains the private, passphrase protected key; another file that contains the corresponding public key:
~/.ssh/admin.id_rsa ~/.ssh/admin.id_rsa.pub
Finally, transfer the public key file to the Git server, e.g. to /tmp/admin.id_rsa.pub.
Client side: Working with roles
I prefer to separate my work as an administrator (see above) from the work I do as a normal user. For this reason I create a second RSA key to distinguish the two roles
ssh-keygen -t rsa -f ~/.ssh/patrick.id_rsa
Again, transfer the public key file to the Git server, e.g. to /tmp/patrick.id_rsa.pub
Now your ~/.ssh directory on the client side should look like something like this:
-rw------- 1 patrick staff 3311 21 Sep 16:55 admin.id_rsa -rw-r--r-- 1 patrick staff 740 21 Sep 16:55 admin.id_rsa.pub -rw------- 1 patrick staff 3311 21 Sep 16:55 patrick.id_rsa -rw-r--r-- 1 patrick staff 740 21 Sep 16:55 patrick.id_rsa.pub
Finally, do a bit of SSH configuration by adding the following stuff to ~/.ssh/config
. The discussion of what these configuration options mean can be found on this wiki on the OpenSSH page.
Host gitolite-admin HostName git.herzbube.ch User gitolite3 UseKeychain yes AddKeysToAgent yes IdentityFile ~/.ssh/admin.id_rsa IdentitiesOnly yes Host gitolite-user HostName git.herzbube.ch User gitolite3 UseKeychain yes AddKeysToAgent yes IdentityFile ~/.ssh/patrick.id_rsa IdentitiesOnly yes
That's it for the moment for the client machine.
Server side: Installation + configuration
This is the package to install for getting gitolite onto the system:
gitolite3
Debconf questions + answers:
- Administrator's SSH key = /tmp/admin.id_rsa.pub
Installing the gitolite3
package adds a new system user gitolite3
, which is not able to log into the system because there is no password. The system user's home directory is
/var/lib/gitolite3
which also happens to be the folder where gitolite places its configuration and the repositories it manages.
The individual parts in this folder are:
- Subfolder
.ssh
- Contains stuff that is needed by SSH, when remote users access repositories hosted on the server via SSH. Notably, the SSH public keys are stored in
.ssh/authorized_keys
. The first key that is added here is the administrator's SSH key that was specified to the Debconf question. - Subfolder
repositories
- Contains the Git repositories managed by gitolite. Package installation creates the
gitolite-admin.git
and thetesting.git
repositories. - File
.gitolite.rc
- gitolite configuration file.
- Subfolder
.gitolite
- More gitolite configuration stuff (not yet clear what exactly), and the default log file location
.gitolite/logs
- File
projects.list
- A simple text file that lists Git repositories. This file is used by Gitweb.
Make the following changes to .gitolite.rc
:
UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb.*', LOG_DEST => 'syslog', [...] ENABLE => [ [...] # creates git-daemon-export-ok files; if you don't use git-daemon, comment this out 'daemon', # creates projects.list file; if you don't use gitweb, comment this out 'gitweb', [...] ], [...] $UNSAFE_PATT = qr([`~#\$\&|<>]); # ------------------------------------------------------------------------------ # per perl rules, this should be the last line in such a file: 1;
Discussion:
- The UMASK is set like this so that repositories and their content are created group-readable. Later we are going to add the web server user
www-data
to thegitolite3
group, which will allow the web server to access the repositories managed by gitolite. - GIT_CONFIG_KEYS is set like this so that all git-config variables that are specific to gitweb can be set in the repositories'
config
file - LOG_DEST : Tells gitolite to write logging to syslog. By default gitolite is logging to files that are located in
/var/lib/gitolite3/.gitolite/logs
. Note: I am also adding a "gitolite" entry to/etc/rsyslog.d/pelargir.conf
(see the Syslog wiki page) - The "daemon" and "gitweb" entries below the "ENABLE" dictionary (?) must be uncommented (= active) for gitweb and git-daemon support. This should already be the case in a default Debian setup. Note: Support for git-daemon must be enabled even though I am not actually running git-daemon, because I want gitolite to create the file
git-daemon-export-ok
for me inside all public repositories, because I have gitweb configured to list only repositories with that file inside. - UNSAFE_PATT : This must be added at the very end of the configuration file, just before the "1;" line. Gitolite, by default, does not allow the following characters in the value of a git-config variable:
` ~ # $ & ( ) | ; < >
. As the documentation says "This is due to unspecified paranoia". I want to allow paranthesis in the description of my repositories, which is why I have to redefine UNSAFE_PATT.
Change the following filesystem permissions so that gitweb can access the repositories (as mentioned above, the web server user www-data
has to be added to the gitolite3
group for this to work):
chmod 640 projects.list chmod 750 repositories
Client side: Clone admin repo
To perform the final administrative work steps, switch back to your local machine and issue the following command:
git clone gitolite-admin:gitolite-admin.git
Due to our SSH setup (see further up), the host alias "gitolite-admin" now causes SSH to use the proper "admin" identity to perform the clone. Afterwards the Git clone is connected to the remote origin "gitolite-admin", which means that when we perform the next git-push or git-pull operation SSH will again use the configuration for the "gitolite-admin" alias, i.e. the "admin" identity.
Note: On my Mac, a GUI dialog pops up where you can enter the passphrase you used to protect the private RSA key. Even nicer is that the dialog offers to store the passphrase in your Mac OS X login keychain so that in the future you don't have to enter the passphrase at all. If this is too much convenience, you can always decline.
You now have a clone of the administration repository gitolite-admin.git, in which you can change things to setup the actual production repositories and those users that will have write access.
Here is the initial content of the gitolite-admin/conf/gitolite.conf
file::
repo gitolite-admin RW+ = admin repo testing RW+ = @all
Add a user
Edit gitolite-admin/conf/gitolite.conf
. Add the user to an existing group, or make a new group, or just add the user to an existing repo
section. For instance:
# On my server I use this group because I always have write access to my own repos # (and I don't host foreign repos) @gods = patrick
Goto gitolite-admin/keydir
and add the user's public key to the file that is named "username.pub"; for instance
cp ~/.ssh/patrick.id_rsa.pub patrick.pub
Commit changes locally
git add . git commit -m "added user patrick & granted access to all repos"
Push changes to the server; the user will now be added on the remote side (e.g. the public key will be added to /var/lib/gitolite3/.ssh/authorized_keys
)
git push
Remove a user
TODO
Rename a user
WARNING: This is untested. That being said, there's no reason why this procedure should not work.
- Change the user's name where it occurs in
gitolite-admin/conf/gitolite.conf
- Rename the user's public key (inside
gitolite-admin/keydir
) withgit mv
- Commit & push changes
Change a user's public key
TODO
Add a repository
Edit gitolite-admin/conf/gitolite.conf
. Add a new repo
section, or add the name of the new repository to an existing repo
section. Simply having a repository name appear somewhere in the configuration will cause the repository to be created when you push changes to the server. For instance:
repo foo RW+ = @gods R = gitweb daemon owner = yourname desc = bla bla
Notes:
- "RW+" is gitolite's way of expressing "full access". It is possible to express much more fine-grained access rules, but the only ones that I care about at the moment besides "RW+" are
- "R" = Read-only access
- "-" = No access
- Making the repository readable by the special user "gitweb" adds the repository to the project list located in the file
/var/lib/gitolite3/projects.list
on the server side - Making the repository readable by the special user "daemon" creates the file
git-daemon-export-ok
in the repository directory on the server side. Only if this file is present will the git-daemon grant access to the repository. I don't actually use git-daemon, but I have configured gitweb to also check for the presence of this file. - "owner = yourname" adds the owner's name to the repository's
config
file. Thegit-config
setting is "gitweb.owner". Gitweb will display this name in its repository listing. - "desc = bla bla" adds a description to the repository's
config
file. Thegit-config
setting is "gitweb.description". Gitweb will display this description in its repository listing.
Now commit changes locally, then push the changes to the server.
Next, create a new repository locally, and then push it to the server:
mkdir foo cd foo git init git remote add origin gitolite-user:foo.git # do some work, git add and commit files git push --all git push --tags
Alternatively, to "import" an existing repository (even a bare one) into gitolite, just execute the last two commands:
cd foo git remote add origin gitolite-user:foo.git git push --all git push --tags
Note: We used the "gitolite-user" host alias that causes SSH to use the options it has stored for that alias in its config file ~/.ssh/config. See further up for details on how to work with different roles (i.e. admin and user role).
The repository is now ready to be cloned on the local side:
git clone gitolite-user:foo.git
Remove a repository
Edit gitolite-admin/conf/gitolite.conf
; remove the section for the repository, and also all references to the repository from all group definitions. After you commit and push this change, the repository will no longer be accessible via gitolite.
Gitolite did not remove the physical repository, though. This needs to be done separately by logging into the server and rm -rf
'ing the physical repository directory.
You may also need to remove other references (e.g. in projects.list
).
Smart HTTP / gitweb
Summary
In this section I show how to set up a single Apache vhost for two things:
- Smart HTTP access to repositories for git clients
- Browser-based interactive access via gitweb
Apache/gitolite integration
Further up in the gitolite configuration section we have made sure that file system permissions are set up so that the gitolite3
group has read-only access to all repositories managed by gitolite.
To take advantage of this, we can now add the Apache server's system user www-data
to the gitolite3
group:
adduser www-data gitolite3
gitweb
Debian package to install:
gitweb
gitweb is a CGI script that is located in
/usr/lib/cgi-bin/gitweb.cgi
gitweb configuration (esp. /etc/gitweb.conf, but also per-repository configuration) is documented in
man gitweb man gitweb.conf
Gitweb stores its configuration in the following file. Note: The file is a fragment of Perl code, so you can put fancy stuff in it if you know enough Perl.
/etc/gitweb.conf
The following things have to be changed in that file:
$projectroot = "/var/lib/gitolite3/repositories"; $projects_list = "/var/lib/gitolite3/projects.list"; # html text to include at home page $home_text = "/var/www/git.herzbube.ch/gitweb-indextext.html"; # Only export repos that we want to be publicly visible. # gitolite is creating the "git-daemon-export-ok" file inside a # repo if in gitolite.conf the repository is made readable by the # special user "daemon". $export_ok = "git-daemon-export-ok"; # Make wider for long project descriptions $projects_list_description_width = 70; # Site name $site_name = "Git trees on git.herzbube.ch"; # Allow for pretty URLs, e.g. http://git.herzbube.ch/gitweb.cgi/acexpander.git # instead of http://git.herzbube.ch/gitweb.cgi?p=acexpander.git;a=summary # Requires mod_rewrite configuration in the web server configuration (for # details see "man gitweb"). $feature{'pathinfo'}{'default'} = [1]; # List of URLs that can be used to clone a repository. Will be displayed in the # project summary. Repositories can override this if they have a "cloneurl" file # in their top-level directory (that file's content is displayed), or if their # config file contains the gitweb.url setting. @git_base_url_list = ("https://git.herzbube.ch/git");
Write the header file /var/www/git.herzbube.ch/gitweb-indextext.html
(the following text is based on what can be found at git.kernel.org):
<p> To clone one of these trees, install <a href="http://git-scm.com/">git</a>, and run: <blockquote> <code>git clone http://git.herzbube.ch/git/</code> + project path. </blockquote> The clone URL for a specific project is available on the summary page of that project. </p> <p> For more information about <a href="http://git-scm.com/">git</a>, see an <a href="http://git-scm.com/documentation">overview</a> of available documentation, the <a href="http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html">tutorial</a> or the <a href="http://www.kernel.org/pub/software/scm/git/docs">man pages</a>. </p>
Smart HTTP
The Debian package
git
includes a CGI script that provides so-called "Smart HTTP" access to git repositories. The script is located here
/usr/lib/git-core/git-http-backend
A bit of documentation for git-http-backend
is available in the Pro Git book, and there's also the man page
man git-http-backend
git-http-backend
is capable of providing write access to repositories, but I am not using this at the moment.
git-http-backend
is so smart that the only configuration it requires can be done within the Apache configuration. See the next section for the details.
Apache vhost configuration
Both Smart HTTP and gitweb are made available under a single Apache vhost: https://git.herzbube.ch/.
Here is the configuration:
# -------------------------------------------------------------------------------- # git.herzbube.ch # -------------------------------------------------------------------------------- <VirtualHost *:80> ServerName git.herzbube.ch Redirect permanent "/" "https://git.herzbube.ch/" </VirtualHost> # -------------------------------------------------------------------------------- # SSL Host # -------------------------------------------------------------------------------- <VirtualHost *:443> ServerName git.herzbube.ch ServerAdmin webmaster@herzbube.ch ErrorLog ${APACHE_LOG_DIR}/git.herzbube.ch/error.log CustomLog ${APACHE_LOG_DIR}/git.herzbube.ch/access.log combined DocumentRoot /usr/share/gitweb Alias /robots.txt /var/www/git.herzbube.ch/robots.txt <Directory /usr/share/gitweb> # The following rules are required for pretty URLs. They are # taken directly from "man gitweb". RewriteEngine On RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule ^.* /gitweb.cgi/$0 [L,PT] </Directory> # Set up environment variables to configure git-http-backend. # Only repositories with the git-daemon-export-ok marker file in them are # served. Defining the environment variable GIT_HTTP_EXPORT_ALL would serve # all repositories, regardless of the presence of the marker file. SetEnv GIT_PROJECT_ROOT /var/lib/gitolite3/repositories ScriptAlias /git/ /usr/lib/git-core/git-http-backend/ <LocationMatch "^/git/"> # We can safely grant all access here because git-http-backend by # default enables write access only for authenticated clients. # Because we don't configure an authentication mechanism here, # the client can never become authenticated. Require all granted </LocationMatch> Include conf-available/pelargir-herzbube.ch-vhosts-ssl.conf </VirtualHost>
git-over-HTTP client access
The Apache vhost configuration above allows clients to clone a repository "foo" with the following command:
git clone https://git.herzbube.ch/git/foo.git