PythonAdministration
Administrating a Python installation boils down to the following questions
- How do I install, upgrade or uninstall Python on a system
- How do I maintain several versions of Python in parallel on the same system
- How do I install, upgrade or uninstall Python modules, possibly for only a specific version of Python
This page tries to answer these questions, first in a general section that explains things that are the same for all system types, then in more specific chapters that cover the peculiarities of the various system types.
Managing Python packages and modules
References
- Python Package Index (PyPi)
- A useful overview of Python packaging and installation tools
How Python locates packages and modules
Python locates packages and modules along a search path. This is a list of directories stored at runtime in the variable sys.path
. When it is started, the Python runtime initializes this variable with the following values:
- The current directory (".")
- The content of the environment variable
PYTHONPATH
. This has the same syntax as the shell variable PATH. - 0-n installation-dependent default paths. Usually this includes what is commonly called the
site-packages
folder - see the next section for details.
A program that knows what it is doing can change the content of sys.path
to influence where modules are searched for.
site-packages
A Python installation usually includes a so-called site-packages
folder. The actual folder name may be different.
The site-packages
folder is a location where Python packages and modules can be installed so that when Python programs execute they can easily find their dependencies.
Refresh: The section "How to define modules" on the wiki page LearningPython explains the difference between modules and packages and how they are represented in the filesystem.
pip
pip
is the Python package manager. pip
manages the site-packages
folder. pip
gets its packages from a package repository called the "Python Package Index", which is commonly abbreviated to PyPI. The URL is https://pypi.org/.
Modern Python versions (>= 3.4) include pip
by default, for older Python versions pip
must be manually installed. This section from the "Python Packaging User Guide" has the details.
Note: Packages managed by pip
are not the same as packages seen by the Python interpreter at runtime, i.e. packages imported by Python code. Quoting from the Python Packaging User Guide: "It is common in the Python community to refer to a distribution using the term “package”. Using the term “distribution” is often not preferred, because it can easily be confused with a Linux distribution, or another larger software distribution like Python itself."
Install a package:
pip install foo
Install a package for the current user only:
pip install --user foo
Show the user-specific base folder:
python -m site --user-base
Uninstall a package:
pip uninstall foo
List the installed packages:
pip list
List installed but outdated packages:
pip list --outdated
Upgrade an installed but outdated package:
pip install --upgrade foo
Search online for available packages:
pip search foo
Package formats
There are several types of package formats:
- Source distribution (sdist): A distribution format (usually generated using
python setup.py sdist
) that provides metadata and the essential source files needed for installing bypip
, or for generating a so-called "Built Distribution". A source distribution requires a build step before it can be installed. - Built distribution: A distribution format containing files and metadata that only need to be moved to the correct location on the target system, to be installed.
- Wheel: A "Built Distribution" format introduced by PEP 427, which is intended to replace the "Egg" format.
- The "Egg" format is another "Built Distribution" format introduced by
setuptools
. - Binary distribution: A specific kind of "Built Distribution" that contains compiled extensions.
Packaging modules
TODO
- Keywords: distutils, setuptools, easy_install
- http://pypi.python.org/pypi/setuptools
- easy_install appears to be distributed together with setuptools
- easy_install is comparable to CPAN
- Is it EasyInstall or easy_install?
- Cheese Shop or cheeseshop refer to PyPi
- Command to install pydot: "easy_install pydotpy"
Linux/Debian
Debian provides specific packages for many important Python modules. For these modules, the Python site administration task therefore simply means install, upgrade or remove Debian packages via aptitude
or another package management frontend.
TODO: Research how to install custom modules, i.e. modules that are not available as Debian packages.
macOS
System version
macOS already comes with Python pre-installed in
/usr/bin/python
This is backed by a framework located here:
/System/Library/Frameworks/Python.framework
This system Python version tends to be quite old. For instance, macOS 10.14.1 still comes with Python 2.7.10, which was released around mid-2015.
Python.org distributions
Newer versions of Python for macOS can be obtained from python.org. These are installer packages. I have not tested any of these recently.
Homebrew version
The way how Homebrew organizes Python is documented here.
Homebrew offers two formulae:
python
installs the most recent Python 3.x version. The executable ispython3
.python@2
installs the most recent Python 2.x version. The executable ispython2
. The formula also providespython
, thus overriding the system-provided Python.
The framework for both formulae is located here: /usr/local/Frameworks/Python.framework
.
site-package
folders are:
/usr/local/lib/python2.7/site-packages /usr/local/lib/python3.7/site-packages
Windows/Cygwin
TODO
- Download setuptools .egg file that matches the installed Python version; download location is the setuptools page on PyPi
- Install by running
sh /path/to/egg-file
- The command
easy_install
should now be available in the path
Windows "pure"
TODO