ruk·si
/notes

🐍 Python
Distribution

Updated at 2022-05-11 23:22

This note is how to prepare your Python package for distribution so it can be e.g. downloaded through pip.

Python packages are distributed in two main formats:

  • A source distribution contains all the code (Python or otherwise) to build the package.
  • A wheel is a prebuilt Python package. It's actually a zip file with the prebuilt code in it. A wheel will be used if package manager finds a build compatible with your system which avoids building the project yourself.
mypackage-0.0.1.tar.gz               # a source distribution aka. sdist
mypackage-0.0.1-py3-none-any.whl     # a wheel, a type of built distribution

Package Definition

First there was distutils, then became setuptools and now there are dozens of tools for building Python for distribution.

When using setuptools, the most common way to package Python code for distribution, setup.py-style project definition is legacy and nowadays you should use pyproject.toml and setup.cfg to specify how your package is to be prepared for distribution.

Build Instructions

pyproject.toml should list the minimal dependencies of the used build system and specify how to run the build system.

# pyproject.toml
[build-system]
requires = ["setuptools>=40.8.0"]
build-backend = "setuptools.build_meta"

pyproject.toml TL;DR:

  • requires specifies what is needed for building this package
    • setuptools package implements the build_sdist for building source distributions
    • wheel package implements the build_wheel for building wheel built distributions
    • wheel is a dependency of setuptools so you can omit it from the requirements
    • setuptools 40.8.0 is the first version of setuptools that offers a PEP 517 backend
  • build-backend specifies what method is used to perform the build
  • Learn more from PEP 517 and PEP 518

Package Metadata

setuptools allows you to define setup.cfg to specify metadata for the package. These are the details that will be visible at e.g. the PyPI website.

Some tools read their configuration from setup.cfg like isort in the next example.

# setup.cfg
[metadata]
name = mypackage
version = attr: mypackage.__version__
description = My package description
long_description = file: README.md
long_description_content_type = text/markdown
author = Me McMe
author_email = me@example.com
url = https://github.com/me/mypackage
license = MIT

[options]
python_requires = >=3.7
packages =
    mypackage
install_requires =
    requests

[options.extras_require]
dev =
    black
    flake8
    flake8-isort
    isort
    mypy
    pytest
    pytest-cov

[isort]
profile = black

~/projects/mypackage/
    mypackage/__init__.py
    mypackage/core.py
    pyproject.toml
    setup.cfg

# start the development...
pip install -e .[dev]

pip install build
python -m build

Note that distributed package dependencies are listed here in the install_requires. The usual locked requirements.txt shouldn't be used in this context as we want to have as broad dependency versions as possible vs. for a deployed software we want to have as tight versions as possible.

Compatibility depends mainly on Python implementation and operating system. Some packages don't have even those restrictions, especially if they are pure-Python.

pandas-1.4.2-cp39-cp39-win32.whl
    = Pandas 1.4.2 for CPython 3.9 on Windows 32-bit

*-py3-none-any.whl
    = the package is _probably_ pure-Python and the wheel is just zipped Python code

If you don't want to use prebuild packages, you can do pip install --no-binary=:all:. You rarely want to do that as some builds require further non-Python dependencies that you'd then need to install. And some package builds can take minutes. But sometimes the wheel might be broken so this becomes handy.

Sources