🐍 Python

Updated at 2022-05-12 02:22

Python packages are distributed in two main formats:

  • A source distribution contains all the code (Python or otherwise) to build the package.
  • A wheel is a prebuilt Python package. It's actually a zip file with the prebuilt code in it. A wheel will be used if package manager finds a build compatible with your system which avoids building the project yourself.
mypackage-0.0.1.tar.gz               # a source distribution aka. sdist
mypackage-0.0.1-py3-none-any.whl     # a wheel, a type of built distribution

Package Definition

First there was distutils, then became setuptools and now there are dozens of tools for building Python for distribution.

When using setuptools, the most common way to package Python code for distribution, project definition is legacy and nowadays you should use pyproject.toml and setup.cfg to specify how your package is to be prepared for distribution.

Build Instructions

pyproject.toml should list the minimal dependencies of the used build system and specify how to run the build system.

# pyproject.toml
requires = ["setuptools>=40.8.0"]
build-backend = "setuptools.build_meta"

pyproject.toml TL;DR:

  • requires specifies what is needed for building this package
    • setuptools package implements the build_sdist for building source distributions
    • wheel package implements the build_wheel for building wheel built distributions
    • wheel is a dependency of setuptools so you can omit it from the requirements
    • setuptools 40.8.0 is the first version of setuptools that offers a PEP 517 backend
  • build-backend specifies what method is used to perform the build
  • Learn more from PEP 517 and PEP 518

Package Metadata

setuptools allows you to define setup.cfg to specify metadata for the package. These are the details that will be visible at e.g. the PyPI website.

Some tools read their configuration from setup.cfg like isort in the next example.

# setup.cfg
name = mypackage
version = attr: mypackage.__version__
description = My package description
long_description = file:
long_description_content_type = text/markdown
author = Me McMe
author_email =
url =
license = MIT

python_requires = >=3.7
packages =
install_requires =

dev =

profile = black
# start the development...
pip install -e .[dev]
pip install build
python -m build

Note that distributed package dependencies are listed here in the install_requires. The usual locked requirements.txt shouldn't be used in this context as we want to have as broad dependency versions as possible vs. for a deployed software we want to have as tight versions as possible.

Compatibility depends mainly on Python implementation and operating system. Some packages don't have even those restrictions, especially if they are pure-Python.

    = Pandas 1.4.2 for CPython 3.9 on Windows 32-bit

    = the package is _probably_ pure-Python and the wheel is just zipped Python code

If you don't want to use prebuild packages, you can do pip install --no-binary=:all:. You rarely want to do that as some builds require further non-Python dependencies that you'd then need to install. And some package builds can take minutes. But sometimes the wheel might be broken so this becomes handy.