Building Scientific Python Packages

Packaging Python code, especially for scientific applications, has historically been hard, with poor documentation, many obscure fragile hacks into a rigid system, and challenges distributing this to users. All that has dramatically improved over the last few years with a series of PEPs (Python Enhancement Proposals) creating a unified standard to build on, innovative packages that have built on these, and extensive documentation around best practices. We will be looking at how the Scientific Python Development Guide [1], along with its template and repo-review [2] tooling, teaches about modern scientific Python software development. This is part of an effort to document recommendations for Python development with several key tools, like uv, ruff, cibuildwheel [3], and scikit-build-core [4].

The Scientific Python Development Guide was started as a guide in the Scikit-HEP organization, written to facilitate the growing number of packages with a growing number of maintainers. Over time, a cookiecutter-based template was added, and a tool to evaluate repository’s configuration and compare it to the guide was added. At the first Scientific Python Developer Summit in 2023, the guide was moved to the Scientific-Python organization, the tools were reworked and generalized, automation was expanded, parts of the NSLS-II guidelines were merged in, the three separate components were combined into one repo, and the guide became part of the Scientific Python organization. Since then, the guide has remained one of the most up-to-date resources available, with new packaging changes like Trusted Publishing and SPDX license identifiers being added days and even sometimes hours after they are announced. We will look at some of the recommendations, especially newer ones.

The cookiecutter, now part of the guide as well, provides a fast start for users who have read the guide. It supports both cookiecutter and copier programs, and can be used with “update” tooling (cruft or copier). It supports around 10 build backends, including the most popular pure Python and compiled extension backends. You can select classic or vcs versioning, and can pick between several licenses. We will see how easy it is to start a project with all tooling in place.

Repo-review is a framework for writing checks for configuration; it is written in Python and can be run using WebAssembly. The sp-repo-review plugin contains all the checks described in the guide, with cross-links to take a user to the badge in the guide for each check. Some projects, like AstroPy, have added this to their CI, though it can be used entirely from the guide’s website.

A dedicated section on binary package will include scikit-build-core, the native build backend that can adapt CMake projects into Python wheels. When combined with pybind11 [5], you can write working compiled extensions with just three files and a small handful of lines. Combined with cibuildwheel, the wheel builder tool used throughout the Python ecosystem, and a good CI service like GitHub Actions, building redistributable wheels for all major platforms, and even WebAssembly, iOS, and Android, is easy. All the required code to get started can easily be shown on a poster.

References 1. The Scientific Python Development Guide, Henry Schreiner, et. al, June 2023. Retrieved 2025-7-17 2. https://learn.scientific-python.org/development 3. repo-review, Henry Schreiner, et. al, June 2023. https://repo-review.readthedocs.io 4. cibuildwheel, Joe Rickerby, Henry Schreiner, et. al, 2016. https://cibuildwheel.pypa.io 5. scikit-build-core, Henry Schreiner, et. al, 2022.. https://scikit-build-core.readthedocs.io 6. pybind11, Wenzel Jakob, Henry Schreiner, et. al, 2017. https://pybind11.readthedocs.io