Development

Development#

Repository layout#

polypix/
  __init__.py         Python API and input validation
  _core.pyi           private compiled-extension typing stub
  py.typed            PEP 561 marker
rust/
  revisit.rs          segmented revisit and gap reduction
  geometry.rs         spherical-polygon validation and containment
  lib.rs              PyO3 bindings and native-buffer safety
  ring/               HEALPix RING coverage, planning, and threading
  reduce.rs           dense and selected coverage reductions
tests/                behavior tests and independent geometry fixtures
benchmarks/           CodSpeed public-call regression benchmarks
examples/             the two documented case studies and their shared helpers
tools/                external-oracle fixture generation and CI guards
docs/                 Sphinx/MyST documentation sources
adr/                  architecture decision records, kept out of the site

Build model#

Polypix is a mixed Python/Rust project built by Maturin. PyO3 exposes the Rust kernel as polypix._core, and the Python package wraps it in the NumPy-first public API. The RING kernel is Polypix’s own, with no HEALPix runtime dependency.

Wheels ship the compiled kernel, and NumPy is the only runtime dependency. HEALPix C++, CFITSIO, CMake, and a C++ compiler appear nowhere in the build or runtime chain.

Development environment#

Pixi provisions Python, NumPy, Rust, Maturin, and pytest, then installs Polypix in editable mode, so pixi run test on a fresh clone both builds the extension and runs the suite. The checked-in Pixi platforms are Linux x86-64 and macOS. On Windows and Linux ARM64, use pip install -e . instead, which CI exercises as well.

pixi run wheel builds a local release wheel. Release wheels for every supported platform come from the GitHub Actions release workflow described below.

Common commands#

Command

Does

pixi run test

Run the test suite

pixi run lint

Ruff, formatting, mypy, and stub checks

pixi run maturin develop

Build the extension in place

pixi run wheel

Build a release-mode wheel

pixi run --environment bench bench

Run the CodSpeed benchmarks locally

pixi run --environment docs docs-build

Build the documentation

pixi run --environment docs docs-doctest

Test interactive documentation examples

pixi run --environment docs docs-serve

Preview with autobuild

pixi run --environment docs docs-figures

Regenerate the example figures

pixi run --environment docs docs-clean

Remove generated documentation output

Documentation#

Sources live in docs/, configured by docs/conf.py. Sphinx renders the Markdown through MyST with sphinx-book-theme. The API reference is generated by sphinx.ext.autodoc from the docstrings in polypix/__init__.py, so the signatures and validation rules on the site are the ones in the source rather than a second copy kept in step by hand.

docs-build and docs-serve both run docs-figures first. That step executes both case studies and redraws the center-versus-overlap diagrams, writing their maps and measurement tables into docs/assets/generated/. The remaining guide figures are stable assets checked into that directory. After changing an example or coverage behavior, regenerate the dynamic figures yourself; docs-serve will not do it on reload.

Each example also runs standalone, writing its maps to the working directory:

pixi run --environment docs docs-communications
pixi run --environment docs docs-earth-observation

The communications example pulls in Astroz, a docs-only binary dependency. The published build runs on Linux x86-64 with glibc 2.36 or newer, which says nothing about Polypix’s own runtime support.

.github/workflows/docs.yml builds the same site on pull requests and publishes site/ to GitHub Pages on pushes to main.

Benchmarks and fixtures#

CodSpeed reports deterministic instruction regressions for single-threaded benchmarks on pull requests and pushes to main. Benchmarks marked parallel are excluded, because Valgrind simulation serializes worker threads; the same workflow runs tools/check_parallel_speedup.py natively as a coarse wall-time guard on multicore scaling.

When documentation publishes a performance figure, cite both the benchmark source and the CodSpeed history. Link the source to the release tag or commit used for the figure, not to a moving local copy. Round simulation results and describe them as reference workloads; they are regression evidence, not a universal wall-clock promise. Case-study pipeline profiles may retain their stage table when it is pinned to the exact GitHub documentation run that produced it and labeled illustrative.

Cross-library benchmarks live in a separate comparison repository that is not public yet. Until it is linked here, this repository makes no cross-library performance claim.

The independent center-and-corner fixtures are regenerated in a temporary environment containing healpy:

python tools/generate_ring_geometry_fixtures.py

healpy is an external oracle, not a dependency of any kind.

Release builds#

.github/workflows/release.yml builds a source distribution and CPython 3.12 stable-ABI wheels for Linux x86-64 and ARM64, macOS 11+ on x86-64 and ARM64, and Windows x86-64. Pull requests build a smaller native-platform smoke matrix; releases and manual runs build everything.

The version comes from Cargo.toml. Keep the Polypix entry in Cargo.lock in step with it.

  1. Open a release pull request setting the version, dating and completing the CHANGELOG.md section, and updating license notices if they changed. Run the test, lint, and docs commands, and merge only once the required workflows pass. For every breaking change, include a before/after migration snippet in the changelog.

  2. On main, run Build and publish manually as a dry run. Manual runs build and test the full wheel matrix without publishing.

  3. Draft a GitHub release tagged v<version> at the release commit, using the changelog section as its notes. Publishing it rebuilds the artifacts and uploads them to PyPI through trusted publishing.

  4. Verify from a clean environment:

    python -m pip install --no-cache-dir polypix==<version>
python -c "import polypix as px; print(px.__version__)"

Check the PyPI metadata and wheel set before you walk away. Published files are immutable, so a mistake means a new patch release.

License and notices#

Apache-2.0. The cell-corner transform includes a small BSD-3-Clause adaptation from Astrometry.net. Keep LICENSE, THIRD_PARTY_NOTICES.md, and Cargo.lock current whenever native dependencies or adapted code change, and check the locked Rust dependency graph before publishing.

Design constraints#

Keep the Python layer thin: normalize inputs and build results in Python, do the expensive geometry, HEALPix, and parallel work in Rust, and return NumPy arrays rather than Python lists for anything large.

Adding a public function means writing it in polypix/__init__.py with a complete docstring, adding it to the lists in docs/api.md, and covering it in the tests and benchmarks. Write the docstring so that it stands on its own: a reader arriving at one function should not have to go and read another one to find out what the arguments accept.

Project documents#

The architecture decision records explain choices that would be awkward to undo, and they live in the adr/ directory of the repository instead of on this site: