Skip to content

Conversation

planetf1
Copy link
Contributor

Title: build: Enable future-proof builds for all architectures and Python versions

Body:

This PR updates the cibuildwheel configuration in pyproject.toml to provide pre-compiled binary wheels for a wide range of platforms and to ensure support for future Python versions.

Problem

Previously, the project's wheel-building configuration was restrictive, causing two main issues:

  1. No Apple Silicon Wheels: Wheels were not being built for the arm64 architecture on macOS, forcing users on Apple Silicon Macs to build the library from source. This resulted in a slow installation process that required a local Rust toolchain.
  2. Limited Python Version Support: Wheels were only built for Python 3.9. Users on any newer Python version (3.10, 3.11, etc.) also had to build from source.

Solution

The [tool.cibuildwheel] section in pyproject.toml has been updated to:

  1. Enable Automatic Python Version Detection: The explicit build list has been commented out. cibuildwheel will now default to its automatic behavior, building for all available Python versions on the CI runner that are compatible with the project's requires-python = ">=3.9" setting. This makes the process future-proof.
  2. Define Target Architectures: The archs option has been added for both macOS and Linux to ensure wheels are built for all intended platforms.
  3. Improve Documentation: Comments have been added to pyproject.toml to clearly explain the build configuration and how to switch back to a pinned list of versions if desired.

Enabled Builds

This change enables the following builds:

  • Python Versions:

    • Now: Wheels will be built for all currently available and supported Python versions on the GitHub Actions runners (typically 3.9, 3.10, 3.11, 3.12, 3.13).
    • Future: When future Python versions (e.g., 3.14) are added to the runners, wheels will be built for them automatically, with no further changes needed.
  • Architectures:

    • macOS: x86_64 (Intel) and arm64 (Apple Silicon).
    • Linux: x86_64 and aarch64.

Python Wheel Compatibility

It's important to note that Python wheels are specific to the Python version they were compiled for. A wheel's filename contains a "Python tag" (e.g., cp39 for CPython 3.9). When a user on Python 3.12 runs pip install, pip will only look for wheels with a compatible cp312 tag. It will never install an incompatible cp39 wheel on a Python 3.12 environment, as the underlying C-API is different. This change ensures that a correctly tagged wheel will be available for each major Python version.

Impact & Trade-offs

  • End-User Installation: Installation will be significantly faster (seconds instead of minutes) for users on all supported platforms, as they will receive a pre-compiled wheel.
  • CI Build Time: The CI build process will take longer due to the much larger matrix of Python versions and architectures being built. This is a deliberate trade-off for a better user experience.
  • Runtime Performance: There is no change to the library's runtime performance.

In Closing ....

This is an issue I noticed whilst using the library. It feels like a useful improvement, but I don't have experience with building or developing on this codebase, so apologies if I missed something or didn't consider other impacts!

Raising as draft initially to verify CI

@CLAassistant
Copy link

CLAassistant commented Sep 16, 2025

CLA assistant check
All committers have signed the CLA.

@kevinsung
Copy link
Collaborator

Actually, we do build wheels for Apple Silicon and Python versions 3.9 through 3.13, as you can see at https://pypi.org/project/ffsim/#files. If you have an issue with installation, please open an issue describing it with instructions to reproduce it.

@kevinsung kevinsung closed this Sep 16, 2025
@planetf1 planetf1 mentioned this pull request Sep 16, 2025
@kevinsung kevinsung reopened this Sep 16, 2025
pyproject.toml Outdated
Comment on lines 88 to 91
# By default, cibuildwheel builds for all supported Python versions available on
# the CI runner. To build for a specific set of versions, uncomment the
# "build" line below and edit the list.
# build = "cp39-* cp310-* cp311-* cp312-* cp313-*"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only build one wheel that is compatible with all Python versions. Please delete this comment.

pyproject.toml Outdated
test-requires = "pytest"
test-command = "pytest {project}/tests"

[tool.cibuildwheel.macos]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use cibuildwheel for macos. Please delete these lines.

@planetf1
Copy link
Contributor Author

Closing until a working solution is tested as much as possible on a fork.

@planetf1 planetf1 closed this Sep 17, 2025
@planetf1
Copy link
Contributor Author

planetf1 commented Sep 17, 2025

The current build is using manylinux2014 -- this is based on centos7 (hence the patch for yum in the builds). Unfortunately the required tools (rust etc) don't appear in the repos for arm. Trying to create a config which does not affect any platform except aarch64 (and uses manylinux_2_28). Currently paused on runner availability for testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants