Add multi-architecture (amd64 & arm64) builds #8389

sheurich · 2025-09-09T17:04:43Z

Summary

This PR modifies the Boulder build system to support both amd64 and arm64 architectures while defaulting to building only the current host architecture. This enables efficient local development and provides parallel multi-architecture CI builds.

Changes Made

tools/container-build.sh

Architecture detection: Automatically detects host architecture using uname -m
Single-arch builds: Builds only for the detected architecture by default
Override support: DOCKER_DEFAULT_PLATFORM environment variable for manual overrides

Containerfile

Platform-aware Go downloads: Uses TARGETPLATFORM build arg to pass correct platform to fetch-and-verify-go.sh
Cross-platform compatibility: Supports both linux/amd64 and linux/arm64 platforms

tools/make-deb.sh

Package Architecture: Uses the ARCH environment variable to set the package architecture.
Proper Debian architecture mapping: Correctly maps to amd64/arm64 for .deb packages

GitHub Actions Workflow Changes

This PR also implements the multi-architecture support in the CI/CD pipelines:

release.yml:
- The release workflow is split into three jobs: build-artifacts, create-release, and push-images.
- The build matrix is updated to build for both amd64 and arm64 architectures on corresponding runners.
- The push-images job now pushes multi-platform container images to ghcr.io with a manifest list.
try-release.yml:
- The workflow is updated to build and test for both amd64 and arm64 architectures.
.dockerignore and .gitignore:
- Updated to ignore the .github directory in docker builds and build artifacts from git.

Testing

ARM64 build on Apple Silicon: GO_VERSION=1.24.6 ./tools/container-build.sh
AMD64 override on ARM host: DOCKER_DEFAULT_PLATFORM=linux/amd64 GO_VERSION=1.24.6 ./tools/container-build.sh
Artifact generation: Produces properly named .amd64.tar.gz/.arm64.tar.gz and .amd64.deb/.arm64.deb files
Docker image tagging: Creates both architecture-specific and generic tags

Usage Examples

# Default: Build current architecture
GO_VERSION=1.24.6 ./tools/container-build.sh

# Override: Build specific architecture
DOCKER_DEFAULT_PLATFORM=linux/amd64 GO_VERSION=1.24.6 ./tools/container-build.sh

This change implements the parallel multi-architecture CI builds described above, while improving local development efficiency.

Fixes #8388

- Added TARGETPLATFORM argument to Containerfile for architecture-specific builds. - Updated container-build.sh to detect architecture and set appropriate platform. - Modified make-deb.sh to dynamically set the architecture in the .deb package.

sheurich · 2025-09-09T17:05:21Z

This PR introduces a breaking change in artifact naming that we should discuss:

Current Change:

Before: boulder-1.25.0.xxx-commit.x86_64.tar.gz and boulder-1.25.0.xxx-commit.x86_64.deb
After: boulder-1.25.0.xxx-commit.amd64.tar.gz and boulder-1.25.0.xxx-commit.amd64.deb

The Question:

Should we maintain backward compatibility by keeping x86_64 naming for AMD64 artifacts?

Considerations:

Arguments for standardized naming (amd64):

Consistent with Docker/Debian conventions
Cleaner, more predictable naming scheme

Arguments for backward compatibility (x86_64):

Won't break existing CI/CD pipelines
Won't break download scripts expecting current names

Potential Impact:

Any automation that downloads artifacts by name
CI/CD systems that expect specific filename patterns
Documentation referencing artifact names

Implementation Options:

Keep current PR as-is (breaking change, but cleaner)
Preserve x86_64 naming for AMD64 while using arm64 for ARM
Add both naming schemes temporarily with deprecation timeline

What's your preference? Will any existing systems be impacted by this naming change?

jprenken

This looks great to me.

I slightly favour moving to standardized naming (amd64). We'll probably need to change a few things internally, but just a few. I think that's worth it.

Between this and #8386, whichever merges last should be modified to (ideally) handle uploading both architectures' images when tagging a release, or at least make sure builds are in amd64 as a temporary quick fix.

jsha

This looks great, thanks for putting in time to make the dev experience better @sheurich !

I'm fine with changing the platform name; we can update our build scripts in prod pretty easily.

jsha · 2025-09-10T00:02:18Z

tools/container-build.sh

+    --tag "boulder:${VERSION}-${ARCH}" \
    --tag "boulder:${VERSION}" \
    --tag "boulder:${COMMIT_ID}" \
    --tag boulder \


I didn't give a ton of thought to these tags when we first wrote container-build.sh. Looking at them now it seems that there will be collisions if we start uploading multiple architectures from this script. For instance, for any given version boulder:${VERSION} could be one arch or another.

Probably a single --tag "boulder:${VERSION}-${ARCH}" would suffice and keep things simple. What do you think @jprenken ?

@sheurich I see that you repeat the string "boulder:${VERSION}-${ARCH}" here and on the two docker run commands below. Let's put that into a single TAG env var and use that in each location for consistency.

Updated this. A follow-on step involving docker buildx imagetools create -t letsencrypt/boulder:${VERSION} letsencrypt/boulder:${VERSION}-amd64 letsencrypt/boulder:${VERSION}-arm64 will be needed to glue the images into a single multi-arch.

jsha · 2025-09-10T00:05:35Z

tools/make-deb.sh

+# Determine architecture - use ARCH env var if set, otherwise detect from uname
+if [ -n "${ARCH:-}" ]; then
+    DEB_ARCH="${ARCH}"
+else
+    case "$(uname -m)" in
+        "x86_64") DEB_ARCH="amd64" ;;
+        "aarch64"|"arm64") DEB_ARCH="arm64" ;;
+        *) echo "Unsupported architecture: $(uname -m)" && exit 1 ;;
+    esac
+fi
+


make-deb.sh is a transitional tool to keep building .debs until we transition to running containers.

I think it's okay to assume $ARCH will always be set, because the only place we call it from is container-build.sh, which sets it.

jprenken · 2025-09-11T17:41:29Z

Between this and #8386, whichever merges last should be modified to (ideally) handle uploading both architectures' images when tagging a release, or at least make sure builds are in amd64 as a temporary quick fix.

#8386 just merged, so if possible, let's get that into this PR. Sorry about the hassle!

sheurich · 2025-09-11T21:51:03Z

@jprenken et al I think this last set of changes should address the feedback and adds full multi-arch image builds in ghcr.io.

sheurich · 2025-09-12T09:54:46Z

The comments in the try-release and release workflows indicate a single Go version is used for release and multiple versions are possible for try-release. Is this still correct? I would like to externalize the GO_VERSION to simplify the workflows.

If both release and try-release will only need one version, either:

Read the version specified in go.mod.
OR
Use the convention of a .go-version file in the repo root with contents like:

1.25.0

If one or both of the workflows depend on having multiple versions:

Use a .github/go-versions.json file with contents like:

{
  "versions": ["1.25.0", "1.24.6"]
}

The release workflow can enforce a single version by choosing only the first entry if desired.

This improves the local development experience by providing stable, predictable tags to use for testing, without affecting the architecture-specific tags required by the CI/release process.

sheurich requested a review from a team as a code owner September 9, 2025 17:04

sheurich requested a review from jprenken September 9, 2025 17:04

jprenken previously approved these changes Sep 9, 2025

View reviewed changes

jprenken requested review from jsha, aarongable and beautifulentropy September 9, 2025 23:24

jsha requested changes Sep 10, 2025

View reviewed changes

sheurich dismissed jprenken’s stale review via 556ccf8 September 10, 2025 12:04

sheurich added 2 commits September 10, 2025 10:18

refactor: streamline Docker build and .deb packaging process

d3cfabe

refactor: clarify architecture and platform determination

5d7d235

sheurich force-pushed the multiplat branch from f2c5239 to 5d7d235 Compare September 10, 2025 14:18

sheurich added 5 commits September 11, 2025 14:05

Merge branch 'main' into multiplat

0a46355

ci: build and publish multi-arch amd64/arm64 artifacts

f135e1f

build: Use commit timestamp for reproducible builds

908a45a

ci: Use dynamic repository owner for GHCR image paths

f8226f8

fix: Correct GO_VERSION build arg in release workflow

09027e8

sheurich requested review from jsha and jprenken September 11, 2025 21:51

Merge branch 'main' into multiplat

e9b99ba

jprenken previously approved these changes Sep 12, 2025

View reviewed changes

sheurich dismissed jprenken’s stale review via 5a96126 September 12, 2025 09:24

sheurich added 2 commits September 12, 2025 06:11

build: Add generic tags to container image

cdcbea6

This improves the local development experience by providing stable, predictable tags to use for testing, without affecting the architecture-specific tags required by the CI/release process.

build: parameterize build and deploy base images

33ad853

sheurich force-pushed the multiplat branch from 5a96126 to 33ad853 Compare September 12, 2025 10:11

sheurich requested a review from jprenken September 17, 2025 16:19

Merge branch 'main' into multiplat

a89a461

jprenken approved these changes Sep 23, 2025

View reviewed changes

Merge branch 'main' into multiplat

8c799d9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add multi-architecture (amd64 & arm64) builds #8389

Add multi-architecture (amd64 & arm64) builds #8389

Uh oh!

sheurich commented Sep 9, 2025 •

edited

Loading

Uh oh!

sheurich commented Sep 9, 2025

Uh oh!

jprenken left a comment

Uh oh!

jsha left a comment

Uh oh!

jsha Sep 10, 2025

Uh oh!

sheurich Sep 10, 2025

Uh oh!

jsha Sep 10, 2025

Uh oh!

jprenken commented Sep 11, 2025

Uh oh!

sheurich commented Sep 11, 2025

Uh oh!

sheurich commented Sep 12, 2025

Uh oh!

Uh oh!

Uh oh!

Add multi-architecture (amd64 & arm64) builds #8389

Are you sure you want to change the base?

Add multi-architecture (amd64 & arm64) builds #8389

Uh oh!

Conversation

sheurich commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes Made

GitHub Actions Workflow Changes

Testing

Usage Examples

Uh oh!

sheurich commented Sep 9, 2025

Current Change:

The Question:

Considerations:

Implementation Options:

Uh oh!

jprenken left a comment

Choose a reason for hiding this comment

Uh oh!

jsha left a comment

Choose a reason for hiding this comment

Uh oh!

jsha Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

sheurich Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

jsha Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

jprenken commented Sep 11, 2025

Uh oh!

sheurich commented Sep 11, 2025

Uh oh!

sheurich commented Sep 12, 2025

Uh oh!

Uh oh!

sheurich commented Sep 9, 2025 •

edited

Loading