Skip to content
This repository was archived by the owner on Jul 23, 2025. It is now read-only.

Conversation

@bbrowning
Copy link

What this PR does

This adjusts the publish-dockers workflow to also publish arm64 images. It attempts to build all templates in both amd64 and arm64, but allows the arm64 builds to fail since some of our templates (specifically meta-reference-gpu) fail to build on arm64. After building and pushing the architecture-specific images, it pushes a manifest list for all the architectures we built to the default image location.

Before, we'd end up with images like distribution-remote-vllm that contained only the amd64 image. Now, distribution-remote-vllm contains a manifest list that contains both amd64 and arm64 images. We still publish single manifest, architecture-specific images at distribution-remote-vllm-amd64 and distribution-remote-vllm-arm64 for really old clients, someone that specifically needs an arch-specific image, and because it's the simplest way to run these GitHub workflows as individual steps where we separate arch-specific image building from manifest list publishing in GitHub Actions.

This also makes the DockerHub organization name a build parameter that defaults to llamastack which I used when testing this to publish the images to my own repository.

The main benefit of this change is it allows arm64 users (whether on linux arm64 or Macs using something like podman desktop) to easily run Llama Stack container images without needing to use emulation.

Test Plan

I ran this workflow on both a pre-release and a release pypi package to verify the workflow runs without error in both cases. It properly pushes and tags amd64 and arm64 images for all default distros exception meta-reference-gpu. That distro only builds under amd64 today, and it gracefully handles the build failure for that distro under arm64 and falls back to just pushing the amd64 image. Note that nowhere are we specifically telling the workflow which images work under amd64 and which under arm64 - it just attempts to build all the templates for both, treating any amd64 build failures as fatal (because all of our bits should work under amd64) and treating and arm64 build failures as non-fatal and continuing on.

Here's a link to the set of images built for distribution-starter as an example of this working - https://hub.docker.com/r/bbrowning/distribution-starter/tags . I ran this workflow twice - once for a 0.2.11rc1 release to test the logic that pulls from test pypi, and once for a 0.2.11 release to test the production release logic.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jun 19, 2025
This adjusts the publish-dockers workflow to also publish arm64
images. It attempts to build all templates in both amd64 and arm64, but
allows the arm64 builds to fail since some of our templates
(specifically meta-reference-gpu) fail to build on arm64. After building
and pushing the architecture-specific images, it pushes a manifest list
for all the architectures we built to the default image location.

Before, we'd end up with images like `distribution-remote-vllm` that
contained only the amd64 image. Now, `distribution-remote-vllm` contains
a manifest list that contains both amd64 and arm64 images. We still
publish single manifest, architecture-specific images at
`distribution-remote-vllm-amd64` and `distribution-remote-vllm-arm64`
for really old clients, someone that specifically needs an arch-specific
image, and because it's the simplest way to run these GitHub workflows
as individual steps where we separate arch-specific image building from
manifest list publishing in GitHub Actions.

Signed-off-by: Ben Browning <[email protected]>
@bbrowning bbrowning force-pushed the push-arm64-images-v2 branch from 62fb024 to 40e3d4f Compare June 22, 2025 17:57
@bbrowning
Copy link
Author

Rebased and updated to Python 3.11, with example manifest list and amd64/arm64 Llama Stack v0.2.12 images pushed to https://hub.docker.com/u/bbrowning (ie https://hub.docker.com/r/bbrowning/distribution-starter/tags) from this PR.

@zdtsw
Copy link

zdtsw commented Jun 23, 2025

probably a not related question: any plan to move from dockerhub to quay.io?
the "pull rate limits" might be a concern for users.

@bbrowning
Copy link
Author

I'd personally be in favor of publishing images to both quay.io and dockerhub, but agree it's a bit of a separate issue from choosing the architectures we publish and may need a separate discussion around orgs, credentials, etc.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants