Docker release #1007

starius · 2025-09-28T18:03:46Z

This pull request introduces a new make docker-release target to enable reproducible builds of Loop releases. The goal is to ensure that anyone can build the same binaries from the same source code.

Key Changes:

Reproducible Builds:
- The release.sh script has been updated to produce reproducible artifacts. This includes:
  - Building Go in a reproducible way.
  - Creating reproducible .zip and .tar.gz archives.
  - Ensuring the list of tags in the manifest is reproducible.
- The script now clones the repository before building to ensure a clean build environment.
docker-release Target:
- A new docker-release target has been added to the Makefile.
- This target uses a Docker container to build the release, ensuring a consistent build environment across different platforms.
- The build process now verifies the signature of the tag being built. It imports GPG key of Alex Bosworth to do it.
Makefile Improvements:
- The print function in the Makefile has been fixed to work correctly on all platforms.
Documentation:
- The new docker-release target and the reproducible build process are now documented.

Reproducible builds

I propose the reviewers to build the PR and compare hashes with me.

The following command assumes git describe --abbrev=10 produces v0.31.2-beta-141-gaec03c1e0b. If this is not true for you, run git pull --tags https://github.com/lightninglabs/loop.

I built the commit of the PR using:

make docker-release tag=v0.31.2-beta-141-gaec03c1e0b

Also built without Docker. Remember to move output dir v0.31.2-beta-141-gaec03c1e0b if you already built with Docker, otherwise it will. Also remember to install Go 1.24.6 and GPG key of Alex Bosworth (gpg --keyserver keys.openpgp.org --recv-keys DE23E73BFA8A0AD5587D2FCDE80D2F3F311FD87E).

./release.sh v0.31.2-beta-141-gaec03c1e0b

Output of both approaches is the same:

$ cat loop-v0.31.2-beta-141-gaec03c1e0b/manifest-v0.31.2-beta-141-gaec03c1e0b.txt 
735591581251182667179e17e7425eb7ac122407ea60ea0c3db21f3cab9b33d3  loop-darwin-amd64-v0.31.2-beta-141-gaec03c1e0b.tar.gz
c46905e5288487986d72721d771c41d1c99a81f526c66a915e1f44d32f281d9c  loop-darwin-arm64-v0.31.2-beta-141-gaec03c1e0b.tar.gz
6c086fe56068af876b2d743d36d0f9f62a329b348d426654ab11927c0de11dcb  loop-freebsd-amd64-v0.31.2-beta-141-gaec03c1e0b.tar.gz
cd13bfb65dcc47bc4c24d2441cc277cbff530bf733a313c1b3883fc038069be6  loop-freebsd-arm-v0.31.2-beta-141-gaec03c1e0b.tar.gz
72c2b2b48108ab0bf2243750c947985fa6964bedd07ce8136c43a4d38703b97b  loop-linux-386-v0.31.2-beta-141-gaec03c1e0b.tar.gz
7959fa6fde240e4c31e2b8f9d6910c6e497c3d0254c97027d823011aa0c6817a  loop-linux-amd64-v0.31.2-beta-141-gaec03c1e0b.tar.gz
bfa10a4b716f676c4f12c11fa52931a1c806e2675e089bf36f6763eb181d9a1e  loop-linux-arm64-v0.31.2-beta-141-gaec03c1e0b.tar.gz
c99caf51d6abb184e7f6aca1ea0276dff0521820de0f87835b25dbf3f33e6afd  loop-linux-armv6-v0.31.2-beta-141-gaec03c1e0b.tar.gz
770c174b52de69c581417881e12be85b3a374d5dd8532a37d0575660553caf88  loop-linux-armv7-v0.31.2-beta-141-gaec03c1e0b.tar.gz
21bb024453355eb91edbbe0f716948e26b6ded7313032b7afad31ad48690b89f  loop-source-v0.31.2-beta-141-gaec03c1e0b.tar.gz
bfecc7ba9eb1daa70eecdb0de4ef720d20bda983765dc1a4d32852198e35ecd4  loop-windows-amd64-v0.31.2-beta-141-gaec03c1e0b.zip
2546a07b86f0db19adf05a9b8a6661a8ee31b0704ef7f0661bbc59213e1c4d74  vendor.tar.gz

$ sha256sum loop-v0.31.2-beta-141-gaec03c1e0b/manifest-v0.31.2-beta-141-gaec03c1e0b.txt 
f3cad408a41a04035dfbc0bda14be51ca33019356008c96eac5f554489ce65e9  loop-v0.31.2-beta-141-gaec03c1e0b/manifest-v0.31.2-beta-141-gaec03c1e0b.txt

I checked this on multiple machines with amd64 and arm64 architectures and Debian and Ubuntu OS and used both make docker-release and release.sh directly. Outputs are exactly the same!

v0.31.3-beta tag

Tag v0.31.3-beta doesn't have a description, so git describe ignores it. Also the tag is not signed:

$ git tag -v v0.31.3-beta
error: v0.31.3-beta: cannot verify a non-tag object of type commit.

IMHO we should remove v0.31.3-beta and make a signed tag v0.31.4-beta with a description and make it a release.

Pull Request Checklist

Update release_notes.md if your PR contains major features, breaking changes or bugfixes

hieblmi · 2025-09-29T11:05:54Z

git describe produces v0.31.2-beta-133-g6402a76e, the trailing e is missing from your PR description.

"make docker-release tag=..." produces the same binaries regardless on the platform and the installed Go version, because it uses Docker.

Previous version produces escapped sequnces as text (Debian 12).

To test "make docker-release" it is convenient to pass a regular commit to it (an output of `git describe`). In this commit release.sh is changed to detect such "tags" and skip tag signature verification for it. Also if release.sh is used without an argument, a unique directory name is generated from the current time.

Create a temporary directory inside loop/ directory and make a clone of Git repo to that dir. This ensures that the version built is exactly what is committed without any unstaged files and not dirty.

Took the code for reproducible packing from LND.

Added -trimpath to remove absolute paths from the binary.

Collect artifacts in "tmp-..." instead of final place. If build fails in the middle, we don't want to leave the artifacts directory with partial result. Also remove the build directory (with Git clone) in the end.

When doing git-clone to a subdir, do not keep tags, since there can be local tags affecting `git describe` and buildvcs info. Instead pull the tags from upstream.

Without --abbrev flag it may produce different number of hash characters on different instances. It depends on the state of Git repo itself.

Otherwise it asks confirmation to remove tmp-.../.git dir.

starius · 2025-09-29T15:42:44Z

$ git describe --help
...
--abbrev=<n>
   Instead of using the default number of hexadecimal digits (which will vary
   according to the number of objects in the repository with a default of 7) of the
   abbreviated object name, use <n> digits, or as many digits as needed to form a
   unique object name. An <n> of 0 will suppress long format, only showing the closest
   tag.

So the length of this hash depends on the state of the Git repo.

I updated the PR, replaced git describe with git describe --abbrev=10 and updated all the hashes.

Please try again!

Make sure we use GNU gzip. Mac by default uses BSD gzip.

starius · 2025-09-30T02:38:40Z

@hieblmi Thanks for reporting!

I updated docs/release.md for Mac tools installation:

brew install gnu-tar gzip

Note that you also need to install (GNU) gzip from brew.

I found the root cause of the discrepancies: GNU and BSD versions of gzip tool produces different outputs. By default Mac uses BSD gzip, but you can install package gzip from brew, it brings you GNU gzip (available as ggzip command).

I updated the script to find the correct tool similar to how it finds GNU Tar. I updated the hashes.

Could you rebuilt it both on Docker and using ./release.sh and compare hashes, please?

Another thing. I copy-pasted the code for reproducible tar_gz packing from LND. I suspect that LND's release script might have the same issue on Mac.

Could you try to build LND release using make release please?
You can follow the instruction from release v0.19.3-beta. Basically you need to install go1.23.12, checkout the release tag and run make release tag=v0.19.3-beta sys=linux-amd64 and make sure the produced file lnd-linux-amd64-v0.19.3-beta.tar.gz has SHA-256 hash 4f61f0606a528c01293b164cd46c5845c20e7251e947fd1e5a9b9ec6973ac07f. If there is a mismatch, probably LND release script needs the same patching to find GNU gzip.

hieblmi · 2025-09-30T08:29:16Z

Thanks for the investigation! Now it all checks out.

The docker release produces:

- Producing TAR.GZ file /repo/tmp-loop-v0.31.2-beta-138-g6d8872f995-20250930-080946/loop-freebsd-arm-v0.31.2-beta-138-g6d8872f995.tar.gz
- Producing manifest-v0.31.2-beta-138-g6d8872f995.txt
71123f601dc7efd1711d1fd8481636e448ac059eef338ee73656d2ca1036dfc3  manifest-v0.31.2-beta-138-g6d8872f995.txt
- Moving artifacts directory to final place /repo/loop-v0.31.2-beta-138-g6d8872f995
- Removing the subdir used for building /repo/tmp-build-20250930-080920
 ~/projects/go/src/github.com/loop  docker-release *2 ?1  sha256sum loop-v0.31.2-beta-138-g6d8872f995/manifest-v0.31.2-beta-138-g6d8872f995.txt                         ✔  7m 36s  10:16:55 
71123f601dc7efd1711d1fd8481636e448ac059eef338ee73656d2ca1036dfc3  loop-v0.31.2-beta-138-g6d8872f995/manifest-v0.31.2-beta-138-g6d8872f995.txt

The ./release.sh produces:

\e[0;32m- Producing manifest-v0.31.2-beta-138-g6d8872f995.txt\e[0m
71123f601dc7efd1711d1fd8481636e448ac059eef338ee73656d2ca1036dfc3  manifest-v0.31.2-beta-138-g6d8872f995.txt
\e[0;32m- Moving artifacts directory to final place /Users/bob/projects/go/src/github.com/loop/loop-v0.31.2-beta-138-g6d8872f995\e[0m
\e[0;32m- Removing the subdir used for building /Users/bob/projects/go/src/github.com/loop/tmp-build-20250930-102308\e[0m
 ~/projects/go/src/github.com/loop  docker-release *2 ?1  sha256sum loop-v0.31.2-beta-138-g6d8872f995/manifest-v0.31.2-beta-138-g6d8872f995.txt                         ✔  3m 19s  10:26:27 
71123f601dc7efd1711d1fd8481636e448ac059eef338ee73656d2ca1036dfc3  loop-v0.31.2-beta-138-g6d8872f995/manifest-v0.31.2-beta-138-g6d8872f995.txt

If Go binary itself comes from `go install golang.org/dl/go1.24.6@latest` it embeds a different BuildID. We unset BuildID to enforce reproducibility.

sputn1ck

LGTM!

docker-release and release.sh produce the same output for me, matching you current pr description
f3cad408a41a04035dfbc0bda14be51ca33019356008c96eac5f554489ce65e9 manifest-v0.31.2-beta-141-gaec03c1e0b.txt

starius marked this pull request as ready for review September 29, 2025 02:34

hieblmi self-requested a review September 29, 2025 10:04

starius added 11 commits September 29, 2025 11:24

make: add target "docker-release"

f70678f

"make docker-release tag=..." produces the same binaries regardless on the platform and the installed Go version, because it uses Docker.

make: fix green prints

8a35705

Previous version produces escapped sequnces as text (Debian 12).

release.sh: git clone before building

0e30dc9

Create a temporary directory inside loop/ directory and make a clone of Git repo to that dir. This ensures that the version built is exactly what is committed without any unstaged files and not dirty.

release.sh: make reproducible .zip and .tar.gz

9e8cde5

Took the code for reproducible packing from LND.

release.sh: build Go in reproducible way

9c13803

Added -trimpath to remove absolute paths from the binary.

release.sh: collect artifacts in intermediate dir

a889fbe

Collect artifacts in "tmp-..." instead of final place. If build fails in the middle, we don't want to leave the artifacts directory with partial result. Also remove the build directory (with Git clone) in the end.

release.sh: make the list of tags reproducible

ec53b0d

When doing git-clone to a subdir, do not keep tags, since there can be local tags affecting `git describe` and buildvcs info. Instead pull the tags from upstream.

release.sh: print sha256 hash of manifest file

2f3edb9

docs: describe "make docker-release"

35d0fa2

release.sh: use git describe --abbrev=10

2464e94

Without --abbrev flag it may produce different number of hash characters on different instances. It depends on the state of Git repo itself.

starius force-pushed the docker-release branch from 6402a76 to 2464e94 Compare September 29, 2025 14:49

release.sh: remove the clonned dir with -rf

a34b46c

Otherwise it asks confirmation to remove tmp-.../.git dir.

release.sh: reproducible gzip on Mac

6d8872f

Make sure we use GNU gzip. Mac by default uses BSD gzip.

starius requested review from alexbosworth, bhandras and sputn1ck September 30, 2025 13:01

starius added 2 commits September 30, 2025 11:37

release.sh: check Go version, provide GO_CMD var

260b90d

release.sh: fix green and red messages on Mac

25c5618

alexbosworth approved these changes Sep 30, 2025

View reviewed changes

release.sh: set -buildid to ''

aec03c1

If Go binary itself comes from `go install golang.org/dl/go1.24.6@latest` it embeds a different BuildID. We unset BuildID to enforce reproducibility.

sputn1ck approved these changes Sep 30, 2025

View reviewed changes

starius merged commit 9f8f4e6 into lightninglabs:master Sep 30, 2025
4 checks passed

starius deleted the docker-release branch September 30, 2025 19:15

starius mentioned this pull request Oct 4, 2025

release.sh: make sure GNU gzip is used lightningnetwork/lnd#10268

Open

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Docker release #1007

Docker release #1007

Uh oh!

starius commented Sep 28, 2025 •

edited

Loading

Uh oh!

hieblmi commented Sep 29, 2025

Uh oh!

starius commented Sep 29, 2025

Uh oh!

starius commented Sep 30, 2025

Uh oh!

hieblmi commented Sep 30, 2025

Uh oh!

sputn1ck left a comment

Uh oh!

Uh oh!

Uh oh!

Docker release #1007

Docker release #1007

Uh oh!

Conversation

starius commented Sep 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Key Changes:

Reproducible builds

v0.31.3-beta tag

Pull Request Checklist

Uh oh!

hieblmi commented Sep 29, 2025

Uh oh!

starius commented Sep 29, 2025

Uh oh!

starius commented Sep 30, 2025

Uh oh!

hieblmi commented Sep 30, 2025

Uh oh!

sputn1ck left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

starius commented Sep 28, 2025 •

edited

Loading