actions: debootstrap: Add parent-suite property to indicate which suite a downstream is based on #424

obbardc · 2023-08-04T08:21:43Z

No description provided.

sjoerdsimons · 2023-08-04T08:27:09Z

Would be great if you could add a test for this case

obbardc · 2023-08-04T16:19:23Z

Would be great if you could add a test for this case

as discussed, next step is to add some CI tests to build old/new debian/ubuntu/kali releases :-)

obbardc · 2023-08-10T10:06:53Z

cc @elboulangero for the Kali rolling tests. Is that good enough ?

elboulangero · 2023-08-15T09:09:41Z

cc @elboulangero for the Kali rolling tests. Is that good enough ?

It's excellent, thanks very much for this work. Tested it, works as expected for Kali.

tests/kali/test.yaml

sjoerdsimons

lots of questins; need some pondering if this is the right shape

sjoerdsimons · 2024-01-13T15:41:16Z

actions/debootstrap_action.go

-	cmdline = append(cmdline, "/usr/share/debootstrap/scripts/unstable")
+
+	if len(d.Script) > 0 {
+		if _, err := os.Stat(d.Script); err != nil {


this should be done in verify not at run time

.github/workflows/ci.yaml

sjoerdsimons · 2024-01-13T15:49:36Z

.github/workflows/ci.yaml

+          - { name: "debian (bookworm armhf)", case: "debian", variables: "-t architecture:armhf -t suite:bookworm" }
+          - { name: "debian (trixie amd64)", case: "debian", variables: "-t architecture:amd64 -t suite:trixie" }
+          - { name: "debian (trixie arm64)", case: "debian", variables: "-t architecture:arm64 -t suite:trixie" }
+          - { name: "debian (trixie armhf)", case: "debian", variables: "-t architecture:armhf -t suite:trixie" }


this massively extends the number of test jobs; Does testing trixie on arm64/armhf give us extra data? Also i'm a tad wary about it breaking CI non-deterministically

as was done to tune our runs; for trixie for now please just build amd64 and only on kvm

yeah probably best to remove trixie

sjoerdsimons

So spending a bunch of time today thinking about it and remembering what the issue is again for background

Fundamentally the main part of this problem happens because the newer debootstrap scripts for Debian forced installing usr-is-merged (and to not install usrmerge), unless it detected a known "old" debian suite. Note that this didn't generally impact e.g. Ubuntu, tanglu or trisquel (as their debootstrap scripts don't do that).

Fundamentally the fix we did in debos was to exclude usr-is-merged unless we know the bootstrap distro was a newer debian to not break existing image builds for stable debian releases. For extra fun this never impacted apertis v2024 (bookworm based) as its init-system-helpers hard depend on usr-is-merged and the --exclude switch only applies to the initial package selection not on the expanded dependency list (cause usr-is-merged to be installed on bootstrap in apertis even though it was excluded). On Debian >= bookworm and some derivatives init-system-helpers depends on usrmerge | usr-is-merged , which with the exclusion by both debos and the debootstrap scripts meant neither got installed hence the issues :).

However it should be noted that this kind of gymnastics from debootstrap is exceedingly rare as should be debos doing these workaround.

Slightly orthogonally debos always forced the Debian debootstrap scripts (unstable specifically but they're all the same for any recentish debian release). Even though e.g. kali, ubuntu, tanglu, etc do have their own custom (to a smaller or bigger extend) debootstrap scripts.

So overall i think the step forward for debos should be to be able to use more appropriate debootstrap scripts (e.g. the ubuntu ones for ubuntu); e.g. either matched on the suite (if it's has it's own script directly) or by a property that indicates which suite to use by debootstrap (usually the direct parent, but there could be a longer lineage) or for more custom handling a deboostrap script as part of the recipe (for those needing a custom scripts, but don't have it available in debootstrap).

Most of that matches the patches you've already done; Though to paint some bikesheds:

Change parent-suite into debootstrap-suite instead, with the semantic that the configured argument is what "suite" to use for debootstrap purposes (which primarily selects the right "builtin" debootstrap script but optionally does workarounds).
Change script into debootstrap-script with the argument being the path to the debootstrap script inside the recipe origin rather then a full path.

Having both a debootstrap-script and debootstrap-suite should cause an error as it makes no real sense. If you use a script and need workarounds they should be done in the custom script instead at which point the debootstrap-suite setting has no meaning anymore :)

Also at the vary least if the suite has a builtin debootstrap script then having a debootstrap-suite should at the very least be a warning but i'd even error on the side of it being an error as it's both confusing and a very weird situation.

Having a debootstrap-script in that situation should not be as problematic as e.g. someone is building a variant not supported in the debootstrap scripts. But the output should clearly indicate a custom script is used (probably in the debootstrap output)

For the current behaviour; we should use the current default "unstable" script as always (but also indicate the fallback in the output and potentially suggest to user to set debootstrap-suite) if suite doesn't have it's own script and neither of the new properties were given. In other words automatic fallback only happens if the user didn't explicitly ask for a specific suite or script for debootstrapping

For the usr-is-merged workaround we should only apply it when we detect a known old distribution (based on the suite used debootstrapping) but assume anything not specifically recognized doesn't need this workaround. This is similar to how debootstrap does this (though it only goes from the remote codename rather then the script name) and will be more future proof.

Your current patchset already gets quite closed to this so the changes should be minimal. For e.g. kali that means there is no reason to set a debootstrap-suite as it already has its scripts in debootstrap and we'll pick those up now. For apertis v2024 the same goes (though building will correctly suggest it should be for consistency), while apertis v2023 (and older) now need a debootstrap-suite setting to build image on a newer debian base/debos, but I think that's fine.

sjoerdsimons · 2024-01-21T09:01:29Z

actions/debootstrap_action.go

-func (d *DebootstrapAction) isLikelyOldSuite() bool {
-	switch strings.ToLower(d.Suite) {
+// Check if suite is something before usr-is-merged was introduced
+func shouldExcludeUsrIsMerged(suite string) bool {


We should flip this function around so the default is false as it won't be needed for newer suites (whose set will increase) but will be for older debian ones (whose set won't change).. It also should default to false for newer debian derivatives (using >= bookworm) .

fwiw debootstrap has the following list: etch*|lenny|squeeze|wheezy|jessie*|stretch|buster|bullseye

no need to deal with jessie* though given it is completely unsupported since 2020

sjoerdsimons · 2024-01-21T09:29:31Z

actions/debootstrap_action.go

+- parent-suite -- release code name which this suite is based on. Useful for downstreams which do
+  not use debian codenames for their suite names (e.g. "stable").


Fwiw this could be used for non-debian parents as well.. E.g. parent-suite could be jammy for ubuntu derivatives (or kali for kali derivatives). We should require the parent-suite to be known by debootstrap (as in a script exists) if given. (potentially this requirement should only exist if there is no script for suite, but in that case this property is useless).

sjoerdsimons · 2024-01-21T13:19:58Z

.github/workflows/ci.yaml

+          - { name: "debian (bookworm armhf)", case: "debian", variables: "-t architecture:armhf -t suite:bookworm" }
+          - { name: "debian (trixie amd64)", case: "debian", variables: "-t architecture:amd64 -t suite:trixie" }
+          - { name: "debian (trixie arm64)", case: "debian", variables: "-t architecture:arm64 -t suite:trixie" }
+          - { name: "debian (trixie armhf)", case: "debian", variables: "-t architecture:armhf -t suite:trixie" }


as was done to tune our runs; for trixie for now please just build amd64 and only on kvm

Allow downstream distros to indicate which suite the bootstrapping should be done for. For now, only use this property to gate the debootstrap workaround for excluding usr-is-merged package to the parent suite which should be beneficial to downstreams. Fixes: go-debos/debos!361 Signed-off-by: Christopher Obbard <[email protected]>

Allow the debootstrap script to be set by the user to a full path. If unset, use the existing behaviour of using the unstable debootstrap script. Signed-off-by: Christopher Obbard <[email protected]>

…suite If the debootstrap script property is unspecified, set the script to be the suite property, falling back to the parent suite if the script doesn't exist and finally falling back to unstable if the parent suite doesn't have a custom script. Signed-off-by: Christopher Obbard <[email protected]>

Signed-off-by: Christopher Obbard <[email protected]>

obbardc · 2025-08-11T14:29:54Z

Looks like I messed up the rebase; took some time and sorted it out. I will look at the remaining comments next.

obbardc requested a review from sjoerdsimons August 4, 2023 08:21

obbardc self-assigned this Aug 4, 2023

obbardc mentioned this pull request Aug 4, 2023

New debootstrap is unable to bootstrap pre-bookworm debian derivatives #361

Open

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch from 38c4eb0 to 2bab6e7 Compare August 4, 2023 09:46

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch from 2bab6e7 to 34532f0 Compare August 10, 2023 10:05

obbardc changed the title ~~actions: debootstrap: Add property parent-suite to indicate which suite bootstrapping should be done for~~ actions: debootstrap: Add parent-suite property to indicate which suite a downstream is based on Aug 10, 2023

elboulangero reviewed Aug 15, 2023

View reviewed changes

tests/kali/test.yaml Outdated Show resolved Hide resolved

elboulangero reviewed Aug 15, 2023

View reviewed changes

tests/kali/test.yaml Outdated Show resolved Hide resolved

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch from 34532f0 to 99fcb9e Compare August 15, 2023 09:33

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch from 99fcb9e to 8eecdbb Compare January 10, 2024 14:35

obbardc added this to the v1.1.4 milestone Jan 10, 2024

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch 3 times, most recently from 7b53bef to 7b6ce52 Compare January 10, 2024 16:46

sjoerdsimons reviewed Jan 13, 2024

View reviewed changes

sjoerdsimons linked an issue Jan 21, 2024 that may be closed by this pull request

New debootstrap is unable to bootstrap pre-bookworm debian derivatives #361

Open

sjoerdsimons requested changes Jan 21, 2024

View reviewed changes

sjoerdsimons mentioned this pull request Feb 24, 2025

actions: Add mmdebstrap action #515

Merged

obbardc moved this from Review required to Stalled in Debos&Fakemachine development Aug 8, 2025

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch 2 times, most recently from 91cec41 to e75a972 Compare August 11, 2025 14:01

obbardc added 5 commits August 11, 2025 15:20

actions: debootstrap: Allow Script property to be set

d050df4

Allow the debootstrap script to be set by the user to a full path. If unset, use the existing behaviour of using the unstable debootstrap script. Signed-off-by: Christopher Obbard <[email protected]>

ci: build more debian suites

6da9132

Signed-off-by: Christopher Obbard <[email protected]>

ci: build apertis

48b108c

Signed-off-by: Christopher Obbard <[email protected]>

ci: build kali

6698804

Signed-off-by: Christopher Obbard <[email protected]>

obbardc force-pushed the wip/obbardc/debootstrap-parent-suite branch from e75a972 to 6698804 Compare August 11, 2025 14:28

		- parent-suite -- release code name which this suite is based on. Useful for downstreams which do
		not use debian codenames for their suite names (e.g. "stable").

actions: debootstrap: Add parent-suite property to indicate which suite a downstream is based on #424

Are you sure you want to change the base?

actions: debootstrap: Add parent-suite property to indicate which suite a downstream is based on #424

Uh oh!

Conversation

obbardc commented Aug 4, 2023

Uh oh!

sjoerdsimons commented Aug 4, 2023

Uh oh!

obbardc commented Aug 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

obbardc commented Aug 10, 2023

Uh oh!

elboulangero commented Aug 15, 2023

Uh oh!

Uh oh!

Uh oh!

sjoerdsimons left a comment

Choose a reason for hiding this comment

Uh oh!

sjoerdsimons Jan 13, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sjoerdsimons Jan 13, 2024

Choose a reason for hiding this comment

Uh oh!

sjoerdsimons Jan 21, 2024

Choose a reason for hiding this comment

Uh oh!

obbardc Feb 27, 2024

Choose a reason for hiding this comment

Uh oh!

sjoerdsimons left a comment

Choose a reason for hiding this comment

Uh oh!

sjoerdsimons Jan 21, 2024

Choose a reason for hiding this comment

Uh oh!

sjoerdsimons Jan 21, 2024

Choose a reason for hiding this comment

Uh oh!

sjoerdsimons Jan 21, 2024

Choose a reason for hiding this comment

Uh oh!

obbardc commented Aug 11, 2025

Uh oh!

Uh oh!

obbardc commented Aug 4, 2023 •

edited

Loading