Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions docs/kb/dasharo-binaries-paths-convention.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Dasharo Binaries Paths Convention

Dasharo firmware binaries are available to download on [dl.3mdeb.com][dl.3mdeb.com]
and [dlui.dasharo.com][dlui.dasharo.com]. These sources are being managed
- manually
- using semi-manual automation
- fully automated CI

The objective of this document is to describe a standard convention of the
directory tree and binary names used in Dasharo sources. It will make it easier
to navigate them and find the releases one is in interested in.

## Directories Convention

The standardised path to a release directory is as follows:

`/<vendor>/<model>/<framework>[_payload]/<version>/`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this have to be directly connected with naming convention or even be part of naming convention as part of subsection, otherwise it would be quickly out of sync

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better if it never went out of sync by being as universal as possible. That's why I've been asking about creating more variants of Dasharo with the same framework+payload. A lot of projects (I've counted 10 repositories) depend on the directory structure and updating it will soon, and would in the future, be costly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's why such a scheme should be implemented programmatically (being derived from variables) instead of being hardcoded. Of course, wherever it makes sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the same time it should be compatile with humans too. People use dl.3mdeb.com or dlui.3mdeb.com. It should be easy to manually find what youre looking for there or just explore what's available while understanding what you're looking on

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, what would be your expectation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure what you're asking for. Expectation from who, what?

My task is to clean up the directory structure because it was evolving for some time and got inconsistent. To do that I propose some structure that I believe would help with that.

From the directory structure I expect for it to be able to fit our needs for clearly separating all current and hopefully future Dasharo releases, and to be friendly for humans to navigate.

From you I expect you to give your opinions and whether the suggested solution is acceptable or not, what should be changed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this have to be directly connected with naming convention or even be part of naming convention as part of subsection, otherwise it would be quickly out of sync

I wonder: why and what exactly is missing.

  1. I am sure we will be changing these names. Putting them in the paths will make us changing the paths more often over and over again.
  2. The given framework/payload/platform/version should be unique (possible exception: initial commit for dasharo naming convention v2 #1154 (comment) ?). Matching to business names can be made elsewhere (e.g. in the release page, in the newsletter).
  3. In a way, we already include this naming convention in this scheme, as we store different kind of releases in different places. Maybe this should be emphasized in the document - connect the editions with the storage.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sure we will be changing these names. Putting them in the paths will make us changing the paths more often over and over again.

Yes. Exceptions appear from time to time. We are also not very strict about following our own naming rules when it comes to paths and files. E.g. MTL iGPU and dGPU was one of that cases where out of a sudden binaries changed their paths because nobody could distinguish dPGU from iGPU, despite it is unique (TNx is dGPU, TU is iGPU) - I was very dissatisfied with that. We often create mess ourselves when it comes to paths.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E.g. MTL iGPU and dGPU was one of that cases where out of a sudden binaries changed their paths because nobody could distinguish dPGU from iGPU, despite it is unique (TNx is dGPU, TU is iGPU)

To help with that issue I'm suggesting here to separate every unique model into its own directory. The binaries can, and should, be the same for compatible boards, but separating the models explicitly in the directory should help with similar scenarios in the future. It should be trivial to modify all the uploading scripts to upload a single binary into multiple directories if it works for more than one device model.


where:
- `<vendor>` - hardware vendor, like `novacustom`, `protectli`, `hardkernel`,
`pcengines`
- `<model>` - Precise device model, like `v540tu`, `vp6670`, `odroid-h4`,
`apu4`
- `<framework>` - firmware framework, like `coreboot`, `slimbootloader`, `heads`
- `[_payload]` - optional, firmware payload, like `_uefi`, `_heads`, `_seabios`
- `<version>` - Dasharo version, like `v0.9.0`, `v1.7.2-rc1`, compliant with
the [versioning scheme](https://docs.dasharo.com/dev-proc/versioning/).

Examples:
- Dasharo (coreboot+Heads) Community Package for Novacustom V540TNx 14th
Gen laptops v1.0.0
+ (historically kept in one path due to compatibility):
+ `/novacustom/v540tnd/coreboot_heads/1.0.0/`
+ `/novacustom/v540tne/coreboot_heads/1.0.0/`
- Dasharo (coreboot+UEFI) Pro Package for Protectli V1000-series v0.9.3
+ (historically kept in two directories, a common `protectli_vault_jsl`
path,
and in more specific variants like `protectli_vault_jsl_v1210` due to
temporal compatibility):
+ `/protectli/v1210/coreboot_uefi/v0.9.3/`
+ `/protectli/v1410/coreboot_uefi/v0.9.3/`
+ `/protectli/v1610/coreboot_uefi/v0.9.3/`
- Dasharo (coreboot+SeaBIOS) Community Package for PC Engines APU4 v24.08.00.01
+ (unique versioning scheme)
+ `/pcengines/apu4/coreboot_seabios/v24.08.00.01/`

### Motivation

#### <framework>[_payload]

Historically, the firmware framework and payload were generally not present in
the paths, which looked like `/<platform_family>/<version>/`.
When the support for new frameworks and payloads appeared, new directories
started to appear, like `/<platform_family>/heads/<version>/` or
`/<platform_family>/slimbootloader/<version>/` which made the paths
inconsistent with the older releases.

To fix this inconsistency issue the framework and (optional) payload will
always be there, even if a platform has only support for a single
framework/payload combination to reduce the ambiguity.

It's argueable that a firmware framework without a payload makes little sense
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I woudn't say it makes little sense, when edk2 can be used as a complete end-to-end firmware solution

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know, @miczyg1 told me some time ago that it doesn't make sense, maybe I misunderstood it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@miczyg1 do you agree this description should be removed?

as it won't allow the device to boot. Dasharo Naming Convention allows for
such cases though, so to reduce the risk of needing to come up with hacks like
`none` payload and inconsistent lengths of the path to a binary, the
`[_payload]` field is a part of the same directory name as the `<framework>`.

#### <vendor>/<model>

The idea to separate the `<platform_family>` into the vendor and specific models
has a couple reasons behind it:
1. the number of combinations of vendor, microarchitecture and model family
resulted in too many directories on a single level
1. knowing just a model name it could be not that obvious which family
it is supposed to belong to
1. the firmware binaries for all the models from a family could be, or not be
compatible with the whole family, it could also change with releases and
require changing the directory structure
1. it was not clear how to release the firmware for just one device from a
family

By separating the `vendor` and `model` into independent fields all the issues
above are solved. An additional bonus is that the user searching for the
binaries doesn't need to know technical details like the microarchitecture
of their CPU, just the name of the producer and the model of the device.

[dl.3mdeb.com]: https://dl.3mdeb.com
[dlui.dasharo.com]: https://dlui.dasharo.com

## Binary Names Convention

To reduce the confusion and risk of mixing up binaries and allow for creating
multiple variants of a single binary without inconsistencies, the name of the
binary is also being standardised, and looks like:

`<vendor>_<model>_<framework>[_<payload>]_<version>[_extra].<extension>`

where:
- `<vendor>` - hardware vendor, like `novacustom`, `protectli`, `hardkernel`,
`pcngines`
- `<model>` - Precise device model, like `v540tu`, `vp6670`, `odroid-h4`,
`apu-4`
- `<framework>` - firmware framework, like `coreboot`, `slimbootloader`, `heads`
- `[_payload]` - optional, firmware payload, like `_uefi`, `_heads`, `_seabios`
- `<version>` - Dasharo version, like `v0.9.0`, `v1.7.2-rc1`
- `[_extra]` - optional additional info, like `_dev_signed`, `_btg_provisioned`,
`_eom`
- `<extension>` - `rom` for normal binaries, `cap` for UEFI Capsules, `cab` for
FWUPD cabinets etc.

### Motivation

This is especially important for the 3mdeb employees or users which flash their
devices often. Every binary should contain all the information required to
distinguish it from any other binary to reduce the risk of bricking the device
or other mishaps during flashing.
Loading