Support arbitrary registration URL

[josegomezr]

Add support for operatingSystem.packages.sccRegistrationUrl to allow for registration against an RMT/SMT/SUMA.

Similar to suse-edge/misc#112

[atanasdinov]

Thank you for your contribution, however, I have to note that this registration only concerns the intermediate container used for RPM resolution, and not the final provisioned node(s).

What is the problem that we're trying to solve?

[josegomezr]

Enable the full process to be able to talk with a SUMA/RMT/SCC Instance.

I understand that this container calls suseconnect to get repositories added via the services returned by the registration server (scc right now) and then will download all RPMs (from the default location: CDN).

By allowing the registration URL just before the download, the downloads can be served by SUMA/RMT/SMT instead.

Or at least is my understanding on when each stage is called 😅

[jdob]

This is awesome, thanks for the contribution. I'll do a quick test on my own to verify the templating part works correctly. In the meantime, I have a few notes on areas that aren't necessarily obvious that need to be updated before it can land:

• pkg/image/validation/os.go - I'd like to see some sort of validation on the URL. Nothing major, but you can use some built in URL functions to make sure it's valid.
• pkg/image/testdata/full-valid-example.yaml - The intention there is to mention every attribute to demonstrate its usage and be able to unit test the parsing. Please add an entry there (line 78 has a usage of sccRegistrationCode, you can put it under there).
• pkg/image/definition_test.go - Make sure to update this with the assertion that the new field is parsed correctly.
• docs/building-images.md - This is our documentation for the definition file. Please update as appropriate (you can follow the model for sccRegistrationCode).
• RELEASE_NOTES.md - Under "Image Definition Changes", add an entry for this new field with a brief description.

[jdob]

@fdegir This is an outside contribution, so not on our roadmap, but can you put this on your todo list of things to add to CI?

[jdob]

Light testing looked good to me. I don't have a local server to test against, but I was able to get the resolution script to show:

suseconnect -r INTERNAL-USE-ONLY-LOL-NO --url "http://foo"

[jdob]

One more thing @josegomezr, please add mention of the new template variable and a brief description here.

[rdoxenham]

Thanks for the contribution @josegomezr - another thing to mention in the documentation would be that by default we register against SCC/CDN, and that if the URL is omitted, this what we use by default. Thanks!

[josegomezr]

I must confess I'm not super inspired on this writing part, I'm very open to suggestions 😅

[rdoxenham]

@atanasdinov @dbw7 @ipetrov117 - would you mind re-reviewing this? I think we should look to merge it if possible. Thank you!

[ipetrov117]

Thank you for your contribution @josegomezr and apologies for jumping late on this.

Sadly, the suggested implementation does not handle all the use-cases that come with the sccRegistrationUrl property introduction.

Ran some tests with this implementation against a local RMT server and found the following things to be working or in need of improvement.

Working:

1. RMT server with http URL is successfully called and used in the RPM resolution process. Image is successfully built and booted.

In need of improvement:

1. Currently, sccRegistrationCode and sccRegistrationUrl are always bundled together.
2. The implementation does not really support https remote server urls.

TL;DR
Regarding (1), bundling these two properties together would mean that if users want to connect a local registration server, they must always provide a registration code. To my knowledge this is not always the case. RMT for instance does not require each connecting machine to provide a registration code. The registration code is placed onto the RMT server itself and connecting machines only need to provide the RMT server URL. IMO we should not bundle these two properties together.

Regarding (2), in the Building Images documentation, we want to suggest the following:

When using sccRegistrationUrl over https it's important that the eib recognizes the CA of the registrationUrl.

There is actually no mechanism that EIB offers to add a custom certificate to the container that is running the RPM resolution process (in order to ensure a consistent RPM resolution process a new container is created inside of the EIB container, check our design doc about this if you have not already).

Yes, we have the certificates directory, however, it is not used to introduce certificates inside of the EIB container, or the RPM resolution container that is inside of it. It is used to introduce the certificates during the image boot. @atanasdinov @jdob please keep me honest.

So with today's implementation, if a user goes and provides sccRegistrationUrl: https://registration-server.like-rmt-smt.here, during the RPM resolution process they would always get the following error:

SUSEConnect error: Post "https://<RMT_SERVER>/connect/subscriptions/systems": tls: failed to verify certificate: x509: certificate signed by unknown authority

To resolve this, the implementation would need to be enhanced to provide the certificate in some way.

@atanasdinov and @jdob, I would love to hear what you think, but from a first glance at the problem I see two approaches:

1. Use the rmt-client-setup script inside of the RPM resolution container to pull the certificates for the RMT server and connect to it (as described in the official doc).

2. Introduce a mechanism that users could leverage to provide custom certificates to the RPM resolution container. In theory we could support a configuration similar to:

   operatingSystem:
     packages:
       scc:
         registrationCode: <code>
         registrationUrl: <url>
          # Directory holding registration server specific certificates.
          # Created by the user within the Image Configure Directory
         certificateDir: <path>

   With this approach, we would need to implement a mechanism to retrieve the certificates from certificateDir and transfer them to the RPM resolution container. This would ensure that the necessary certificates are always available in the container before attempting to connect to the registration server over https.

Let me know if I missed or misunderstood something, but IMO we should not merge this implementation until we tackle/discuss the above points.

[josegomezr]

Leaving a note to self here: there seems to be support for populating the cert pool with arbitrary certificate and/or a directory!

https://github.com/SUSE/connect-ng/blob/9e380ad555245d18d16bd7e2843594f50def2f7a/internal/connect/cert_pool.go#L25