Add bootstate endpoint to registry server

[Nuckal777]

Proposed Changes

• This is a straightforward implementation, which adds a Kubernetes Client to the HTTP server. If this is not desired, we should align on how to transport the event of calling the endpoint into the controller.
• In contrast to Define /bootstate call back endpoint in manager to track if Server successfully booted #399 the systemUUID is moved into the JSON payload, similar to the /register endpoint.

Fixes #399.

[afritzler]

Adding a k8sClient to the registry server will make it harder in the future to factor out the registry and deploy it somewhere were it might not have access to the metal-operator api server. How about instead the ServerReconciler probes the /bootstate endpoint periodically since we are doing a periodic retry on all Server objects?

[Nuckal777]

How about instead the ServerReconciler probes the /bootstate endpoint periodically since we are doing a periodic retry on all Server objects?

This would introduce a gap in which the bootstate can be lost:

• Server boots up, calls the bootstate endpoint and receives an HTTP 200.
• Some time passes, while the bootstate only resides in memory
• Server controller reconciles the Server object and applies the condition

Restarting the registry in the second step would loose the bootstate event.

[hardikdr]

Thanks for the PR @Nuckal777, dropped some nit inline.

Thinking about it a little more, I’d propose a following holistic way of handling the first-boot flow. Instead of tying it to a single signal, we could make it more flexible, for example, let the boot-operator set conditions like IPXEScriptFetched and IgnitionDataFetched, and have the /bootstate POST update a BootStateReceived condition on the ServerBootConfig.

Then, the metal-operator could decide which of these conditions to treat as the actual boot completion using a boot-completion-condition flag. This would also make it easier to support things like NetBootOnce and NetBootAlways policies on the ServerClaim side later even when /bootstate POST call is not configured in the Ignition. Wdyt?

[hardikdr]

Optional: how about camel-case for BootState references instead?

[hardikdr]

It would be interesting to explore the trade offs between having this condition on Server vs ServerBootConfig CR. I am somehow leaning more towards ServerBootConfig.

[Nuckal777]

Hmm, ServerBootConfiguration might be more fitting as their lifetimes are bound to a ServerClaim and the discovery boot. 🤔

[Nuckal777]

Instead of tying it to a single signal, we could make it more flexible...

Agree, iirc it was part of the initial discussion on the topic that the owner of a ServerClaim could choose what is considered to be a successful network boot.