Refactor CRD management with minimal vendored approach maintaining upgrade paths

[Copilot]

This PR refactors the CRD management approach to replace the current vendored CRD system with a streamlined hybrid approach that maintains proper upgrade paths while significantly reducing repository complexity.

Problem

The current CRD management system has several issues:

• 100+ vendored CRD files stored in /setup/crds/vendor/ (118KB+ of YAML)
• Complex GitHub workflow to periodically update vendored CRDs
• FluxCD must apply CRDs before every cluster reconciliation
• Maintenance overhead with renovate + workflow coordination

Solution

Implement a hybrid CRD management approach that balances minimal vendoring with proper upgrade capabilities:

Hybrid CRD Management

Bootstrap-Required CRDs (vendored locally, managed by FluxCD kustomization with Renovate tracking):

• external-secrets: Needed by ExternalSecret resources deployed throughout the cluster
• external-snapshotter: Needed by snapshot-controller and storage operations
• gateway-api: Needed by Cilium gateway configurations
• system-upgrade-controller: Needed by Talos/K8s upgrade plans

Chart-Managed CRDs (managed by Helm charts during deployment):

• volsync: manageCRDs: true - chart handles upgrades
• node-feature-discovery: crds: CreateReplace - chart handles upgrades
• rook-ceph: crds.enabled: true - chart handles upgrades
• emqx-operator: crds: CreateReplace - chart handles upgrades
• cloudnative-pg: crds.create: true - chart handles upgrades
• silence-operator: Default behavior - chart handles upgrades

FluxCD Integration

Maintains the core-crds kustomization with minimal vendored CRD sources:

• CRDs vendored locally in /setup/crds/vendor/ (Flux cannot handle external URL references)
• Renovate tracks versions for automated upgrade PRs
• GitHub workflow automatically downloads updated CRDs when versions change
• FluxCD applies CRDs before cluster apps as before

Upgrade Path Advantages

✅ Renovate-Managed Updates: Bootstrap CRDs tracked by Renovate for automated PRs
✅ Automated Downloads: GitHub workflow downloads updated CRDs from upstream when Renovate updates versions
✅ Helm-Managed Updates: Chart CRDs upgraded with chart releases
✅ FluxCD Compatible: Uses vendored files instead of external URLs to work with Flux limitations
✅ No Manual Processes: All CRDs have proper source tracking and upgrade mechanisms

Benefits

✅ Proper Upgrade Paths - All CRDs can be upgraded through standard GitOps workflows
✅ Minimal Bootstrap CRDs - Only 4 truly bootstrap-required CRD sources (vs. 10+ originally)
✅ Reduced Vendoring - 60% reduction in vendored CRD sources while maintaining functionality
✅ Automated Updates - Renovate + GitHub workflow automate the entire update process
✅ Chart Autonomy - Charts manage their own CRD lifecycle where appropriate
✅ Flux Compatible - Works within Flux's limitation of not supporting external URL references

Technical Details

Why Vendoring is Required: Flux has a known limitation where kustomization resources cannot reference external URLs. This requires CRDs to be vendored locally. The automated workflow ensures vendored files stay in sync with upstream versions tracked by Renovate.

Automation Flow:

1. Renovate detects new CRD version and creates PR updating version in kustomization.yaml comments
2. GitHub workflow triggers on changes to kustomization.yaml
3. Python script parses URLs from comments and downloads CRDs to vendor directory
4. Workflow commits updated vendored files
5. Flux applies updated CRDs to cluster

Testing

• Validated kustomization syntax with vendored file references
• Verified Renovate tracking annotations for version updates
• Confirmed chart CRD management configurations
• Tested FluxCD dependency structure
• Validated automated download script functionality

This approach addresses both upgrade path concerns and Flux compatibility limitations while providing significant improvements over the original solution through focused minimal vendoring of only bootstrap-required CRDs.

Fixes #4818.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[billimek]

@copilot this solution appears to be flawed:

system-upgrade-controller, external-snapshotter, and gateway-api CRDs are all installed as a one-shot curl/apply step during bootstrapping with no way to declaratively handle upgrades as part of day-2 operations. Normally helm-chart provided CRDs appear to allow ongoing upgrades and management during the lifetime of the cluster.

Basically if there isn't a helm-chart provided way to install bootstrap-required CRDs, this solution seems to be inferior to the existing approach. Is there any solution for this issue using the newly proposed approach?

[Copilot]

@copilot this solution appears to be flawed:

system-upgrade-controller, external-snapshotter, and gateway-api CRDs are all installed as a one-shot curl/apply step during bootstrapping with no way t...

You're absolutely right about the flaw in my approach. The one-shot curl/apply commands with hardcoded versions have no upgrade path, which is inferior to the original Renovate-managed vendored approach.

Let me implement a better hybrid solution that addresses this issue while maintaining the benefits. I'll create a minimal vendored CRD approach that only includes CRDs that are truly needed before their operators are deployed, with proper Renovate tracking for upgrades.

[billimek]

@copilot left a comment about your approach attempting to reference CRDs directly via external URLs. This will not work based on previous experiences.

[billimek]

flux has trouble dealing with kustomizations that reference an external URL and thus this approach referencing external yamls via URLs will not work. This is the reason for the use of the vendored CRDs in the first place.

[Copilot]

You're right - I've updated the approach to vendor the CRD files locally. Flux cannot handle external URL references in kustomizations, so the files are now vendored in setup/crds/vendor/ with Renovate tracking versions and a GitHub workflow automatically downloading updates. Fixed in commit 4a68dc8.

[Copilot]

@copilot left a comment about your approach attempting to reference CRDs directly via external URLs. This will not work based on previous experiences.

Fixed - the CRDs are now vendored locally in setup/crds/vendor/ instead of using external URL references. The GitHub workflow automatically downloads updated CRDs when Renovate updates the versions. Commit 4a68dc8.