Skip to content

Conversation

cam72cam
Copy link
Member

@cam72cam cam72cam commented May 15, 2025

We load the full provider schemas into OpenTofu, regardless of what small subsection we actually use. I'm experimenting with filtering out resources that are not used by the configuration or state.

The current machinery around provider initialization and schemas is ... janky. This prototype digs into that jank to show the promise of the idea.

Contrived Example:

terraform {
        required_providers {
                aws = { source = "hashicorp/aws" }
                awscc = { source = "hashicorp/awscc" }
                azurerm = { source = "hashicorp/azurerm" }
                google = { source = "hashicorp/google" }
                google-beta = { source = "hashicorp/google-beta" }
        }
}

resource "local_file" "myfile" {
  content  = "stuff"
  filename = "myfile"
}

output "foo" {
        value = "bar"
}

https://gist.github.com/cam72cam/aa55271ebe2bbafc9586d1673a007581

  • A full 2 seconds faster
  • 148MB peak down to 46MB peak
  • Schema memory usage down from ~70MB to ~1MB

This also reduces the risk of #2730

Target Release

1.11.0

Checklist

  • I have read the contribution guide.
  • I have not used an AI coding assistant to create this PR.
  • I have written all code in this PR myself OR I have marked all code I have not written myself (including modified code, e.g. copied from other places and then modified) with a comment indicating where it came from.
  • I (and other contributors to this PR) have not looked at the Terraform source code while implementing this PR.

Go checklist

  • I have run golangci-lint on my change and receive no errors relevant to my code.
  • I have run existing tests to ensure my code doesn't break anything.
  • I have added tests for all relevant use cases of my code, and those tests are passing.
  • I have only exported functions, variables and structs that should be used from other packages.
  • I have added meaningful comments to all exported functions, variables, and structs.

Website/documentation checklist

  • I have locally started the website as described here and checked my changes.

cam72cam added 3 commits May 15, 2025 14:10
This would have potentially caused more iterations of the loop, which is
already approaching n*2 time.

Given that the loop would already run until all possible nodes would be
pruned, this change just adjusts it to prune more items in the same
iteration instead of breaking it into more iterations.

Signed-off-by: Christian Mesh <[email protected]>
Copy link

Reminder for the PR assignee: If this is a user-visible change, please update the changelog as part of the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant