diff --git a/docs/2.0/docs/accountfactory/guides/drift-remediation-with-async-module.md b/docs/2.0/docs/accountfactory/guides/drift-remediation-with-async-module.md index 5870c1670..b46975469 100644 --- a/docs/2.0/docs/accountfactory/guides/drift-remediation-with-async-module.md +++ b/docs/2.0/docs/accountfactory/guides/drift-remediation-with-async-module.md @@ -134,7 +134,7 @@ For `_envcommon/landingzone/root-pipelines-apply-role.hcl`, ensure that you have } ``` -For `_envcommon/landingzone/root-pipelines-plan-role.hcl`, ensure that you have at leasat the following permissions: +For `_envcommon/landingzone/root-pipelines-plan-role.hcl`, ensure that you have at least the following permissions: ```hcl "CloudWatchEventsReadOnlyAccess" = { diff --git a/docs/2.0/docs/patcher/concepts/index.md b/docs/2.0/docs/patcher/concepts/index.md index cf7ab8275..5c3d31149 100644 --- a/docs/2.0/docs/patcher/concepts/index.md +++ b/docs/2.0/docs/patcher/concepts/index.md @@ -1,9 +1,37 @@ # What is Gruntwork Patcher? -Gruntwork Patcher automates the process of keeping your infrastructure code ([Terragrunt](https://terragrunt.gruntwork.io/) and [OpenTofu](https://opentofu.org/)) up to date, including applying patches to ensure compatibility with backward-incompatible module releases. +Gruntwork Patcher automates the process of keeping your infrastructure code ([Terragrunt](https://terragrunt.gruntwork.io/), [OpenTofu](https://opentofu.org/), and [Terraform](https://terraform.io)) up to date, including applying patches to ensure compatibility with backward-incompatible module releases. + +We use the term "Patcher" to refer to the [Patcher CLI](https://github.com/gruntwork-io/patcher-cli), the public [Patcher GitHub Action](https://github.com/gruntwork-io/patcher-action), and the collection of tools (like Terrapatch[https://github.com/gruntwork-io/terrapatch-cli]) that enable Patcher to be useful. + +## Why use Patcher? Manually identifying updates and assessing whether they can be safely applied can consume significant engineering time for each module dependency. Patcher eliminates this inefficiency by streamlining the update process. -You can use Gruntwork Patcher to manage dependencies on the Gruntwork IaC Library, which includes patches for recent breaking changes to Gruntwork modules. Patcher also supports updating dependencies for your own modules or open-source projects, using semantic versioning to identify safe updates and highlight those that need manual review. +Patcher supports keeping any set of OpenTofu/Terraform modules up to date, whether they be your inhouse modules, third-party open source modules, or modules from the [Gruntwork IaC Library](library/concepts/overview). + +Patcher specializes in keeping infrastructure code up to date and currently supports automatic updates for: + +- OpenTofu modules +- Terraform modules +- Terragrunt units +- Terragrunt stacks + +## Two update modes + +When most teams think about updating their infrastructure code, there are two core use cases they look to solve: + +1. **Legacy upgrade.** You wish to take a repo or set of files that are significantly out of date and bring them up to date with the latest OpenTofu/Terraform module versions. +2. **Ongoing updates.** You wish to streamline the process of keeping a repo or set of files up to date over time. + +Patcher can help with both of these use cases. + +For legacy upgrades, the Patcher CLI offers an [interactive mode[(guides/update)] where you can browse all modules in the current working directory and below, browse available updates, and upgrade modules one at a time. We've found this approach works well with a modest set of updates, however for significantly out of date repos or files or a large number of files, you may wish to consider alternative approaches. + +For ongoing updates, you can use Patcher in conjunction with our published GitHub Action to automatically scan your repo's modules and [open pull requests](guides/ongoing-updates) with updated module versions. Opening one pull request for every update would be cumbersome, so Patcher supports a variety of [grouping modes](/concepts/grouping) that allow you to streamline your workflow. + +## How does Patcher work? + +Patcher handles both non-breaking and breaking changes. For non-breaking changes, for the [Gruntwork Iac Library](library/concepts/overview), Patcher uses a set of pre-generated changelog files to identify the "next safe change," enabling upgrades to automatically bypass "empty" version upgrades where a module is available at a newer version but in fact has had no actual file changes. Upon request, we can work with you to implement these pre-generated changelogs in your organization. -Gruntwork Patcher provides a clear, README-driven workflow for breaking changes without available patches. When Patcher detects breaking changes, it updates the relevant dependency to the version containing those changes and generates a README file in the folder with the updated file. This README outlines the release notes and details the breaking changes. Users must review the README, address any necessary actions, and remove the file before rerunning Patcher. +For breaking changes, Patcher offers a systematic approach to doing code transformations -- we call these "patches" -- so that module consumers can automatically apply breaking changes to their modules. Or if Patcher detects a breaking change but a patch does not exist, Patcher updates the relevant module to the next breaking change and generates a README file in the folder with the updated file that outlines the release notes and details the breaking changes. Users review the README, address any necessary actions, and remove the file before re-running Patcher. \ No newline at end of file diff --git a/docs/2.0/docs/patcher/guides/ongoing-updates.md b/docs/2.0/docs/patcher/guides/ongoing-updates.md new file mode 100644 index 000000000..bd8775370 --- /dev/null +++ b/docs/2.0/docs/patcher/guides/ongoing-updates.md @@ -0,0 +1,312 @@ +# Ongoing Updates + +## Overview + +As we mentioned in the [Patcher overview](../concepts/index.md), Patcher supports two primary modes of working: + +1. **Legacy upgrade.** You wish to take a repo or set of files that are significantly out of date and bring them up to date with the latest OpenTofu/Terraform module versions. +2. **Ongoing updates.** You wish to streamline the process of keeping a repo or set of files up to date over time. + +In this guide, we'll use Patcher to help you with the second mode. The basic idea is that we'll set up a GitHub Action that scans your entire repository for outdated dependencies and automatically creates pull requests to keep everything up to date. + +This approach is ideal for teams that want to: +- Maintain current dependencies across their entire codebase +- Receive regular automated updates without manual intervention +- Keep security vulnerabilities and technical debt to a minimum + +## Which CI systems are supported? + +Patcher CLI is a binary and is therefore agnostic to the underlying CI system, however the GitHub Action we describe here currently only works for GitHub.com or GitHub Enterprise. + +:::info +We'll be adding equivalent support for GitLab soon. In the meantime, for non-GitHub users, feel free to adapt the code at https://github.com/gruntwork-io/patcher-action to suit your needs. +::: + +## Implementation + +We're about to create a GitHub Actions workflow that consists of two main jobs: + +1. **`patcher-report`**: Scans your entire repository for outdated dependencies +2. **`patcher-update`**: Creates individual pull requests for each dependency that needs updating + +:::info +Individual pull requests can quickly get overwhelming. Once you've got this working, check out our page on [grouping](../concepts/grouping.md) to see how to consolidate many updates into a single pull request. +::: + +### Step-by-step setup for GitHub.com users + +#### 1. Create a GitHub token +- Create a token that can read patcher-cli and terrapatch-cli releases in your org (or `gruntwork-io`) (see [GitHub Personal Access Token Setup](#github-personal-access-token-setup) below). +- Save it as a GitHub Actions secret named `PATCHER_CI_TOKEN`. + +:::warn +Most of the time when users have challenges getting Patcher to work, it's because their GitHub token is not set up correctly. Be sure to follow these steps carefully, but if you'd like to validate that your GitHub token is configured correctly, consider copying and pasting our [Validate GitHub Access Token](https://github.com/gruntwork-io/patcher-action/blob/main/.github/workflows/validate-github-access.yml) workflow. +::: + +#### 2. Add the GitHub Actions workflow file +- In the repo in which you want Patcher to run, create a new file `.github/workflows/patcher-updates.yml`. +- Populate it using one of the [GitHub Actions Workflow Examples](#github-actions-workflow-examples) below. + +#### 3. Choose when to run Patcher +- There are three non-mutually exclusive options for when Patcher will run: + 1. Schedule (recommended): Run Patcher on a weekly basis (or other cadence that fits your workflow). + 2. Manual: Use `workflow_dispatch` for ad-hoc runs. This lets you trigger Patcher on demand from the GitHub UI to test changes, validate permissions, or run outside your normal schedule. + 3. New module release: Use the `repository_dispatch` trigger to run when a new module release is published. + + See the examples below for a scheduled run and a repository_dispatch trigger you can reuse. For most teams, a weekly schedule plus the ability to run manually works well. + +:::info +Almost all customers opt for the scheduled approach, with a periodic manual run. +::: + +#### 4. Run and review! +- Manually run the GitHub Actions workflow the first time to validate configuration. +- Review the generated PRs and merge as desired. + +To run manually: in your repo, go to Actions → select “Patcher Updates” → Run workflow → choose the branch → Run workflow. + +### Step-by-step setup for GitHub Enterprise users + +Many GitHub Enterprise users self-host GitHub Enterprise and wish to host all binaries internally. If you'd like to fully self-host Patcher, then you'll eventually follow the same steps as above in [Step-by-step setup for GitHub.com users](#step-by-step-setup-for-githubcom-users), however first we need to copy over some assets from Gruntwork to your environment. + +#### 1. Self-host the relevant Gruntwork repos + +The first step is to [self-host the IaC Library](../../library/guides/self-hosting.md). Even if you don't use the Gruntwork IaC Library for its modules, we can use the [repo-copier](https://github.com/gruntwork-io/repo-copier) tool in the linked page to copy private repos to any VCS system of your choice. + +To run Patcher locally, you'll need to configure repo-copier to mirror each of the following repos: + +1. https://github.com/gruntwork-io/patcher-cli +2. https://github.com/gruntwork-io/terrapatch-cli +3. https://github.com/gruntwork-io/patcher-action + +#### 2. Follow the steps above + +Follow the steps above in [Step-by-step setup for GitHub.com users](#step-by-step-setup-for-githubcom-users), but don't start testing things (step 4 above), until we've made the modifications below. + +#### 3. Specify your GitHub Enterprise settings + +Update the GitHub Action workflow as described in [For GitHub Enterprise Users](#for-github-enterprise-users) to work with your specific GitHub Enterprise setup. This will: + +- Specify your GitHub Enterprise settings +- Use your local copy of our Patcher GitHub Action (versus our version hosted at GitHub.com); remember that thanks to repo-copier, they should be exactly the same, except that yours is hosted internally. + +#### 4. Make sure your runners will work with the `patcher-action`. + +Our published `patcher-action` specifies the GitHub Actions workflow will `runs-on: ubuntu-latest`. In the GitHub Enterprise environment, this exact runner tag *might* be available for you, but if it's not, you'll either need to: + +1. Change the GitHub Actions workflow `runs-on:` value to a runner you have internaly, or +2. Tag your existing runners with `ubuntu-latest` so it gets picked up by the new GitHub Action. + +#### 5. Make your internal GitHub Action accessible + +By default, repos in GitHub Enterprise are not accessible to GitHub Actions run in other repos. We will need to change this setting by going to your internal `patcher-action` rep → click Settings → click Actions, then General → scroll down to "Access" and select the desired access level (we recommend "Accessible from repositories in the 'your_org' organization"). + +## GitHub Actions Workflow Examples + +We've included example GitHub Actions workflow files both for GitHub.com users and GitHub Enterprise users (both hosted and self-hosted). + +### For GitHub.com users + +Create a new file `.github/workflows/patcher-updates.yml` in your repository: + +```yml +name: Patcher Updates + +on: + workflow_dispatch: + repository_dispatch: + types: [new_module_release] + schedule: + # Run every Monday at 04:15 UTC + - cron: "15 4 * * 1" + +permissions: + contents: write + pull-requests: write + +jobs: + patcher-report: + runs-on: ubuntu-latest + outputs: + dependencies: ${{ steps.get-deps.outputs.dependencies }} + steps: + - uses: actions/checkout@v4 + - id: get-deps + uses: gruntwork-io/patcher-action@v2 + with: + patcher_command: report + read_token: ${{ secrets.PATCHER_CI_TOKEN }} + working_dir: ./ + + patcher-update: + needs: [patcher-report] + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + dependency: ${{ fromJson(needs.patcher-report.outputs.dependencies) }} + steps: + - uses: actions/checkout@v4 + with: + # Make sure Patcher has enough Git history to correctly determine changes + fetch-depth: 0 + + - uses: gruntwork-io/patcher-action@v2 + with: + patcher_command: update + update_token: ${{ secrets.PATCHER_CI_TOKEN }} + working_dir: ./ + dependency: ${{ matrix.dependency }} + pull_request_title: "[Patcher] Update ${{ matrix.dependency }}" + pull_request_branch: "patcher-updates-${{ matrix.dependency }}" +``` +### For GitHub Enterprise users + +Use the same GitHub Action above, but configure two jobs within the same workflow and point to your GitHub Enterprise instance and organization via the enterprise inputs. If you mirror the action internally, replace the action reference to your mirrored copy. + +Here is an example showing two jobs in a single workflow: + +```yml +jobs: + # 1) Report job (Enterprise): discover outdated dependencies + patcher-report: + runs-on: ubuntu-latest + outputs: + dependencies: ${{ steps.get-deps.outputs.dependencies }} + steps: + - uses: actions/checkout@v4 + - id: get-deps + uses: my-enterprise-org/patcher-action@v2 + with: + patcher_command: report + read_token: ${{ secrets.PATCHER_CI_TOKEN }} + github_base_url: "https://github.company.com" + github_org: "my-enterprise-org" + patcher_git_repo: "patcher-cli" + terrapatch_git_repo: "terrapatch-cli" + # Optional: defaults to github_org if not provided + terrapatch_github_org: "my-enterprise-org" + working_dir: ./ + + # 2) Update job (Enterprise): open one PR per dependency + patcher-update: + needs: [patcher-report] + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + dependency: ${{ fromJson(needs.patcher-report.outputs.dependencies) }} + steps: + - uses: actions/checkout@v4 + with: + # Ensure enough history for accurate diffs + fetch-depth: 0 + - uses: gruntwork-io/patcher-action@v2 + with: + patcher_command: update + update_token: ${{ secrets.PATCHER_CI_TOKEN }} + working_dir: ./ + dependency: ${{ matrix.dependency }} + pull_request_title: "[Patcher] Update ${{ matrix.dependency }}" + pull_request_branch: "patcher-updates-${{ matrix.dependency }}" +``` + +## GitHub Personal Access Token Setup + +You'll need to manually create a GitHub token that has permission to access the Patcher CLI and Terrapatch CLI tools. + +You could create either a [fine-grained or classic token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#types-of-personal-access-tokens), and we recommend creating a fine-grained token as follows: + +#### 1. Navigate to GitHub Settings + - Click your profile picture → **Settings** → **Developer settings** → **Personal access tokens** → **Fine-grained tokens** + +#### 2. Create New Token + - Click **Generate new token** + - Select **Fine-grained personal access token** + +#### 3. Configure Token Settings + - **Token name**: Choose a descriptive name (e.g., `PATCHER_CI_TOKEN`) + - **Expiration**: Set to **90 days** or longer based on your security policy + - **Resource owner**: Select the GitHub organization that holds the patcher-cli and terrapatch-cli tools. + - If you are accessing these via GitHub.com, the organization is `gruntwork-io`. + - If you are accessing these via self-hosted GitHub Enterprise, the organization is whatever GitHub organization has the `patcher-cli` and `terrapatch-cli` repos. + + :::warn + It's easy to not select the right organization! Be sure to select the right GitHub org -- not your username -- that actually holds the repos you're looking to access. + ::: + +#### 4. Configure Repository Access + Configure access to the following repositories: + - The **patcher-cli repository** (typically `gruntwork-io/patcher-cli` or your custom org) + - The **terrapatch-cli repository** (typically `gruntwork-io/terrapatch-cli` or your custom org) + +#### 5. Set Required Permissions + Under "Permissions", configure these **Repository permissions**: + - **Contents**: **Read** access + - **Metadata**: **Read** access + - **Actions**: **Read** access (for downloading releases) + + :::info + The GitHub Action will also need the permission to open pull requests, however it will get that from the `GITHUB_TOKEN` that is automatically generated for each GitHub Actions workflow run. Therefore this token can be read-only. + ::: + +#### 6. Generate and Store Token + - Click **Generate token** + - **Copy the token immediately** (you won't be able to see it again) + - Store it as a GitHub Actions secret named `PATCHER_CI_TOKEN` in the repository where the workflow runs. + + :::warn + Keep your token secure and never commit it to your repository. Always store it as a GitHub secret. + ::: + +Your `PATCHER_CI_TOKEN` token is now ready for use! + +## Key Configuration Options + +For more information on the configuration options in our standard GitHub Action, see the README at https://github.com/gruntwork-io/patcher-action. + +### Customization Options + +### Filtering Dependencies +If you want to exclude certain directories or files, you can add filtering: + +```yml +- uses: gruntwork-io/patcher-action@v2 + with: + patcher_command: report + read_token: ${{ secrets.PATCHER_CI_TOKEN }} + exclude_dirs: "examples/**,tests/**" + working_dir: ./ +``` + +### Update Strategies +Control how aggressively Patcher updates dependencies: + +```yml +- uses: gruntwork-io/patcher-action@v2 + with: + patcher_command: update + update_token: ${{ secrets.PATCHER_CI_TOKEN }} + update_strategy: next-safe # or "next-breaking" + # ... other parameters +``` + +### Dry Run Mode +Test your workflow without creating actual pull requests: + +```yml +- uses: gruntwork-io/patcher-action@v2 + with: + patcher_command: update + update_token: ${{ secrets.PATCHER_CI_TOKEN }} + dry_run: true + # ... other parameters +``` + +## Environment-Specific Updates + +If you need to promote updates across multiple environments (dev → stage → prod) rather than updating everything at once, see the [Setting up Promotion Workflows](/2.0/docs/patcher/guides/promotion-workflows) guide. That approach provides: + +- Environment-specific scanning and updates +- Controlled promotion between environments +- Validation gates between each environment diff --git a/docs/2.0/docs/patcher/guides/promotion-workflows.md b/docs/2.0/docs/patcher/guides/promotion-workflows.md index a2f18d488..b38b12cac 100644 --- a/docs/2.0/docs/patcher/guides/promotion-workflows.md +++ b/docs/2.0/docs/patcher/guides/promotion-workflows.md @@ -1,12 +1,25 @@ # Using Patcher Promotion Workflows +:::info +This guide covers setting up promotion workflows to move updates across multiple environments (dev → stage → prod). If you want to keep your entire codebase up to date without environment-specific promotion, see [GitHub Actions for Continuous Updates](/2.0/docs/patcher/guides/github-actions-continuous-updates). +::: + :::info As of July 2024, Gruntwork officially supports Patcher Promotion Workflows using GitHub Actions. Support for other CI systems will be introduced in future releases. ::: **Related content**: * [Concepts - Patcher Workflows](/2.0/docs/patcher/concepts/promotion-workflows) -* [Architecture - Overview](/2.0/docs/patcher/architecture) +* [Architecture - Overview](/2.0/docs/patcher/architecture) + +## When to Use Promotion Workflows + +Patcher supports two primary use cases: + +1. **Upgrading very out-of-date infrastructure code**: For codebases that haven't been updated in a while +2. **Keeping infrastructure code up to date over time**: For ongoing maintenance and security + +Patcher excels at the second use case. Promotion workflows are specifically designed for teams that need to validate changes across multiple environments before reaching production. If you simply want to keep your entire codebase current without environment-specific validation, consider using the [simpler continuous updates approach](/2.0/docs/patcher/guides/github-actions-continuous-updates) instead. ## Prerequisites ### Infrastructure as Code diff --git a/sidebars/docs.js b/sidebars/docs.js index 5831b4a4c..5b0579fd2 100644 --- a/sidebars/docs.js +++ b/sidebars/docs.js @@ -571,6 +571,11 @@ const sidebar = [ type: "category", collapsed: true, items: [ + { + label: "Ongoing Updates", + type: "doc", + id: "2.0/docs/patcher/guides/ongoing-updates", + }, { label: "Setting up Promotion Workflows", type: "doc",