Skip to content

2025 06 03 parallel credentials #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

2025 06 03 parallel credentials #11

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
457ee69
Implement parallel token fetching with TypeError retry logic
Resonance1584 Jun 3, 2025
9d29f27
Implement parallel token fetching with TypeError retry logic
Resonance1584 Jun 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 70 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,32 +11,89 @@ When installed on a repository or organization, it allows for usage of [Gruntwor

Direct installation of this action by third parties isn't recommended. The Gruntwork maintainers will set up the integration in the [Pipelines Workflows](https://github.com/gruntwork-io/pipelines-workflows) repository.

To understand how this action can be customized, know that an optional `FALLBACK_TOKEN` can be provided to the action to replace integration with the Gruntwork.io app.
This action can fetch multiple tokens in parallel, improving workflow performance. You can provide multiple token requests as a JSON array, and each token request can have its own fallback token.

This allows customers to opt-out of using the Gruntwork.io app, and instead use a static PAT that they provision using the guidance [here](https://docs.gruntwork.io/pipelines/security/machine-users).
This allows customers to opt-out of using the Gruntwork.io app for specific tokens, and instead use static PATs that they provision using the guidance [here](https://docs.gruntwork.io/pipelines/security/machine-users).

## How it Works

At a high level, this action does the following:

1. Uses the GitHub `@actions/core` library to fetch a JWT authenticating a workflow as being run within the context of a particular repository using GitHub servers.
2. Uses that JWT to attempt to fetch a token from Gruntwork servers that authenticates the workflow as being allowed to access resources Gruntwork allows, and resources that the Gruntwork.io app has been granted access to.
1. If the token is successfully fetched, it is set as an output variable for the workflow to use in subsequent steps.
2. If the token cannot be fetched, the action will attempt to use the `FALLBACK_TOKEN` provided as an input to the action.
3. If the `FALLBACK_TOKEN` is not provided, the action will fail the workflow.
2. Uses that JWT to attempt to fetch tokens from Gruntwork servers that authenticate the workflow as being allowed to access resources Gruntwork allows, and resources that the Gruntwork.io app has been granted access to.
1. If the tokens are successfully fetched, they are set as output variables for the workflow to use in subsequent steps.
2. If a token cannot be fetched, the action will attempt to use the `fallback_secret` provided for that specific token.
3. If a `fallback_secret` is not provided for a required token, the action will fail the workflow.
3. All token requests are processed in parallel for improved performance.

As a consequence of running this action, the workflow will have a token that can be used to access relevant resources in GitHub, scoped to the permissions required for particular steps in the workflow.
As a consequence of running this action, the workflow will have tokens that can be used to access relevant resources in GitHub, scoped to the permissions required for particular steps in the workflow.

e.g.
## Retry Logic and Resilience

This action includes robust retry logic to handle transient network issues:

- **Network Errors**: Automatically retries on `TypeError` exceptions thrown by `fetch()` due to network connectivity issues
- **Server Errors**: Retries on HTTP 5xx server errors and 429 rate limiting responses
- **Retry Strategy**: Up to 3 attempts with random backoff (0-3 seconds) between retries
- **Parallel Resilience**: Each token fetch has independent retry logic, so one token's network issues don't affect others
- **Graceful Fallback**: If all retries fail, the action falls back to the provided `fallback_secret` for that specific token

This ensures workflows remain stable even in environments with intermittent network connectivity or temporary service disruptions.

## Examples

### Single Token (Legacy Compatible)
```yml
- name: Fetch Multiple Tokens
id: pipelines-tokens
uses: gruntwork-io/pipelines-credentials@main
with:
PIPELINES_TOKEN_PATHS: |
[
{
"name": "gruntwork_read",
"path": "pipelines-read/gruntwork-io",
"fallback_secret": "${{ secrets.PIPELINES_READ_TOKEN }}"
}
]
```

### Multiple Tokens
```yml
- name: Fetch Gruntwork Read Token
id: pipelines-gruntwork-read-token
- name: Fetch Multiple Tokens
id: pipelines-tokens
uses: gruntwork-io/pipelines-credentials@main
with:
PIPELINES_TOKEN_PATH: "pipelines-read/gruntwork-io"
FALLBACK_TOKEN: ${{ secrets.PIPELINES_READ_TOKEN }}
PIPELINES_TOKEN_PATHS: |
[
{
"name": "gruntwork_read",
"path": "pipelines-read/gruntwork-io",
"fallback_secret": "${{ secrets.PIPELINES_READ_TOKEN }}"
},
{
"name": "customer_org_read",
"path": "pipelines-read/${{ github.repository_owner }}",
"fallback_secret": "${{ secrets.PIPELINES_READ_TOKEN }}"
},
{
"name": "pr_create",
"path": "propose-infra-change/${{ github.repository_owner }}",
"fallback_secret": "${{ secrets.PR_CREATE_TOKEN }}"
}
]
```

### Using the Tokens
```yml
- name: Use tokens
env:
GRUNTWORK_TOKEN: ${{ fromJSON(steps.pipelines-tokens.outputs.PIPELINES_TOKENS).gruntwork_read }}
CUSTOMER_TOKEN: ${{ fromJSON(steps.pipelines-tokens.outputs.PIPELINES_TOKENS).customer_org_read }}
PR_TOKEN: ${{ fromJSON(steps.pipelines-tokens.outputs.PIPELINES_TOKENS).pr_create }}
run: |
echo "Using tokens for various operations"
```

Will result in the workflow being able to access a token at `${{ steps.pipelines-gruntwork-read-token.outputs.PIPELINES_TOKEN }}` that can be used to read relevant resources in the `gruntwork-io` organization, scoped to the ability to clone select repositories and fetch the [pipelines-cli](https://github.com/gruntwork-io/pipelines-cli) binary.
The workflow will have access to tokens at `${{ fromJSON(steps.pipelines-tokens.outputs.PIPELINES_TOKENS).token_name }}` where `token_name` corresponds to the `name` field in the token request configuration.

195 changes: 147 additions & 48 deletions action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,100 +1,199 @@
name: Pipelines Credentials
description: Fetch Pipelines Credentials
inputs:
PIPELINES_TOKEN_PATH:
required: true
FALLBACK_TOKEN:
PIPELINES_TOKEN_PATHS:
required: true
description: 'JSON array of token path objects with "name", "path", and "fallback_secret" fields'
api_base_url:
default: "https://api.prod.app.gruntwork.io/api/v1"
outputs:
PIPELINES_TOKEN:
value: ${{ steps.get_token.outputs.PIPELINES_TOKEN }}
PIPELINES_TOKENS:
value: ${{ steps.get_tokens.outputs.PIPELINES_TOKENS }}

runs:
using: composite
steps:
- name: Fetch Pipelines Token
id: get_token
- name: Fetch Pipelines Tokens
id: get_tokens
uses: actions/github-script@v7
env:
FALLBACK_TOKEN: ${{ inputs.FALLBACK_TOKEN }}
PIPELINES_TOKEN_PATH: ${{ inputs.PIPELINES_TOKEN_PATH }}
PIPELINES_TOKEN_PATHS: ${{ inputs.PIPELINES_TOKEN_PATHS }}
API_BASE_URL: ${{ inputs.api_base_url }}
with:
script: |
try {
const aud = "https://api.prod.app.gruntwork.io"
const apiBaseURL = process.env.API_BASE_URL
const tokenRequests = JSON.parse(process.env.PIPELINES_TOKEN_PATHS)

console.log(`API Base URL: ${apiBaseURL}`)
console.log(`Token requests: ${JSON.stringify(tokenRequests, null, 2)}`)

// Validate token requests
for (const request of tokenRequests) {
if (!request.name) {
throw new Error(`Token request missing required 'name' field: ${JSON.stringify(request)}`)
}
if (!request.path) {
throw new Error(`Token request missing required 'path' field: ${JSON.stringify(request)}`)
}
}

const idToken = await core.getIDToken(aud)

const isRetryableError = (response) => {
return response.status >= 500 || response.status === 429
const isRetryableError = (error, response = null) => {
// Retry on network errors (TypeError from fetch)
if (error instanceof TypeError) {
return true
}
// Retry on server errors or rate limiting
if (response && (response.status >= 500 || response.status === 429)) {
return true
}
return false
}

const loginWithRetries = async (tries) => {
const providerTokenResponse = await fetch(`${apiBaseURL}/tokens/auth/login`, {
method: "POST",
headers: {
"Authorization": `Bearer ${idToken}`
}
})
const loginURL = `${apiBaseURL}/tokens/auth/login`
try {
console.log(`Attempting to login to: ${loginURL}`)
const providerTokenResponse = await fetch(loginURL, {
method: "POST",
headers: {
"Authorization": `Bearer ${idToken}`
}
})

if (providerTokenResponse.ok) {
return providerTokenResponse
} else {
if (tries > 0 && isRetryableError(null, providerTokenResponse)) {
console.log(`Failed to get provider token from ${loginURL}: ${providerTokenResponse.status} ${providerTokenResponse.statusText}. Retrying...`)

if (providerTokenResponse.ok) {
return providerTokenResponse
} else {
if (tries > 0 && isRetryableError(providerTokenResponse)) {
console.log(`Failed to get provider token: ${providerTokenResponse.status} ${providerTokenResponse.statusText}. Retrying...`)
// Random backoff between 0 and 3 seconds
await new Promise(resolve => setTimeout(resolve, Math.floor(Math.random() * 3000)))

return loginWithRetries(tries - 1)
} else {
return providerTokenResponse
}
}
} catch (error) {
if (tries > 0 && isRetryableError(error)) {
console.log(`Network error getting provider token from ${loginURL}: ${error.message}. Retrying...`)

// Random backoff between 0 and 3 seconds
await new Promise(resolve => setTimeout(resolve, Math.floor(Math.random() * 3000)))

return loginWithRetries(tries - 1)
} else {
return providerTokenResponse
throw error
}
}
}

const providerTokenResponse = await loginWithRetries(3)
let providerToken = null

if (providerTokenResponse.ok) {
const providerTokenJson = await providerTokenResponse.json()
const pipelinesTokenResponse = await fetch(`${apiBaseURL}/tokens/pat/${process.env.PIPELINES_TOKEN_PATH}`, {
method: "GET",
headers: {
"Authorization": `Bearer ${providerTokenJson.token}`
providerToken = providerTokenJson.token
console.log("Successfully obtained provider token")
} else {
console.log(`Failed to get provider token: ${providerTokenResponse.status} ${providerTokenResponse.statusText}`)
}

// Fetch all tokens in parallel
const fetchToken = async (tokenRequest) => {
const { name, path, fallback_secret } = tokenRequest

console.log(`Processing token request for '${name}' with path '${path}'`)

const fetchTokenWithRetries = async (tries) => {
const tokenURL = `${apiBaseURL}/tokens/pat/${path}`
try {
console.log(`Attempting to fetch ${name} token from: ${tokenURL}`)
const pipelinesTokenResponse = await fetch(tokenURL, {
method: "GET",
headers: {
"Authorization": `Bearer ${providerToken}`
}
})

if (pipelinesTokenResponse.ok) {
const pipelinesTokenJson = await pipelinesTokenResponse.json()
console.log(`Successfully fetched ${name} token from GitHubApp`)
return {
name,
token: pipelinesTokenJson.token,
source: 'github_app'
}
} else {
if (tries > 0 && isRetryableError(null, pipelinesTokenResponse)) {
console.log(`Failed to get ${name} token from ${tokenURL}: ${pipelinesTokenResponse.status} ${pipelinesTokenResponse.statusText}. Retrying...`)

// Random backoff between 0 and 3 seconds
await new Promise(resolve => setTimeout(resolve, Math.floor(Math.random() * 3000)))

return fetchTokenWithRetries(tries - 1)
} else {
console.log(`Failed to get ${name} token from ${tokenURL}: ${pipelinesTokenResponse.status} ${pipelinesTokenResponse.statusText}`)
return null
}
}
} catch (error) {
if (tries > 0 && isRetryableError(error)) {
console.log(`Network error getting ${name} token from ${tokenURL}: ${error.message}. Retrying...`)

// Random backoff between 0 and 3 seconds
await new Promise(resolve => setTimeout(resolve, Math.floor(Math.random() * 3000)))

return fetchTokenWithRetries(tries - 1)
} else {
console.log(`Failed to get ${name} token from ${tokenURL}: ${error}`)
return null
}
}
}

if (providerToken) {
const result = await fetchTokenWithRetries(3)
if (result) {
return result
}
})
}

if (pipelinesTokenResponse.ok) {
const pipelinesTokenJson = await pipelinesTokenResponse.json()
console.log("Setting PIPELINES_TOKEN to GitHubApp token")
core.setOutput('PIPELINES_TOKEN', pipelinesTokenJson.token)
// Fall back to the provided fallback token
console.log(`Using fallback token for ${name}`)

return
} else {
console.log(`Failed to get pipelines token: ${pipelinesTokenResponse.status} ${pipelinesTokenResponse.statusText}`)
if (!fallback_secret) {
const errMsg = `The pipelines-credentials GitHub Action was unable to dynamically fetch credentials for '${name}' using the Gruntwork.io GitHub App, and no fallback token was provided. Ensure that the Gruntwork.io app is installed, or that a fallback token is provided for '${name}'.`
core.setFailed(errMsg)
throw new Error(errMsg)
}

} else {
console.log(`Failed to get provider token: ${providerTokenResponse.status} ${providerTokenResponse.statusText}`)
return {
name,
token: fallback_secret.trim(),
source: 'fallback'
}
}

} catch (error) {
console.log(`Failed to get pipelines token: ${error}`)
}
// Fetch all tokens in parallel
const tokenResults = await Promise.all(tokenRequests.map(fetchToken))

console.log("Setting PIPELINES_TOKEN to fallback token")
// Convert to object with token names as keys
const tokensOutput = {}
tokenResults.forEach(result => {
tokensOutput[result.name] = result.token
})

if (! process.env.FALLBACK_TOKEN) {
const errMsg = "The pipelines-credentials GitHub Action was unable to dynamically fetch credentials using the Gruntwork.io GitHub App, and no FALLBACK_TOKEN was provided. Ensure that the Gruntwork.io app is installed, or that a FALLBACK_TOKEN is provided."
core.setOutput('PIPELINES_TOKENS', JSON.stringify(tokensOutput))
console.log(`Successfully fetched ${tokenResults.length} tokens: ${tokenResults.map(r => `${r.name} (${r.source})`).join(', ')}`)

core.setFailed(errMsg)

throw new Error(errMsg)
} catch (error) {
console.log(`Failed to get pipelines tokens: ${error}`)
core.setFailed(error.message)
throw error
}

core.setOutput('PIPELINES_TOKEN', process.env.FALLBACK_TOKEN.trim())