Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[bug] aws-region config parameter in workflow, not used with project level roles causes GitHub Action failure #1469

Closed
ben-of-codecraft opened this issue May 15, 2024 · 5 comments · Fixed by #1492
Closed

[bug] aws-region config parameter in workflow, not used with project level roles causes GitHub Action failure #1469

ben-of-codecraft opened this issue May 15, 2024 · 5 comments · Fixed by #1492

Comments

@ben-of-codecraft
Copy link
Contributor

ben-of-codecraft commented May 15, 2024 •
edited

When using the following set up:

  • no-backend
  • multiple projects
  • no assume role in workflow

Specifying the region in the digger action does not work and will not be used when populateKeys method is called to get AWS credentials based on the project role. This results in the following error:

Failed to get keys from role: populateKeys: Could not retrieve keys from provider failed to retrieve credentials, operation error STS: AssumeRoleWithWebIdentity, failed to resolve service endpoint, endpoint rule error, Invalid Configuration: Missing Region
failed to fetch AWS keys, Failed to get (state) keys from role: populateKeys: Could not retrieve keys from provider failed to retrieve credentials, operation error STS: AssumeRoleWithWebIdentity, failed to resolve service endpoint, endpoint rule error, Invalid Configuration: Missing Region

digger_workflow.yml

name: Terraform Deployment

on:
  pull_request:
    branches: [ "main" ]
    types: [ opened, synchronize ]
  issue_comment:
    types: [created]
  workflow_dispatch:

jobs:
  terraform-deploy:
    runs-on: ubuntu-latest
    
    permissions:
      contents: write      # required to merge PRs
      actions: write       # required for plan persistence
      id-token: write      # required for workload-identity-federation
      pull-requests: write # required to post PR comments
      statuses: write      # required to validate combined PR status

    steps:
      - uses: actions/checkout@v4
      - name: Setup GitHub Env
        run: |
          echo '${{secrets.TF_GIT_CONFIG}}' > ~/.gitconfig
      - name: digger run
        uses: diggerhq/[email protected]
        with:      
          setup-terraform: true  
          disable-locking: true                                    
          aws-region: us-west-2   
          no-backend: true
                                   
        env:
          GITHUB_CONTEXT: ${{ toJson(github) }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

With multi-account digger.yml:

telemetry: false
traverse_to_nested_projects: true

projects:
    - name: aws-base-dev
      dir: dev
      include_patterns: ["./modules/**"]
      workflow_file: digger_workflow.yml
      aws_role_to_assume:
        state: arn:aws:iam::xxx:role/xxx-oidc-role
        command: arn:aws:iam::xxx:role/xxx-oidc-role


    - name: aws-base-prod
      dir: prod
      include_patterns: ["./modules/**"]
      workflow_file: digger_workflow.yml
      aws_role_to_assume:
        state: arn:aws:iam::xxx:role/xxx-oidc-role
        command: arn:aws:iam::xxx:role/xxx-oidc-role
     ```
@ben-of-codecraft
Copy link
Contributor Author

The current workaround is adding the AWS_REGION to the environment variable for the workflow so it is picked up by the credential call

See below for an example:

jobs:
  terraform-deploy:
    runs-on: ubuntu-latest
    env:
        AWS_REGION: 'us-west-2'
    permissions:
      contents: write      # required to merge PRs
      actions: write       # required for plan persistence
      id-token: write      # required for workload-identity-federation
      pull-requests: write # required to post PR comments
      statuses: write      # required to validate combined PR status

    steps:
      - uses: actions/checkout@v4
      - name: Setup GitHub Env
        run: |
          echo '${{secrets.TF_GIT_CONFIG}}' > ~/.gitconfig
      - name: digger run
        uses: diggerhq/[email protected]
        with:
          setup-terraform: true  
          disable-locking: true                                    
          aws-region: us-west-2   
          no-backend: true
                                   
        env:
          GITHUB_CONTEXT: ${{ toJson(github) }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    ```

@motatoes
Copy link
Contributor

I think I have seen this behaviour before. Not sure why AWS sts client does not set a default region while setting a role. Need to dig further into it.

@ben-of-codecraft
Copy link
Contributor Author

One thing I am testing is this in the digger.yml to see if it fills the need without having to change the workflow file. Because if I update the workflow file then I lose the ability to have cross-region support out of the same repo. Where if I can assign it as an env variable in the workflow I can make magic happen

telemetry: false
traverse_to_nested_projects: true

projects:
    - name: "{{github.org}}-{{github.repo}}-dev"
      dir: dev
      include_patterns: [
        "./modules/**",
        "./config/dev/**",
      ]
      workflow_file: digger_workflow.yml
      aws_role_to_assume:
        state: {{output.stsStateRoleDev}}
        command: {{output.stsCommandRoleDev}}              

    - name: "{{github.org}}-{{github.repo}}-prod"
      dir: prod
      include_patterns: [
        "./modules/**",
        "./config/prod/**",
      ]
      workflow_file: digger_workflow.yml
      aws_role_to_assume:
        state: {{output.stsStateRoleProd}}
        command: {{output.stsCommandRoleProd}}   

workflows:
    default:
        env_vars:
            state: 
                - name: AWS_REGION
                  value: {{config.region}}

            commands:
                - name: AWS_REGION
                  value: {{config.region}}

@ben-of-codecraft
Copy link
Contributor Author

I finally got around to testing this today and using this technique to set the region did not work:

workflows:
  default:
    env_vars:
      state:
        - name: AWS_REGION
          value: "us-west-2"
      commands:
        - name: AWS_REGION
          value: "us-west-2"

@ben-of-codecraft
Copy link
Contributor Author

ben-of-codecraft commented May 20, 2024 •
edited

I think I found the problem, when examining an environment with the workflow vars set when I looked at initialize digger config:

I see this:

Run diggerhq/[email protected]
  with:
    setup-terraform: true
    disable-locking: true
    no-backend: true
    setup-aws: false
    aws-region: us-east-1
    setup-google-cloud: false
    setup-azure: false
    setup-terragrunt: false
    setup-opentofu: false
    terragrunt-version: v0.[5]
    terraform-version: v1.5.5
    configure-checkout: true

However, I am passing in us-west-2 and the credentials I have created are not in that region. So maybe this is part of the problem, I am digging in further to see if I can figure anything out.

This was referenced May 21, 2024
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@motatoes @ben-of-codecraft