Using AWS KMS in Terraform Cloud

[julian-koho]

Hello, I’m facing an issue currently where I get the following error

Error: Error getting data key: 0 successful groups required, got 0 
with data.sops_file.infra-secrets
on main.tf line 5, in data “sops_file” “infra-secrets”:
data “sops_file” “infra-secrets” {

This appears to be tied to a lack of AWS permissions to use the key. I’m not sure what I’m missing or need to do to make this function. I’ve tried the following:

1. Declare an AWS Provider with the appropriate configuration to assume directly in to the role:

provider “aws” {
   assume_role {
    role_arn     = var.role_arn
    session_name = “Terraform”
    external_id  = var.external_id
  }
  access_key = var.access_key
  secret_key = var.secret_key
  region     = var.aws_region
}

variable “access_key” {}
variable “secret_key” {}
variable “aws_region” {}
variable "role_arn" {}
variable "external_id" {}

With the variables assigned via the terraform cloud UI, this pattern works for our other use cases. In addition, because I was concerned about a provider initialization order issue, I put this AWS provider above sops, and put in the following to ensure that AWS was fully functional before Sops attempted to decrypt:

data “aws_caller_identity” “current” {}

data “sops_file” “infra-secrets” {
  source_file = “infra-secrets.yaml”
  depends_on = [
    data.aws_caller_identity.current,
  ]
}

2. Alternatively use the terraform cloud variable manager to set environment variiables:

AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
AWS_DEFAULT_REGION=...

The user here has the permission for the role specified in the associated sops file:

sops:
    kms:
    -   arn: arn:aws:kms:us-west-2:{account-id}:key/{key-id}
        created_at: ‘...’
        enc: ...
        aws_profile: “arn:aws:iam::{account-id}:role/{role-id}”

These two methods did not resolve the issue, so I feel like I'm either missing something foundational, or there is unexpected behaviour here.

My theories are around the way the provider (via sops, via the aws SDK) is acting is somehow misbehaving, but I admit that my attempts to follow the code and debug stalled out a bit in the middle of the sops decrypt package.

[multani]

For 1., this is never going to work: the sops provider (actually sops itself) doesn't know anything about the AWS provider and the settings you configure in the AWS provider are only local to that provider.

So, if the AWS provider manages to assume a new role and get temporary credentials for this role, it will not set anything "outside" of the provider and you can't reuse it for sops.

For 2., you are almost there:

• If you want to use aws_profile, I suppose it has to be a profile that you configure inside your AWS configuration file. The profile may then contain the name of the role it will try to assume.
• If you want to actually use an AWS role here, then instead of using aws_profile, simply use role: "arn:aws:iam::{account-id}:role/{role-id}" and it should do what you want to do. You can read more here: Assuming roles and using KMS in various AWS accounts