Cost estimates for ECS service not shown

[la-kurt]

For some reason, infracost does not compute our ECS fargate services in the cost breakdown. Might it be because we use for_each? Cause I noticed it works fine in one of our projects that don't use for_each for the ecs_service module.

Code snippet

module "ecs_taskdef" {
  source   = "../../shared/ecs_taskdef"
  for_each = module.ecs_taskdef_config_gen.config
  ...
}

module "ecs_service" {
  source   = "../../shared/ecs_service"
  for_each = module.ecs_taskdef

  cluster_id                = module.ecs_cluster.cluster_id
  taskdef_arn               = each.value.arn
  subnet_ids                = module.network.subnet_ids
  target_group_arn          = module.lb.target_groups[each.key].arn
  ...
}

Partial output of infracost breakdown --path . --show-skipped

 OVERALL TOTAL                                                                                                     $52.31 
──────────────────────────────────
86 cloud resources were detected:
∙ 20 were estimated, 19 of which include usage-based costs, see https://infracost.io/usage-file
∙ 66 were free:
  ∙ 10 x aws_iam_role
  ∙ 10 x aws_lb_listener_rule
  ∙ 10 x aws_lb_target_group
  ∙ 4 x aws_iam_policy
  ∙ 4 x aws_iam_role_policy_attachment
  ∙ 3 x aws_s3_bucket_versioning
  ∙ 2 x aws_lb_listener
  ∙ 2 x aws_route_table_association
  ∙ 2 x aws_s3_bucket_ownership_controls
  ∙ 2 x aws_s3_bucket_policy
  ∙ 2 x aws_s3_bucket_public_access_block
  ∙ 2 x aws_security_group
  ∙ 2 x aws_subnet
  ∙ 1 x aws_acm_certificate
  ∙ 1 x aws_acm_certificate_validation
  ∙ 1 x aws_db_subnet_group
  ∙ 1 x aws_ecs_cluster
  ∙ 1 x aws_ecs_cluster_capacity_providers
  ∙ 1 x aws_ecs_task_definition
  ∙ 1 x aws_internet_gateway
  ∙ 1 x aws_route_table
  ∙ 1 x aws_s3_bucket_server_side_encryption_configuration
  ∙ 1 x aws_secretsmanager_secret_version
  ∙ 1 x aws_vpc

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project                                              ┃ Monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ atomio-group/infra-scripts/environments/some-env     ┃ $52          ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛

No ecs_service can be found in the list of computed resources, as well as in the skipped resources list.
However, when I try to run it against a plan json, infracost is able to show a more accurate cost estimation.

Partial output of infracost breakdown --path test.json --show-skipped

 module.ecs_service["a"].aws_ecs_service.main                                                                                 
 ├─ Per GB per hour                                                                            1  GB                             $3.73 
 └─ Per vCPU per hour                                                                        0.5  CPU                           $16.99 
                                                                                                                                       
 module.ecs_service["b"].aws_ecs_service.main                                                                                  
 ├─ Per GB per hour                                                                            1  GB                             $3.73 
 └─ Per vCPU per hour                                                                        0.5  CPU                           $16.99 
                                                                                                                                       
 module.ecs_service["c"].aws_ecs_service.main                                                                                  
...

 OVERALL TOTAL                                                                                                                 $259.55 
──────────────────────────────────
132 cloud resources were detected:
∙ 39 were estimated, 27 of which include usage-based costs, see https://infracost.io/usage-file
∙ 93 were free:
  ∙ 13 x aws_iam_policy
  ∙ 13 x aws_iam_role_policy_attachment
  ∙ 10 x aws_ecs_task_definition
  ∙ 10 x aws_iam_role
  ∙ 10 x aws_lb_listener_rule
  ∙ 10 x aws_lb_target_group
  ∙ 3 x aws_s3_bucket_versioning
  ∙ 2 x aws_lb_listener
  ∙ 2 x aws_route_table_association
  ∙ 2 x aws_s3_bucket_ownership_controls
  ∙ 2 x aws_s3_bucket_policy
  ∙ 2 x aws_s3_bucket_public_access_block
  ∙ 2 x aws_security_group
  ∙ 2 x aws_subnet
  ∙ 1 x aws_acm_certificate
  ∙ 1 x aws_acm_certificate_validation
  ∙ 1 x aws_db_subnet_group
  ∙ 1 x aws_ecs_cluster
  ∙ 1 x aws_ecs_cluster_capacity_providers
  ∙ 1 x aws_internet_gateway
  ∙ 1 x aws_route_table
  ∙ 1 x aws_s3_bucket_server_side_encryption_configuration
  ∙ 1 x aws_secretsmanager_secret_version
  ∙ 1 x aws_vpc

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project                                                        ┃ Monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ atomio-group/infra-scripts/environments/some-env/test.json     ┃ $260         ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛

Notice the disparity in # of detected resources, as well as total cost.

[aliscott]

Thanks @la-kurt. For this case we'd need to see the following additional code to be able to debug the cause:

1. The block of code where the aws_ecs_service.main resource is defined
2. The block of code showing the module.ecs_taskdef_config_gen.config output, and any variables/locals that module uses.

[la-kurt]

@aliscott

ecs_service module (main.tf)

resource "aws_ecs_service" "main" {
  name                               = "${var.name}-${var.code}-service"
  cluster                            = var.cluster_id
  task_definition                    = var.taskdef_arn
  desired_count                      = var.desired_count
  deployment_minimum_healthy_percent = var.min_health
  deployment_maximum_percent         = var.max_health
  scheduling_strategy                = "REPLICA"
  enable_execute_command             = true
  health_check_grace_period_seconds  = var.health_check_grace_period

  network_configuration {
    security_groups  = var.security_group_ids
    subnets          = var.subnet_ids
    assign_public_ip = true
  }

  load_balancer {
    target_group_arn = var.target_group_arn
    container_name   = "${var.name}-container"
    container_port   = 80
  }

  lifecycle {
    ignore_changes = [desired_count, capacity_provider_strategy]
  }

  tags = merge(var.tags, {})
}

ecs_taskdef_config_gen

This module is just where we generate values for the task def configuration, such as env vars, image name, cpu/mem config, etc. just so our task def module wouldn't be too big. Here we read from a couple of data sources, such as aws_secretsmanager_secret_versions, and generate other secrets using the random_password resource.

outputs.tf

output "config" {
  value = local.config
}

main.tf

...
locals {
...
  env = {
  for k, v in var.servers : k => {
    ... # this is just where we generate env vars for the task def, for example:
    PROJECT = var.project
    ENABLE_TRACING = var.enable_tracing
  }
  }

  settings = {
    for k, v in var.servers : k => merge(
      var.default_settings,
      lookup(v, "settings", {}),
      {
        execution_role_arn = data.aws_iam_role.ecs_task_execution_role.arn
        enable_apm         = lookup(v, "enable_apm", false)
      }
    )
  }

  config = { for k, v in var.servers : k => {
    settings = local.settings[k]
    image    = "${data.aws_ecr_repository.repos[k].repository_url}:${v.version}"
    env      = local.env[k]
  } 
}

Here's how the task def module uses the config_gen module

module "ecs_taskdef" {
  source   = "../../shared/ecs_taskdef"
  for_each = module.ecs_taskdef_config_gen.config

  name     = each.key
  code     = var.code
  settings = each.value.settings
  image    = each.value.image
  env      = each.value.env
  region   = var.region
...

ecs_taskdef module

...
resource "aws_ecs_task_definition" "taskdef" {
  family                   = "${var.name}-${var.code}-task"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = var.settings.cpu
  memory                   = var.settings.memory
  execution_role_arn       = var.settings.execution_role_arn
  task_role_arn            = var.task_role.arn

  container_definitions = jsonencode(concat(
    [{
      name        = "${var.name}-container"
      image       = var.image
      essential   = true
      environment = [for k, v in var.env : { name = k, value = v }]
      portMappings = [{
        protocol      = "tcp"
        containerPort = 80
        hostPort      = 80
      }]
      mountPoints      = [for k, v in var.efs_ids : { sourceVolume = v.name, containerPath = v.mount_path }]
      logConfiguration = lookup(local.log_driver_configs, var.log_provider, local.awslogs_config)
      linuxParameters = {
        initProcessEnabled = true
      }
    }],
    var.enable_firelens ? [local.firelens_container] : [],
    var.enable_apm ? [lookup(local.apm_sidecar_containers, var.apm_provider, {})] : []
  ))

  dynamic "volume" {
    for_each = var.efs_ids
    content {
      name = volume.value.name

      efs_volume_configuration {
        file_system_id     = volume.key
        root_directory     = volume.value.root_dir
        transit_encryption = "ENABLED"
      }
    }
  }

  tags = merge(var.tags, {})
}

I'm not sure if I could share the variables we use here publicly, but do let me know if the code snippets here aren't enough. Thanks

[aliscott]

Thanks @la-kurt. I've not yet been able to reproduce it. Are you able to provide the code for the var.servers in the ecs_taskdef_config_gen module.

Also can I confirm what version of Infracost you are using?

[la-kurt]

@aliscott

var.servers

servers = {
  a = {
    version    = "latest"
    bucket     = true
    bucket_acl = "public-read"
    enable_apm = true

    env = {
      INVITE_DEFAULT_EXPIRY = "1209600"
      DB_LOG_LEVEL          = "info"
      ENABLE_V3             = "false"
    }
  }
  b = {
    version    = "0.8.4-8"
    bucket     = true
    bucket_acl = "public-read"

    env = {
      ENABLE_V2           = "true"
    }
  }
  c = {
    version         = "latest"
    image_base_name = "c"

    env = {
    }
  }
   ...
}

❯ infracost -v
Infracost v0.10.24

[la-kurt]

@aliscott another thing that could be causing this: our task def container definitions are marked as sensitive because we use sensitive values in the environment variables.

[aliscott]

Thanks @la-kurt. From what I can tell so far is Infracost is unable to tell that the aws_ecs_service.main resource is FARGATE as opposed to EC2.

Infracost does this by checking the following:

1. If the aws_ecs_service resource has launch_type = "FARGATE"
2. If the aws_ecs_service resource has a capacity_provider_strategy for Fargate.
3. If the aws_ecs_cluster resource has a default_capacity_provider_strategy or capacity_providers for Fargate. (for users using the AWS Terraform provider v4.
4. If the aws_ecs_cluster_capacity_providers has a default_capacity_provider_strategy or capacity_providers for Fargate. (for users using the AWS Terraform provider v5.

So my thoughts so far are since 1 or 2 aren't specified in your Terraform code, then Infracost is trying to work out if it's a Fargate service by checking the aws_ecs_cluster resource that's referenced in the service, and for some reason it can't evaluate that reference.

Are you able to share the code for the aws_ecs_cluster resource and the output that provides the module.ecs_cluster.cluster_id output?

[la-kurt]

sure @aliscott

aws_ecs_cluster + capacity provider

resource "aws_ecs_cluster" "main" {
  name = "${var.code}-cluster"

  setting {
    name  = "containerInsights"
    value = "enabled"
  }

  configuration {
    execute_command_configuration {
      logging = "OVERRIDE"
      log_configuration {
        cloud_watch_log_group_name = aws_cloudwatch_log_group.ecs_exec.name
      }
    }
  }

  tags = var.tags
}

resource "aws_ecs_cluster_capacity_providers" "capacity_providers" {
  cluster_name = aws_ecs_cluster.main.name

  capacity_providers = ["FARGATE", "FARGATE_SPOT"]
  default_capacity_provider_strategy {
    capacity_provider = var.production ? "FARGATE" : "FARGATE_SPOT"
    weight            = 1
    base              = 0
  }
}

cluster_id output

output "cluster_id" {
  value = aws_ecs_cluster.main.id
}

[aliscott]

Thanks @la-kurt. I've managed to reproduce this.

There seems to be two issues. I'm resolving one as part of #2606. The other will probably be a bit more complicated to resolve. I've created another issue for it here: #2605.