Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Adding Pod Identity / Access Entries option for Karpenter #400

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

feat: Adding Pod Identity / Access Entries option for Karpenter #400

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8c83eda
Adding Pod Identity option for Karpenter
rodrigobersa Jun 9, 2024
f5e9ed6
Pre-commit
rodrigobersa Jun 9, 2024
e589037
Reversing karpenter source to github
rodrigobersa Jun 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ module "eks" {
| <a name="module_external_secrets"></a> [external\_secrets](#module\_external\_secrets) | aws-ia/eks-blueprints-addon/aws | 1.1.1 |
| <a name="module_gatekeeper"></a> [gatekeeper](#module\_gatekeeper) | aws-ia/eks-blueprints-addon/aws | 1.1.1 |
| <a name="module_ingress_nginx"></a> [ingress\_nginx](#module\_ingress\_nginx) | aws-ia/eks-blueprints-addon/aws | 1.1.1 |
| <a name="module_karpenter"></a> [karpenter](#module\_karpenter) | aws-ia/eks-blueprints-addon/aws | 1.1.1 |
| <a name="module_karpenter"></a> [karpenter](#module\_karpenter) | ../terraform-aws-eks-blueprints-addon | n/a |
| <a name="module_karpenter_sqs"></a> [karpenter\_sqs](#module\_karpenter\_sqs) | terraform-aws-modules/sqs/aws | 4.0.1 |
| <a name="module_kube_prometheus_stack"></a> [kube\_prometheus\_stack](#module\_kube\_prometheus\_stack) | aws-ia/eks-blueprints-addon/aws | 1.1.1 |
| <a name="module_metrics_server"></a> [metrics\_server](#module\_metrics\_server) | aws-ia/eks-blueprints-addon/aws | 1.1.1 |
Expand All @@ -120,6 +120,7 @@ module "eks" {
| [aws_cloudwatch_event_target.karpenter](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) | resource |
| [aws_cloudwatch_log_group.aws_for_fluentbit](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource |
| [aws_cloudwatch_log_group.fargate_fluentbit](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource |
| [aws_eks_access_entry.node](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_entry) | resource |
| [aws_eks_addon.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) | resource |
| [aws_iam_instance_profile.karpenter](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) | resource |
| [aws_iam_policy.fargate_fluentbit](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
Expand Down Expand Up @@ -225,6 +226,7 @@ module "eks" {
| <a name="input_helm_releases"></a> [helm\_releases](#input\_helm\_releases) | A map of Helm releases to create. This provides the ability to pass in an arbitrary map of Helm chart definitions to create | `any` | `{}` | no |
| <a name="input_ingress_nginx"></a> [ingress\_nginx](#input\_ingress\_nginx) | Ingress Nginx add-on configurations | `any` | `{}` | no |
| <a name="input_karpenter"></a> [karpenter](#input\_karpenter) | Karpenter add-on configuration values | `any` | `{}` | no |
| <a name="input_karpenter_create_access_entry"></a> [karpenter\_create\_access\_entry](#input\_karpenter\_create\_access\_entry) | Determines whether to create Karpenter Access Entry for Cluster Access Management API. | `bool` | `false` | no |
| <a name="input_karpenter_enable_instance_profile_creation"></a> [karpenter\_enable\_instance\_profile\_creation](#input\_karpenter\_enable\_instance\_profile\_creation) | Determines whether Karpenter will be allowed to create the IAM instance profile (v1beta1) or if Terraform will (v1alpha1) | `bool` | `true` | no |
| <a name="input_karpenter_enable_spot_termination"></a> [karpenter\_enable\_spot\_termination](#input\_karpenter\_enable\_spot\_termination) | Determines whether to enable native node termination handling | `bool` | `true` | no |
| <a name="input_karpenter_node"></a> [karpenter\_node](#input\_karpenter\_node) | Karpenter IAM role and IAM instance profile configuration values | `any` | `{}` | no |
Expand Down
25 changes: 23 additions & 2 deletions main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2749,7 +2749,7 @@ locals {
input_karpenter_node_instance_profile_name = try(var.karpenter_node.instance_profile_name, local.karpenter_node_iam_role_name)
# This is the name passed to the Karpenter Helm chart - either the profile the module creates, or one provided by the user
output_karpenter_node_instance_profile_name = try(aws_iam_instance_profile.karpenter[0].name, var.karpenter_node.instance_profile_name, "")
karpenter_namespace = try(var.karpenter.namespace, "karpenter")
karpenter_namespace = try(var.karpenter.namespace, "kube-system")

karpenter_set = [
# TODO - remove at next breaking change
Expand Down Expand Up @@ -3006,6 +3006,21 @@ resource "aws_iam_instance_profile" "karpenter" {
tags = merge(var.tags, try(var.karpenter_node.instance_profile_tags, {}))
}

resource "aws_eks_access_entry" "node" {
count = var.enable_karpenter && var.karpenter_create_access_entry ? 1 : 0

cluster_name = var.cluster_name
principal_arn = local.create_karpenter_node_iam_role ? aws_iam_role.karpenter[0].arn : var.karpenter.node_iam_role_arn
type = "EC2_LINUX"

tags = var.tags

depends_on = [
# If we try to add this too quickly, it fails. So .... we wait
module.karpenter_sqs
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it work if karpenter_enable_spot_termination = false?
Because sqs were not created in this case.

]
}

module "karpenter" {
source = "aws-ia/eks-blueprints-addon/aws"
version = "1.1.1"
Expand All @@ -3021,7 +3036,7 @@ module "karpenter" {
namespace = local.karpenter_namespace
create_namespace = try(var.karpenter.create_namespace, true)
chart = try(var.karpenter.chart, "karpenter")
chart_version = try(var.karpenter.chart_version, "0.35.0")
chart_version = try(var.karpenter.chart_version, "0.37.0")
repository = try(var.karpenter.repository, "oci://public.ecr.aws/karpenter")
values = try(var.karpenter.values, [])

Expand Down Expand Up @@ -3058,6 +3073,12 @@ module "karpenter" {
)
set_sensitive = try(var.karpenter.set_sensitive, [])

# Pod Identity
enable_pod_identity = try(var.karpenter.enable_pod_identity, false)
create_pod_identity_association = try(var.karpenter.create_pod_identity_association, false)
cluster_name = var.cluster_name
service_account = local.karpenter_service_account_name

# IAM role for service account (IRSA)
set_irsa_names = ["serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn"]
create_role = try(var.karpenter.create_role, true)
Expand Down
18 changes: 14 additions & 4 deletions tests/complete/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,8 @@ module "eks_blueprints_addons" {
vpc-cni = {
most_recent = true
}
kube-proxy = {}
kube-proxy = {}
eks-pod-identity-agent = {}
adot = {
most_recent = true
service_account_role_arn = module.adot_irsa.iam_role_arn
Expand Down Expand Up @@ -162,8 +163,11 @@ module "eks_blueprints_addons" {

enable_karpenter = true
karpenter_enable_instance_profile_creation = true
# ECR login required
karpenter_create_access_entry = true
karpenter = {
enable_pod_identity = true
create_pod_identity_association = true
# ECR login required
repository_username = data.aws_ecrpublic_authorization_token.token.user_name
repository_password = data.aws_ecrpublic_authorization_token.token.password
}
Expand Down Expand Up @@ -266,12 +270,17 @@ module "eks" {
instance_type = "m5.large"

min_size = 1
max_size = 10
max_size = 5
desired_size = 1
}
}

tags = local.tags
tags = merge(local.tags, {
# NOTE - if creating multiple security groups with this module, only tag the
# security group that Karpenter should utilize with the following tag
# (i.e. - at most, only one security group should have this tag in your account)
"karpenter.sh/discovery" = local.name
})
}

################################################################################
Expand All @@ -298,6 +307,7 @@ module "vpc" {

private_subnet_tags = {
"kubernetes.io/role/internal-elb" = 1
"karpenter.sh/discovery" = local.name
}

tags = local.tags
Expand Down
6 changes: 6 additions & 0 deletions variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -454,6 +454,12 @@ variable "karpenter_enable_instance_profile_creation" {
default = true
}

variable "karpenter_create_access_entry" {
description = "Determines whether to create Karpenter Access Entry for Cluster Access Management API."
type = bool
default = false
}

variable "karpenter_sqs" {
description = "Karpenter SQS queue for native node termination handling configuration values"
type = any
Expand Down
Loading