Skip to content

Jenkins X 3.x Infrastructure Git Template for Terraform and Google Cloud Platform for managing cloud resources

License

Notifications You must be signed in to change notification settings

jx3-gitops-repositories/jx3-terraform-gke

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Google Terraform Quickstart template

Use this template to easily create a new Git Repository for managing Jenkins X cloud infrastructure needs.

We recommend using Terraform to manange the infrastructure needed to run Jenkins X. There are a number of cloud resources which may need to be created such as:

  • Kubernetes cluster
  • Storage buckets for long term storage of logs
  • IAM Bindings to manage permissions for applications using cloud resources

Jenkins X likes to use GitOps to manage the lifecycle of both infrastructure and cluster resources. This requires two Git Repositories to achieve this:

  • Infrastructure git repository: infrastructure resources will be managed by Terraform and will keep resources in sync.
  • Cluster git repository: the Kubernetes specific cluster resources will be managed by Jenkins X and keep resources in sync.

Prerequisites

gcloud auth application-default login
  • Google cloud container services has to be enabled
gcloud services enable container.googleapis.com

Git repositories

We use 2 git repositories:

  • Infrastructure git repository for the Terraform configuration to setup/upgrade/modify your cloud infrastructure (kubernetes cluster, IAM accounts, IAM roles, buckets etc)
  • Cluster git repository to contain the helmfile.yaml file to define the helm charts to deploy in your cluster

We use separate git repositories since the infrastructure tends to change rarely; whereas the cluster git repository changes alot (every time you add a new quickstart, import a project, release a project etc).

Often different teams look after infrastructure; or you may use tools like Terraform Cloud to process changes to infrastructure & review changes to infrastructure more closely than promotion of applications.

Getting started

Note: remember to create the Git repositories below in your Git Organisation rather than your personal Git account else this will lead to issues with ChatOps and automated registering of webhooks.

  1. Create an Infrastructure git repo from this GitHub Template https://github.com/jx3-gitops-repositories/jx3-terraform-gke/generate. If you are following from the Jenkins X website then this initial step may have already happened.

    Note: Ensure Owner is the name of the Git Organisation that will hold the GitOps repositories used for Jenkins X.

  2. Create a Cluster git repository; choosing your desired secrets store, either Google Secret Manager or Vault:

    Note: Ensure Owner is the name of the Git Organisation that will hold the GitOps repositories used for Jenkins X.

  3. You need to configure the git URL of your Cluster git repository (which contains helmfile.yaml) into the Infrastructure git repository (which contains main.tf).

So from inside a git clone of the Infrastructure git repository (which already has the files main.tf and values.auto.tfvars inside) you need to link to the other Cluster repository (which contains helmfile.yaml) by committing the required terraform values from below to your values.auto.tfvars, e.g.

cat <<EOF >> values.auto.tfvars    
jx_git_url = "https://github.com/$git_owner_from_cluster_template_above/$git_repo_from_cluster_template_above"
gcp_project = "my-gcp-project"
EOF

If using Google Secret Manager (not Vault) cluster template from above enable it for Terraform using:

cat <<EOF >> values.auto.tfvars 
gsm = true
EOF

The contents of your values.auto.tfvars file should look something like this (the last line will be omitted if not using gsm)....

resource_labels = { "provider" : "jx" }
jx_git_url = "https://github.com/myowner/myname-cluster"
gcp_project = "my-gcp-project"
gsm = true
  1. commit and push any changes to your Infrastructure git repository:
git commit -a -m "fix: configure cluster repository and project"
git push
  1. Now define 2 environment variables to pass the bot user and token into Terraform:
export TF_VAR_jx_bot_username=my-bot-username
export TF_VAR_jx_bot_token=my-bot-token
  1. Now, initialise, plan and apply Terraform:
terraform init
terraform plan
terraform apply

Connect to the cluster

$(terraform output connect)

Tail the Jenkins X installation logs

$(terraform output follow_install_logs)

Once finished you can now move into the Jenkins X Developer namespace

jx ns jx

and create or import your applications

jx project

Terraform Inputs

For the full list of terraform inputs see the documentation for jenkins-x/terraform-google-jx

Name Description Type Default Required
apex_domain The apex / parent domain to be allocated to the cluster string "" no
apex_domain_gcp_project The GCP project the parent domain is managed by, used to write recordsets for a subdomain if set. Defaults to current project. string "" no
apex_domain_integration_enabled Add recordsets from a subdomain to a parent / apex domain bool true no
artifact_description artifact registry repository Description string "jenkins-x Docker Repository" no
artifact_enable Create artifact registry repository bool true no
artifact_location artifact registry repository Location string "us-central1" no
artifact_repository_id artifact registry repository Name, Defaul Cluster Name string "" no
autoscaler_location_policy location policy for primary node pool string "ANY" no
autoscaler_max_node_count Maximum number of cluster nodes number 5 no
autoscaler_min_node_count Minimum number of cluster nodes number 3 no
cluster_location The location (region or zone) in which the cluster master will be created. If you specify a zone (such as us-central1-a), the cluster will be a zonal cluster with a single cluster master. If you specify a region (such as us-west1), the cluster will be a regional cluster with multiple masters spread across zones in the region string "us-central1-a" no
cluster_name Name of the Kubernetes cluster to create string "" no
force_destroy Flag to determine whether storage buckets get forcefully destroyed bool false no
gcp_project The name of the GCP project to use string n/a yes
gsm Enables Google Secrets Manager, not available with JX2 bool false no
initial_cluster_node_count initial number of cluster nodes number 3 no
initial_primary_node_pool_node_count initial number of pool nodes number 1 no
jx_bot_token Bot token used to interact with the Jenkins X cluster git repository string n/a yes
jx_bot_username Bot username used to interact with the Jenkins X cluster git repository string n/a yes
jx_git_url URL for the Jenins X cluster git repository string n/a yes
kuberhealthy Enables Kuberhealthy helm installation bool false no
lets_encrypt_production Flag to determine wether or not to use the Let's Encrypt production server. bool true no
master_authorized_networks List of master authorized networks. If none are provided, disallow external access (except the cluster node IPs, which GKE automatically allowlists). list(object({ cidr_block = string, display_name = string }))
[
{
"cidr_block": "0.0.0.0/0",
"display_name": "any"
}
]
no
node_disk_size Node disk size in GB string "100" no
node_disk_type Node disk type, either pd-standard or pd-ssd string "pd-standard" no
node_machine_type Node type for the Kubernetes cluster string "n1-standard-2" no
node_preemptible Use preemptible nodes bool false no
node_spot Use spot nodes bool false no
resource_labels Set of labels to be applied to the cluster map(string) {} no
subdomain Optional sub domain for the installation string "" no
tls_email Email used by Let's Encrypt. Required for TLS when parent_domain is specified string "" no

Cleanup

To remove any cloud resources created here run:

terraform destroy

Contributing

When adding new variables please regenerate the markdown table

terraform-docs markdown table .

and replace the Inputs section above

Formatting

When developing please remember to format codebase before raising a pull request

terraform fmt -check -diff -recursive

About

Jenkins X 3.x Infrastructure Git Template for Terraform and Google Cloud Platform for managing cloud resources

Topics

Resources

License

Stars

Watchers

Forks