terraform-aws-elasticsearch

Terraform module to create Amazon Elasticsearch Service clusters, following the Well-Architected Framework and best AWS practices.

Amazon Elasticsearch Service is a fully managed service that makes it easy to deploy, operate, and scale Elasticsearch clusters in the AWS Cloud. Elasticsearch is a popular open-source search and analytics engine for use cases such as log analytics, real-time application monitoring, and clickstream analysis. With Amazon ES, you get direct access to the Elasticsearch APIs; existing code and applications work seamlessly with the service.

Examples

Check the examples folder where you can see how to configure a public ES cluster, and another example showing how to set it with VPC options.

Usage

You can use this module to create your Amazon ES cluster by defining each parameters blocks as follows:

module "aws_es" {

  source = "git::https://github.com/lgallard/terraform-aws-elasticsearch.git"

  domain_name           = "elasticsearch_public"
  elasticsearch_version = "7.1"

  cluster_config = {
    dedicated_master_enabled = "true"
    instance_count           = "3"
    instance_type            = "r5.large.elasticsearch"
    zone_awareness_enabled   = "true"
    availability_zone_count  = "3"
  }

  ebs_options = {
    ebs_enabled = "true"
    volume_size = "25"
  }

  encrypt_at_rest = {
    enabled    = "true"
    kms_key_id = "alias/aws/es"
  }

  log_publishing_options = {
    enabled                  = "true"
    log_type                 = "INDEX_SLOW_LOGS"
  }

  advanced_options = {
    "rest.action.multi.allow_explicit_index" = "true"
  }

  node_to_node_encryption_enabled                = "true"
  snapshot_options_automated_snapshot_start_hour = "23"

  tags = {
    Owner = "sysops"
    env   = "dev"
  }

Note: You can also define the above ElasticSearch cluster using just the module variables. Instead of defining a cluster_config block (list of map), you can set each of the cluster_config_* variables, as shown below:

module "aws_es" {

  source = "git::https://github.com/lgallard/terraform-aws-elasticsearch.git"

  domain_name           = "elasticsearch_public"
  elasticsearch_version = "7.1"

  cluster_config_dedicated_master_enabled = true
  cluster_config_instance_count           = "3"
  cluster_config_instance_type            = "r5.large.elasticsearch"
  cluster_config_zone_awareness_enabled   = "true"
  cluster_config_availability_zone_count  = "3"

  ebs_options_ebs_enabled = true
  ebs_options_volume_size = "25"

  encrypt_at_rest_enabled    = true
  encrypt_at_rest_kms_key_id = "alias/aws/es"

  log_publishing_options_enabled  = true
  log_publishing_options_log_type = "INDEX_SLOW_LOGS"

  advanced_options = {
    "rest.action.multi.allow_explicit_index" = "true"
  }

  node_to_node_encryption_enabled                = "true"
  snapshot_options_automated_snapshot_start_hour = "23"

  tags = {
    Owner = "sysops"
    env   = "dev"
  }

Providers

Name	Version
aws	>= 2.69.0

Inputs

Name	Description	Type	Default	Required
access_policies	IAM policy document specifying the access policies for the domain	string	""	no
advanced_options	Key-value string pairs to specify advanced configuration options. Note that the values for these configuration options must be strings (wrapped in quotes) or they may be wrong and cause a perpetual diff, causing Terraform to want to recreate your Elasticsearch domain on every apply	map(string)	{}	no
advanced_security_options	Options for fine-grained access control	any	{}	no
advanced_security_options_enabled	Whether advanced security is enabled (Forces new resource)	bool	false	no
advanced_security_options_internal_user_database_enabled	Whether the internal user database is enabled. If not set, defaults to false by the AWS API.	bool	false	no
advanced_security_options_master_user_arn	ARN for the master user. Only specify if internal_user_database_enabled is not set or set to false)	string	n/a	yes
advanced_security_options_master_user_password	The master user's password, which is stored in the Amazon Elasticsearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true.	string	n/a	yes
advanced_security_options_master_user_username	The master user's username, which is stored in the Amazon Elasticsearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true.	string	n/a	yes
cluster_config	Cluster configuration of the domain	map	{}	no
cluster_config_availability_zone_count	Number of Availability Zones for the domain to use with	number	3	no
cluster_config_dedicated_master_count	Number of dedicated master nodes in the cluster	number	3	no
cluster_config_dedicated_master_enabled	Indicates whether dedicated master nodes are enabled for the cluster	bool	true	no
cluster_config_dedicated_master_type	Instance type of the dedicated master nodes in the cluster	string	"r5.large.elasticsearch"	no
cluster_config_instance_count	Number of instances in the cluster	number	3	no
cluster_config_instance_type	Instance type of data nodes in the cluster	string	"r5.large.elasticsearch"	no
cluster_config_warm_count	The number of warm nodes in the cluster	number	n/a	no
cluster_config_warm_enabled	Indicates whether to enable warm storage	bool	false	no
cluster_config_warm_type	The instance type for the Elasticsearch cluster's warm nodes	string	n/a	no
cluster_config_zone_awareness_enabled	Indicates whether zone awareness is enabled. To enable awareness with three Availability Zones	bool	false	no
cognito_options	Options for Amazon Cognito Authentication for Kibana	map	{}	no
cognito_options_enabled	Specifies whether Amazon Cognito authentication with Kibana is enabled or not	bool	false	no
cognito_options_identity_pool_id	ID of the Cognito Identity Pool to use	string	""	no
cognito_options_role_arn	ARN of the IAM role that has the AmazonESCognitoAccess policy attached	string	""	no
cognito_options_user_pool_id	ID of the Cognito User Pool to use	string	""	no
create_service_link_role	Create service link role for AWS Elasticsearch Service	bool	true	no
domain_endpoint_options	Domain endpoint HTTP(S) related options.	any	{}	no
domain_endpoint_options_enforce_https	Whether or not to require HTTPS	bool	false	no
domain_endpoint_options_tls_security_policy	The name of the TLS security policy that needs to be applied to the HTTPS endpoint. Valid values: Policy-Min-TLS-1-0-2019-07 and Policy-Min-TLS-1-2-2019-07	string	"Policy-Min-TLS-1-2-2019-07"	no
domain_name	Name of the domain	string	n/a	yes
ebs_enabled	Whether EBS volumes are attached to data nodes in the domain	bool	true	no
ebs_options	EBS related options, may be required based on chosen instance size	map	{}	no
ebs_options_iops	The baseline input/output (I/O) performance of EBS volumes attached to data nodes. Applicable only for the Provisioned IOPS EBS volume type	number	0	no
ebs_options_volume_size	The size of EBS volumes attached to data nodes (in GB). Required if ebs_enabled is set to true	number	10	no
ebs_options_volume_type	The type of EBS volumes attached to data nodes	string	"gp2"	no
elasticsearch_version	The version of Elasticsearch to deploy.	string	"7.1"	no
encrypt_at_rest	Encrypt at rest options. Only available for certain instance types	map	{}	no
encrypt_at_rest_enabled	Whether to enable encryption at rest	bool	true	no
encrypt_at_rest_kms_key_id	The KMS key id to encrypt the Elasticsearch domain with. If not specified then it defaults to using the aws/es service KMS key	string	"alias/aws/es"	no
log_publishing_options	Options for publishing slow logs to CloudWatch Logs	map	{}	no
log_publishing_options_cloudwatch_log_group_arn	iARN of the Cloudwatch log group to which log needs to be published	string	""	no
log_publishing_options_enabled	Specifies whether given log publishing option is enabled or not	bool	true	no
log_publishing_options_log_type	A type of Elasticsearch log. Valid values: INDEX_SLOW_LOGS, SEARCH_SLOW_LOGS, ES_APPLICATION_LOGS	string	"INDEX_SLOW_LOGS"	no
log_publishing_options_retention	Retention in days for the created Cloudwatch log group	number	90	no
node_to_node_encryption	Node-to-node encryption options	map	{}	no
node_to_node_encryption_enabled	Whether to enable node-to-node encryption	bool	true	no
snapshot_options	Snapshot related options	map	{}	no
snapshot_options_automated_snapshot_start_hour	Hour during which the service takes an automated daily snapshot of the indices in the domain	number	0	no
tags	A mapping of tags to assign to the resource	map	{}	no
timeouts	Timeouts map.	map	{}	no
timeouts_update	How long to wait for updates.	string	n/a	no
vpc_options	VPC related options, see below. Adding or removing this configuration forces a new resource	map	{}	no
vpc_options_security_group_ids	List of VPC Security Group IDs to be applied to the Elasticsearch domain endpoints. If omitted, the default Security Group for the VPC will be used	list	[]	no
vpc_options_subnet_ids	List of VPC Subnet IDs for the Elasticsearch domain endpoints to be created in	list	[]	no

Outputs

Name	Description
arn	Amazon Resource Name (ARN) of the domain
domain_id	Unique identifier for the domain
endpoint	Domain-specific endpoint used to submit index, search, and data upload requests
kibana_endpoint	Domain-specific endpoint for kibana without https scheme
vpc_options_availability_zones	If the domain was created inside a VPC, the names of the availability zones the configured subnet_ids were created inside
vpc_options_vpc_id	If the domain was created inside a VPC, the ID of the VPC