MariaDB Enterprise Cluster / ColumnStore Engine / MaxScale Proxy

Summary

MariaDB ColumnStore is a columnar storage engine that utilizes a massively parallel distributed data architecture. It was built by porting InfiniDB to MariaDB and has been released under the GPL license.

MariaDB ColumnStore is designed for big data scaling to process petabytes of data, linear scalability and exceptional performance with real-time response to analytical queries. It leverages the I/O benefits of columnar storage, compression, just-in-time projection, and horizontal and vertical partitioning to deliver tremendous performance when analyzing large data sets.

This is a Terraform and Ansible project to provision a high availability MariaDB ColumnStore deployment on Amazon Web Services. This automation project will create the following system:

• 3 MariaDB Nodes For Durability & Performance
• 1 S3 Bucket For Object Storage (Data)
• 1 Multi-Attach io2 SSD GFS2 Volume (Shared Metadata)
• 2 MaxScale Nodes For High Availability

Prerequisites:

• Amazon Web Services (AWS) Account
• Install Terraform ^†
• Install Ansible ^‡
• MariaDB Enterprise Token

^† Requires Terraform v0.14.4 or above.
^‡ Requires Full Ansible 2.10.5 or above. (Not Ansible-Core)

Instructions:

Open a terminal window and clone the repository:

1. git clone https://github.com/mariadb-corporation/columnstore-ansible.git
2. cd into the newly cloned folder
3. Edit variables.tf and supply your own variables.
4. terraform init
5. terraform plan (Optional)
6. terraform apply --auto-approve
7. ansible-playbook provision.yml

Further information can be found on our official deployment guide.

Current Approved AWS Image(s)

AMI OS	AMI ID	Region	Zone
CentOS 7	ami-0bc06212a56393ee1	us-west-2	us-west-2c

MCS Command-line Instructions

From your primary node:

Set API Code:

mcs cluster set api-key --key <api_key>

Get Status:

mcs cluster status

Start Cluster:

mcs cluster start

Stop Cluster:

mcs cluster stop

Add Node:

mcs cluster node add --node <node>

Remove Node:

mcs cluster node remove --node <node>

Mode Set Read Only:

mcs cluster set mode --mode readonly

Mode Set Read/Write:

mcs cluster set mode --mode readwrite

Other CLI Tools

• core Change directory to /var/log/mariadb/columnstore/corefiles
• dbrm Change directory to /var/lib/columnstore/data1/systemFiles/dbrm
• extentSave Backup extent map
• tcrit Tail crit.log
• tdebug Tail debug.log
• terror Tail error.log
• tinfo Tail info.log
• twarning Tail warning.log

REST-API Instructions

Format of url endpoints for REST API:

https://{server}:{port}/cmapi/{version}/{route}/{command}

Examples urls for available endpoints:

• https://127.0.0.1:8640/cmapi/0.4.0/cluster/status
• https://127.0.0.1:8640/cmapi/0.4.0/cluster/start
• https://127.0.0.1:8640/cmapi/0.4.0/cluster/shutdown
• https://127.0.0.1:8640/cmapi/0.4.0/cluster/node
• https://127.0.0.1:8640/cmapi/0.4.0/cluster/mode-set

Request Headers Needed:

• 'x-api-key': 'somekey123'
• 'Content-Type': 'application/json'

_{Note: x-api-key can be set to any value of your choice during the first call to the server. Subsequent connections will require this same key}

Examples using curl:

Get Status:

$ curl -s https://127.0.0.1:8640/cmapi/0.4.0/cluster/status --header 'Content-Type:application/json' --header 'x-api-key:somekey123' -k | jq .

Start Cluster:

$ curl -s -X PUT https://127.0.0.1:8640/cmapi/0.4.0/cluster/start --header 'Content-Type:application/json' --header 'x-api-key:somekey123' --data '{"timeout":20}' -k | jq .

Stop Cluster:

$ curl -s -X PUT https://127.0.0.1:8640/cmapi/0.4.0/cluster/shutdown --header 'Content-Type:application/json' --header 'x-api-key:somekey123' --data '{"timeout":20}' -k | jq .

Add Node:

$ curl -s -X PUT https://127.0.0.1:8640/cmapi/0.4.0/cluster/node --header 'Content-Type:application/json' --header 'x-api-key:somekey123' --data '{"timeout":20, "node": "<replace_with_desired_hostname>"}' -k | jq .

Remove Node:

$ curl -s -X DELETE https://127.0.0.1:8640/cmapi/0.4.0/cluster/node --header 'Content-Type:application/json' --header 'x-api-key:somekey123' --data '{"timeout":20, "node": "<replace_with_desired_hostname>"}' -k | jq .

Mode Set:

$ curl -s -X PUT https://127.0.0.1:8640/cmapi/0.4.0/cluster/mode-set --header 'Content-Type:application/json' --header 'x-api-key:somekey123' --data '{"timeout":20, "mode": "readwrite"}' -k | jq .