Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rds_cluster: add support for I/O-Optimized storage configuration for aurora clusters #2063

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

mandar242
Copy link
Contributor

@mandar242 mandar242 commented Apr 26, 2024

SUMMARY

Adds support for I/O-Optimized storage configuration for aurora clusters

Fixes #2038

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

rds_cluster

ADDITIONAL INFORMATION

Copy link

github-actions bot commented Apr 26, 2024

Docs Build 📝

Thank you for contribution!✨

The docsite for this PR is available for download as an artifact from this run:
https://github.com/ansible-collections/amazon.aws/actions/runs/9423606388

You can compare to the docs for the main branch here:
https://ansible-collections.github.io/amazon.aws/branch/main

File changes:

  • M collections/amazon/aws/rds_cluster_module.html
Click to see the diff comparison.

NOTE: only file modifications are shown here. New and deleted files are excluded.
See the file list and check the published docs to see those files.

diff --git a/home/runner/work/amazon.aws/amazon.aws/docsbuild/base/collections/amazon/aws/rds_cluster_module.html b/home/runner/work/amazon.aws/amazon.aws/docsbuild/head/collections/amazon/aws/rds_cluster_module.html
index 2ad4874..5644e9c 100644
--- a/home/runner/work/amazon.aws/amazon.aws/docsbuild/base/collections/amazon/aws/rds_cluster_module.html
+++ b/home/runner/work/amazon.aws/amazon.aws/docsbuild/head/collections/amazon/aws/rds_cluster_module.html
@@ -916,11 +916,15 @@ see <a class="reference internal" href="#ansible-collections-amazon-aws-rds-clus
 </div></td>
 <td><div class="ansible-option-cell"><p>Specifies the storage type to be associated with the DB cluster.</p>
 <p>This setting is required to create a Multi-AZ DB cluster.</p>
-<p>When specified, a value for the <em>iops</em> parameter is required.</p>
-<p>Defaults to <code class="docutils literal notranslate"><span class="pre">io1</span></code>.</p>
+<p>For multi-AZ DB clusters, <code class="ansible-option docutils literal notranslate"><strong><a class="reference internal" href="#ansible-collections-amazon-aws-rds-cluster-module-parameter-storage-type"><span class="std std-ref"><span class="pre">storage_type</span></span></a></strong></code> defaults to <code class="ansible-value docutils literal notranslate"><span class="pre">io1</span></code> and a value for the <code class="ansible-value docutils literal notranslate"><span class="pre">iops</span></code> parameter is required.</p>
+<p>For Aurora DB clusters, <code class="ansible-option docutils literal notranslate"><strong><a class="reference internal" href="#ansible-collections-amazon-aws-rds-cluster-module-parameter-storage-type"><span class="std std-ref"><span class="pre">storage_type</span></span></a></strong></code> defaults to <code class="ansible-value docutils literal notranslate"><span class="pre">aurora</span></code> standard.</p>
+<p>For mysql and postgres DB clusters, <code class="ansible-option docutils literal notranslate"><strong><a class="reference internal" href="#ansible-collections-amazon-aws-rds-cluster-module-parameter-storage-type"><span class="std std-ref"><span class="pre">storage_type</span></span></a></strong></code> defaults to <code class="ansible-value docutils literal notranslate"><span class="pre">io1</span></code>.</p>
+<p>Support for <code class="ansible-value docutils literal notranslate"><span class="pre">aurora</span></code> and <code class="ansible-value docutils literal notranslate"><span class="pre">aurora-iopt1</span></code> was added in release 8.1.0.</p>
 <p class="ansible-option-line"><strong class="ansible-option-choices">Choices:</strong></p>
 <ul class="simple">
 <li><p><code class="ansible-option-choices-entry docutils literal notranslate"><span class="pre">&quot;io1&quot;</span></code></p></li>
+<li><p><code class="ansible-option-choices-entry docutils literal notranslate"><span class="pre">&quot;aurora&quot;</span></code></p></li>
+<li><p><code class="ansible-option-choices-entry docutils literal notranslate"><span class="pre">&quot;aurora-iopt1&quot;</span></code></p></li>
 </ul>
 </div></td>
 </tr>

Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/befc075d1e684814bcc4d5940953e0e4

✔️ ansible-galaxy-importer SUCCESS in 4m 47s
✔️ build-ansible-collection SUCCESS in 14m 52s
✔️ ansible-test-splitter SUCCESS in 5m 37s
✔️ integration-amazon.aws-1 SUCCESS in 38m 26s
✔️ integration-amazon.aws-2 SUCCESS in 11m 05s
✔️ integration-amazon.aws-3 SUCCESS in 10m 23s
✔️ integration-amazon.aws-4 SUCCESS in 12m 34s
✔️ integration-amazon.aws-5 SUCCESS in 18m 42s
✔️ integration-amazon.aws-6 SUCCESS in 10m 08s
Skipped 38 jobs

plugins/modules/rds_cluster.py Outdated Show resolved Hide resolved
plugins/modules/rds_cluster.py Show resolved Hide resolved
plugins/modules/rds_cluster.py Outdated Show resolved Hide resolved
plugins/modules/rds_cluster.py Outdated Show resolved Hide resolved
plugins/modules/rds_cluster.py Outdated Show resolved Hide resolved
Copy link
Contributor

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/ansible-collections/amazon.aws for 2063,e49f461d8a066fe96ae98b91f61ae83f6881be65

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from e49f461 to d38569d Compare May 7, 2024 00:47
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/66b14acb07a145348bcbdc345973fec1

✔️ ansible-galaxy-importer SUCCESS in 5m 58s
✔️ build-ansible-collection SUCCESS in 15m 45s
✔️ ansible-test-splitter SUCCESS in 6m 48s
✔️ integration-amazon.aws-1 SUCCESS in 39m 52s
✔️ integration-amazon.aws-2 SUCCESS in 11m 54s
✔️ integration-amazon.aws-3 SUCCESS in 10m 27s
✔️ integration-amazon.aws-4 SUCCESS in 13m 09s
integration-amazon.aws-5 RETRY_LIMIT in 1m 43s
✔️ integration-amazon.aws-6 SUCCESS in 8m 36s
✔️ integration-community.aws-1 SUCCESS in 21m 35s
✔️ integration-community.aws-2 SUCCESS in 7m 20s
Skipped 36 jobs

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from 2d84ddd to 097bacd Compare May 8, 2024 06:32
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/c4f7e8e8c71b4ebe98b0604aa7725bc9

✔️ ansible-galaxy-importer SUCCESS in 5m 22s
✔️ build-ansible-collection SUCCESS in 15m 55s
✔️ ansible-test-splitter SUCCESS in 5m 38s
✔️ integration-amazon.aws-1 SUCCESS in 42m 22s
✔️ integration-amazon.aws-2 SUCCESS in 15m 48s
✔️ integration-amazon.aws-3 SUCCESS in 12m 48s
✔️ integration-amazon.aws-4 SUCCESS in 12m 14s
✔️ integration-amazon.aws-5 SUCCESS in 19m 46s
✔️ integration-amazon.aws-6 SUCCESS in 10m 43s
Skipped 38 jobs

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from 097bacd to db7aac9 Compare May 9, 2024 21:52
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/d21d8be9b3334c4ca31a2425b5cf4cc9

✔️ ansible-galaxy-importer SUCCESS in 5m 55s
✔️ build-ansible-collection SUCCESS in 15m 49s
✔️ ansible-test-splitter SUCCESS in 6m 26s
✔️ integration-amazon.aws-1 SUCCESS in 37m 28s
✔️ integration-amazon.aws-2 SUCCESS in 13m 02s
✔️ integration-amazon.aws-3 SUCCESS in 11m 21s
✔️ integration-amazon.aws-4 SUCCESS in 12m 40s
✔️ integration-amazon.aws-5 SUCCESS in 20m 09s
✔️ integration-amazon.aws-6 SUCCESS in 8m 50s
Skipped 38 jobs

plugins/modules/rds_cluster.py Outdated Show resolved Hide resolved
plugins/modules/rds_cluster.py Outdated Show resolved Hide resolved
plugins/modules/rds_cluster.py Outdated Show resolved Hide resolved
plugins/modules/rds_cluster.py Outdated Show resolved Hide resolved
plugins/modules/rds_cluster.py Outdated Show resolved Hide resolved
plugins/modules/rds_cluster.py Outdated Show resolved Hide resolved
@@ -118,10 +118,49 @@
that:
- not _result_create_db_cluster.changed

# aurora-iopt1 test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please check also that storage_type defaults to aurora when it is not specified and the engine is aurora?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified manually that it Sets config to

Cluster storage configuration
Aurora Standard

The issue with the tasks for amazon.aws.rds_cluster and amazon.aws.rds_cluster_info is, they do not return storage_type when storage_type param is not set or even when set storage_type=aurora.

Also tried with aws cli command, however does not return StorageType as well https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-clusters.html

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you handle the defaults in the module logic, instead of leaving them to the API, you should be able to access the returned parameters (like here https://github.com/ansible-collections/amazon.aws/pull/2063/files#diff-41d80755edc72340b8c75ea1d8c215e8e9b974de3ffe70e2bd0538c37baa2f0aR1329).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tried handling as suggested, still unable to get the value of stroage_type in return values.
probably I'm missing something?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I understand. I also tried with different versions of botocore. We should probably add a note in the RETURN block for the storage type. Does this behavior occur with the default storage type value on aurora engines or also on mysql/postgres?

@tremble Any suggestions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alinabuzachis @tremble considering that api does not return the required value/param, and the I have manually verified the results, should we revert this and continue to progress the pr?

Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/a5815310243e46c48578f2908be507c0

✔️ ansible-galaxy-importer SUCCESS in 7m 45s
✔️ build-ansible-collection SUCCESS in 16m 12s
✔️ ansible-test-splitter SUCCESS in 6m 45s
✔️ integration-amazon.aws-1 SUCCESS in 36m 59s
✔️ integration-amazon.aws-2 SUCCESS in 11m 12s
✔️ integration-amazon.aws-3 SUCCESS in 9m 06s
✔️ integration-amazon.aws-4 SUCCESS in 12m 13s
✔️ integration-amazon.aws-5 SUCCESS in 16m 28s
✔️ integration-amazon.aws-6 SUCCESS in 8m 03s
Skipped 38 jobs

This comment was marked as outdated.

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from 5c0abe2 to 55d85ef Compare May 15, 2024 05:27

This comment was marked as outdated.

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from 55d85ef to 1ac2e41 Compare May 15, 2024 19:09

This comment was marked as outdated.

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from 1ac2e41 to 6ed2bf6 Compare May 16, 2024 18:23

This comment was marked as outdated.

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from 6ed2bf6 to 13a43ea Compare May 20, 2024 23:58

This comment was marked as outdated.

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from 13a43ea to acfb347 Compare May 22, 2024 16:07

This comment was marked as outdated.

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from acfb347 to 770a272 Compare May 22, 2024 20:48

This comment was marked as outdated.

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from 770a272 to 6f91772 Compare May 23, 2024 20:31

This comment was marked as outdated.

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from 437696a to cfc0203 Compare May 28, 2024 19:57

This comment was marked as outdated.

This comment was marked as outdated.

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from d4d47a9 to e2ecf30 Compare May 30, 2024 22:27

This comment was marked as outdated.

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from e2ecf30 to f29f59a Compare June 4, 2024 19:04
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/17dacf81114a46c0a8689c00bcc18601

✔️ ansible-galaxy-importer SUCCESS in 5m 56s
✔️ build-ansible-collection SUCCESS in 17m 05s
✔️ ansible-test-splitter SUCCESS in 8m 04s
✔️ integration-amazon.aws-1 SUCCESS in 34m 37s
✔️ integration-amazon.aws-2 SUCCESS in 9m 53s
✔️ integration-amazon.aws-3 SUCCESS in 8m 57s
✔️ integration-amazon.aws-4 SUCCESS in 12m 58s
✔️ integration-amazon.aws-5 SUCCESS in 17m 12s
✔️ integration-amazon.aws-6 SUCCESS in 8m 15s
Skipped 38 jobs

@mandar242 mandar242 force-pushed the rds_cluster_storage_type_support_upadte branch from f29f59a to 54618ac Compare June 7, 2024 22:01
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/cc49f72331124d00a22c56ae8675c91b

✔️ ansible-galaxy-importer SUCCESS in 5m 39s
✔️ build-ansible-collection SUCCESS in 18m 48s
✔️ ansible-test-splitter SUCCESS in 7m 01s
✔️ integration-amazon.aws-1 SUCCESS in 37m 01s
✔️ integration-amazon.aws-2 SUCCESS in 11m 33s
✔️ integration-amazon.aws-3 SUCCESS in 10m 04s
✔️ integration-amazon.aws-4 SUCCESS in 13m 03s
✔️ integration-amazon.aws-5 SUCCESS in 19m 02s
✔️ integration-amazon.aws-6 SUCCESS in 12m 28s
✔️ integration-community.aws-1 SUCCESS in 23m 47s
✔️ integration-community.aws-2 SUCCESS in 47m 53s
✔️ integration-community.aws-3 SUCCESS in 7m 11s
✔️ integration-community.aws-4 SUCCESS in 27m 02s
✔️ integration-community.aws-5 SUCCESS in 16m 47s
✔️ integration-community.aws-6 SUCCESS in 27m 42s
✔️ integration-community.aws-7 SUCCESS in 16m 47s
✔️ integration-community.aws-8 SUCCESS in 23m 40s
✔️ integration-community.aws-9 SUCCESS in 21m 23s
✔️ integration-community.aws-10 SUCCESS in 15m 32s
✔️ integration-community.aws-11 SUCCESS in 24m 56s
✔️ integration-community.aws-12 SUCCESS in 17m 57s
✔️ integration-community.aws-13 SUCCESS in 28m 04s
✔️ integration-community.aws-14 SUCCESS in 16m 13s
✔️ integration-community.aws-15 SUCCESS in 32m 35s
✔️ integration-community.aws-16 SUCCESS in 37m 07s
integration-community.aws-17 RETRY_LIMIT in 1m 42s
✔️ integration-community.aws-18 SUCCESS in 18m 35s
✔️ integration-community.aws-19 SUCCESS in 13m 45s
✔️ integration-community.aws-20 SUCCESS in 11m 31s
Skipped 18 jobs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

amazon.aws.rds_cluster module should support storage_type values for Aurora standard and IO optimized
2 participants