Pipelines to detect and correct misconfigurations leading to AWS savings opportunities.
Docker daemon must be installed and running. Please see Install Docker Engine for more information.
Download and install Flowpipe (https://flowpipe.io/downloads) and Steampipe (https://steampipe.io/downloads). Or use Brew:
brew install turbot/tap/flowpipe
brew install turbot/tap/steampipe
Install the AWS plugin with Steampipe:
steampipe plugin install aws
Steampipe will automatically use your default AWS credentials. Optionally, you can setup multiple accounts or customize AWS credentials.
Create a credential_import
resource to import your Steampipe AWS connections:
vi ~/.flowpipe/config/aws.fpc
credential_import "aws" {
source = "~/.steampipe/config/aws.spc"
connections = ["*"]
}
For more information on importing credentials, please see Credential Import.
For more information on credentials in Flowpipe, please see Managing Credentials.
Clone the mod:
mkdir aws-thrifty
cd aws-thrifty
git clone [email protected]:turbot/flowpipe-mod-aws-thrifty.git
Install the dependencies:
flowpipe mod install
Several pipelines have input variables that can be configured to better match your environment and requirements.
Each variable has a default defined in its source file, e.g, s3/s3_buckets_without_lifecycle_policy.fp
(or variables.fp
for more generic variables), but these can be overwritten in several ways:
The easiest approach is to setup your vars file, starting with the sample:
cp thrifty.fpvars.example thrifty.fpvars
vi thrifty.fpvars
flowpipe pipeline run detect_and_correct_ebs_snapshots_exceeding_max_age --var-file=thrifty.fpvars
Alternatively, you can pass variables on the command line:
flowpipe pipeline run detect_and_correct_ebs_snapshots_exceeding_max_age --var=ebs_snapshot_age_max_days=10
Or through environment variables:
export FP_VAR_ebs_snapshot_age_max_days=10
flowpipe pipeline run detect_and_correct_ebs_snapshots_exceeding_max_age
For more information, please see Passing Input Variables
To run your first detection, you'll need to ensure your Steampipe server is up and running:
steampipe service start
To find your desired detection, you can filter the pipeline list
output:
flowpipe pipeline list | grep "detect_and_correct"
Then run your chosen pipeline:
flowpipe pipeline run detect_and_correct_ebs_snapshots_exceeding_max_age
By default the above approach would find the relevant resources and then send a message to your configured notifier.
However; you can request via an Input Step a corrective action to run against each detection result; this behavior is achieved by setting approvers
either as a variable or for a one-off approach, by passing approvers
as an argument.
Note: This approach requires running
flowpipe server
as it uses aninput
step.
flowpipe pipeline run detect_and_correct_ebs_snapshots_exceeding_max_age --host local --arg='approvers=["default"]'
If you're happy to just apply the same action against all detected items, you can apply them without the input
step by overriding the default_action
argument (or the detection specific variable).
flowpipe pipeline run detect_and_correct_ebs_snapshots_exceeding_max_age --arg='default_action="delete_snapshot"'
However; if you have configured a non-empty list for your approvers
variable, you will need to override it as below:
flowpipe pipeline run detect_and_correct_ebs_snapshots_exceeding_max_age --arg='approvers=[]' --arg='default_action="delete_snapshot"'
Finally, each detection pipeline has a corresponding Query Trigger, these are disabled by default allowing for you to configure only those which are required, see the docs for more information.
This repository is published under the Apache 2.0 license. Please see our code of conduct. We look forward to collaborating with you!
Flowpipe and Steampipe are products produced from this open source software, exclusively by Turbot HQ, Inc. They are distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.
Want to help but don't know where to start? Pick up one of the help wanted
issues: