aws-s3-baseline

A micro-baseline to check for insecure or public S3 buckets and bucket objects in your AWS Environment. This InSpec compliance profile verifies that you do not have any insure or open to public S3 Bucket or Bucket Objects in your AWS Environment in an automated way.

Required Gems

This profile requires the following gems:

• inspec
• inspec-bin
• aws-sdk-s3
• concurrent-ruby (v1.1.0 or higher)

Please install these gems in the ruby environment that InSpec is using prior to executing the profile.

Large Buckets and Profile Runtime

The s3-objects-no-public-access control iterates through every object in each bucket in your AWS environment. The runtime will depend on the number of objects in your S3 Buckets.

On average the profile can process around ~1000 objects/sec.

If you have buckets with large numbers of objects, we suggest you script a loop and use the single_bucket input to parallelize the workload.

To see the processing in more details use the -l debug flag to get verbose output.

Then you can load all your HDF JSON results into Heimdall Lite to easily review all your scan results from the multiple runs by loading them in Heimdall.

Getting Started

It is intended and recommended that InSpec and this profile be run from a "runner" host (such as a DevOps orchestration server, an administrative management system, or a developer's workstation/laptop) against the target remotely over AWS CLI.

For the best security of the runner, always install on the runner the latest version of InSpec and supporting Ruby language components.

The latest versions and installation options are available at the InSpec site.

This baseline also requires the AWS Command Line Interface (CLI) which is available at the AWS CLI site.

Getting MFA Aware AWS Access, Secret and Session Tokens

You will need to ensure your AWS CLI environment has the right system environment variables set with your AWS region and credentials and session token to use the AWS CLI and InSpec resources in the AWS environment. InSpec supports the following standard AWS variables:

• AWS_REGION
• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN (optional) - required if MFA is enabled

Notes on MFA

In any AWS MFA enabled environment - you need to use derived credentials to use the CLI. Your default AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY will not satisfy the MFA Policies in AWS environments.

• The AWS documentation is here: https://docs.aws.amazon.com/cli/latest/reference/sts/get-session-token.html
• The AWS profile documentation is here: https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html
• A useful bash script for automating this is here: https://gist.github.com/dinvlad/d1bc0a45419abc277eb86f2d1ce70625

To generate credentials using an AWS Profile you will need to use the following AWS CLI commands.

a. aws sts get-session-token --serial-number arn:aws:iam::<$YOUR-MFA-SERIAL> --token-code <$YOUR-CURRENT-MFA-TOKEN> --profile=<$YOUR-AWS-PROFILE>

b. Then export the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN that was generated by the above command.

Tailoring to Your Environment

The following inputs must be configured in an inputs ".yml" file for the profile to run correctly for your specific environment. More information about InSpec inputs can be found in the InSpec Profile Documentation.

# List of buckets exempted from inspection.
exception_bucket_list:
  - bucket1
  - bucket2
  ...

# Test only one bucket
single_bucket: 'my-bucket'

Note

When you use the single_bucket input, the profile will ONLY scan that bucket.

Usage

# Set required ENV variables
$ export AWS_ACCESS_KEY_ID=key-id
$ export AWS_SECRET_ACCESS_KEY=access-key
$ export AWS_SESSION_TOKEN=session-token # if MFA is enabled

Installing the needed Gems

Plain Old Ruby Environment

• gem install concurrent-ruby

Using a Chef or CINC Omnibus Installation

• chef gem install concurrent-ruby

Running This Baseline Directly from Github

Testing all your buckets except those defined in your excluded buckets

inspec exec https://github.com/mitre/aws-s3-baseline/archive/master.tar.gz --target aws:// --input-file=your_inputs_file.yml --reporter=cli json:your_output_file.json

Testing a single bucket

inspec exec https://github.com/mitre/aws-s3-baseline/archive/master.tar.gz --target aws:// --input single_bucket=your_bucket --reporter=cli json:your_output_file.json

Different Run Options

Full exec options

Running This Baseline from a local Archive copy

If your runner is not always expected to have direct access to GitHub, use the following steps to create an archive bundle of this baseline and all of its dependent tests:

(Git is required to clone the InSpec profile using the instructions below. Git can be downloaded from the Git site.)

When the "runner" host uses this profile baseline for the first time, follow these steps:

Create your Archieve of the Profile

mkdir profiles
cd profiles
git clone https://github.com/mitre/aws-s3-baseline
inspec archive aws-s3-baseline

Run your scan using the Archieved Copy

inspec exec <name of generated archive> --target aws:// --input-file=<path_to_your_inputs_file/name_of_your_inputs_file.yml> --reporter=cli json:<path_to_your_output_file/name_of_your_output_file.json>

Updating your Archieved Copy

For every successive run, follow these steps to always have the latest version of this baseline:

cd aws-s3-baseline
git pull
cd ..
inspec archive aws-s3-baseline --overwrite

Run your updated Archieved Copy

inspec exec <name of generated archive> --target aws:// --input-file=<path_to_your_inputs_file/name_of_your_inputs_file.yml> --reporter=cli json:<path_to_your_output_file/name_of_your_output_file.json>

Using Heimdall for Viewing the JSON Results

The JSON results output file can be loaded into heimdall-lite for a user-interactive, graphical view of the InSpec results.

The JSON InSpec results file may also be loaded into a full heimdall server, allowing for additional functionality such as to store and compare multiple profile runs.

Authors

• Rony Xavier - rx294
• Aaron Lippold - aaronlippold
• Matthew Dromazos - dromazmj

Special Thanks

• Shivani Karikar - karikarshivani

NOTICE

© 2018-2022 The MITRE Corporation.

Approved for Public Release; Distribution Unlimited. Case Number 18-3678.

NOTICE

MITRE hereby grants express written permission to use, reproduce, distribute, modify, and otherwise leverage this software to the extent permitted by the licensed terms provided in the LICENSE.md file included with this project.

NOTICE

This software was produced for the U. S. Government under Contract Number HHSM-500-2012-00008I, and is subject to Federal Acquisition Regulation Clause 52.227-14, Rights in Data-General.

No other use other than that granted to the U. S. Government, or to those acting on behalf of the U. S. Government under that Clause is authorized without the express written permission of The MITRE Corporation.

For further information, please contact The MITRE Corporation, Contracts Management Office, 7515 Colshire Drive, McLean, VA 22102-7539, (703) 983-6000.