Support Settings Configuration through CLI and environment Variables

[mcrowson]

Right now settings are just passed in through the settings file. This is somewhat limiting if there is a value that will be dynamic with deployments (such as a docker image file, etc.) We should support accepting settings through environment variables, the CLI, the current settings file, and any mix of those.

The order of precedence with mixed should be CLI -> environment -> config file.

If possible, the implementation of this ticket would not be to enumerate each settings value for each of these sources, but to have some way of dynamically finding zappa settings and configuring it.

[monkut]

I'm interested in this getting in as a feature.

I've created a cli tool that will generate the zappa_settings.json file dynamically for ci integration.
https://github.com/monkut/zappa-configuration-generator

It's been working for me for a long time, but I would like similar functionality integrated into zappa itself.

[michaelhudelson]

Any updates on this? Super interested in having a feature like this :)

[souravjamwal77]

Hi @michaelhudelson
You can specify additional environment variables into a .env file for your deployment.

@monkut I saw your repo. And yes your configuration can help in CI/CD deployment.

[michaelhudelson]

Hi @souravjamwal77

Thanks for getting back to me :)

I think I misunderstood this ticket - I'm actually looking for something that is to Zappa what Terragrunt is to Terraform.

I want to be able to have a base settings file and then right a layer of customization over it.

If you are interested, I recently made something in a personal project that does this as a POC in about 80 lines of python.

[dougharris]

Another use case for this is not committing secrets to git. For example, when setting a certificate ARN in the settings file, it contains the account number for the AWS account. I’d rather reference an environment variable for that ARN then commit the hardcoded ARN with account number and UUID to git.

[dougharris]

I have the beginning of an implementation that would read settings from environment variables. I'm having difficulty imagining how passing settings as command line options would work. @mcrowson do you have ideas about what the CLI experience would be?

Like if you wanted to pass the value for s3_bucket on the command line, is this what you're thinking?

zappa --s3_bucket mybucket deploy

Or perhaps something like:

zappa --config s3_bucket=mybucket deploy

I'm not sure if/how click's arg processing supports arbitrary extra command line options.

Any suggestions?

[mcrowson]

I am admittedly far from Zappa development these days and have handed the reigns off to others. However you all want to roll with this is up to you.

[javulticat]

@dougharris, perhaps I'm not quite understanding the issue you're describing, but I'm unsure if I see an actual problem that is not already solvable using Zappa as-is. Following best practices, the use of Zappa should never require any secrets to be committed to a repo. Major collaborative version control platforms (e.g., GitHub) have long supported securely storing secrets that can be securely loaded into the ephemeral environments that get created on CI/CD runners during job execution. And then Zappa currently offers a variety of ways to securely set those environment variables in AWS from your CI/CD workflow. And, as some have mentioned above, it's relatively common for users to write a short Python script that can assist in dynamically inserting/generating values into a zappa_settings file in their CI/CD workflow, if necessary.

[dougharris]

@javulticat In my particular use case, my zappa_settings.json has certificate_arn set per the instructions on Deploying to a Domain With AWS Certificate Manager:

"certificate_arn": "arn:aws:acm:us-east-1:<redacted AWS account number>:certificate/<redacted UUID for cert>",

I'd like to make my repo public as an example of a few things in my project, but I'd prefer that this ARN not be visible (in particular, I'd like my account number not to be so easily visible).

Is there a way for the certificate ARN to be pulled from AWS environment variables?

My WIP on this would see ${CERTIFICATE_ARN} as a value in my zappa_settings.json and recognize the syntax to use os.environ.get(CERTIFICATE_ARN) for that value

[dougharris]

My commit there shows the idea. I'd like to add tests before submitting a pull request.