Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more functionality around dependency management and other tweaks #3

Open
wants to merge 29 commits into
base: main
Choose a base branch
from

Conversation

linuxfreakus
Copy link

Not sure how much you want to take pull requests but I found your repo, started using it and liked it... but also wanted more features so I added:

  • ability to put template files and .tfgen.yaml into a .tfgen.d directory to make them easier to work on and have syntax highlighting, etc.
  • added sprig to get more functionality included for go templates
  • added a deps section to the yaml to declare providers and modules, and names of remote states that can be used to filter on when generating templates along with associated helper functions, etc
  • fixed the cobra so that it returns an error exit code if something goes wrong instead of silently failing
  • added a --log-level flag
  • built a new template context struct to add more than just vars to the template context
  • ability to rewrite filenames when generating output based on yaml config
  • probably a few other things I forgot about

The code under the config.go could probably get broken up into some separate files since i added more structs for reading the yaml, but its not too terrible yet.

linuxfreakus and others added 27 commits June 3, 2022 20:21
Updated the config struct to include some new properties that will allow
a root configuration to list 'child' directories and a 'clean_pattern'
which will give users some control over how tfgen will clean up templates.
Removed some of the logic in the function that searches parent
directories. Moved that logic to its own function ('collateConfig') so
it can be used by other functions, including the ones needed for the new
command 'exec_all'.
New commands that will allow tfgent to delete files that were generated
by a template. These commands expect a 'clean_pattern' to be defined,
ideally in the root configuration to allow for the 'clean_all' command,
which will clean the 'child_directories' defined in the root confing.
This should give the user a great deal of control around how they want
things cleaned up when needed.
Thew 'exec_all' command allows users to specify a directory where a root
configuration is (this is required -- it errors out, otherwise). Then,
tfgen will 'exec' as normal for any and all children directories that
are defined by the root config.
Adding the new commands ('exec_all', 'clean', and 'clean_all') to main.
The term 'child_directories' might become confusing later on due to some
other features that might be implemented. Going to change them to
'target_directories' which is a bit more in line with the original
default behavior of tfgen and how it named/identified the 'targetDir'.
New Functionality & Concepts: Child Directories and Clean-Up
@refl3ction
Copy link
Owner

@linuxfreakus thank you for opening this PR! I still own you this review, I'm trying to find some time to do so. I promise I'll try to get back to you this weekend 😁

@refl3ction
Copy link
Owner

refl3ction commented Jul 8, 2022

My thoughts regarding the added functionalities:

Good ✅

  • Configuration inside .tfgen.d
  • Increased functionality for template
  • Log level flag
  • Template context struct
  • exec_all command

Needs discussion

Deps

The deps block IMO added a little bit of complexity to the tool, I understand the problem you tried to solve but maybe we can solve it using a different approach, I'll elaborate.

default_providers, required_providers and extra_providers

Maybe we can keep it simple and just use a template to render them, in addition, when using this feature, we lose the ability to pass in specific configuration to the provider, for example region or allowed_accounts_ids for the aws provider.

default_remote_states and extra_remote_states

I think we can simplify the usage of this feature, since we still need to declare the template for it anyways, I was thinking about doing it like that:

# .tfgen.yaml
deps:
  remote_states:
    filename: _deps.tf
    templates:
      vpc: |
        data "terraform_remote_state" "vpc" {
          backend = "s3"
          config = {
            bucket         = "{{ .Vars.state_s3_bucket }}"
            dynamodb_table = "{{ .Vars.dynamodb_table }}"
            key            = "{{ .Vars.env }}/networking/vpc/terraform.tfstate"
            region         = "{{ .Vars.state_aws_region }}"
          }
        }

And then, instead of declaring it on the config files, the user could provide some arguments to the exec command and tfgen provides then ad-hoc during template generation.

tfgen exec . -dep vpc -dep hosted-zone

Main reason is that, depending on the setup, it's hard to define common dependencies for a set of modules. You can have a compute folder and all the modules inside it can depend on the a vpc or not, or they can depend on a vpc and some other module, or different vpcs.
Thoughts?

Modules

Same as above.

Commands

clean and clean_all

What would be a use case that is worth automating the process of deleting the template files? Specially, because we need to specify all the subdirectores in the config file, I don't know if it's worth adding this complexity. If there is a valid use case for that, we need to output all the files that will be deleted for the user and also prompt for confirmation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants