Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support different versions of dbt in our test matrix #965

Open
1 task
tatiana opened this issue May 15, 2024 · 1 comment
Open
1 task

Support different versions of dbt in our test matrix #965

tatiana opened this issue May 15, 2024 · 1 comment
Assignees
Labels
area:testing Related to testing, like unit tests, integration tests, etc dbt:test Primarily related to dbt test command or functionality epic-assigned execution:virtualenv Related to Virtualenv execution environment
Milestone

Comments

@tatiana
Copy link
Collaborator

tatiana commented May 15, 2024

Context

At the moment, we have:

  • Python [3.8 - 3.11] x Airflow [2.3 - 2.9]

And we want to introduce dbt to the test matrix.

This would increase significantly the amount of tests being run, without a clear benefit.

Goal

Have one set of integration & unit tests for:

  • Airflow [2.3 - 2.9] x Python 3.8 (oldest supported) + latest dbt
  • Airflow [2.3 - 2.9] x Python 3.12 (latest) + latest dbt
    This would reduce the amount of Python versions we are checking against.

Have integration & unit tests for the following combinations:

Acceptance criteria

  • Introduce dbt to our test matrix

It may be worth having an independent virtual env for dbt, to avoid conflicts similar to #944.
Another alternative may be to use constraints, as proposed in #965.

Suppose we decide to have separate virtualenvs for Airflow and db. In that case, we will still have to have at least one integration test with dbt and Airflow in the same venv to validate the behaviour of InvocationMode.DBT_RUNNER: https://astronomer.github.io/astronomer-cosmos/getting_started/execution-modes.html#invocation-modes.

@dosubot dosubot bot added area:testing Related to testing, like unit tests, integration tests, etc dbt:test Primarily related to dbt test command or functionality execution:virtualenv Related to Virtualenv execution environment labels May 15, 2024
@tatiana
Copy link
Collaborator Author

tatiana commented May 15, 2024

Perhaps, instead of running every test against the combination of the matrix, we could use -k and run only tests for specific errors related to some of the older versions.

@tatiana tatiana added this to the 1.5.0 milestone May 17, 2024
@tatiana tatiana added triage-needed Items need to be reviewed / assigned to milestone and removed triage-needed Items need to be reviewed / assigned to milestone labels May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:testing Related to testing, like unit tests, integration tests, etc dbt:test Primarily related to dbt test command or functionality epic-assigned execution:virtualenv Related to Virtualenv execution environment
Projects
None yet
Development

No branches or pull requests

2 participants