Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with ignore_row_if argument in certain expectations #9748

Open
NehaNRane opened this issue Apr 12, 2024 · 0 comments
Open

Issue with ignore_row_if argument in certain expectations #9748

NehaNRane opened this issue Apr 12, 2024 · 0 comments

Comments

@NehaNRane
Copy link

NehaNRane commented Apr 12, 2024

I've encountered a problem with the ignore_row_if argument in three expectations: expect_column_pair_values_a_to_be_greater_than_b, expect_multicolumn_sun_to_equal, and expect_compound_columns_to_be_unique. While the argument works as expected when either_value_is_missing or any_value_is_missing is passed, it behaves unexpectedly when both_values_are_missing / all_values_are_missing or neither / never is used.

Specifically, when using both_values_are_missing / all_values_are_missing or neither / never in the ignore_row_if argument, the following metrics are incorrect:

  • unexpected_count
  • unexpected_percent
  • partial_unexpected_list
  • unexpected_percent_total
  • unexpected_percent_nonmissing

This issue impacts the accuracy of the expectations and could lead to incorrect interpretations of data quality.

Steps to Reproduce:

  1. Use the affected expectations (expect_column_pair_values_a_to_be_greater_than_b, expect_multicolumn_sun_to_equal, expect_compound_columns_to_be_unique).
  2. Pass both_values_are_missing / all_values_are_missing or neither / never in the ignore_row_if argument.
  3. Observe the incorrect values for the mentioned metrics.

Expected Behavior:
The ignore_row_if argument should accurately handle the specified conditions ( both_values_are_missing / all_values_are_missing or neither / never) and produce correct metrics for the affected expectations.

Additional Context:

  • Great Expectations version: 0.18.12
  • Python version: 3.9.6
  • snowflake-connector-python: 3.8.1
  • snowflake-sqlalchemy: 1.5.1
@NehaNRane NehaNRane changed the title ignore_row_if argument in expectation - expect_column_pair_values_a_to_be_greater_than_b is not functioning properly Issue with ignore_row_if argument in certain expectations Apr 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant