Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENH: support writing and filtered reading from bbox columns in GeoParquet #3282

Merged
merged 43 commits into from
May 27, 2024
Merged

ENH: support writing and filtered reading from bbox columns in GeoParquet #3282

merged 43 commits into from
May 27, 2024

Commits on May 20, 2024

  1. REF: frontload metadata validation

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    b77f289 View commit details
    Browse the repository at this point in the history

  2. REF: rename function

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    9a17f22 View commit details
    Browse the repository at this point in the history

  3. REF: frontload metadata validation for _read_feather

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    0725190 View commit details
    Browse the repository at this point in the history

  4. REF: remove duplicate metadata validation code from _arrow_to_pandas

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    196e67d View commit details
    Browse the repository at this point in the history

  5. REF: import parquet as optional dependency, remove redundant line

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    3ac2483 View commit details
    Browse the repository at this point in the history

  6. ENH: write failing tests

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    c618c30 View commit details
    Browse the repository at this point in the history

  7. ENH: add write bbox column to _to_parquet

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    20ae75b View commit details
    Browse the repository at this point in the history

  8. ENH: add bbox filtering to _read_parquet

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    74d4177 View commit details
    Browse the repository at this point in the history

  9. ENH: add read_bbox_column to _read_parquet

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    29046e5 View commit details
    Browse the repository at this point in the history

  10. ENH: update geometry bbox calculation method

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    90560d5 View commit details
    Browse the repository at this point in the history

  11. ENH: fix docstring of bbox tuple order 

    Co-authored-by: Joris Van den Bossche <[email protected]>
    @nicholas-ys-tan @jorisvandenbossche
    nicholas-ys-tan and jorisvandenbossche committed May 20, 2024
    Configuration menu
    Copy the full SHA
    ba31f04 View commit details
    Browse the repository at this point in the history

  12. ENH: update how bbox column gets appended to table

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    381186a View commit details
    Browse the repository at this point in the history

  13. ENH: allow use of both bbox and filters kwarg in read_parquet

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    81e7b38 View commit details
    Browse the repository at this point in the history

  14. ENH: fix failing tests

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    1b95a11 View commit details
    Browse the repository at this point in the history

  15. ENH: change if write bbox column to predicate with default name 'bbox'

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    5b0736b View commit details
    Browse the repository at this point in the history

  16. ENH: add fallback in case user does not have ds.dataset

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    c7c52ea View commit details
    Browse the repository at this point in the history

  17. ENH: changelog and some comments

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    53fb78b View commit details
    Browse the repository at this point in the history

  18. ENH: clean up rebase linting

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    c9c71e2 View commit details
    Browse the repository at this point in the history

  19. ENH: clean up rebase linting in arrow

    @nicholas-ys-tan
    nicholas-ys-tan committed May 20, 2024
    Configuration menu
    Copy the full SHA
    5a23676 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2024

  1. Remove minimum pyarrow version in tests

    @nicholas-ys-tan
    nicholas-ys-tan committed May 25, 2024
    Configuration menu
    Copy the full SHA
    a2cc212 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into issue3252

    @nicholas-ys-tan
    nicholas-ys-tan committed May 25, 2024
    Configuration menu
    Copy the full SHA
    a494c52 View commit details
    Browse the repository at this point in the history

  3. linting

    @nicholas-ys-tan
    nicholas-ys-tan committed May 25, 2024
    Configuration menu
    Copy the full SHA
    40e06b5 View commit details
    Browse the repository at this point in the history

  4. hard code 'bbox' as column name in tests

    @nicholas-ys-tan
    nicholas-ys-tan committed May 25, 2024
    Configuration menu
    Copy the full SHA
    79761b6 View commit details
    Browse the repository at this point in the history

  5. Add tests for filters without a bbox

    @nicholas-ys-tan
    nicholas-ys-tan committed May 25, 2024
    Configuration menu
    Copy the full SHA
    1597a66 View commit details
    Browse the repository at this point in the history

  6. pass in geometadata to prevent repeated decoding

    @nicholas-ys-tan
    nicholas-ys-tan committed May 25, 2024
    Configuration menu
    Copy the full SHA
    c565e6c View commit details
    Browse the repository at this point in the history

  7. refactor _arrow_to_pandas to take geo_metadata

    @nicholas-ys-tan
    nicholas-ys-tan committed May 25, 2024
    Configuration menu
    Copy the full SHA
    b55fcfb View commit details
    Browse the repository at this point in the history

Commits on May 26, 2024

  1. Fix bbox to include all intersections

    @nicholas-ys-tan
    nicholas-ys-tan committed May 26, 2024
    Configuration menu
    Copy the full SHA
    1be2f60 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. Update docstring 

    Co-authored-by: Matt Richards <[email protected]>
    @nicholas-ys-tan @m-richards
    nicholas-ys-tan and m-richards committed May 27, 2024
    Configuration menu
    Copy the full SHA
    f503c22 View commit details
    Browse the repository at this point in the history

  2. Update docstring 

    Co-authored-by: Joris Van den Bossche <[email protected]>
    @nicholas-ys-tan @jorisvandenbossche
    nicholas-ys-tan and jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    7aeefac View commit details
    Browse the repository at this point in the history

  3. Update test description 

    Co-authored-by: Joris Van den Bossche <[email protected]>
    @nicholas-ys-tan @jorisvandenbossche
    nicholas-ys-tan and jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    611c6ba View commit details
    Browse the repository at this point in the history

  4. simplify line 

    Co-authored-by: Joris Van den Bossche <[email protected]>
    @nicholas-ys-tan @jorisvandenbossche
    nicholas-ys-tan and jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    a4583d6 View commit details
    Browse the repository at this point in the history

  5. update bbox_filter line 

    Co-authored-by: Joris Van den Bossche <[email protected]>
    @nicholas-ys-tan @jorisvandenbossche
    nicholas-ys-tan and jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    248f68d View commit details
    Browse the repository at this point in the history

  6. remove redundant kwarg in test 

    Co-authored-by: Joris Van den Bossche <[email protected]>
    @nicholas-ys-tan @jorisvandenbossche
    nicholas-ys-tan and jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    7e40345 View commit details
    Browse the repository at this point in the history

  7. docstring formatting 

    Co-authored-by: Joris Van den Bossche <[email protected]>
    @nicholas-ys-tan @jorisvandenbossche
    nicholas-ys-tan and jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    9662fd2 View commit details
    Browse the repository at this point in the history

  8. docstring formatting 

    Co-authored-by: Joris Van den Bossche <[email protected]>
    @nicholas-ys-tan @jorisvandenbossche
    nicholas-ys-tan and jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    d91a8ef View commit details
    Browse the repository at this point in the history

  9. updated from review

    @nicholas-ys-tan
    nicholas-ys-tan committed May 27, 2024
    Configuration menu
    Copy the full SHA
    fac0e7c View commit details
    Browse the repository at this point in the history

  10. linting geodataframe

    @nicholas-ys-tan
    nicholas-ys-tan committed May 27, 2024
    Configuration menu
    Copy the full SHA
    163b7d7 View commit details
    Browse the repository at this point in the history

  11. remove read_covering_column

    @jorisvandenbossche
    jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    7c1e1a6 View commit details
    Browse the repository at this point in the history

  12. simplify tests a bit

    @jorisvandenbossche
    jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    cecb4d6 View commit details
    Browse the repository at this point in the history

  13. Change read_parquet to read metadata upfront before reading the data

    @jorisvandenbossche
    jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    d06cfcd View commit details
    Browse the repository at this point in the history

  14. fix for older pyarrow

    @jorisvandenbossche
    jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    4fb7e6d View commit details
    Browse the repository at this point in the history

  15. move validation to existing validation function

    @jorisvandenbossche
    jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    efc149c View commit details
    Browse the repository at this point in the history

  16. Merge remote-tracking branch 'upstream/main' into issue3252

    @jorisvandenbossche
    jorisvandenbossche committed May 27, 2024
    Configuration menu
    Copy the full SHA
    7291940 View commit details
    Browse the repository at this point in the history