Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix non CRS incompatibility when saving to postgis #3048

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

igorVinicius
Copy link

Before this fix, when the user tries to write a non CRS geometry on a PostGIS database, the action fails because the SRID was converted to -1 (Geopandas pattern), not 0 (PostGIS pattern).

Well, one important observation: I also think that the test related to the last commit on this issue was not executed, because the test was correctly implemented and it must fail on the previous implementation.

@m-richards
Copy link
Member

m-richards commented Oct 23, 2023

Do you mean test_write_postgis_without_crs? This is run on CI - see here: https://github.com/geopandas/geopandas/actions/runs/6601942151/job/17933478854
So it seems there is something conditional about your situation that the existing test currently doesn't catch.

Edit: looking at the code, are you using if_exists=="append"? I believe this case is not covered by that test. If that is the case, could you contribute a test/ extend the existing test covering the "append" case?

@igorVinicius
Copy link
Author

Hi @m-richards ! Thanks for your response. I did'nt notice that the existing test "test_write_postgis_without_crs" doesn't catch the if_exists conditional.

I will review and submit a new PR.

@m-richards
Copy link
Member

You can submit changes as new commits directly on this PR

@igorVinicius
Copy link
Author

igorVinicius commented Oct 23, 2023

@m-richards , just added "append" on the test. Now we try to save on PostGIS twice: first replacing, second appending.

One thing: somehow, CI/CD pre-commit is failing after linting my changes. Do you know why?

@m-richards
Copy link
Member

One thing: somehow, CI/CD pre-commit is failing after linting my changes. Do you know why?

This is due to the code formatting and linting rules we have as part of geopandas to keep things consistent. You might like to have a look at https://geopandas.org/en/latest/community/contributing.html#style-guide-linting, in particular the bit about pre-commit to keep your commits in sync with the style conventions. But for now, I've just pushed a commit which should resolve the linting, so the tests will run.

geopandas/io/sql.py Outdated Show resolved Hide resolved
@@ -467,12 +467,15 @@ def test_write_postgis_without_crs(self, engine_postgis, df_nybb):
df_nybb.crs = None
with pytest.warns(UserWarning, match="Could not parse CRS from the GeoDataF"):
write_postgis(df_nybb, con=engine, name=table, if_exists="replace")
write_postgis(df_nybb, con=engine, name=table, if_exists="append")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move this after the assertion assert target_srid == 0, "SRID should be 0, found %s" % target_srid and add the code to test the assertion again. This works as is, but it would be nice to keep a test for if_exists="replace" only.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect! Thanks.

geopandas/io/sql.py Outdated Show resolved Hide resolved
# Testing when append table
with pytest.warns(UserWarning, match="Could not parse CRS from the GeoDataF"):
write_postgis(df_nybb, con=engine, name=table, if_exists="append")

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
assert target_srid == 0, "SRID should be 0, found %s" % target_srid

You should add the assertion in too, this current just tests that if_exists="append" doesn't crash, but we want it to not crash and also give the right answer 😄

@@ -483,6 +484,17 @@ def test_write_postgis_without_crs(self, engine_postgis, df_nybb):
with pytest.warns(UserWarning, match="Could not parse CRS from the GeoDataF"):
write_postgis(df_nybb, con=engine, name=table, if_exists="append")

sql = text(
"SELECT Find_SRID('{schema}', '{table}', '{geom_col}')"
" ORDER BY '{geom_col}' DESC;".format(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a reason this order by is needed/ why we cannot reuse the same sql from above?

Could we instead just run conn.execute(sql).fetchall() below and check the srids for each row?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants