Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Marhsmallow-SQLAlchemy + Sharded read/write databases #416

Open
shirblc opened this issue Nov 11, 2021 · 0 comments
Open

Marhsmallow-SQLAlchemy + Sharded read/write databases #416

shirblc opened this issue Nov 11, 2021 · 0 comments

Comments

@shirblc
Copy link

shirblc commented Nov 11, 2021

Hi!

In our project we're using SQLAlchemy's ShardedSession to handle read/write replicas handling. It works fine with SQLAlchemy and Flask-SQLAlchemy (all CRUD operations run on the write replica unless set_shard() is called on the query), but Marshmallow-SQLAlchemy seems to be having issues dumping the results. In particular, it seems to have issues around dumping nested fields.

Since set_shard is a query method, rather than a session method, it's not passed down to the queries Marshmallow-SQLAlchemy makes to get the nested objects. So when Marshmallow-SQLAlchemy dumps the results it either:

  • errors because it's trying to grab one_or_none() and there are two shards to read from
  • correctly grabs it from the default database (which makes the read replica partially useless)

So it seems The question is, is there a way to pass the query's execution_options (where the shard ID is) to the queries Marshmallow-SQLAlchemy makes for the nested objects?

Basic reproduction - https://github.com/shirblc/marshmallow-sqla-sharded

Thank you!

EDIT:
Also, when we've replaced ShardedQuery with our own custom class that overrode one_or_none() to add a shard_id, SQLAlchemy worked fine, but Marshmallow-SQLAlchemy still failed. Is Marshmallow-SQLAlchemy using another query class, despite the Session's configuration?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant