Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FINAL statement not working #231

Open
tvorogme opened this issue Feb 7, 2023 · 2 comments · May be fixed by #233
Open

FINAL statement not working #231

tvorogme opened this issue Feb 7, 2023 · 2 comments · May be fixed by #233

Comments

@tvorogme
Copy link

tvorogme commented Feb 7, 2023 •
edited

Describe the bug

I'm using .final() but there are no changes in query.

Code:

query = AccountActualStatesGQ.get_query(info)  # SQLAlchemy query
query = apply_requested_fields(info, query, AccountActualStates)
qs = query.filter_by(**kwargs)
qs = qs.final() <--

Then I'm compiling query:

logger.warning(f'Query to clickhouse: {qs.statement.compile(compile_kwargs={"literal_binds": True})}')

And got SELECT query without FINAL.

SELECT account_actual_state.parsed_jetton_wallet_balance, account_actual_state.parsed_jetton_wallet_owner_address_address, account_actual_state.parsed_jetton_wallet_jetton_address_address, account_actual_state.parsed_jetton_wallet_data_is_approved, account_actual_state.address, account_actual_state.workchain, account_actual_state.gen_utime 
FROM account_actual_state 
WHERE account_actual_state.parsed_jetton_wallet_owner_address_address = '71050EEDE7F8CAD72DE92F3112C1D58860963267C68EB4AE8BCC45298258A81A' AND account_actual_state.parsed_jetton_wallet_jetton_address_address = 'F4BDD480FCD79D47DBAF6E037D1229115FEB2E7AC0F119E160EBD5D031ABDF2E' AND account_actual_state.parsed_jetton_wallet_data_is_approved = 1 ORDER BY account_actual_state.parsed_jetton_wallet_balance DESC 
 LIMIT 0, 150

I've quickly check debugger, it seems that Query class is from lib, but select statement in it not. When I call .final() in query it works correctly (set final flag in class to True), but in sqlcompiler final_clause = getattr(select, '_final_clause', None) returns None

qs
Out[2]: <clickhouse_sqlalchemy.orm.query.Query at 0x169ad3e20>
qs.statement
Out[3]: <sqlalchemy.sql.selectable.Select object at 0x16a57c9d0>

I've also tried to catch _compile_context call, but it seems that it's not calling. Maybe I do something wrong?

I've figured out that I can compile it in the right way:

context = qs._compile_context()
clause = context.query

try:
  compiled = clause.compile(dialect=engine.dialect, compile_kwargs={"literal_binds": True,
                                                                          "render_postcompile": True})
  logger.warning(f'Query to clickhouse: {compiled}')
except Exception as e:
  compiled = clause.compile(dialect=engine.dialect)
  logger.warning(f'Query to clickhouse: {compiled}')

But actual query that will be sent to clickhouse (when iterate over qs) will not contain FINAL.

Versions

Latest version from git
@tvorogme tvorogme linked a pull request Feb 9, 2023 that will close this issue
Open
@tvorogme
Copy link
Author

tvorogme commented Feb 9, 2023

I'm not sqlalchemy dev and my code is filthy, but it fixes FINAL and other flags in ORM engine.

The problem is that after compiling SQL (only if ORM is used) it creates new Select statement and not copy attributes from original one. I've overloaded 2 methods and add copy from old Select to new all needed attributes. Maybe it could be done in a more elegant way.

@iamkarlson
Copy link

In my project we had to patch clickhouse dialect so it propagates FINAL to the sql queries. Seems like it's working. @xzkostyan would it make too much hassle to merge aforementioned PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ORM missed attributes fix tvorogme/clickhouse-sqlalchemy
2 participants
@iamkarlson @tvorogme