Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not able to create snapshot of a model using dbt-trino==1.7.1 #386

Open
1 task
AviKasliwal04 opened this issue Feb 25, 2024 · 3 comments
Open
1 task

Not able to create snapshot of a model using dbt-trino==1.7.1 #386

AviKasliwal04 opened this issue Feb 25, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@AviKasliwal04
Copy link

Expected behavior

Snapshot of model should be created.

The log should say something on the lines of:
1 of 1 OK snapshotted _<model_name>_......................................... [SUCCESS in 20.57s]

Actual behavior

Getting database error:

Database Error in snapshot _<model_name>_ (snapshots/_<model_name>_snapshot.sql)
  TrinoUserError(type=USER_ERROR, name=SYNTAX_ERROR, message="line 3:76: mismatched input '__dbt_tmp'. Expecting: '.', <EOF>", query_id=_<query_id>_)

Actual query fired:
DROP TABLE IF EXISTS "<catalog_or_db_name>"."<schema_name>"."<model_name>"__dbt_tmp

Should have fired:
DROP TABLE IF EXISTS "<catalog_or_db_name>"."<schema_name>"."<model_name>__dbt_tmp"

Steps To Reproduce

Create a simple model and then a snapshot for that model.

dbt-core == 1.7.1
dbt-trino == 1.7.1
trino version == 438 [Should I bring it down to 435?]

Log output/Screenshots

Screenshot 2024-02-25 at 7 42 59 PM

Operating System

Mac OS 14 Sanoma

dbt version

1.7.1

Trino Server version

438

Python version

3.9.6

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
@AviKasliwal04 AviKasliwal04 added the bug Something isn't working label Feb 25, 2024
@damian3031
Copy link
Member

I'm trying to reproduce this behaviour, but so far, I've had no luck.

So, it fails exactly at
DROP TABLE IF EXISTS "<catalog_or_db_name>"."<schema_name>"."<model_name>"__dbt_tmp

but previously CREATE TABLE succeded, appending __dbt_tmp suffix properly to model name?

@AviKasliwal04
Copy link
Author

I was able to find a way around it by modifying the run_hooks macro. added one run_hooks.sql file in my macros folder with the following code snippet

{% macro run_hooks(hooks, inside_transaction=True) %}
  {% for hook in hooks | selectattr('transaction', 'equalto', inside_transaction)  %}
    {% if not inside_transaction and loop.first %}
      {% call statement(auto_begin=inside_transaction) %}
        commit;
      {% endcall %}
    {% endif %}
    {% set rendered = render(hook.get('sql')) | trim | replace('"', '') %} -- to replace "" from table name
    {% if (rendered | length) > 0 %}
      {% call statement(auto_begin=inside_transaction) %}
        {{ rendered }}
      {% endcall %}
    {% endif %}
  {% endfor %}
{% endmacro %}


{% macro make_hook_config(sql, inside_transaction) %}
    {{ tojson({"sql": sql, "transaction": inside_transaction}) }}
{% endmacro %}


{% macro before_begin(sql) %}
    {{ make_hook_config(sql, inside_transaction=False) }}
{% endmacro %}


{% macro in_transaction(sql) %}
    {{ make_hook_config(sql, inside_transaction=True) }}
{% endmacro %}


{% macro after_commit(sql) %}
    {{ make_hook_config(sql, inside_transaction=False) }}
{% endmacro %}

@AviKasliwal04
Copy link
Author

@damian3031 the above solution worked for my case, but is it the best way, or there is some other easier way, may be by updating some configurations - could not find alter way in docs :/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants