-
Notifications
You must be signed in to change notification settings - Fork 146
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem? Please describe.
We are running into an issue with the dbt-clickhouse adapter when using delete+insert. When running multiple dbt-clickhouse jobs at the same time, we often get a 500 complaining that the temporary xxx_dbt_new_data already exists:
09:36:29 :HTTPDriver for clickhouse:8123 returned response code 500)
09:36:29 Code: 57. DB::Exception: Table default.tablenamet__dbt_new_data already exists. (TABLE_ALREADY_EXISTS) (version 23.3.2.37 (official build))
My guess is that it's related to this row here
{% macro clickhouse__incremental_delete_insert(existing_relation, unique_key, incremental_predicates) %}
{% set new_data_relation = existing_relation.incorporate(path={"identifier": model['name'] + '__dbt_new_data'}) %}
{{ drop_relation_if_exists(new_data_relation) }}
....
{% endmacro %}
Describe the solution you'd like
Would it be a good PR to add a random suffix to the table so that the temp tables created by dbt-clickhouse doesn't collide with each other? Could somebody at the team fix this (seems like a quick fix) or would you accept a PR?
Describe alternatives you've considered
Change our pipelines to make sure we always just run one dbt-clickhouse job at a time.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request