-
Notifications
You must be signed in to change notification settings - Fork 72
[FIX] util.indirect_references: use table_name instead of model_name #141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Otherwise errors (such as the following) will ensue. ``` 2024-09-24 12:11:34,164 2158 ERROR matt-purchase odoo.upgrade.base.tests.test_util: ERROR: TestRecords.test_rename_xmlid Traceback (most recent call last): File "/upgrade-util/src/base/tests/test_util.py", line 1030, in test_rename_xmlid res = util.rename_xmlid(cr, "base.TX1", "base.TX2", on_collision="merge") File "/upgrade-util/src/util/records.py", line 751, in rename_xmlid replace_record_references(cr, (model, old_id), (model, new_id), replace_xmlid=False) File "/upgrade-util/src/util/records.py", line 1308, in replace_record_references return replace_record_references_batch(cr, {old[1]: new[1]}, old[0], new[0], replace_xmlid) File "/upgrade-util/src/util/records.py", line 1490, in replace_record_references_batch explode_execute(cr, query, table=ir.table) File "/upgrade-util/src/util/pg.py", line 352, in explode_execute explode_query_range(cr, query, table, alias=alias, bucket_size=bucket_size), File "/upgrade-util/src/util/pg.py", line 259, in explode_query_range cr.execute(format_query(cr, "SELECT min(id), max(id) FROM {}", table)) File "/tmp/tmpinhjowz1/odoo/pr/153463/odoo/sql_db.py", line 354, in execute res = self._obj.execute(query, params) psycopg2.errors.UndefinedTable: relation "res.partner" does not exist LINE 1: SELECT min(id), max(id) FROM "res.partner" ```
upgradeci retry with always only purchase |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There are still some weird errors: https://upgradeci.odoo.com/upgradeci/run/165581 |
Yes, I'm debugging those. |
From DM discussion, these lines upgrade-util/src/util/records.py Lines 1472 to 1489 in a402a8d
Should be something like: query = cr.mogrify(
format_query(
cr,
"""
UPDATE {table}
SET {column} = (
SELECT jsonb_object_agg(key, COALESCE(
jsonb_object(%s::text[], %s::text[])->>value)::int,
value::int
))
FROM jsonb_each_text({column})
)
WHERE {column} IS NOT NULL
AND {column} @? %s
AND {{parallel_filter}}
""",
table=ir.table,
column=ir.res_id,
),
[
list(id_mapping.keys()),
list(id_mapping.values(),
"$.* ? ({})".format(" || ".join(map("@ == {}".format, id_mapping))),
],
).decode() The key idea: pass the keys and values from id_mapping, to be used as jsonb in the query. Avoiding in the process explicit jsonb dicts in the form |
2b28336
to
508d69e
Compare
src/util/records.py
Outdated
UPDATE {table} | ||
SET {column} = ( | ||
SELECT jsonb_object_agg(key, COALESCE((%s::jsonb->>value)::int, value::int)) | ||
SELECT jsonb_object(ARRAY_AGG(key), %s) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SELECT jsonb_object(ARRAY_AGG(key), %s) | |
SELECT jsonb_object_agg(key, (jsonb_object(%s::text[], %s::text[])->>value)::int, value::int) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this version avoids having literal {
in the query.
Let's try it over my patch.
src/util/records.py
Outdated
column=ir.res_id, | ||
), | ||
[Json(id_mapping), "$.* ? ({})".format(" || ".join(map("@ == {}".format, id_mapping)))], | ||
[list(map(str, id_mapping.values())), "$.* ? ({})".format(" || ".join(map("@ == {}".format, id_mapping)))], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[list(map(str, id_mapping.values())), "$.* ? ({})".format(" || ".join(map("@ == {}".format, id_mapping)))], | |
[list(id_mapping.keys()), list(id_mapping.values()), "$.* ? ({})".format(" || ".join(map("@ == {}".format, id_mapping)))], |
508d69e
to
a518368
Compare
81e900d
to
aeb3308
Compare
Correctly handle changes in company dependent jsonb columns. Use the function `jsonb_object` instead of a casted string, so we don't get literal `{` and `}` into the query. Such characters needs to be escaped to allow the query to be exploded. Also use `cr.mogrify` instead of manual formatting of the jsonpath argument of the query. Co-authored-by: Edoardo Piroli <[email protected]> Co-authored-by: Alvaro Fuentes <[email protected]>
aeb3308
to
e186ed5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@robodoo rebase-ff r+
Merge method set to rebase and fast-forward. |
Otherwise errors (such as the following) will ensue. ``` 2024-09-24 12:11:34,164 2158 ERROR matt-purchase odoo.upgrade.base.tests.test_util: ERROR: TestRecords.test_rename_xmlid Traceback (most recent call last): File "/upgrade-util/src/base/tests/test_util.py", line 1030, in test_rename_xmlid res = util.rename_xmlid(cr, "base.TX1", "base.TX2", on_collision="merge") File "/upgrade-util/src/util/records.py", line 751, in rename_xmlid replace_record_references(cr, (model, old_id), (model, new_id), replace_xmlid=False) File "/upgrade-util/src/util/records.py", line 1308, in replace_record_references return replace_record_references_batch(cr, {old[1]: new[1]}, old[0], new[0], replace_xmlid) File "/upgrade-util/src/util/records.py", line 1490, in replace_record_references_batch explode_execute(cr, query, table=ir.table) File "/upgrade-util/src/util/pg.py", line 352, in explode_execute explode_query_range(cr, query, table, alias=alias, bucket_size=bucket_size), File "/upgrade-util/src/util/pg.py", line 259, in explode_query_range cr.execute(format_query(cr, "SELECT min(id), max(id) FROM {}", table)) File "/tmp/tmpinhjowz1/odoo/pr/153463/odoo/sql_db.py", line 354, in execute res = self._obj.execute(query, params) psycopg2.errors.UndefinedTable: relation "res.partner" does not exist LINE 1: SELECT min(id), max(id) FROM "res.partner" ``` Part-of: #141 Signed-off-by: Christophe Simonis (chs) <[email protected]>
Correctly handle changes in company dependent jsonb columns. Use the function `jsonb_object` instead of a casted string, so we don't get literal `{` and `}` into the query. Such characters needs to be escaped to allow the query to be exploded. Also use `cr.mogrify` instead of manual formatting of the jsonpath argument of the query. closes #141 Signed-off-by: Christophe Simonis (chs) <[email protected]> Co-authored-by: Edoardo Piroli <[email protected]> Co-authored-by: Alvaro Fuentes <[email protected]>
Otherwise errors (such as the following) will ensue. ``` 2024-09-24 12:11:34,164 2158 ERROR matt-purchase odoo.upgrade.base.tests.test_util: ERROR: TestRecords.test_rename_xmlid Traceback (most recent call last): File "/upgrade-util/src/base/tests/test_util.py", line 1030, in test_rename_xmlid res = util.rename_xmlid(cr, "base.TX1", "base.TX2", on_collision="merge") File "/upgrade-util/src/util/records.py", line 751, in rename_xmlid replace_record_references(cr, (model, old_id), (model, new_id), replace_xmlid=False) File "/upgrade-util/src/util/records.py", line 1308, in replace_record_references return replace_record_references_batch(cr, {old[1]: new[1]}, old[0], new[0], replace_xmlid) File "/upgrade-util/src/util/records.py", line 1490, in replace_record_references_batch explode_execute(cr, query, table=ir.table) File "/upgrade-util/src/util/pg.py", line 352, in explode_execute explode_query_range(cr, query, table, alias=alias, bucket_size=bucket_size), File "/upgrade-util/src/util/pg.py", line 259, in explode_query_range cr.execute(format_query(cr, "SELECT min(id), max(id) FROM {}", table)) File "/tmp/tmpinhjowz1/odoo/pr/153463/odoo/sql_db.py", line 354, in execute res = self._obj.execute(query, params) psycopg2.errors.UndefinedTable: relation "res.partner" does not exist LINE 1: SELECT min(id), max(id) FROM "res.partner" ``` Part-of: #141 Signed-off-by: Christophe Simonis (chs) <[email protected]>
Otherwise errors (such as the following) will ensue.