Skip to content

Shared DynamicFilterPhysicalExpr causes recursive queries to fail #16998

@nuno-faria

Description

@nuno-faria

Describe the bug

The SortExec operator will share the same DynamicFilterPhysicalExpr across multiple invocations of with_new_children (e.g., from reset_plan_states), causing queries that repeatedly execute it to fail.

To Reproduce

For instance, this query returns just 2 rows, while it should return 5:

> with recursive r as (
  select 0 as k, 0 as v
    union all
    (
      select *
      from r
      order by v
      limit 1
    )
)
select *
from r
limit 5;
+---+---+
| k | v |
+---+---+
| 0 | 0 |
| 0 | 0 |
+---+---+
2 row(s) fetched.

Expected behavior

The query above should return:

+---+---+
| k | v |
+---+---+
| 0 | 0 |
| 0 | 0 |
| 0 | 0 |
| 0 | 0 |
| 0 | 0 |
+---+---+

Additional context

When comparing the physical plan, we can see that the second one (after one execution) is different than the initial one in the DynamicFilterPhysicalExpr expression:

@@ -1,24 +1,30 @@
 DynamicFilterPhysicalExpr {
   children: [
     Column {
       name: "v", index: 1
     }
   ]
   , remapped_children: None, inner: RwLock {
     data: Inner {
-      generation: 1, expr: Literal {
-        value: Boolean(true), field: Field {
-          name: "lit", data_type: Boolean, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {
+      generation: 2, expr: BinaryExpr {
+        left: Column {
+          name: "v", index: 1
+        }
+        , op: Lt, right: Literal {
+          value: Int64(0), field: Field {
+            name: "lit", data_type: Int64, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {
+            }
           }
         }
+        , fail_on_overflow: false
       }
     }
     , poisoned: false, ..
   }
   , data_type: RwLock {
     data: None, poisoned: false, ..
   }
   , nullable: RwLock {
     data: None, poisoned: false, ..
   }
 }

Updating SortExec::with_new_children to fully clone the filter makes the recursive query run correctly, but I don't know if there are any side effects that I'm not aware.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingregressionSomething that used to work no longer does

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions