WIP: Pin to Adaptive Parquet Predicate Pushdown #18368

alamb · 2025-10-29T21:25:52Z

This is a test branch for performance profiling for the code from @hhhizzz upstream in arrow-rs:

[Parquet] Adaptive Parquet Predicate Pushdown arrow-rs#8733

alamb · 2025-10-30T08:29:47Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1017-gcp #18~24.04.1-Ubuntu SMP Tue Sep 23 17:51:44 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_adapative_scan (c181597) to 0a650a0 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb · 2025-10-30T09:27:34Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃         Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ QQuery 0     │  2738.16 ms │                2692.39 ms │      no change │
│ QQuery 1     │  1227.30 ms │                1313.44 ms │   1.07x slower │
│ QQuery 2     │  2316.04 ms │                2423.93 ms │      no change │
│ QQuery 3     │  1139.86 ms │                1153.79 ms │      no change │
│ QQuery 4     │  2232.67 ms │                2416.56 ms │   1.08x slower │
│ QQuery 5     │ 28051.03 ms │               27834.75 ms │      no change │
│ QQuery 6     │  4186.76 ms │                 166.77 ms │ +25.11x faster │
│ QQuery 7     │  3574.54 ms │                3695.72 ms │      no change │
└──────────────┴─────────────┴───────────────────────────┴────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 45466.36ms │
│ Total Time (alamb_test_adapative_scan)   │ 41697.35ms │
│ Average Time (HEAD)                      │  5683.30ms │
│ Average Time (alamb_test_adapative_scan) │  5212.17ms │
│ Queries Faster                           │          1 │
│ Queries Slower                           │          2 │
│ Queries with No Change                   │          5 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃         Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.09 ms │                   2.53 ms │   1.21x slower │
│ QQuery 1     │    50.87 ms │                  53.88 ms │   1.06x slower │
│ QQuery 2     │   136.47 ms │                 137.52 ms │      no change │
│ QQuery 3     │   161.11 ms │                 167.04 ms │      no change │
│ QQuery 4     │  1025.66 ms │                1092.05 ms │   1.06x slower │
│ QQuery 5     │  1463.22 ms │                1488.92 ms │      no change │
│ QQuery 6     │     2.15 ms │                   2.20 ms │      no change │
│ QQuery 7     │    53.84 ms │                  71.18 ms │   1.32x slower │
│ QQuery 8     │  1425.84 ms │                1470.39 ms │      no change │
│ QQuery 9     │  1772.91 ms │                1825.44 ms │      no change │
│ QQuery 10    │   386.84 ms │                 485.98 ms │   1.26x slower │
│ QQuery 11    │   446.25 ms │                 538.85 ms │   1.21x slower │
│ QQuery 12    │  1409.22 ms │                1556.39 ms │   1.10x slower │
│ QQuery 13    │  2156.43 ms │                2348.62 ms │   1.09x slower │
│ QQuery 14    │  1268.16 ms │                1449.71 ms │   1.14x slower │
│ QQuery 15    │  1222.16 ms │                1263.90 ms │      no change │
│ QQuery 16    │  2630.85 ms │                2759.71 ms │      no change │
│ QQuery 17    │  2644.54 ms │                2674.41 ms │      no change │
│ QQuery 18    │  5335.54 ms │                5054.80 ms │  +1.06x faster │
│ QQuery 19    │   124.73 ms │                 144.01 ms │   1.15x slower │
│ QQuery 20    │  2062.39 ms │                2061.97 ms │      no change │
│ QQuery 21    │  2329.07 ms │                2538.07 ms │   1.09x slower │
│ QQuery 22    │  4344.11 ms │                4171.74 ms │      no change │
│ QQuery 23    │ 12780.49 ms │                1395.75 ms │  +9.16x faster │
│ QQuery 24    │   216.12 ms │                 317.21 ms │   1.47x slower │
│ QQuery 25    │   515.37 ms │                 634.02 ms │   1.23x slower │
│ QQuery 26    │   223.40 ms │                 330.81 ms │   1.48x slower │
│ QQuery 27    │  2970.05 ms │                3137.36 ms │   1.06x slower │
│ QQuery 28    │ 22884.87 ms │               24904.65 ms │   1.09x slower │
│ QQuery 29    │   966.13 ms │                 960.83 ms │      no change │
│ QQuery 30    │  1354.70 ms │                1422.04 ms │      no change │
│ QQuery 31    │  1364.26 ms │                1418.44 ms │      no change │
│ QQuery 32    │  5188.96 ms │                4419.72 ms │  +1.17x faster │
│ QQuery 33    │  5989.10 ms │                5624.95 ms │  +1.06x faster │
│ QQuery 34    │  6149.11 ms │                6013.20 ms │      no change │
│ QQuery 35    │  2001.90 ms │                1997.83 ms │      no change │
│ QQuery 36    │   122.10 ms │                  20.21 ms │  +6.04x faster │
│ QQuery 37    │    53.18 ms │                  19.88 ms │  +2.67x faster │
│ QQuery 38    │   121.17 ms │                  20.21 ms │  +5.99x faster │
│ QQuery 39    │   197.39 ms │                  19.74 ms │ +10.00x faster │
│ QQuery 40    │    41.01 ms │                  21.47 ms │  +1.91x faster │
│ QQuery 41    │    38.54 ms │                  20.74 ms │  +1.86x faster │
│ QQuery 42    │    33.45 ms │                  20.95 ms │  +1.60x faster │
└──────────────┴─────────────┴───────────────────────────┴────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 95665.73ms │
│ Total Time (alamb_test_adapative_scan)   │ 86079.33ms │
│ Average Time (HEAD)                      │  2224.78ms │
│ Average Time (alamb_test_adapative_scan) │  2001.84ms │
│ Queries Faster                           │         11 │
│ Queries Slower                           │         16 │
│ Queries with No Change                   │         16 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 213.40 ms │                 175.23 ms │ +1.22x faster │
│ QQuery 2     │  30.83 ms │                  26.55 ms │ +1.16x faster │
│ QQuery 3     │  46.60 ms │                  40.82 ms │ +1.14x faster │
│ QQuery 4     │  35.00 ms │                  28.36 ms │ +1.23x faster │
│ QQuery 5     │  77.73 ms │                  76.78 ms │     no change │
│ QQuery 6     │  19.33 ms │                  19.25 ms │     no change │
│ QQuery 7     │ 228.06 ms │                 207.04 ms │ +1.10x faster │
│ QQuery 8     │  33.39 ms │                  30.44 ms │ +1.10x faster │
│ QQuery 9     │  98.33 ms │                 100.35 ms │     no change │
│ QQuery 10    │  60.78 ms │                  58.75 ms │     no change │
│ QQuery 11    │  17.41 ms │                  17.02 ms │     no change │
│ QQuery 12    │  51.78 ms │                  49.57 ms │     no change │
│ QQuery 13    │  48.59 ms │                  45.82 ms │ +1.06x faster │
│ QQuery 14    │  14.12 ms │                  12.85 ms │ +1.10x faster │
│ QQuery 15    │  24.95 ms │                  24.12 ms │     no change │
│ QQuery 16    │  25.02 ms │                  24.57 ms │     no change │
│ QQuery 17    │ 156.23 ms │                 145.67 ms │ +1.07x faster │
│ QQuery 18    │ 322.72 ms │                 329.67 ms │     no change │
│ QQuery 19    │  35.83 ms │                  37.15 ms │     no change │
│ QQuery 20    │  50.77 ms │                  48.63 ms │     no change │
│ QQuery 21    │ 344.51 ms │                 305.96 ms │ +1.13x faster │
│ QQuery 22    │  20.72 ms │                  20.63 ms │     no change │
└──────────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1956.11ms │
│ Total Time (alamb_test_adapative_scan)   │ 1825.23ms │
│ Average Time (HEAD)                      │   88.91ms │
│ Average Time (alamb_test_adapative_scan) │   82.96ms │
│ Queries Faster                           │        10 │
│ Queries Slower                           │         0 │
│ Queries with No Change                   │        12 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

hhhizzz · 2025-10-30T09:38:01Z

Got some impressive improvement in some query, I'll take a look for the slower query like QQuery 26 . 1.48x slower didn't like noise.

zhuqi-lucas · 2025-10-30T09:48:58Z

Thank you @alamb @hhhizzz , we may also need to trigger the clickbench_pushdown which is the enabled filter pushdown, in order to compare this PR and main.

alamb · 2025-10-30T11:02:56Z

Thank you @alamb @hhhizzz , we may also need to trigger the clickbench_pushdown which is the enabled filter pushdown, in order to compare this PR and main.

Good call -- I forgot about that one. I just hard coded the pushdown on -- will rerun

alamb · 2025-10-30T11:33:27Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1017-gcp #18~24.04.1-Ubuntu SMP Tue Sep 23 17:51:44 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_adapative_scan (a0cbf52) to 930620a diff using: clickbench_pushdown
Results will be posted here when complete

alamb · 2025-10-30T12:02:48Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_pushdown.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.60 ms │                   2.33 ms │ +1.12x faster │
│ QQuery 1     │    55.17 ms │                  54.34 ms │     no change │
│ QQuery 2     │   136.84 ms │                 140.47 ms │     no change │
│ QQuery 3     │   164.07 ms │                 163.01 ms │     no change │
│ QQuery 4     │  1203.30 ms │                1078.05 ms │ +1.12x faster │
│ QQuery 5     │  1581.61 ms │                1526.89 ms │     no change │
│ QQuery 6     │     2.13 ms │                   2.22 ms │     no change │
│ QQuery 7     │    72.51 ms │                  73.17 ms │     no change │
│ QQuery 8     │  1518.06 ms │                1472.00 ms │     no change │
│ QQuery 9     │  1948.40 ms │                1898.97 ms │     no change │
│ QQuery 10    │   501.07 ms │                 485.25 ms │     no change │
│ QQuery 11    │   566.69 ms │                 556.84 ms │     no change │
│ QQuery 12    │  1659.60 ms │                1541.78 ms │ +1.08x faster │
│ QQuery 13    │  2594.14 ms │                2278.78 ms │ +1.14x faster │
│ QQuery 14    │  1735.34 ms │                1460.55 ms │ +1.19x faster │
│ QQuery 15    │  1327.28 ms │                1258.79 ms │ +1.05x faster │
│ QQuery 16    │  2729.06 ms │                2683.83 ms │     no change │
│ QQuery 17    │  2696.97 ms │                2683.94 ms │     no change │
│ QQuery 18    │  5157.72 ms │                4965.78 ms │     no change │
│ QQuery 19    │   149.40 ms │                 141.88 ms │ +1.05x faster │
│ QQuery 20    │  2012.43 ms │                1967.18 ms │     no change │
│ QQuery 21    │  2460.10 ms │                2352.27 ms │     no change │
│ QQuery 22    │  4271.47 ms │                4077.51 ms │     no change │
│ QQuery 23    │  1155.60 ms │                1176.57 ms │     no change │
│ QQuery 24    │   261.13 ms │                 248.77 ms │     no change │
│ QQuery 25    │   681.20 ms │                 635.93 ms │ +1.07x faster │
│ QQuery 26    │   362.74 ms │                 338.25 ms │ +1.07x faster │
│ QQuery 27    │  3293.90 ms │                3101.06 ms │ +1.06x faster │
│ QQuery 28    │ 23992.63 ms │               23346.17 ms │     no change │
│ QQuery 29    │   991.31 ms │                 981.48 ms │     no change │
│ QQuery 30    │  2225.58 ms │                1465.19 ms │ +1.52x faster │
│ QQuery 31    │  2079.26 ms │                1437.83 ms │ +1.45x faster │
│ QQuery 32    │  5371.03 ms │                4334.00 ms │ +1.24x faster │
│ QQuery 33    │  6696.14 ms │                5615.78 ms │ +1.19x faster │
│ QQuery 34    │  6088.96 ms │                6177.78 ms │     no change │
│ QQuery 35    │  2132.40 ms │                2024.73 ms │ +1.05x faster │
│ QQuery 36    │    25.43 ms │                  26.36 ms │     no change │
│ QQuery 37    │    25.38 ms │                  25.48 ms │     no change │
│ QQuery 38    │    24.88 ms │                  25.47 ms │     no change │
│ QQuery 39    │    25.14 ms │                  24.59 ms │     no change │
│ QQuery 40    │    26.51 ms │                  26.12 ms │     no change │
│ QQuery 41    │    25.45 ms │                  25.53 ms │     no change │
│ QQuery 42    │    24.82 ms │                  25.16 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 90055.45ms │
│ Total Time (alamb_test_adapative_scan)   │ 83928.07ms │
│ Average Time (HEAD)                      │  2094.31ms │
│ Average Time (alamb_test_adapative_scan) │  1951.82ms │
│ Queries Faster                           │         15 │
│ Queries Slower                           │          0 │
│ Queries with No Change                   │         28 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘

alamb · 2025-10-30T12:02:52Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1017-gcp #18~24.04.1-Ubuntu SMP Tue Sep 23 17:51:44 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_adapative_scan (a0cbf52) to 930620a diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

zhuqi-lucas · 2025-10-30T12:19:04Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_pushdown.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.60 ms │                   2.33 ms │ +1.12x faster │
│ QQuery 1     │    55.17 ms │                  54.34 ms │     no change │
│ QQuery 2     │   136.84 ms │                 140.47 ms │     no change │
│ QQuery 3     │   164.07 ms │                 163.01 ms │     no change │
│ QQuery 4     │  1203.30 ms │                1078.05 ms │ +1.12x faster │
│ QQuery 5     │  1581.61 ms │                1526.89 ms │     no change │
│ QQuery 6     │     2.13 ms │                   2.22 ms │     no change │
│ QQuery 7     │    72.51 ms │                  73.17 ms │     no change │
│ QQuery 8     │  1518.06 ms │                1472.00 ms │     no change │
│ QQuery 9     │  1948.40 ms │                1898.97 ms │     no change │
│ QQuery 10    │   501.07 ms │                 485.25 ms │     no change │
│ QQuery 11    │   566.69 ms │                 556.84 ms │     no change │
│ QQuery 12    │  1659.60 ms │                1541.78 ms │ +1.08x faster │
│ QQuery 13    │  2594.14 ms │                2278.78 ms │ +1.14x faster │
│ QQuery 14    │  1735.34 ms │                1460.55 ms │ +1.19x faster │
│ QQuery 15    │  1327.28 ms │                1258.79 ms │ +1.05x faster │
│ QQuery 16    │  2729.06 ms │                2683.83 ms │     no change │
│ QQuery 17    │  2696.97 ms │                2683.94 ms │     no change │
│ QQuery 18    │  5157.72 ms │                4965.78 ms │     no change │
│ QQuery 19    │   149.40 ms │                 141.88 ms │ +1.05x faster │
│ QQuery 20    │  2012.43 ms │                1967.18 ms │     no change │
│ QQuery 21    │  2460.10 ms │                2352.27 ms │     no change │
│ QQuery 22    │  4271.47 ms │                4077.51 ms │     no change │
│ QQuery 23    │  1155.60 ms │                1176.57 ms │     no change │
│ QQuery 24    │   261.13 ms │                 248.77 ms │     no change │
│ QQuery 25    │   681.20 ms │                 635.93 ms │ +1.07x faster │
│ QQuery 26    │   362.74 ms │                 338.25 ms │ +1.07x faster │
│ QQuery 27    │  3293.90 ms │                3101.06 ms │ +1.06x faster │
│ QQuery 28    │ 23992.63 ms │               23346.17 ms │     no change │
│ QQuery 29    │   991.31 ms │                 981.48 ms │     no change │
│ QQuery 30    │  2225.58 ms │                1465.19 ms │ +1.52x faster │
│ QQuery 31    │  2079.26 ms │                1437.83 ms │ +1.45x faster │
│ QQuery 32    │  5371.03 ms │                4334.00 ms │ +1.24x faster │
│ QQuery 33    │  6696.14 ms │                5615.78 ms │ +1.19x faster │
│ QQuery 34    │  6088.96 ms │                6177.78 ms │     no change │
│ QQuery 35    │  2132.40 ms │                2024.73 ms │ +1.05x faster │
│ QQuery 36    │    25.43 ms │                  26.36 ms │     no change │
│ QQuery 37    │    25.38 ms │                  25.48 ms │     no change │
│ QQuery 38    │    24.88 ms │                  25.47 ms │     no change │
│ QQuery 39    │    25.14 ms │                  24.59 ms │     no change │
│ QQuery 40    │    26.51 ms │                  26.12 ms │     no change │
│ QQuery 41    │    25.45 ms │                  25.53 ms │     no change │
│ QQuery 42    │    24.82 ms │                  25.16 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 90055.45ms │
│ Total Time (alamb_test_adapative_scan)   │ 83928.07ms │
│ Average Time (HEAD)                      │  2094.31ms │
│ Average Time (alamb_test_adapative_scan) │  1951.82ms │
│ Queries Faster                           │         15 │
│ Queries Slower                           │          0 │
│ Queries with No Change                   │         28 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘

The result is nice!

alamb · 2025-10-30T12:47:00Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │  2761.31 ms │                2612.38 ms │ +1.06x faster │
│ QQuery 1     │  1346.33 ms │                1347.10 ms │     no change │
│ QQuery 2     │  2498.54 ms │                2479.56 ms │     no change │
│ QQuery 3     │  1152.14 ms │                1204.33 ms │     no change │
│ QQuery 4     │  2253.57 ms │                2231.62 ms │     no change │
│ QQuery 5     │ 27531.42 ms │               28355.44 ms │     no change │
│ QQuery 6     │  4210.59 ms │                4209.56 ms │     no change │
│ QQuery 7     │  3375.54 ms │                3643.74 ms │  1.08x slower │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 45129.44ms │
│ Total Time (alamb_test_adapative_scan)   │ 46083.73ms │
│ Average Time (HEAD)                      │  5641.18ms │
│ Average Time (alamb_test_adapative_scan) │  5760.47ms │
│ Queries Faster                           │          1 │
│ Queries Slower                           │          1 │
│ Queries with No Change                   │          6 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.70 ms │                   2.29 ms │ +1.18x faster │
│ QQuery 1     │    51.06 ms │                  50.60 ms │     no change │
│ QQuery 2     │   138.61 ms │                 135.59 ms │     no change │
│ QQuery 3     │   166.77 ms │                 167.90 ms │     no change │
│ QQuery 4     │  1119.95 ms │                1065.90 ms │     no change │
│ QQuery 5     │  1550.72 ms │                1508.70 ms │     no change │
│ QQuery 6     │     2.25 ms │                   2.27 ms │     no change │
│ QQuery 7     │    55.76 ms │                  55.97 ms │     no change │
│ QQuery 8     │  1480.58 ms │                1436.01 ms │     no change │
│ QQuery 9     │  1860.29 ms │                1809.27 ms │     no change │
│ QQuery 10    │   403.80 ms │                 393.32 ms │     no change │
│ QQuery 11    │   466.42 ms │                 441.63 ms │ +1.06x faster │
│ QQuery 12    │  1417.37 ms │                1371.35 ms │     no change │
│ QQuery 13    │  2157.15 ms │                2131.16 ms │     no change │
│ QQuery 14    │  1301.84 ms │                1274.99 ms │     no change │
│ QQuery 15    │  1281.30 ms │                1202.79 ms │ +1.07x faster │
│ QQuery 16    │  2704.20 ms │                2643.74 ms │     no change │
│ QQuery 17    │  2677.83 ms │                2632.10 ms │     no change │
│ QQuery 18    │  5009.69 ms │                4921.76 ms │     no change │
│ QQuery 19    │   128.81 ms │                 127.11 ms │     no change │
│ QQuery 20    │  2009.43 ms │                2000.36 ms │     no change │
│ QQuery 21    │  2302.65 ms │                2313.71 ms │     no change │
│ QQuery 22    │  4005.96 ms │                3950.86 ms │     no change │
│ QQuery 23    │ 12976.26 ms │               12908.74 ms │     no change │
│ QQuery 24    │   225.06 ms │                 217.97 ms │     no change │
│ QQuery 25    │   522.26 ms │                 501.31 ms │     no change │
│ QQuery 26    │   224.96 ms │                 214.46 ms │     no change │
│ QQuery 27    │  2926.27 ms │                2871.48 ms │     no change │
│ QQuery 28    │ 22735.49 ms │               22368.77 ms │     no change │
│ QQuery 29    │   972.18 ms │                 991.88 ms │     no change │
│ QQuery 30    │  1389.54 ms │                1320.77 ms │     no change │
│ QQuery 31    │  1355.95 ms │                1311.98 ms │     no change │
│ QQuery 32    │  4361.40 ms │                4567.59 ms │     no change │
│ QQuery 33    │  5668.58 ms │                5617.54 ms │     no change │
│ QQuery 34    │  5998.49 ms │                6129.10 ms │     no change │
│ QQuery 35    │  2012.46 ms │                2007.28 ms │     no change │
│ QQuery 36    │   119.65 ms │                 121.07 ms │     no change │
│ QQuery 37    │    52.22 ms │                  54.13 ms │     no change │
│ QQuery 38    │   121.01 ms │                 124.07 ms │     no change │
│ QQuery 39    │   196.61 ms │                 201.45 ms │     no change │
│ QQuery 40    │    41.27 ms │                  45.22 ms │  1.10x slower │
│ QQuery 41    │    39.20 ms │                  41.46 ms │  1.06x slower │
│ QQuery 42    │    33.40 ms │                  33.46 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 94267.38ms │
│ Total Time (alamb_test_adapative_scan)   │ 93289.12ms │
│ Average Time (HEAD)                      │  2192.26ms │
│ Average Time (alamb_test_adapative_scan) │  2169.51ms │
│ Queries Faster                           │          3 │
│ Queries Slower                           │          2 │
│ Queries with No Change                   │         38 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 171.69 ms │                 165.08 ms │     no change │
│ QQuery 2     │  25.99 ms │                  25.81 ms │     no change │
│ QQuery 3     │  37.22 ms │                  38.23 ms │     no change │
│ QQuery 4     │  28.53 ms │                  28.44 ms │     no change │
│ QQuery 5     │  78.32 ms │                  77.63 ms │     no change │
│ QQuery 6     │  19.76 ms │                  19.67 ms │     no change │
│ QQuery 7     │ 224.45 ms │                 228.98 ms │     no change │
│ QQuery 8     │  34.21 ms │                  36.02 ms │  1.05x slower │
│ QQuery 9     │ 103.08 ms │                 110.50 ms │  1.07x slower │
│ QQuery 10    │  60.16 ms │                  61.94 ms │     no change │
│ QQuery 11    │  17.02 ms │                  18.37 ms │  1.08x slower │
│ QQuery 12    │  51.95 ms │                  52.19 ms │     no change │
│ QQuery 13    │  46.85 ms │                  49.00 ms │     no change │
│ QQuery 14    │  13.83 ms │                  14.09 ms │     no change │
│ QQuery 15    │  24.71 ms │                  24.72 ms │     no change │
│ QQuery 16    │  25.55 ms │                  24.62 ms │     no change │
│ QQuery 17    │ 151.95 ms │                 153.78 ms │     no change │
│ QQuery 18    │ 333.56 ms │                 333.10 ms │     no change │
│ QQuery 19    │  36.55 ms │                  37.85 ms │     no change │
│ QQuery 20    │  50.07 ms │                  49.79 ms │     no change │
│ QQuery 21    │ 359.75 ms │                 311.37 ms │ +1.16x faster │
│ QQuery 22    │  20.34 ms │                  21.12 ms │     no change │
└──────────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1915.54ms │
│ Total Time (alamb_test_adapative_scan)   │ 1882.29ms │
│ Average Time (HEAD)                      │   87.07ms │
│ Average Time (alamb_test_adapative_scan) │   85.56ms │
│ Queries Faster                           │         1 │
│ Queries Slower                           │         3 │
│ Queries with No Change                   │        18 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

alamb · 2025-10-30T13:15:42Z

The result is nice!

Yes indeed -- nicely done @hhhizzz

github-actions bot added the common Related to common crate label Oct 29, 2025

WIP: Pin to Adaptive Parquet Predicate Pushdown

a0cbf52

alamb force-pushed the alamb/test_adapative_scan branch from c181597 to a0cbf52 Compare October 30, 2025 11:04

github-actions bot removed the common Related to common crate label Oct 30, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

WIP: Pin to Adaptive Parquet Predicate Pushdown #18368

WIP: Pin to Adaptive Parquet Predicate Pushdown #18368

Uh oh!

alamb commented Oct 29, 2025 •

edited

Loading

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

hhhizzz commented Oct 30, 2025

Uh oh!

zhuqi-lucas commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

zhuqi-lucas commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

WIP: Pin to Adaptive Parquet Predicate Pushdown #18368

Are you sure you want to change the base?

WIP: Pin to Adaptive Parquet Predicate Pushdown #18368

Uh oh!

Conversation

alamb commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

hhhizzz commented Oct 30, 2025

Uh oh!

zhuqi-lucas commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

zhuqi-lucas commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alamb commented Oct 29, 2025 •

edited

Loading