Introduce JIT code generation for performance improvement #1850
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
With JIT codegen, we could generate specific code for each query to reduce branching overhead from the generalized interpret mode execution. Furthermore, we could reduce the memory footprint during the execution by chaining multiple Arrow compute kernels together and reusing the intermediate vectors.
The row format we introduced lately in #1782 is another motivating use case for JIT code generation. We could eliminate most branching during row to batch or batch to row conversion since the branching pattern is set by schema, enjoy the potential advantage of several use cases for row format, and pay the least price during format conversion.
Describe the solution you'd like
- Create a generalized code constructing API that could fulfill our needs on expressing query-specific exec functions.
- Utilize Cranelift Code Generator, which is stable enough and can fulfill our use cases currently, eliminating branch and glue code fragments into runnable functions.
Describe alternatives you've considered
Build code generation runtime over LLVM instead of Cranelift. However, I think LLVM is over-killing since most of the performance-intensive computations are done at the Arrow kernel.
Additional context
We also have an item in our Roadmap: "Push-based query execution and code generation."