Skip to content

Rewrite ParquetRecordBatchStream (async API) in terms of the PushDecoder #8677

@alamb

Description

@alamb

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

We added the ParquetPushDecoder in #7997

One of the rationales is to avoid duplicating the control logic between the Async Reader and the Sync Reader.

However, it actually (temporarily) makes the problem worse by adding a 3rd copy of the control logic ala the xkcd standards effect

image

Describe the solution you'd like

  1. Refactor the ParquetRecordBatchStream to use ParquetPushDecoder

Describe alternatives you've considered

Additional context

Metadata

Metadata

Assignees

Labels

enhancementAny new improvement worthy of a entry in the changelog

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions