[Discuss] Release cadence / patch releases / Long Term Supported (lts) minor releases

[andygrove]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

• We release a major DataFusion version with breaking changes every two weeks.
• This creates work for downstream projects.

Describe the solution you'd like

• In parallel with the frequent major releases, offer "long-term support" for one major version for an extended period (perhaps three months?).
• This would allow downstream projects to pick up bug fixes and performance improvements without much work and then schedule major upgrades once per quarter.
• Other projects can upgrade every two weeks if they wish to.

Describe alternatives you've considered
Keep things as they are.

Additional context

[tustvold]

Is there a risk this would encourage people to only update once every 3 months? I'm not sure we're at a level of API stability where performing 3 months of upgrades in one go wouldn't be a fraught experience?

[andygrove]

Is there a risk this would encourage people to only update once every 3 months? I'm not sure we're at a level of API stability where performing 3 months of upgrades in one go wouldn't be a fraught experience?

Yes, that is definitely a risk.

[alamb]

I would be in favor of a LTS branch for sure -- the thing that is needed is the maintainer bandwidth to manage it (for example, to backport API compatible changes)

cc @maxburke

[alamb]

I think the upgrade situation is better than it was previously. For example we now document major upgrades and

• https://datafusion.apache.org/library-user-guide/upgrading.html

However, it still requires lots of work