Skip to content

[lake/iceberg] Implement IcebergLakeCatalog for PK Tables #1372

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Aug 8, 2025

Conversation

MehulBatra
Copy link
Contributor

Purpose

Add support for IcebergLakeCatalog for fluss primary key tables

Linked issue: close #1337

Brief change log

  • Added new file for IcebergLakeCatalog implementing LakeCatalog
  • Modified IcebergLakeStorage.createLakeCatalog method
  • In progress

Tests

In progress

API and Format

In progress

Documentation

In progress

@MehulBatra MehulBatra changed the title in progress [lake/iceberg] Implement IcebergLakeCatalog for PK Tables Jul 20, 2025
@MehulBatra MehulBatra marked this pull request as draft July 20, 2025 15:23
@MehulBatra MehulBatra force-pushed the iceberg-catalog-setup-pktables branch from b57ff0b to 69df64b Compare July 24, 2025 20:31
@MehulBatra MehulBatra marked this pull request as ready for review July 27, 2025 21:17
@MehulBatra
Copy link
Contributor Author

@luoyuxia Please help me with the review, whenever you get time!

@luoyuxia luoyuxia requested a review from Copilot July 28, 2025 05:43
@luoyuxia luoyuxia self-assigned this Jul 28, 2025
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements IcebergLakeCatalog to add support for Fluss primary key tables in Iceberg lake storage. The implementation provides the infrastructure to create Iceberg tables that correspond to Fluss primary key tables with proper schema conversion and partitioning.

  • Adds a new IcebergLakeCatalog class implementing the LakeCatalog interface
  • Updates IcebergLakeStorage to return the new catalog implementation instead of throwing UnsupportedOperationException
  • Defines Iceberg version property in the Maven POM

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
IcebergLakeStorage.java Updates createLakeCatalog method to return IcebergLakeCatalog instance
IcebergLakeCatalog.java New implementation of LakeCatalog interface for Iceberg integration
pom.xml Adds iceberg.version property definition

Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MehulBatra Thanks for the pr. Left some comments. PTAL.
Also, please don't forget to add test to cover it.

Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MehulBatra Thanks for the pr. Left minor comments. PTAL

@MehulBatra MehulBatra requested a review from luoyuxia August 6, 2025 19:20
Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MehulBatra Left some comments again. PTAL

@MehulBatra MehulBatra requested a review from luoyuxia August 7, 2025 18:28
@luoyuxia
Copy link
Contributor

luoyuxia commented Aug 8, 2025

@MehulBatra I append a some commit to improve some code, PTAL. If you have no question, I'll merge later.

@MehulBatra
Copy link
Contributor Author

@MehulBatra I append a some commit to improve some code, PTAL. If you have no question, I'll merge later.

Thanks @luoyuxia for the review and feedback!

Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging......

@luoyuxia luoyuxia merged commit 0cba53b into apache:main Aug 8, 2025
3 checks passed
@SML0127
Copy link
Contributor

SML0127 commented Aug 9, 2025

@MehulBatra Thanks for the great work. I’ll follow up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Iceberg catalog should support primary key table
3 participants