Feature Request / Improvement

In the current IRC (RestCatalog) commit process, if a client receives a commitStateUnknown status, it won't try to reconcile the state to see if the commit was successful. This is different from how HiveCatalog or GlueCatalog handles things.

The primary reason for this is that the IRC commit is broken into multiple updates, and operations like schema evolution don't create a new snapshot, making it difficult to confirm success. Since the metadata.json is created directly in the catalog, it's not easy to reconcile and confirm if the request was successful.

However, for simpler cases like adding a new snapshot, we could reconcile the state. We could cross-reference the update request with the expected snapshot to be added, then reload the table and check if that snapshot is present.

While this needs more discussion and review, a good starting point would be to implement this reconciliation process for new snapshot additions and expand from there, as this will certainly be useful for clients to not give up and watermark stuff at their end.

Query engine

None

Willingness to contribute

• I can contribute this improvement/feature independently
• I would be willing to contribute this improvement/feature with guidance from the Iceberg community
• I cannot contribute this improvement/feature at this time