Skip to content

Conversation

jlon
Copy link

@jlon jlon commented Jul 14, 2025

What type of PR is this:

  • Feature

Which issues of this PR fixes :

When importing the primary key table, it supports ignoring deleted Row data, that is, changing DELETE to UPSERT. Especially in the case of FLink-CDC, when these parameters are turned on, the complete data can be retained when imported into Starrocks.
Fixes #

Problem Summary(Required) :

Checklist:

  • [ yes] I have added test cases for my bug fix or my new feature
  • [ yes ] This pr will affect users' behaviors
  • This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function

@jaogoy
Copy link
Collaborator

jaogoy commented Jul 28, 2025

What's the difference with #405

@jaogoy jaogoy requested review from banmoy and Copilot July 28, 2025 09:32
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for ignoring delete operations when importing data into primary key tables by introducing a new sink.ignore.delete configuration option. When enabled, DELETE operations are converted to UPSERT operations, which is particularly useful for Flink-CDC scenarios where complete data retention is desired.

Key changes:

  • Adds new configuration option sink.ignore.delete with default value false
  • Updates transformer interfaces and implementations to accept the new ignoreDelete parameter
  • Modifies transformation logic to convert DELETE operations to UPSERT when the option is enabled

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file
File Description
StarRocksSinkOptions.java Adds new SINK_IGNORE_DELETE configuration option and getter method
StarRocksIRowTransformer.java Updates interface to include ignoreDelete parameter in transform method
StarRocksTableRowTransformer.java Implements logic to convert DELETE to UPSERT when ignoreDelete is true
StarRocksGenericRowTransformer.java Implements logic to convert DELETE to UPSERT when ignoreDelete is true
RowDataSerializationSchema.java Adds ignoreDelete field and passes it to transformer
SinkFunctionFactory.java Passes ignoreDelete option to serialization schema constructor
StarRocksDynamicSinkFunction.java Updates transform call to include ignoreDelete parameter
StarRocksDynamicSinkFunctionV2.java Updates transform call to include ignoreDelete parameter
StarRocksTableRowTransformerTest.java Updates test to include new ignoreDelete parameter
StarRocksGenericRowTransformerTest.java Updates test to include new ignoreDelete parameter
Comments suppressed due to low confidence (2)

src/main/java/com/starrocks/connector/flink/row/sink/StarRocksTableRowTransformer.java:108

  • Missing opening brace for the else block. The else statement on line 104 should have an opening brace after 'else {'.
        }

src/main/java/com/starrocks/connector/flink/row/sink/StarRocksGenericRowTransformer.java:62

  • Missing opening brace for the else block. The else statement on line 58 should have an opening brace after 'else {'.
        }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants