POC: conditional creation of internal topics in Kafka Connect #20384
+31
−7
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a small POC to demonstrate the possibility of disabling the creation of internal topics in Kafka Connect.
It might be useful in cases where internal topics contain information about a large number of connectors, and one or more of the original topics were accidentally removed or renamed in the configuration.
The new mode would provide the ability to:
This way, there is a chance to troubleshoot the root cause instead of unexpected state of all connectors (for example, if the config topic is still present but the offset topic was accidentally removed or renamed, all the connectors will start from scratch).
Out of scope for this POC: