-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Remove performance regression when we're subscribing with a regex #5202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
🎉 All Contributor License Agreements have been signed. Ready to merge. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR addresses a performance regression when subscribing with regex patterns in scenarios where few topics in the cluster match the pattern. The main optimization replaces the inefficient approach of copying all cached topics to a list with direct iteration over the metadata cache, and introduces deduplication to handle multiple cache entries with the same topic name.
- Replaces topic list creation with direct cache iteration using
TAILQ_FOREACH
- Adds a hash map for deduplication to avoid processing duplicate topic names
- Introduces early exit logic for already-matched topics to improve efficiency
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
src/rdkafka_metadata.c
Outdated
if (matched) | ||
/* | ||
* Just remove it from unmatched. | ||
* Topic was already add to |
Copilot
AI
Sep 23, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Line 1298 contains a grammatical error: 'add' should be 'added'.
* Topic was already add to | |
* Topic was already added to |
Copilot uses AI. Check for mistakes.
continue; | ||
RD_MAP_SET(&map, topic, topic); | ||
|
||
mdt = &rkmce->rkmce_mtopic; |
Copilot
AI
Sep 23, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] Extra space between 'mdt' and '=' creates inconsistent formatting. Should be single space like other assignments.
mdt = &rkmce->rkmce_mtopic; | |
mdt = &rkmce->rkmce_mtopic; |
Copilot uses AI. Check for mistakes.
f34492b
to
933adfa
Compare
… none of many topics in the cluster are matching it
933adfa
to
2834d65
Compare
and none of many topics in the cluster are matching it.