Skip to content

Conversation

JAicewizard
Copy link
Contributor

I wasn't fully done with the PR, here are the implementations for parallel row and chunk sources (I will add parallel chunk tmrw).

The chunk version is, as expected, marginally better (~10%). I think this can be further improved by having faster chunk access in one way or another, as the bottleneck is still just setting the values and everything in involved with that

@taniabogatsch taniabogatsch added the feature / enhancement Code improvements or a new feature label Jan 28, 2025
@JAicewizard JAicewizard force-pushed the improve_appender_speed branch from 3dc890e to 9a1f723 Compare February 12, 2025 16:22
@JAicewizard
Copy link
Contributor Author

@taniabogatsch I think the last remaining question is, is the schema actually usefull? Should it be kept in the API or removed.

Besides that I think this is ready for a review. (I will remove the draft once the schema issue is resolved)

@taniabogatsch taniabogatsch self-requested a review February 17, 2025 09:43
@taniabogatsch
Copy link
Collaborator

Hi @JAicewizard. Thanks for the PR. This time, I'll not merge anything before it is ready. ;) We've just moved to v2 on main. Let me know once you've resolved all merge conflicts, and then I'll have a look at reviewing it and also at the schema question.

@taniabogatsch taniabogatsch added the changes requested Changes have been requested to a PR or issue label Mar 17, 2025
@JAicewizard
Copy link
Contributor Author

I have a very busy couple of weeks ahead, generally this semester is quite full with projects so little time for this ATM. (I've tried to bump my duckdb extension to 1.2 for the past couple of weeks as well, that has been eating all my duckdb-time for now)

@taniabogatsch
Copy link
Collaborator

No worries - I hope the projects go well, and we'll get back to this PR whenever your schedule frees up again. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changes requested Changes have been requested to a PR or issue feature / enhancement Code improvements or a new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants