Skip to content

Conversation

Centril
Copy link
Contributor

@Centril Centril commented Sep 1, 2025

Description of Changes

Aternative to and closes #3210.
This version relies on pending_schema_changes.
The first commit adds clear_table to the datastore that's efficient and can be exposed to the module ABI in a follow up.
The second commit fixes drop_table.

API and ABI breaking changes

None

Expected complexity level and risk

3?

Testing

test_drop_table_is_transactional is amended to check TxData.

Copy link
Contributor

@Shubham8287 Shubham8287 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks Good, thanks!
Considering there will be a follow-up PR to fix this for commitlog replay.
This PR will make commitlog replay logic to empty dropped tables but we still need a patch to actually drop them from memory.

@Shubham8287 Shubham8287 mentioned this pull request Sep 2, 2025
1 task
Copy link
Contributor

@gefjon gefjon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holding off on approving until we get the replay parts of this merged and tested, but I like this.

@bfops bfops added the release-any To be landed in any release window label Sep 3, 2025
@Centril
Copy link
Contributor Author

Centril commented Sep 5, 2025

Holding off on approving until we get the replay parts of this merged and tested, but I like this.

I've fixed the replay parts, but we cannot integration test this until the add-column PR as simply dropping a table is not allowed.

@Centril Centril requested a review from gefjon September 5, 2025 13:11
@Centril Centril force-pushed the centril/clear-table-and-fix-drop-table branch from 08b70cf to 813f34c Compare September 8, 2025 12:15
let (tx_table, tx_blob_store, delete_table) = self
.tx_state
.get_table_and_blob_store_or_create_from(table_id, commit_table);
let mut rows_removed = tx_table.clear(tx_blob_store);
Copy link
Contributor

@Shubham8287 Shubham8287 Sep 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For tx_table, shouldn’t it be enough to just remove it from the BTreeMap of TxState::insert_tables?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it wasn't for the blob store you could do that. We could as an optimization check whether the table layout is fixed, in which case there cannot be any blob store references, making the removal sound.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, I don’t fully understand how we’re using blob_store, but for TxState::blob_store, won’t MutTxId::commit eventually drop it anyway?

Do you think is it a problem to occupy extra bytes in blob_store during the lifecycle of tx (until it commits or rollback)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-any To be landed in any release window
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants