[Backport] Improve the find_received_certificates process (#4708)
#4762
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bart-linera
force-pushed
the
improve_find_received_certificates_conway
branch
7 times, most recently
from
f4b3760 to
824cee5
Compare
Add some debugging Restructure the code; use the new flow for background sync Reduce some of the logs to trace Update Cargo.locks Fix cancellation, remove sync_sleep_ms Refactor the code into smaller functions Fix web and a comment Fix CLI.md Retry downloading blocks if we receive invalid certificates Move ReceivedLogs and ValidatorTrackers to separate files; add comments Perform find_received_certificates in batches Use a similar strategy when downloading certificates as when downloading blobs Don't try downloading certificates from validators who don't have them Improve logging Change the order of methods Fix CLI.md and web Address review comments Shuffle the nodes with each iteration, so that we download from different nodes every time Remove a superfluous check Improve a comment Fix a Clippy issue Do not reuse faulty validators Remove unnecessary pub(super) Add a unit test for LazyBatches Fix formatting Improve the unit test Address some of the review comments Improve batching Fix potential infinite loop when a committee is missing Address more comments Fix CLI.md Fix web Improve batching: take more than 1 block per chain at once Make test exercise number of blocks per chain Add a test with two validators Move ValidatorTracker to the bottom of the file Optimize ValidatorTracker Address review comments Fix a type conversion Update a Cargo.lock Improve the ValidatorTrackers API Add a tracker filtering test
bart-linera
force-pushed
the
improve_find_received_certificates_conway
branch
from
824cee5 to
f342866
Compare
|
Compatibility test is not passing |
|
I'll re-run it to see if it's a legitimate issue, or just flakiness. |
ma2bd
approved these changes
Oct 11, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
find_received_certificatescurrently downloads all of the sender certificates for a chain from all the validators, before processing them. This is very inefficient with chains receiving lots of messages.Proposal
Collect the certificates to be downloaded from a quorum, and then try to download each certificate once. Process the certificates every time a download for a chain finishes.
This is the backport of #4708 onto
testnet_conway.Test Plan
CI, manual testing if syncing with Testnet Conway is improved
Release Plan
testnetbranch, thenLinks