Just chiming in here to share some of the takeaways that the ProbeLab team has gathered when analysing the IDONTWANT addition to GossipSub. Takeaways that should help motivate the extra effort to implement extra complex changes in the internal structure to enable the canceling any ongoing RPC.

The main highlight that we could extract from the traces produced by hermes is that around 70% of the duplicates that it received came from peers that it had already notified with a previous IDONTWANT message.
This could indicate that the single check for "unwanted" msg IDs before calling the SendRPC call isn't enough to prevent duplicates, as:

The transmission of IDONTWANTs isn't arriving by the time we start transmitting bytes
The IDONWANT study isn't implemented and therefore, there is no cancellation of the msg transmission as the spec suggests.

The following figure shows the CDF of the elapsed time between the IDONTWANT notification sent to a peer (for a msg ID) and the arrival of the duplicated message from the same peer (and the same msg ID), supporting that the remote node should have enought time to cancel any ongoing transmission.

As a result of the study and a follow-up discussion with @MarcoPolo and @cskiraly on how could we address the issue, the idea of shifting GossipSub from a single-stream protocol to a multiple-stream one (on per msg) has popped up. Change that could:

enable concurrent RPCs to the same Peer
cancel/reset any stream where we know we are sending a specific message.

Cancel queued Publish RPCs if IDONTWANT arrives in the meantime #611

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Cancel queued `Publish` RPCs if `IDONTWANT` arrives in the meantime #611