[Spike] Direct TX #2514

alpe · 2025-08-04T08:50:17Z

🚧 WIP - early version

Introduces

DirectTxSequencer - an adapter to the existing sequencer that prepends direct-TX on next batch request
DirectTxReaper - worker that pulls direct-TX from the DA and sends them to the DirectTxSequencer
coresequencer.DirectTX - transfer object
DirectTXBatchQueue - persistent queue for DirectTX
- pb.DirectTX - protobuf to persistent objects in DirectTXBatchQueue to maintain state

Overview

github-actions · 2025-08-04T08:50:38Z

The latest Buf updates on your PR. Results from workflow CI and Release / buf-check (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`⏩ skipped`	`✅ passed`	`✅ passed`	Aug 4, 2025, 8:50 AM

gemini-code-assist

Summary of Changes

Hello @alpe, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

I've been working on a significant spike to introduce a mechanism for "direct transactions." The core idea is to allow certain transactions to be submitted directly to the Data Availability (DA) layer, bypassing the traditional mempool. My changes introduce a DirectTxReaper that actively fetches these transactions from DA and a DirectTxSequencer that prioritizes their inclusion in blocks. This provides a robust way to ensure transactions are processed even if they don't go through the standard mempool, which is particularly useful for forced inclusion scenarios. I've also updated the node's configuration to support parameters for controlling this forced inclusion behavior and added an end-to-end test to validate the entire flow.

Highlights

Direct Transaction Reaper: I've introduced the DirectTxReaper component, which actively monitors the Data Availability (DA) layer for direct transactions. This reaper is responsible for fetching these transactions and submitting them to the sequencer for inclusion, ensuring that transactions posted directly to DA are eventually processed.
Direct Transaction Sequencer: I've implemented a new DirectTxSequencer interface and a concrete implementation that wraps an existing sequencer. This new sequencer prioritizes direct transactions retrieved by the DirectTxReaper before considering transactions from the regular mempool. This design allows for flexible integration without altering the underlying sequencer logic.
Configurable Forced Inclusion: I've updated the node's configuration to include ForcedInclusionConfig parameters, such as MaxInclusionDelay and MinDADelay. These settings will allow for fine-grained control over when direct transactions, once seen on the DA layer, must be included in a block.
End-to-End Testing for Direct Transactions: I've added a new end-to-end test (direct_tx_da_e2e_test.go) to validate the entire flow of direct transaction force inclusion. This test simulates submitting a transaction directly to the DA layer and then verifies its eventual inclusion in a block, confirming the functionality works as intended.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a 'Direct TX' feature, allowing transactions to be submitted directly to the Data Availability layer. The implementation is a good starting point for this spike, but it contains several critical issues that need to be addressed. These include potential panics, incorrect hashing logic, and bugs in data processing loops. There are also several hardcoded values that should be made configurable for better flexibility. The new tests have some issues as well, such as incorrect mock implementations. Addressing these points will significantly improve the robustness and correctness of this new feature.

gemini-code-assist · 2025-08-04T08:52:40Z

block/direct_tx.go

+	//}
+	h := m.headerCache.GetItem(daHeight)
+	if h == nil {
+		panic("header not found in cache") // todo (Alex): not sure if headers are always available before data. better assume not


Using panic here can crash the entire node if a header is not found in the cache. While the todo comment acknowledges this, it's a critical issue. The function should handle this case gracefully by logging an error and returning false, especially since it's possible for data to be available before its corresponding header.

Suggested change

panic("header not found in cache") // todo (Alex): not sure if headers are always available before data. better assume not

m.logger.Debug("header not found in cache, height:", daHeight)

return false

gemini-code-assist · 2025-08-04T08:52:40Z

block/direct_tx.go

+		//return false
+	}
+	for _, tx := range unsignedData.Txs {
+		txHash := sha256.New().Sum(tx)


The transaction hashing logic is incorrect. sha256.New().Sum(tx) computes the hash of an empty input and appends the result to tx, which is not the intended behavior. To compute the SHA256 hash of the transaction, you should use sha256.Sum256(tx).

txHashBytes := sha256.Sum256(tx) txHash := txHashBytes[:]

gemini-code-assist · 2025-08-04T08:52:40Z

block/direct_tx.go

+		txHash := sha256.New().Sum(tx)
+		d := DirectTransaction{
+			TxHash:            txHash,
+			TX:                unsignedData.Txs[0],


There is a bug in this loop. The code is iterating over unsignedData.Txs with the loop variable tx, but inside the DirectTransaction struct, it's always assigning unsignedData.Txs[0]. This means every transaction in the DirectTransaction list will have the content of the first transaction. It should use the loop variable tx instead.

TX: tx,

gemini-code-assist · 2025-08-04T08:52:40Z

block/direct_tx_reaper.go

+			if !has {
+				newTxs = append(newTxs, sequencer.DirectTX{
+					TX:              tx,
+					ID:              result.IDs[i],


There's a critical bug in how the blob ID is being assigned to direct transactions. The code uses the transaction's index within a blob (i) to look up an ID from result.IDs, which is a list of blob IDs. This is incorrect because all transactions within a single blob should share the same blob ID. Furthermore, if a blob contains more transactions than there are total blob IDs for that height, this will cause a panic due to an index out-of-range error.

The logic should be to iterate through blobs with their index to get the corresponding blob ID, and then assign that same blob ID to all transactions found within that blob.

gemini-code-assist · 2025-08-04T08:52:40Z

block/direct_tx_reaper_test.go

+func (m *MockDirectTxSequencer) SubmitDirectTxs(ctx context.Context, txs [][]byte) error {
+	args := m.Called(ctx, txs)
+	return args.Error(0)


The mock implementation of SubmitDirectTxs has an incorrect signature. The interface expects ...sequencer.DirectTX, but the mock is defined with [][]byte. This will cause a compilation error because the mock does not satisfy the DirectTxSequencer interface. The test needs to be updated to use the correct type.

Suggested change

func (m *MockDirectTxSequencer) SubmitDirectTxs(ctx context.Context, txs [][]byte) error {

args := m.Called(ctx, txs)

return args.Error(0)

func (m *MockDirectTxSequencer) SubmitDirectTxs(ctx context.Context, txs ...sequencer.DirectTX) error {

args := m.Called(ctx, txs)

return args.Error(0)

gemini-code-assist · 2025-08-04T08:52:41Z

apps/testapp/cmd/run.go

@@ -88,12 +88,12 @@ var RunCmd = &cobra.Command{
 		if err != nil {
 			return err
 		}
-
+		directTXSeq := single.NewDirectTxSequencer(sequencer, logger, datastore, 100) // todo (Alex): find a good default


The maxSize for the DirectTxSequencer is hardcoded to 100. This should be exposed as a configuration parameter to avoid magic numbers in the code and allow for easier adjustments.

gemini-code-assist · 2025-08-04T08:52:41Z

block/manager.go

+		DAIncludedHeight: m.daIncludedHeight.Load(),
+		Id:               []byte(m.genesis.ChainID),
+		LastBatchData:    m.lastBatchData,
+		MaxBytes:         defaultMaxBytes, // todo (Alex): do we need to reserve some space for headers and other data?


The defaultMaxBytes constant is hardcoded. This value should be configurable, for example, through the node's configuration file. This would provide more flexibility for different deployment scenarios.

gemini-code-assist · 2025-08-04T08:52:41Z

sequencers/single/direct_tx_queue.go

+	err = bq.db.Delete(ctx, ds.NewKey(key))
+	if err != nil {
+		// Log the error but continue
+		fmt.Printf("Error deleting processed batch: %v\n", err)


Using fmt.Printf for logging is not ideal. It bypasses the structured logger, making logs harder to parse, filter, and manage, especially in a production environment. Please use the provided logger instance instead.

gemini-code-assist · 2025-08-04T08:52:41Z

sequencers/single/direct_tx_queue.go

+			fmt.Printf("Error reading entry from datastore: %v\n", result.Error)
+			continue
+		}
+		var pbBatch pb.DirectTX
+		err := proto.Unmarshal(result.Value, &pbBatch)
+		if err != nil {
+			fmt.Printf("Error decoding batch for key '%s': %v. Skipping entry.\n", result.Key, err)
+			continue


Error conditions are being logged using fmt.Printf. This should be replaced with the structured logger (bq.logger if available, or passed in) to ensure consistent and manageable logging. Direct prints to stdout/stderr should be avoided.

gemini-code-assist · 2025-08-04T08:52:41Z

test/e2e/direct_tx_da_e2e_test.go

+	require.Eventually(t, func() bool {
+		if evm.CheckTxIncluded(t, tx.Hash()) {
+			return true
+		}
+		//go func() { // submit any TX to trigger block creation
+		//	tx := evm.GetRandomTransaction(t, TestPrivateKey, TestToAddress, DefaultChainID, DefaultGasLimit, &globalNonce)
+		//	evm.SubmitTransaction(t, tx)
+		//}()
+		return false
+	}, 1*time.Second, 500*time.Millisecond, "Transaction should be included in a block")


The timeout for this require.Eventually is set to 1 second, which might be too short for an end-to-end test that involves DA layer interaction and block production. This could lead to flaky tests. Consider increasing the timeout to a more generous value, like 10 or 15 seconds, to ensure the test is reliable.

alpe · 2025-08-04T09:28:39Z

alex_questions.md

+  random bytes
+- Should we always include direct TX although they may be duplicates to mempool TX? spike: yes
+- Should we fill the block space with directTX if possible or reserve space for mempool TX
+


What should we do when a sequencer was not able to add a direct-TX within the time window?

tac0turtle · 2025-08-07T08:54:38Z

core/sequencer/direct_tx.go

+
+	// SubmitDirectTxs adds direct transactions to the sequencer.
+	// This method is called by the DirectTxReaper.
+	SubmitDirectTxs(ctx context.Context, txs ...DirectTX) error


is this for submission to the DA layer?

This method is for submitting direct-tx from the DA to the sequencer. The name is analogue to SubmitBatchTxs which is used to submit tx from the mempool to the sequencer. The name is not very intuitive though.

alpe added 3 commits August 1, 2025 14:46

x

2067294

x

adaea04

x

5fcbe58

github-project-automation bot added this to Evolve Aug 4, 2025

github-actions bot assigned alpe Aug 4, 2025

gemini-code-assist bot reviewed Aug 4, 2025

View reviewed changes

alpe commented Aug 4, 2025

View reviewed changes

tac0turtle reviewed Aug 7, 2025

View reviewed changes

	panic("header not found in cache") // todo (Alex): not sure if headers are always available before data. better assume not
	m.logger.Debug("header not found in cache, height:", daHeight)
	return false

[Spike] Direct TX #2514

Are you sure you want to change the base?

[Spike] Direct TX #2514

Uh oh!

Conversation

alpe commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Uh oh!

github-actions bot commented Aug 4, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

alpe Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

tac0turtle Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

alpe Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alpe commented Aug 4, 2025 •

edited

Loading