Skip to content

Conversation

@rymnc
Copy link
Member

@rymnc rymnc commented Jul 2, 2025

Added

A new benchmarking group for niop operations in the alu.rs file. The changes add comprehensive benchmarks for various operations (ADD, SUB, MUL, EXP, SLL, XNOR) across different operand widths (U8, U16, U32)

Additions to benchmarking:

  • A new niop benchmarking group was added to evaluate the performance of narrow integer operations (niop) with different operand widths (U8, U16, U32).
  • Introduced the niop_bench macro to simplify the creation of benchmarks for operations like ADD, SUB, MUL, EXP, SLL, and XNOR.
  • Benchmarks iterate over generated operand pairs for each width, ensuring a variety of test cases for each operation.

Breaking

  • Used a new fuel-vm 0.64.0 release which brings breaking changes: https://github.com/FuelLabs/fuel-vm/releases/tag/v0.64.0
  • The default gas cost has been changed according to new optimizations and a new benchmark machine for a local node.
  • Receipts now are behind an Arc pointer to avoid unnecessary cloning. It affects transaction states and preconfirmations.
  • The deploy_contract_with_id now works with [u8] instead of the Contract type.

Changed

Actualized benchmarks with the performance of the network. With this change we should be able to run benchmarks on nightly bases.

Added several optimizations to the code base:

  • Executor now reuses the memory instance between blocks production and validation avoiding allocations.
  • Receipts are not cloned anymore from the VM. Instead, they are extracted from the VM and stored ina s shared pointer.
  • The gas usage for script is discovered in a faster way.
  • Create transaction doesn't require a cloning of the contract bytecode anymore to calculate its ContractId.
  • Serialization and deserialization of the bytes became much faster for canonical, postcard, bincode codecs.

@rymnc rymnc added the no changelog Skip the CI check of the changelog modification label Jul 2, 2025
@rymnc rymnc self-assigned this Jul 2, 2025
@rymnc rymnc marked this pull request as ready for review July 2, 2025 16:32
Copy link
Member

@MitchTurner MitchTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s there somewhere I can read up on what niop is? I've been out of the loop on it.

edit: The description helps haha:

A new niop benchmarking group was added to evaluate the performance of narrow integer operations (niop) with different operand widths (U8, U16, U32).

Still unfamiliar with what it's for, but this helps me approximate :)

);

{
let mut niop = c.benchmark_group("niop");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think this would be more readable as niop_benchmark_group. It's confusing passing niop to something named niop_bench

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in ee93d38

Copy link
Member Author

@rymnc rymnc Jul 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

@xgreenx xgreenx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, how we convert results of these benchmark into the actual cost? Because we have only one gas cost which we need to fill, but we have 18 benchmarks for it. I see how we can use it in sanity checks, but I don't understand how does it work for the collect binary=D

@rymnc
Copy link
Member Author

rymnc commented Jul 3, 2025

Hmm, how we convert results of these benchmark into the actual cost? Because we have only one gas cost which we need to fill, but we have 18 benchmarks for it. I see how we can use it in sanity checks, but I don't understand how does it work for the collect binary=D

checking

@rymnc
Copy link
Member Author

rymnc commented Jul 5, 2025

Hmm, how we convert results of these benchmark into the actual cost? Because we have only one gas cost which we need to fill, but we have 18 benchmarks for it. I see how we can use it in sanity checks, but I don't understand how does it work for the collect binary=D

the collect binary groups benchmark groups and takes the average from them i believe.

fn into_relative_costs(self, mut costs: Costs) -> Costs {
let baseline = self.get_baseline();
let State {
mut ids, groups, ..
} = self;
let relative = groups.into_iter().filter_map(|(name, samples)| {
let relative = samples
.into_iter()
.find(|sample| sample.starts_with(&name) && ids.contains_key(sample))
.and_then(|sample| ids.remove(&sample))
.map(|mean| Cost::Relative(map_to_ratio(baseline, mean)))?;
Some((name, relative))
});
costs.0.extend(relative);
costs
}

xgreenx
xgreenx previously approved these changes Jul 26, 2025
@xgreenx xgreenx requested a review from a team as a code owner September 17, 2025 00:44
@xgreenx xgreenx removed the no changelog Skip the CI check of the changelog modification label Oct 9, 2025
@rymnc rymnc enabled auto-merge (squash) October 10, 2025 08:51
@xgreenx xgreenx disabled auto-merge October 10, 2025 09:13
@xgreenx xgreenx merged commit df471f9 into master Oct 10, 2025
37 checks passed
@xgreenx xgreenx deleted the chore/update-benchmarks branch October 10, 2025 09:13
@github-actions github-actions bot mentioned this pull request Oct 18, 2025
xgreenx added a commit that referenced this pull request Oct 18, 2025
## Version 0.47.0

### Breaking
- [3014](#3014): Extend
`CoinConfig` to also support being configured with private keys.
- [3050](#3050): - Used a new
`fuel-vm 0.64.0` release which brings breaking changes:
https://github.com/FuelLabs/fuel-vm/releases/tag/v0.64.0
- The default gas cost has been changed according to new optimizations
and a new benchmark machine for a local node.
- Receipts now are behind an `Arc` pointer to avoid unnecessary cloning.
It affects transaction states and preconfirmations.
- The `deploy_contract_with_id` now works with `[u8]` instead of the
`Contract` type.
- [3083](#3083): Disable
gossip subscription for transactions by default. It can be enabled using
--subscribe-to-transactions flag.
- [3094](#3094): Fixed the bug
where we charged for additional 40 entries on the GraphQL query
- [3099](#3099): Use
hash-based `balance_root` and `state_root` fields in tx inputs and
outputs
- [3110](#3110): Bump Rust
version to 1.90.0 and fuel-vm to 0.65.0
- [3117](#3117): The algorithm
to calculate the genesis state root and the balance root was changed to
not use SMTs. This change doesn't affect mainnet, testnet, and devnet,
because these networks didn't use any balances or storage slots in the
state config. But the change is breaking for anyone who used them.

### Added
- [3023](#3023): Add a method
to wait for `gas_price` to finish syncing upon receiving a new block,
and modify the test accordingly.
- [3049](#3049): Support
awating of DA block height on pre defined block production.
- [3050](#3050): A new
benchmarking group for `niop` operations in the `alu.rs` file. The
changes add comprehensive benchmarks for various operations (`ADD`,
`SUB`, `MUL`, `EXP`, `SLL`, `XNOR`) across different operand widths
(`U8`, `U16`, `U32`)
  
  ### Additions to benchmarking:
  
* A new `niop` benchmarking group was added to evaluate the performance
of narrow integer operations (`niop`) with different operand widths
(`U8`, `U16`, `U32`).
* Introduced the `niop_bench` macro to simplify the creation of
benchmarks for operations like `ADD`, `SUB`, `MUL`, `EXP`, `SLL`, and
`XNOR`.
* Benchmarks iterate over generated operand pairs for each width,
ensuring a variety of test cases for each operation.
- [3059](#3059): Enable debug
logging in predicates and scripts using ECAL. Upgrades fuel-vm to
0.63.0.
- [3085](#3085): Add
scaffolding for the new block aggregator service
- [3092](#3092): Add adapter
for the block aggregator DB
- [3096](#3096): Add
subscription manager for core service
- [3097](#3097): Add block
source adapter using fuel storage and block importer

### Changed
- [3050](#3050): Actualized
benchmarks with the performance of the network. With this change we
should be able to run benchmarks on nightly bases.
  
  Added several optimizations to the code base:
- Executor now reuses the memory instance between blocks production and
validation avoiding allocations.
- Receipts are not cloned anymore from the VM. Instead, they are
extracted from the VM and stored ina s shared pointer.
  - The gas usage for script is discovered in a faster way.
- `Create` transaction doesn't require a cloning of the contract
bytecode anymore to calculate its `ContractId`.
- Serialization and deserialization of the bytes became much faster for
canonical, `postcard`, `bincode` codecs.
- [3104](#3104): Automatically
adjust relayer log page size to prevent sync failures from oversized
Ethereum RPC responses.

### Removed
- [3117](#3117): Removed SMT
usage from STF and benchmarks.

## What's Changed
* feat: Support private keys in coin config and log these if present by
@netrome in #3014
* Update network versions on README by @github-actions[bot] in
#3076
* Support awating of DA block height on pre defined block production by
@Guocork in #3049
* Configurable --subscribe-to-transactions by @Dentosal in
#3083
* Update network versions on README by @github-actions[bot] in
#3082
* Block Aggregator Service Scaffolding by @MitchTurner in
#3085
* Update network versions on README by @github-actions[bot] in
#3095
* chore(gas_price_service): add a method to wait until storages are
synced by @Guocork in #3023
* Fixed calculation of complexity by @xgreenx in
#3094
* Block aggregator db adapter by @MitchTurner in
#3092
* Enable debug logging in predicates and scripts using ECAL by @Dentosal
in #3059
* Add new block subscription to inner service by @MitchTurner in
#3096
* Block source for aggregator by @MitchTurner in
#3097
* Fixed flaky syscall integration tests by @xgreenx in
#3103
* Add dynamic adjustment for relayer log page size. by @its-saeed in
#3104
* chore(benchmarks): add benchmarks for niop opcode in all operation
modes by @rymnc in #3050
* chore(codeowners): update with current codeowners by @rymnc in
#3114
* Bump rust to 1.90 and fuel-vm 0.65.0 by @zees-dev in
#3110
* Use hash-based balance_root and state_root fields by @Dentosal in
#3099
* Removed SMT usage from STF and benchmarks by @xgreenx in
#3117

## New Contributors
* @its-saeed made their first contribution in
#3104
* @zees-dev made their first contribution in
#3110

**Full Changelog**:
v0.46.0...v0.47.0

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants