Skip to content

Conversation

almilosz
Copy link
Contributor

@almilosz almilosz commented Oct 13, 2025

Description

Add new sample for js api similar to python one.
Add yargs to manage the sample arguments.
Expose PerfMetrics.add() as JavaScript doesn't support operator overloading to return custom objects like Python does.
Run sample in tests/python_tests/samples/test_benchmark_genai.py

Ticket: CVS-172877

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a JavaScript benchmark sample for LLM performance testing, similar to the existing Python benchmark. It introduces the yargs package for command-line argument parsing and exposes a new add() method on the PerfMetrics class to enable metric aggregation in JavaScript.

Key changes:

  • Added benchmark_genai.js sample with comprehensive CLI options for benchmarking LLMs
  • Exposed PerfMetrics.add() method in JavaScript bindings to support metric aggregation
  • Updated documentation to improve clarity and consistency

Reviewed Changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
samples/js/text_generation/benchmark_genai.js New benchmarking sample with CLI argument parsing and performance metric collection
src/js/src/perf_metrics.cpp Added add() method binding to enable PerfMetrics aggregation
src/js/include/perf_metrics.hpp Added method declarations for add() and get_value()
src/js/src/helper.cpp Added template specialization to unwrap PerfMetrics objects
src/js/include/helper.hpp Added unwrap template function declaration
src/js/lib/pipelines/llmPipeline.ts Added TypeScript interface for the new add() method
src/js/tests/module.test.js Added test coverage for the new add() method
samples/js/package.json Added yargs dependency for CLI argument parsing
samples/js/text_generation/README.md Updated documentation for the new benchmark sample
samples/python/text_generation/README.md Minor grammar and clarity improvements
Files not reviewed (1)
  • samples/js/package-lock.json: Language not supported

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@github-actions github-actions bot added the category: GGUF GGUF file reader label Oct 14, 2025
indirect=["convert_model"],
)
def test_js_sample_benchmark_genai(self, convert_model, prompt, sample_args):
if sys.platform == 'darwin':
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Log fragment from ticket.
Log from test that was failing

Copy link
Contributor

@Retribution98 Retribution98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good, only dependencies is required to be updated or discussed

Copy link
Contributor

@Retribution98 Retribution98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes were made with upcoming releases of openvino-node and openvino-genai-node in mind.

Copy link
Contributor

@Retribution98 Retribution98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Wovchena Wovchena requested a review from Copilot October 16, 2025 11:20
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 11 out of 12 changed files in this pull request and generated 3 comments.

Files not reviewed (1)
  • samples/js/package-lock.json: Language not supported

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@almilosz almilosz added this pull request to the merge queue Oct 20, 2025
@almilosz almilosz added this to the 2025.4 milestone Oct 21, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Oct 21, 2025
@Retribution98 Retribution98 added this pull request to the merge queue Oct 21, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Oct 21, 2025
@almilosz almilosz added this pull request to the merge queue Oct 21, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 21, 2025
@almilosz almilosz added this pull request to the merge queue Oct 22, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 22, 2025
@Retribution98 Retribution98 added this pull request to the merge queue Oct 22, 2025
Merged via the queue into openvinotoolkit:master with commit 5099a65 Oct 22, 2025
185 of 198 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants