-
Notifications
You must be signed in to change notification settings - Fork 290
[JS API] Add text_generation/benchmark_genai.js sample #2826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[JS API] Add text_generation/benchmark_genai.js sample #2826
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds a JavaScript benchmark sample for LLM performance testing, similar to the existing Python benchmark. It introduces the yargs
package for command-line argument parsing and exposes a new add()
method on the PerfMetrics
class to enable metric aggregation in JavaScript.
Key changes:
- Added
benchmark_genai.js
sample with comprehensive CLI options for benchmarking LLMs - Exposed
PerfMetrics.add()
method in JavaScript bindings to support metric aggregation - Updated documentation to improve clarity and consistency
Reviewed Changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
File | Description |
---|---|
samples/js/text_generation/benchmark_genai.js | New benchmarking sample with CLI argument parsing and performance metric collection |
src/js/src/perf_metrics.cpp | Added add() method binding to enable PerfMetrics aggregation |
src/js/include/perf_metrics.hpp | Added method declarations for add() and get_value() |
src/js/src/helper.cpp | Added template specialization to unwrap PerfMetrics objects |
src/js/include/helper.hpp | Added unwrap template function declaration |
src/js/lib/pipelines/llmPipeline.ts | Added TypeScript interface for the new add() method |
src/js/tests/module.test.js | Added test coverage for the new add() method |
samples/js/package.json | Added yargs dependency for CLI argument parsing |
samples/js/text_generation/README.md | Updated documentation for the new benchmark sample |
samples/python/text_generation/README.md | Minor grammar and clarity improvements |
Files not reviewed (1)
- samples/js/package-lock.json: Language not supported
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
indirect=["convert_model"], | ||
) | ||
def test_js_sample_benchmark_genai(self, convert_model, prompt, sample_args): | ||
if sys.platform == 'darwin': |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks good, only dependencies is required to be updated or discussed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes were made with upcoming releases of openvino-node
and openvino-genai-node
in mind.
Co-authored-by: Kirill Suvorov <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 11 out of 12 changed files in this pull request and generated 3 comments.
Files not reviewed (1)
- samples/js/package-lock.json: Language not supported
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
5099a65
Description
Add new sample for js api similar to python one.
Add yargs to manage the sample arguments.
Expose
PerfMetrics.add()
as JavaScript doesn't support operator overloading to return custom objects like Python does.Run sample in tests/python_tests/samples/test_benchmark_genai.py
Ticket: CVS-172877