Skip to content

light: perf test initial commit #752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

light: perf test initial commit #752

wants to merge 5 commits into from

Conversation

@mitzkia
Copy link
Contributor

@mitzkia mitzkia commented Aug 5, 2025 •
edited
Loading

  • How to start perf measurement:
$ /usr/bin/poetry -C /source/tests/light run python -m pytest /source/tests/light/performance_tests/test_performance.py --installdir=/install --msg-size=1024 --input-msg-type=csv --run-time=30 --filterx-rule=empty,filterx-builtin-simple-functions --destination=empty,file-dst,clickhouse-dst,clickhouse-dst-for-filterx-builtin-simple-functions --flow-control=flow-control-off,flow-control-on

@mitzkia
Copy link
Contributor Author

mitzkia commented Aug 5, 2025 •
edited
Loading

The PR is not yet finished, but it works with limitations.
The perf results can be found with a grep:

$ grep -RnHi "INFO - Stats result" reports/last/

@mitzkia mitzkia marked this pull request as ready for review August 15, 2025 14:07
@mitzkia mitzkia force-pushed the light_performance_test branch from ae26b91 to 21504b8 Compare August 18, 2025 05:16
@mitzkia mitzkia marked this pull request as draft August 18, 2025 05:17
@mitzkia mitzkia force-pushed the light_performance_test branch 3 times, most recently from 087f4db to 9856796 Compare August 18, 2025 07:20
@alltilla alltilla self-requested a review August 26, 2025 09:32
@mitzkia
light/loggen: Proper check if loggen is already stopped
994dd41 
Signed-off-by: Andras Mitzki <[email protected]>
@mitzkia
light/fixtures: Add new execution start parameters
e5c5679 
- these parameters are valid only for performance test execution


Signed-off-by: Andras Mitzki <[email protected]>
@mitzkia
light/performance: Add various log generator functions:
0beae92 
- supported log types:
    - cef message
    - kv message
    - leef message
    - csv message
    - json message
    - parsed cef json message
    - parsed leef json message
    - eventlog xml message

Signed-off-by: Andras Mitzki <[email protected]>
@mitzkia
light/performance: Add initial performance test
ded62fd 
- tests can be executed from command line with a following extra parameters:

```
... --msg-size=1024 --input-msg-type=csv --run-time=30 --filterx-rule=empty,filterx-builtin-simple-functions --destination=empty,file-dst,clickhouse-dst,clickhouse-dst-for-filterx-builtin-simple-functions --flow-control=flow-control-off,flow-control-on
```

or

```
... --msg-size=1024 --input-msg-type=csv --msg-counter=100000 --filterx-rule=empty,filterx-builtin-simple-functions --destination=empty,file-dst,clickhouse-dst,clickhouse-dst-for-filterx-builtin-simple-functions --flow-control=flow-control-off,flow-control-on
```

Signed-off-by: Andras Mitzki <[email protected]>
@mitzkia mitzkia force-pushed the light_performance_test branch from 9856796 to ded62fd Compare September 15, 2025 08:54
@mitzkia
fixup
cfce6a8 
Signed-off-by: Andras Mitzki <[email protected]>
@mitzkia mitzkia marked this pull request as ready for review September 16, 2025 08:36
@axoflow axoflow deleted a comment from sonarqubecloud bot Sep 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alltilla alltilla Awaiting requested review from alltilla

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mitzkia