Skip to content

Conversation

JoshuaMoelans
Copy link
Member

@JoshuaMoelans JoshuaMoelans commented Jul 4, 2025

Cherry-picked from #1297 (since we don't need the full alternative API for now). Merging into #1272 so we can send integers as int64.

Will initially be used mainly for the new Logs feature, but we should introduce (mostly test) this for other data we send too.


Introduces sentry_value_new_(u)int64()


#skip-changelog

Copy link

github-actions bot commented Jul 4, 2025

Messages
📖 Do not forget to update Sentry-docs with your feature once the pull request gets approved.

Generated by 🚫 dangerJS against 84fb966

@JoshuaMoelans JoshuaMoelans marked this pull request as ready for review July 4, 2025 11:49
@JoshuaMoelans
Copy link
Member Author

@sentry review

Copy link

seer-by-sentry bot commented Jul 7, 2025

On it! We are reviewing the PR and will provide feedback shortly.

Copy link

seer-by-sentry bot commented Jul 7, 2025

PR Description

This pull request introduces support for 64-bit integer types (signed and unsigned) within the Sentry native SDK's value system. This enhancement allows the SDK to accurately capture and transmit larger numerical values that exceed the range of the existing 32-bit integer type, improving data fidelity for applications dealing with such values.

Click to see more

Key Technical Changes

  1. Added new value types: SENTRY_VALUE_TYPE_INT64 and SENTRY_VALUE_TYPE_UINT64 are added to the sentry_value_type_t enum.
  2. Implemented new value creation functions: sentry_value_new_int64() and sentry_value_new_uint64() are introduced to create sentry_value_t instances holding 64-bit integer values.
  3. Implemented accessor functions: sentry_value_as_int64() and sentry_value_as_uint64() are added to retrieve the 64-bit integer values from a sentry_value_t.
  4. JSON serialization support: The sentry__jsonwriter_write_int64() and sentry__jsonwriter_write_uint64() functions are implemented to serialize the new value types into JSON format.
  5. Msgpack serialization support: The value_to_msgpack function is updated to handle the new value types when serializing to msgpack format.
  6. Internal representation: The thing_t struct's union is extended to accommodate int64_t and uint64_t values.
  7. Type conversion: Implemented type conversion from int32 and double to int64 and uint64.

Architecture Decisions

The design maintains the existing tagged-pointer approach for sentry_value_t, extending the thing_t union to store the 64-bit integer values on the heap. This approach avoids increasing the size of the sentry_value_t struct itself, preserving ABI compatibility. The new value types are also frozen by default, similar to doubles and strings.

Dependencies and Interactions

These changes primarily affect the sentry_value API and its internal implementation. They also interact with the JSON and msgpack serialization components. No new external dependencies are introduced. The changes are designed to be backward-compatible with existing code that uses the sentry_value API.

Risk Considerations

  1. Potential for data loss during type conversion: Converting from double to uint64_t involves truncation, which can lead to loss of precision. The code includes a TODO to address range validation, but this needs careful consideration.
  2. Buffer overflow during string conversion: The snprintf calls in sentry__jsonwriter_write_int64 and sentry__jsonwriter_write_uint64 use a fixed-size buffer. While the size is likely sufficient, it's crucial to ensure that it can accommodate the largest possible 64-bit integer value to prevent buffer overflows. The buffer size in sentry__value_stringify should also be consistent with the JSON writer.
  3. Double precision loss: Converting int64 and uint64 to double can result in precision loss due to the limited precision of double floating point numbers. This is handled by returning NaN if outside the double precision range, but this behavior needs to be clearly documented.

Notable Implementation Details

  1. The THING_TYPE constants are defined using #define which can be error-prone. Consider using an enum.
  2. The _int64_data and _uint64_data field names in the thing_t struct are not very descriptive. Consider renaming them to _int64 and _uint64 for clarity.
  3. The tests include cross-type conversion tests, but more comprehensive boundary and edge case testing is needed, especially for double to uint64 conversion and for very large int64/uint64 values.

@JoshuaMoelans JoshuaMoelans requested a review from supervacuus July 7, 2025 15:27
Copy link
Collaborator

@supervacuus supervacuus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a couple of issues that require further deliberation. We can pair up on this if you'd like.

@JoshuaMoelans JoshuaMoelans requested a review from supervacuus July 8, 2025 15:15
@JoshuaMoelans JoshuaMoelans merged commit 940bd6a into joshua/feat/logs_logger_module Jul 14, 2025
51 of 55 checks passed
@JoshuaMoelans JoshuaMoelans deleted the joshua/feat/logs_sentry_value_t_API_single_commit branch July 14, 2025 12:49
jpnurmi pushed a commit that referenced this pull request Jul 25, 2025
* add (u)int64 sentry_value_t type

* add value_to_msgpack missing switch cases

* remove undefined behavior test (C99 6.3.1.4)

* avoid Windows sized integer name collision

* cleanup & apply code review feedback

* more cleanup & remove type coercion
@jpnurmi
Copy link
Collaborator

jpnurmi commented Jul 25, 2025

Looks like sentry__value_from_json also needs some changes to avoid sentry__strtod_c and be able to handle > 2^53.

JoshuaMoelans added a commit that referenced this pull request Jul 28, 2025
* add sentry log API + send first logs

* fix log_level_as_string

* attach attributes to logs

* attach formatted message + args

* add to example

* add more attributes

* cleanup

* windows warning-as-error

* windows warning-as-error v2

* windows warning-as-error v2 (final)

* add unit tests for initial logs

* memleak attempted fix

* memleak attempted fix 2

* cleanup

* use `sentry_level_t` instead of new log level enum

* add SENTRY_LEVEL_TRACE to sentry_logger

* quick anti-brownout fix
- see #1274

* fix missing SENTRY_LEVEL_INFO string return

* fix logger level check + add test

* cleanup logs parameter extraction

* warn-as-error fix

* const char* fix

* static function

* feat(logs): add (u)int64 sentry_value_t type (#1301)

* add (u)int64 sentry_value_t type

* add value_to_msgpack missing switch cases

* remove undefined behavior test (C99 6.3.1.4)

* avoid Windows sized integer name collision

* cleanup & apply code review feedback

* more cleanup & remove type coercion

* update logs param conversion

* own uint64 string

* apply suggestions from code review
JoshuaMoelans added a commit that referenced this pull request Sep 23, 2025
* add sentry logs option

* add sentry logs option to example

* feat(logs): add sentry log API + send first logs (#1272)

* add sentry log API + send first logs

* fix log_level_as_string

* attach attributes to logs

* attach formatted message + args

* add to example

* add more attributes

* cleanup

* windows warning-as-error

* windows warning-as-error v2

* windows warning-as-error v2 (final)

* add unit tests for initial logs

* memleak attempted fix

* memleak attempted fix 2

* cleanup

* use `sentry_level_t` instead of new log level enum

* add SENTRY_LEVEL_TRACE to sentry_logger

* quick anti-brownout fix
- see #1274

* fix missing SENTRY_LEVEL_INFO string return

* fix logger level check + add test

* cleanup logs parameter extraction

* warn-as-error fix

* const char* fix

* static function

* feat(logs): add (u)int64 sentry_value_t type (#1301)

* add (u)int64 sentry_value_t type

* add value_to_msgpack missing switch cases

* remove undefined behavior test (C99 6.3.1.4)

* avoid Windows sized integer name collision

* cleanup & apply code review feedback

* more cleanup & remove type coercion

* update logs param conversion

* own uint64 string

* apply suggestions from code review

* fixed log parameter conversion

* update example to avoid warning-as-error

* feat(logs): batching (#1338)

* initial queue attempt

* add timer

* prototype double buffer approach

* update logs unit tests for batching

* replace timer with bgworker

* add first integration tests

* update example.c with correct log thread amounts

* cleanup

* add wait for 'adding' logs in flush logic

* go back to single queue for performance testing

* add time checks

* add ToDos + cleanup sentry_value_t cloning

* initial attempt

* cond_wait for timer + 'adding' spinlock

* add sleep for tests

* add sleep for tests

* force flush before attempting timer_worker shutdown

* add proper cond_wait for 'adding' counter

* revert to manual flush on shutdown instead of timer thread flush

* add separate timer_stop atomic

* cleanup + replace 'adding' cond_wait by pure spinlock

* change bgworker for simpler thread implementation

* cleanup

* fix memleak

* fixes

* cleanup

* cleanup

* windows fixes

* update shutdown order

* fixes

* explicit check to stop timer task

* windows cleanup

* loosen threaded test assertion for CI
- too much variability in thread scheduling, so we can expect pretty much anything

* add continue for unexpected logs flush trigger instead of attempting flush

* Windows re-add condition variable trigger case

* feat(logs): add `before_send_log` (#1355)

* add `before_send_log` callback

* add `before_send_log` callback tests

* (temporary) add debug for calling sentry_options_free

* remove early return

* add late return

* cleanup

* fix ownership issues in single buffer batching (#1362)

* let the producer thread sleep for 1ms between logs

* fix two missing NULL checks in the json serializer

* clean up logging and early exits in `enqueue_log_single()`

* clean up ownerships in logs

* eliminate clones (we expect that everything outlives the logs being sent except local construction)
* use incref everywhere where we ref global state. this was the cause of the UAF, partially solved with the clones but a few were missing. no reason to clone if we do not want to disconnect for a particular object graph
* clarify that add_attribute expects ownership
* minimize scope_mut by moving os_context out
* raise that log output in throughput tests add to variability (stdout logging should be turned off when running a limit)

* log error in case we weren't able to start the log batching worker

* fix clang-cl warning

* ci: fix failing mingw build (#1361)

* ci: fix failing mingw build

* split `ASM_MASM_COMPILER` and `_FLAGS`

* add `ASM_MASM_FLAGS` in `mingw` install step

* specify the `CMAKE_ASM_MASM_COMPILER` as a `FILEPATH`

* clean up CMAKE_DEFINES construction so it is easier to diff in the future

* fix `LLVM_MINGW_INSTALL_PATH` to be referenced locally rather than $env

(cherry picked from commit 519554f)

* use UNREACHABLE macro to fix anal warnings

* batching double buffered (#1365)

* first attempt at double buffered

* remove the sleep from the windows thread func

* clean up thread waiting in the example

* adapt the double buffer to use retries, fix remaining issues, clean up and write inline docs

* return early in example on sentry_init error.

* fix formatting via shorter name for thread gate atomic

* improve inline docs of log_buffer_t members

* fetch os_context from scope

* move scope/options data retrieval into separate function + add expected keys to test

* update logs API to return status code

* cleanup

* add log-event trace connection test

* remove duplicate test

* specify macOS SDKROOT

---------

Co-authored-by: JoshuaMoelans <[email protected]>

* add flush retry for missed flush requests

* move flush retry into flush function

* add docs

---------

Co-authored-by: Mischan Toosarani-Hausberger <[email protected]>

* update CHANGELOG.md

* use `trace_id` from scoped spans for logs

* fix copy-paste leftover + docs

* add log_sleep for thread test + variable NUM_LOGS

* no `usleep` on windows :(

* fix seconds->milliseconds

* cleanup

* test(logs): add 32-bit vargs test (#1370)

* add vargs conversion test

* add ifdef for 32-bit systems

* Update tests/unit/test_logs.c

Co-authored-by: Mischan Toosarani-Hausberger <[email protected]>

* add comment

* fix comment

---------

Co-authored-by: Mischan Toosarani-Hausberger <[email protected]>

* Apply suggestions from code review

Co-authored-by: Mischan Toosarani-Hausberger <[email protected]>

* post-merge cleanup

* pin ruamel version

* let's unpin ruamel.yaml.clib to get 0.2.14

which seemingly fixes the missing header:

https://sourceforge.net/p/ruamel-yaml-clib/tickets/47/#de77

* add empty payload check

* log output of logger tests if we fail to find the pre-crash marker

* fix: move `is_tsan` marker into the `has_crashpad` condition...

...so we can ignore in which module a `crashpad` test runs.

* fix: update `has_crashpad` condition comment

* really only move `is_tsan`, but keep the module level `pytestmark`

* CHANGELOG.md update

* CHANGELOG.md update

* CHANGELOG.md update

---------

Co-authored-by: Mischan Toosarani-Hausberger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants