Skip to content

Conversation

umaannamalai
Copy link
Contributor

This PR updates sync and async Bedrock instrumentation and tests to pull token counts directly from the response object. If a customer has an LLM token counting callback function registered, this will still take priority over the response token count values.

It also removes token count calculations from error cases as the agent will consistently stop reporting token counts when a error was met during the creation of a text generation.

@umaannamalai umaannamalai changed the title Add response token count logic to Gemini instrumentation. Add response token count logic to Bedrock instrumentation. Sep 19, 2025
Copy link

github-actions bot commented Sep 19, 2025

MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ ACTION actionlint 7 0 0 0.94s
✅ MARKDOWN markdownlint 7 0 0 0 1.3s
✅ PYTHON ruff 946 0 0 0 1.01s
✅ PYTHON ruff-format 946 0 0 0 0.36s
✅ YAML prettier 15 0 0 0 1.48s
✅ YAML v8r 15 0 0 5.66s
✅ YAML yamllint 15 0 0 0.64s

See detailed reports in MegaLinter artifacts

MegaLinter is graciously provided by OX Security

@mergify mergify bot added the tests-failing Tests failing in CI. label Sep 19, 2025
@codecov-commenter
Copy link

codecov-commenter commented Sep 19, 2025

Codecov Report

❌ Patch coverage is 89.79592% with 10 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (feature-llm-token-counts@7d5adac). Learn more about missing BASE report.

Files with missing lines Patch % Lines
newrelic/hooks/external_botocore.py 89.79% 6 Missing and 4 partials ⚠️
Additional details and impacted files
@@                     Coverage Diff                     @@
##             feature-llm-token-counts    #1504   +/-   ##
===========================================================
  Coverage                            ?   81.65%           
===========================================================
  Files                               ?      207           
  Lines                               ?    24037           
  Branches                            ?     3802           
===========================================================
  Hits                                ?    19628           
  Misses                              ?     3132           
  Partials                            ?     1277           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hmstepanek hmstepanek marked this pull request as ready for review October 1, 2025 00:28
@hmstepanek hmstepanek requested a review from a team as a code owner October 1, 2025 00:28
Copy link
Contributor

@hmstepanek hmstepanek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I said this in a comment too but I'm going to stop reviewing this for the moment.

Can you for the sake of being able to easily identify what's changed here: re-order these changes in the file such that they end up being diffed against the original function. Right now it's really hard to compare to see what's changed because the diff isn't showing a 1:1 comparison. I know it's a pain but in a larger PR like this with lots of files changed it makes it a ton easier to review.


bedrock_attrs["input"] = request_body.get("inputText")

def extract_bedrock_titan_text_model_streaming_response(response_body, bedrock_attrs):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you for the sake of being able to easily identify what's changed here: re-order these changes in the file such that they end up being diffed against the original function. Right now it's really hard to compare to see what's changed because the diff isn't showing a 1:1 comparison. I know it's a pain but in a larger PR like this with lots of files changed it makes it a ton easier to review.

@mergify mergify bot added the merge-conflicts Merge conflicts detected. label Oct 7, 2025
@mergify mergify bot removed the merge-conflicts Merge conflicts detected. label Oct 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tests-failing Tests failing in CI.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants