Watsonx Semantic Conventions [INSTA-21674] #2792

elinacse · 2025-09-22T12:22:21Z

Changes

Please provide a brief description of the changes here.

Note: if the PR is touching an area that is not listed in the existing areas, or the area does not have sufficient domain experts coverage, the PR might be tagged as experts needed and move slowly until experts are identified.

Merge requirement checklist

CONTRIBUTING.md guidelines followed.
Change log entry added, according to the guidelines in When to add a changelog entry.
- If your PR does not need a change log, start the PR title with [chore]
Links to the prototypes or existing instrumentations (when adding or changing conventions)

linux-foundation-easycla · 2025-09-22T12:22:28Z

The committers listed above are authorized under a signed CLA.

✅ login: elinacse / name: elina priyadarshinee (aa9abea, 41c08ce, 1b3e11a, 9097f67, 3d924e4, 393e8ec)

gyliu513 · 2025-09-22T12:59:14Z

.chloggen/watsonx-sem-convention.yaml

+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+# The values here must be integers.
+issues: [123]


gyliu513 · 2025-09-22T13:01:15Z

.chloggen/watsonx-sem-convention.yaml

+  Added specific semantic conventions for IBM WatsonX AI including:
+  - WatsonX-specific attributes like project_id, space_id, version, and trace_id
+  - WatsonX-specific LLM attributes like decoding_method, random_seed, max_new_tokens, min_new_tokens, and repetition_penalty
+  - WatsonX-specific metrics for completions duration, exceptions, responses, and tokens
+  - Dedicated span definition for WatsonX client operations
+  - Comprehensive documentation with examples


This may bring some confuse, can you remove it?

gyliu513 · 2025-09-22T13:02:21Z

docs/attributes-registry/gen-ai.md

+
+## GenAI Attributes
+
+This document defines the attributes used to describe telemetry in the context of Generative Artificial Intelligence (GenAI) Models requests and responses.


This table is generated automatically, check https://github.com/open-telemetry/semantic-conventions/pull/2353/files and https://github.com/open-telemetry/semantic-conventions/blob/main/CONTRIBUTING.md#2-update-the-markdown-files

yes I have executed table generation command and it created watsonx.md file under registry folder .

Actually the file should be deleted as it is replaced by registry/attributes/*.md which is auto generated.

gyliu513 · 2025-09-22T13:03:11Z

model/gen-ai/watsonx.yaml

+    brief: >
+      This group defines attributes specific to IBM WatsonX AI.
+    attributes:
+      - id: gen_ai.watsonx.request.project_id


s/watsonx/watsonx.ai

gyliu513 · 2025-09-22T13:04:16Z

schema-next.yaml

@@ -0,0 +1,586 @@
+file_format: 1.1.0
+schema_url: https://opentelemetry.io/schemas/next
+versions:


why this file?

removed the file .

gyliu513 · 2025-09-22T13:12:58Z

@elinacse Please also check https://github.com/open-telemetry/semantic-conventions/pull/2046/files#diff-29d3a3c8df003e3fd7ac429797cf584e8faf546768432415a01c289802232aba

Seems we need to follow openai pattern at https://github.com/open-telemetry/semantic-conventions/tree/main/model/openai, we need a folder for watsonxai

thompson-tomo

Pr needs to be cleaned up to minimise unintended changes and removal of old auto generated files.

thompson-tomo · 2025-09-22T14:44:33Z

model/gen-ai/watsonx.yaml

+  - id: metric.gen_ai.watsonx.ai.completions.duration
+    type: metric
+    metric_name: llm.watsonx.ai.completions.duration
+    brief: 'Measures the duration of WatsonX completions operations'
+    instrument: histogram
+    unit: "s"
+    stability: experimental
+    extends: metric_attributes.gen_ai
+
+  - id: metric.gen_ai.watsonx.ai.completions.exceptions
+    type: metric
+    metric_name: llm.watsonx.ai.completions.exceptions
+    brief: 'Counts the number of exceptions that occurred during WatsonX completions operations'
+    instrument: counter
+    unit: "{exception}"
+    stability: experimental
+    extends: metric_attributes.gen_ai
+
+  - id: metric.gen_ai.watsonx.ai.completions.responses
+    type: metric
+    metric_name: llm.watsonx.ai.completions.responses
+    brief: 'Counts the number of responses received from WatsonX completions operations'
+    instrument: counter
+    unit: "{response}"
+    stability: experimental
+    extends: metric_attributes.gen_ai
+
+  - id: metric.gen_ai.watsonx.ai.completions.tokens
+    type: metric
+    metric_name: llm.watsonx.ai.completions.tokens
+    brief: 'Counts the number of tokens processed in WatsonX completions operations'
+    instrument: counter
+    unit: "{token}"
+    stability: experimental
+    extends: metric_attributes.gen_ai
+    attributes:
+      - ref: gen_ai.token.type
+        requirement_level: required


Any reason these are being restricted to watsonx? Could we not have these as general gen-ai metrics?

thompson-tomo · 2025-09-22T14:51:31Z

model/gen-ai/watsonx.yaml

+  - id: span.gen_ai.watsonx.ai.client
+    stability: experimental
+    span_kind: client
+    type: span
+    brief: >
+      Describes an IBM WatsonX AI operation span.
+    attributes:
+      - ref: gen_ai.request.model
+        requirement_level: required
+      - ref: gen_ai.watsonx.ai.request.project_id
+        requirement_level:
+          conditionally_required: if the request includes a project_id
+      - ref: gen_ai.watsonx.ai.request.space_id
+        requirement_level:
+          conditionally_required: if the request includes a space_id
+      - ref: gen_ai.watsonx.ai.request.version
+        requirement_level: recommended
+      - ref: gen_ai.watsonx.ai.response.trace_id
+        requirement_level:
+          conditionally_required: if the response was received and includes a trace_id
+      - ref: gen_ai.usage.input_tokens
+        brief: The number of tokens used in the prompt sent to IBM WatsonX AI.
+      - ref: gen_ai.usage.output_tokens
+        brief: The number of tokens used in the completions from IBM WatsonX AI.
+      - ref: llm.watsonx.ai.decoding_method
+        requirement_level: recommended
+      - ref: llm.watsonx.ai.random_seed
+        requirement_level: recommended
+      - ref: llm.watsonx.ai.max_new_tokens
+        requirement_level: recommended
+      - ref: llm.watsonx.ai.min_new_tokens
+        requirement_level: recommended
+      - ref: llm.watsonx.ai.repetition_penalty
+        requirement_level: recommended


This should be in the gen-ai/spans.yaml file

thompson-tomo · 2025-09-22T14:58:01Z

model/watsonxai/registry.yaml

+      - id: watsonxai.request.version
+        stability: development
+        type: string
+        brief: The version of the model being used.
+        examples: ['1.0', '2.3.1']
+      - id: watsonxai.response.trace_id
+        stability: development
+        type: string
+        brief: The trace ID returned by IBM WatsonX AI.
+        examples: ['wxt-12345678-abcd-1234-efgh-1234567890ab']
+      - id: watsonxai.decoding_method
+        stability: development
+        type: string
+        brief: The decoding method used by WatsonX for generating responses.
+        examples: ['greedy', 'sample']
+      - id: watsonxai.random_seed
+        stability: development
+        type: int
+        brief: The random seed used by WatsonX for deterministic generation.
+        examples: [42]
+      - id: watsonxai.max_new_tokens
+        stability: development
+        type: int
+        brief: The maximum number of new tokens to generate in the response.
+        examples: [100]
+      - id: watsonxai.min_new_tokens
+        stability: development
+        type: int
+        brief: The minimum number of new tokens to generate in the response.
+        examples: [10]
+      - id: watsonxai.repetition_penalty
+        stability: development
+        type: double
+        brief: The penalty applied to repeated tokens in the generated response.
+        examples: [1.2]


These should be reviewed to see if any existing attributes could be used, any new general attributes could be added.

gyliu513 · 2025-09-22T21:41:57Z

docs/gen-ai/README.md

 * [Azure AI Inference](./azure-ai-inference.md): Semantic Conventions for Azure AI Inference.
 * [OpenAI](./openai.md): Semantic Conventions for OpenAI.
 * [AWS Bedrock](./aws-bedrock.md): Semantic Conventions for AWS Bedrock.
+* [IBM WatsonX AI](./watsonx.md): Semantic Conventions for IBM WatsonX AI.


s/watsonx.md/ibm-watsonx-ai.md

gyliu513 · 2025-09-22T21:43:46Z

docs/gen-ai/watsonx.md

+
+### IBM WatsonX AI Client
+
+**Span kind**: [Client](../../specification/trace/api.md#spankind)


You did not add following, how does this table is generated?

 

gyliu513 · 2025-09-22T21:45:59Z

model/gen-ai/watsonx.yaml

@@ -0,0 +1,126 @@
+groups:


I think we do not need this file

gyliu513 · 2025-09-22T21:48:35Z

model/watsonxai/registry.yaml

@@ -0,0 +1,53 @@
+groups:
+  - id: registry.watsonxai


s/registry.watsonxai/registry.ibm.watsonx.ai

gyliu513 · 2025-09-22T21:49:23Z

model/watsonxai/registry.yaml

+    brief: >
+      This group defines attributes for IBM WatsonX AI.
+    attributes:
+      - id: watsonxai.request.project_id


s/watsonxai/ibm.watsonx.ai

lmolkova · 2025-09-23T00:03:41Z

docs/gen-ai/watsonx.md

+
+# IBM WatsonX AI
+
+**Status**: [Experimental](../document-status.md)


Suggested change

**Status**: [Experimental](../document-status.md)

**Status**: [Development](https://opentelemetry.io/docs/specs/otel/document-status)

lmolkova · 2025-09-23T00:04:25Z

docs/gen-ai/watsonx.md

+
+This document defines semantic conventions for IBM WatsonX AI.
+
+## Attributes


please take a look at other semantic conventions for GenAI, databases or messaging. We don't document attributes specific to that system, but the full conventions.

lmolkova · 2025-09-23T00:05:24Z

docs/gen-ai/watsonx.md

+| `gen_ai.watsonx.ai.request.version` | string | The version of the model being used | `1.0`, `2.3.1` | Recommended |
+| `gen_ai.watsonx.ai.response.trace_id` | string | The trace ID returned by IBM WatsonX AI | `wxt-12345678-abcd-1234-efgh-1234567890ab` | Conditionally Required [2] |
+| `llm.watsonx.ai.decoding_method` | string | The decoding method used by WatsonX for generating responses | `greedy`, `sample` | Recommended |
+| `llm.watsonx.ai.random_seed` | int | The random seed used by WatsonX for deterministic generation | `42` | Recommended |


anything watson-specific should be in the ibm.watson* namespace

lmolkova · 2025-09-23T00:07:01Z

docs/gen-ai/watsonx.md

+
+| Metric Name | Description | Unit | Instrument | Attribute Key | Attribute Values | Requirement Level |
+| --- | --- | --- | --- | --- | --- | --- |
+| `llm.watsonx.ai.completions.duration` | Measures the duration of WatsonX completions operations | s | Histogram | | | Recommended |


please unify this with generic GenAI metrics - they already capture everything you define here https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-metrics.md

lmolkova · 2025-09-23T00:07:25Z

docs/gen-ai/watsonx.md

+
+tracer = trace.get_tracer("example-watsonx-client")
+
+with tracer.start_as_current_span(


we don't usually add manual instrumentation code samples into semantic conventions.

thompson-tomo · 2025-09-25T07:16:40Z

docs/gen-ai/gen-ai-metrics.md

+| Name     | Instrument Type | Unit (UCUM) | Description    | Stability | Entity Associations |
+| -------- | --------------- | ----------- | -------------- | --------- | ------ |
+| `ibm.watsonx.ai.completions.tokens` | Counter | `{token}` | Number of tokens processed by IBM WatsonX.AI completions. | ![Development](https://img.shields.io/badge/-development-blue) |  |
+
+| Attribute  | Type | Description  | Examples  | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability |
+|---|---|---|---|---|---|
+| [`gen_ai.operation.name`](/docs/registry/attributes/gen-ai.md) | string | The name of the operation being performed. | `chat`; `text_completion` | `Required` | ![Development](https://img.shields.io/badge/-development-blue) |
+| [`gen_ai.token.type`](/docs/registry/attributes/gen-ai.md) | string | The type of token being counted. | `input`; `output` | `Required` | ![Development](https://img.shields.io/badge/-development-blue) |
+| [`gen_ai.request.model`](/docs/registry/attributes/gen-ai.md) | string | The name of the GenAI model a request is being made to. | `ibm/granite-13b-chat-v2` | `Recommended` | ![Development](https://img.shields.io/badge/-development-blue) |


How does this differ to the tables above?

thompson-tomo · 2025-09-25T07:17:25Z

docs/gen-ai/gen-ai-metrics.md

+| Name     | Instrument Type | Unit (UCUM) | Description    | Stability | Entity Associations |
+| -------- | --------------- | ----------- | -------------- | --------- | ------ |
+| `ibm.watsonx.ai.completions.responses` | Counter | `{response}` | Number of responses generated by IBM WatsonX.AI completions. | ![Development](https://img.shields.io/badge/-development-blue) |  |
+
+| Attribute  | Type | Description  | Examples  | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability |
+|---|---|---|---|---|---|
+| [`gen_ai.operation.name`](/docs/registry/attributes/gen-ai.md) | string | The name of the operation being performed. | `chat`; `text_completion` | `Required` | ![Development](https://img.shields.io/badge/-development-blue) |
+| [`gen_ai.request.model`](/docs/registry/attributes/gen-ai.md) | string | The name of the GenAI model a request is being made to. | `ibm/granite-13b-chat-v2` | `Recommended` | ![Development](https://img.shields.io/badge/-development-blue) |
+| [`gen_ai.response.finish_reasons`](/docs/registry/attributes/gen-ai.md) | string[] | Array of reasons the model stopped generating tokens. | `["stop"]`; `["length"]` | `Recommended` | ![Development](https://img.shields.io/badge/-development-blue) |


How does this differ to the tables above?

thompson-tomo · 2025-09-25T07:18:15Z

docs/gen-ai/gen-ai-metrics.md

+| Name     | Instrument Type | Unit (UCUM) | Description    | Stability | Entity Associations |
+| -------- | --------------- | ----------- | -------------- | --------- | ------ |
+| `ibm.watsonx.ai.completions.exceptions` | Counter | `{exception}` | Number of exceptions encountered during IBM WatsonX.AI completions. | ![Development](https://img.shields.io/badge/-development-blue) |  |
+
+| Attribute  | Type | Description  | Examples  | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability |
+|---|---|---|---|---|---|
+| [`gen_ai.operation.name`](/docs/registry/attributes/gen-ai.md) | string | The name of the operation being performed. | `chat`; `text_completion` | `Required` | ![Development](https://img.shields.io/badge/-development-blue) |
+| [`gen_ai.request.model`](/docs/registry/attributes/gen-ai.md) | string | The name of the GenAI model a request is being made to. | `ibm/granite-13b-chat-v2` | `Recommended` | ![Development](https://img.shields.io/badge/-development-blue) |
+| [`error.type`](/docs/registry/attributes/error.md) | string | Describes a class of error the operation ended with. | `timeout`; `rate_limit_exceeded`; `invalid_request` | `Required` | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |


How does this differ to the tables above?

thompson-tomo · 2025-09-25T07:29:35Z

model/gen-ai/metrics.yaml

+  - id: metric.ibm.watsonx.ai.completions.tokens
+    type: metric
+    metric_name: ibm.watsonx.ai.completions.tokens
+    annotations:
+      code_generation:
+        metric_value_type: int
+    brief: 'Number of tokens processed by IBM WatsonX.AI completions.'
+    instrument: counter
+    unit: "{token}"
+    stability: development
+    attributes:
+      - ref: gen_ai.operation.name
+        requirement_level: required
+      - ref: gen_ai.token.type
+        requirement_level: required
+      - ref: gen_ai.request.model
+        requirement_level: recommended


Can we not be using gen_ai.client.token.usage?

watsonxSemConv

9097f67

elinacse requested review from a team as code owners September 22, 2025 12:22

github-project-automation bot added this to Semantic Conventions Triage Sep 22, 2025

github-project-automation bot moved this to Untriaged in Semantic Conventions Triage Sep 22, 2025

github-actions bot added enhancement New feature or request area:gen-ai labels Sep 22, 2025

codeUpdate

aa9abea

gyliu513 reviewed Sep 22, 2025

View reviewed changes

elinacse added 2 commits September 22, 2025 19:12

modelAdded

41c08ce

watsonxChanges

3d924e4

thompson-tomo reviewed Sep 22, 2025

View reviewed changes

gyliu513 reviewed Sep 22, 2025

View reviewed changes

lmolkova requested changes Sep 23, 2025

View reviewed changes

github-project-automation bot moved this from Untriaged to Blocked in Semantic Conventions Triage Sep 23, 2025

elinacse changed the title ~~Watsonx Semantic Conventions [INSTA-55630]~~ Watsonx Semantic Conventions [INSTA-21674] Sep 24, 2025

elinacse added 2 commits September 25, 2025 08:18

watsonxSem-INSTA-21674

1b3e11a

watsonxChanges

393e8ec

thompson-tomo reviewed Sep 25, 2025

View reviewed changes


		## GenAI Attributes

		This document defines the attributes used to describe telemetry in the context of Generative Artificial Intelligence (GenAI) Models requests and responses.


		### IBM WatsonX AI Client

		Span kind: [Client](../../specification/trace/api.md#spankind)


		# IBM WatsonX AI

		Status: [Experimental](../document-status.md)

	Status: [Experimental](../document-status.md)
	Status: [Development](https://opentelemetry.io/docs/specs/otel/document-status)


		This document defines semantic conventions for IBM WatsonX AI.

		## Attributes


		tracer = trace.get_tracer("example-watsonx-client")

		with tracer.start_as_current_span(

Watsonx Semantic Conventions [INSTA-21674] #2792

Are you sure you want to change the base?

Watsonx Semantic Conventions [INSTA-21674] #2792

Uh oh!

Conversation

elinacse commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Merge requirement checklist

Uh oh!

linux-foundation-easycla bot commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gyliu513 Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gyliu513 commented Sep 22, 2025

Uh oh!

thompson-tomo left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elinacse commented Sep 22, 2025 •

edited

Loading

linux-foundation-easycla bot commented Sep 22, 2025 •

edited

Loading

gyliu513 Sep 22, 2025 •

edited

Loading

thompson-tomo left a comment •

edited

Loading