Skip to content

Commit 70972cb

Browse files
ishleenk17Copilotmykola-elasticcolleenmcginnis
authored andcommitted
VertexAI Initial draft- prompt response logging (#46383)
* VertexAI Initial draft- prompt response logging * Add changelog * make update * Add test file * Add documentation * Update docs * Update documentaion and make update * Resolve lint errors * Updated test file * Remove test file * Add test file * update test file * Update x-pack/metricbeat/module/gcp/vertexai_logs/_meta/docs.md Co-authored-by: Copilot <[email protected]> * Update the partiotin logic in BigQuery * Add support of logging time filtering * reshuffle imports in vertexai_logs_test.go * Add string datatype for request id * Address comments * Remove the otel_log field * Update vertexai_logs.go * Update vertexai_logs_test.go * Replace run with read * Linter error * linter fix * LINTER ERROR * rerun make update --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Mykola Kmet <[email protected]> Co-authored-by: Colleen McGinnis <[email protected]> (cherry picked from commit e3c6cce)
1 parent c50e2cc commit 70972cb

File tree

18 files changed

+1120
-2
lines changed

18 files changed

+1120
-2
lines changed

.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,7 @@ CHANGELOG*
234234
/x-pack/metricbeat/module/coredns @elastic/obs-infraobs-integrations
235235
/x-pack/metricbeat/module/enterprisesearch @elastic/app-search-team
236236
/x-pack/metricbeat/module/gcp @elastic/obs-ds-hosted-services @elastic/obs-infraobs-integrations
237+
/x-pack/metricbeat/module/gcp/vertexai_logs @elastic/obs-infraobs-integrations
237238
/x-pack/metricbeat/module/gcp/billing @elastic/obs-infraobs-integrations
238239
/x-pack/metricbeat/module/gcp/cloudrun_metrics @elastic/obs-infraobs-integrations
239240
/x-pack/metricbeat/module/gcp/cloudsql_mysql @elastic/obs-infraobs-integrations

CHANGELOG.next.asciidoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -339,6 +339,8 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
339339
- Preserve queries for debugging when `merge_results: true` in SQL module {pull}42271[42271]
340340
- Collect more fields from ES node/stats metrics and only those that are necessary {pull}42421[42421]
341341
- Add support for Kafka 4.0 in the Kafka module. {pull}44723[44723]
342+
- Add NTP response validation for system/ntp module. {pull}46184[46184]
343+
- Add vertexai_logs metricset to GCP for prompt response collection from VertexAI service. {pull}46383[46383]
342344

343345
*Metricbeat*
344346

docs/reference/metricbeat/exported-fields-gcp.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1201,3 +1201,87 @@ Google Cloud Storage metrics
12011201
type: long
12021202

12031203

1204+
## vertexai_logs [_vertexai_logs]
1205+
1206+
```{applies_to}
1207+
stack: beta 9.2.0
1208+
```
1209+
1210+
Google Cloud Vertex AI Prompt Response Logs metrics
1211+
1212+
**`gcp.vertexai_logs.endpoint`**
1213+
: The Vertex AI API endpoint URL used for the request.
1214+
1215+
type: keyword
1216+
1217+
1218+
**`gcp.vertexai_logs.deployed_model_id`**
1219+
: The ID of the deployed model that processed the request.
1220+
1221+
type: keyword
1222+
1223+
1224+
**`gcp.vertexai_logs.logging_time`**
1225+
: Timestamp when the AI interaction was logged.
1226+
1227+
type: date
1228+
1229+
1230+
**`gcp.vertexai_logs.request_id`**
1231+
: Unique identifier for the AI request.
1232+
1233+
type: double
1234+
1235+
1236+
**`gcp.vertexai_logs.request_payload`**
1237+
: Array of request payload strings containing user prompts and inputs.
1238+
1239+
type: text
1240+
1241+
Field is not indexed.
1242+
1243+
1244+
**`gcp.vertexai_logs.response_payload`**
1245+
: Array of response payload strings containing AI model outputs.
1246+
1247+
type: text
1248+
1249+
Field is not indexed.
1250+
1251+
1252+
**`gcp.vertexai_logs.model`**
1253+
: Name of the AI model used (e.g., gemini-2.5-pro).
1254+
1255+
type: keyword
1256+
1257+
1258+
**`gcp.vertexai_logs.model_version`**
1259+
: Version of the AI model used.
1260+
1261+
type: keyword
1262+
1263+
1264+
**`gcp.vertexai_logs.api_method`**
1265+
: The API method called (e.g., generateContent, predict).
1266+
1267+
type: keyword
1268+
1269+
1270+
**`gcp.vertexai_logs.full_request`**
1271+
: Complete request object containing all request details in JSON format.
1272+
1273+
type: object
1274+
1275+
1276+
**`gcp.vertexai_logs.full_response`**
1277+
: Complete response object containing all response details in JSON format.
1278+
1279+
type: object
1280+
1281+
1282+
**`gcp.vertexai_logs.metadata`**
1283+
: Additional metadata associated with the AI interaction in JSON format.
1284+
1285+
type: object
1286+
1287+
Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
---
2+
mapped_pages:
3+
- https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-gcp-vertexai_logs.html
4+
applies_to:
5+
stack: beta 9.2.0
6+
---
7+
8+
% This file is generated! See scripts/docs_collector.py
9+
10+
# Google Cloud Platform vertexai_logs metricset [metricbeat-metricset-gcp-vertexai_logs]
11+
12+
The `vertexai_logs` metricset is designed to collect Vertex AI prompt-response logs from GCP BigQuery. BigQuery is a fully-managed, serverless data warehouse that stores detailed logs of interactions with Vertex AI models.
13+
14+
Vertex AI logs export to BigQuery enables you to export detailed Google Cloud Vertex AI interaction data (such as prompts, responses, model usage, and metadata) automatically to a BigQuery dataset that you specify. Then you can access your Vertex AI logs from BigQuery for detailed analysis and monitoring using Metricbeat. This enables comprehensive tracking of AI model usage, performance monitoring, and cost analysis.
15+
16+
The logs include detailed information about:
17+
- API endpoints and deployed models
18+
- Request and response payloads
19+
- Model versions and API methods used
20+
- Request metadata and timing information
21+
22+
23+
## Metricset-specific configuration notes [_metricset_specific_configuration_notes_14]
24+
25+
* **table_id**: (Required) Full table identifier in the format `project_id.dataset_id.table_name` that contains the Vertex AI logs data. You can copy this from the "Details" tab when viewing your table in the BigQuery web console, under the "Table ID" field.
26+
27+
28+
## Configuration example [_configuration_example_22]
29+
30+
```yaml
31+
- module: gcp
32+
metricsets:
33+
- vertexai_logs
34+
period: 10m
35+
project_id: "your project id"
36+
credentials_file_path: "your JSON credentials file path"
37+
table_id: "your_project.your_dataset.your_vertex_ai_logs_table"
38+
```
39+
40+
## Sample Event
41+
42+
Here is a sample event for `vertexai_logs`:
43+
44+
```json
45+
{
46+
"@timestamp": "2023-12-01T10:30:45.000Z",
47+
"cloud": {
48+
"provider": "gcp",
49+
"project": {
50+
"id": "my-gcp-project"
51+
}
52+
},
53+
"gcp": {
54+
"vertexai_logs": {
55+
"endpoint": "https://us-central1-aiplatform.googleapis.com",
56+
"deployed_model_id": "1234567890123456789",
57+
"logging_time": "2023-12-01T10:30:45.000Z",
58+
"request_id": 98765432101234567,
59+
"request_payload": ["What is machine learning?"],
60+
"response_payload": ["Machine learning is a subset of artificial intelligence..."],
61+
"model": "gemini-2.5-pro",
62+
"model_version": "1.0",
63+
"api_method": "generateContent",
64+
"full_request": {
65+
"inputs": ["What is machine learning?"],
66+
"parameters": {
67+
"temperature": 0.7
68+
}
69+
},
70+
"full_response": {
71+
"outputs": ["Machine learning is a subset of artificial intelligence..."],
72+
"usage": {
73+
"input_tokens": 5,
74+
"output_tokens": 50
75+
}
76+
},
77+
"metadata": {
78+
"user_id": "user123",
79+
"session_id": "session456"
80+
}
81+
}
82+
}
83+
}
84+
```
85+
86+
## Fields [_fields]
87+
88+
For a description of each field in the metricset, see the [exported fields](/reference/metricbeat/exported-fields-gcp.md) section.
89+
90+
Here is an example document generated by this metricset:
91+
92+
```json
93+
{
94+
"@timestamp": "2025-09-02T10:14:50.313Z",
95+
"agent": {
96+
"hostname": "metricbeat-host",
97+
"name": "metricbeat-host"
98+
},
99+
"cloud": {
100+
"account": {
101+
"id": "elastic-beats"
102+
},
103+
"provider": "gcp",
104+
"project": {
105+
"id": "elastic-beats"
106+
}
107+
},
108+
"event": {
109+
"dataset": "gcp.vertexai_logs",
110+
"duration": 123456789,
111+
"module": "gcp"
112+
},
113+
"gcp": {
114+
"vertexai_logs": {
115+
"endpoint": "https://us-central1-aiplatform.googleapis.com/v1/projects/elastic-beats/locations/us-central1/endpoints/123456789",
116+
"deployed_model_id": "model-deployment-123",
117+
"logging_time": "2025-09-02T10:14:50.313Z",
118+
"request_id": 98765432101234567,
119+
"model": "gemini-2.5-pro",
120+
"model_version": "001",
121+
"api_method": "generateContent",
122+
"request_payload": [
123+
"What is the weather like today?"
124+
],
125+
"response_payload": [
126+
"I don't have access to real-time weather information. Please check a weather service or app for current conditions."
127+
],
128+
"full_request": {
129+
"contents": [
130+
{
131+
"parts": [
132+
{
133+
"text": "What is the weather like today?"
134+
}
135+
],
136+
"role": "user"
137+
}
138+
],
139+
"generationConfig": {
140+
"temperature": 0.7,
141+
"maxOutputTokens": 1024
142+
}
143+
},
144+
"full_response": {
145+
"candidates": [
146+
{
147+
"content": {
148+
"parts": [
149+
{
150+
"text": "I don't have access to real-time weather information. Please check a weather service or app for current conditions."
151+
}
152+
],
153+
"role": "model"
154+
},
155+
"finishReason": "STOP",
156+
"safetyRatings": [
157+
{
158+
"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
159+
"probability": "NEGLIGIBLE"
160+
}
161+
]
162+
}
163+
],
164+
"usageMetadata": {
165+
"promptTokenCount": 8,
166+
"candidatesTokenCount": 24,
167+
"totalTokenCount": 32
168+
}
169+
},
170+
"metadata": {
171+
"region": "us-central1",
172+
"zone": "us-central1-a"
173+
}
174+
}
175+
},
176+
"metricset": {
177+
"name": "vertexai_logs",
178+
"period": 300000
179+
},
180+
"service": {
181+
"type": "gcp"
182+
}
183+
}
184+
```

docs/reference/metricbeat/metricbeat-module-gcp.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -363,6 +363,16 @@ metricbeat.modules:
363363
exclude_labels: false
364364
period: 1m
365365
collect_dataproc_user_labels: true
366+
367+
- module: gcp
368+
metricsets:
369+
- vertexai_logs
370+
period: 300s # 5 minutes
371+
project_id: "your-project-id"
372+
table_id: "your-project-id.dataset.id.table_name"
373+
credentials_file_path: "/path/to/service-account.json"
374+
# credentials_json: '{"type": "service_account", ...}'
375+
time_lookback_hours: 1 # How many hours back to look for initial data fetch
366376
```
367377

368378

@@ -380,3 +390,4 @@ The following metricsets are available:
380390
* [metrics](/reference/metricbeat/metricbeat-metricset-gcp-metrics.md)
381391
* [pubsub](/reference/metricbeat/metricbeat-metricset-gcp-pubsub.md)
382392
* [storage](/reference/metricbeat/metricbeat-metricset-gcp-storage.md)
393+
* [vertexai_logs](/reference/metricbeat/metricbeat-metricset-gcp-vertexai_logs.md) {applies_to}`stack: beta 9.2.0`

docs/reference/metricbeat/metricbeat-modules.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ This section contains detailed information about the metric collecting modules c
3535
| [Elasticsearch](/reference/metricbeat/metricbeat-module-elasticsearch.md) | ![No prebuilt dashboards](images/icon-no.png "") | [ccr](/reference/metricbeat/metricbeat-metricset-elasticsearch-ccr.md)<br>[cluster_stats](/reference/metricbeat/metricbeat-metricset-elasticsearch-cluster_stats.md)<br>[enrich](/reference/metricbeat/metricbeat-metricset-elasticsearch-enrich.md)<br>[index](/reference/metricbeat/metricbeat-metricset-elasticsearch-index.md)<br>[index_recovery](/reference/metricbeat/metricbeat-metricset-elasticsearch-index_recovery.md)<br>[index_summary](/reference/metricbeat/metricbeat-metricset-elasticsearch-index_summary.md)<br>[ingest_pipeline](/reference/metricbeat/metricbeat-metricset-elasticsearch-ingest_pipeline.md) {applies_to}`stack: beta`<br>[ml_job](/reference/metricbeat/metricbeat-metricset-elasticsearch-ml_job.md)<br>[node](/reference/metricbeat/metricbeat-metricset-elasticsearch-node.md)<br>[node_stats](/reference/metricbeat/metricbeat-metricset-elasticsearch-node_stats.md)<br>[pending_tasks](/reference/metricbeat/metricbeat-metricset-elasticsearch-pending_tasks.md)<br>[shard](/reference/metricbeat/metricbeat-metricset-elasticsearch-shard.md) |
3636
| [Envoyproxy](/reference/metricbeat/metricbeat-module-envoyproxy.md) | ![No prebuilt dashboards](images/icon-no.png "") | [server](/reference/metricbeat/metricbeat-metricset-envoyproxy-server.md) |
3737
| [Etcd](/reference/metricbeat/metricbeat-module-etcd.md) | ![No prebuilt dashboards](images/icon-no.png "") | [leader](/reference/metricbeat/metricbeat-metricset-etcd-leader.md)<br>[metrics](/reference/metricbeat/metricbeat-metricset-etcd-metrics.md) {applies_to}`stack: beta`<br>[self](/reference/metricbeat/metricbeat-metricset-etcd-self.md)<br>[store](/reference/metricbeat/metricbeat-metricset-etcd-store.md) |
38-
| [Google Cloud Platform](/reference/metricbeat/metricbeat-module-gcp.md) | ![Prebuilt dashboards are available](images/icon-yes.png "") | [billing](/reference/metricbeat/metricbeat-metricset-gcp-billing.md)<br>[carbon](/reference/metricbeat/metricbeat-metricset-gcp-carbon.md) {applies_to}`stack: beta`<br>[compute](/reference/metricbeat/metricbeat-metricset-gcp-compute.md)<br>[dataproc](/reference/metricbeat/metricbeat-metricset-gcp-dataproc.md)<br>[firestore](/reference/metricbeat/metricbeat-metricset-gcp-firestore.md)<br>[gke](/reference/metricbeat/metricbeat-metricset-gcp-gke.md)<br>[loadbalancing](/reference/metricbeat/metricbeat-metricset-gcp-loadbalancing.md)<br>[metrics](/reference/metricbeat/metricbeat-metricset-gcp-metrics.md)<br>[pubsub](/reference/metricbeat/metricbeat-metricset-gcp-pubsub.md)<br>[storage](/reference/metricbeat/metricbeat-metricset-gcp-storage.md) |
38+
| [Google Cloud Platform](/reference/metricbeat/metricbeat-module-gcp.md) | ![Prebuilt dashboards are available](images/icon-yes.png "") | [billing](/reference/metricbeat/metricbeat-metricset-gcp-billing.md)<br>[carbon](/reference/metricbeat/metricbeat-metricset-gcp-carbon.md) {applies_to}`stack: beta`<br>[compute](/reference/metricbeat/metricbeat-metricset-gcp-compute.md)<br>[dataproc](/reference/metricbeat/metricbeat-metricset-gcp-dataproc.md)<br>[firestore](/reference/metricbeat/metricbeat-metricset-gcp-firestore.md)<br>[gke](/reference/metricbeat/metricbeat-metricset-gcp-gke.md)<br>[loadbalancing](/reference/metricbeat/metricbeat-metricset-gcp-loadbalancing.md)<br>[metrics](/reference/metricbeat/metricbeat-metricset-gcp-metrics.md)<br>[pubsub](/reference/metricbeat/metricbeat-metricset-gcp-pubsub.md)<br>[storage](/reference/metricbeat/metricbeat-metricset-gcp-storage.md)<br>[vertexai_logs](/reference/metricbeat/metricbeat-metricset-gcp-vertexai_logs.md) {applies_to}`stack: beta 9.2.0` |
3939
| [Golang](/reference/metricbeat/metricbeat-module-golang.md) | ![Prebuilt dashboards are available](images/icon-yes.png "") | [expvar](/reference/metricbeat/metricbeat-metricset-golang-expvar.md)<br>[heap](/reference/metricbeat/metricbeat-metricset-golang-heap.md) |
4040
| [Graphite](/reference/metricbeat/metricbeat-module-graphite.md) | ![No prebuilt dashboards](images/icon-no.png "") | [server](/reference/metricbeat/metricbeat-metricset-graphite-server.md) |
4141
| [HAProxy](/reference/metricbeat/metricbeat-module-haproxy.md) | ![Prebuilt dashboards are available](images/icon-yes.png "") | [info](/reference/metricbeat/metricbeat-metricset-haproxy-info.md)<br>[stat](/reference/metricbeat/metricbeat-metricset-haproxy-stat.md) |

docs/reference/toc.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -954,6 +954,7 @@ toc:
954954
- file: metricbeat/metricbeat-metricset-gcp-metrics.md
955955
- file: metricbeat/metricbeat-metricset-gcp-pubsub.md
956956
- file: metricbeat/metricbeat-metricset-gcp-storage.md
957+
- file: metricbeat/metricbeat-metricset-gcp-vertexai_logs.md
957958
- file: metricbeat/metricbeat-module-golang.md
958959
children:
959960
- file: metricbeat/metricbeat-metricset-golang-expvar.md

x-pack/metricbeat/include/list.go

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

x-pack/metricbeat/metricbeat.reference.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -697,6 +697,18 @@ metricbeat.modules:
697697
period: 1m
698698
collect_dataproc_user_labels: true
699699

700+
- module: gcp
701+
metricsets:
702+
- vertexai_logs
703+
period: 300s # 5 minutes
704+
project_id: "your-project-id"
705+
table_id: "your-project-id.dataset.id.table_name"
706+
credentials_file_path: "/path/to/service-account.json"
707+
# credentials_json: '{"type": "service_account", ...}'
708+
time_lookback_hours: 1 # How many hours back to look for initial data fetch
709+
710+
711+
700712
#-------------------------------- Golang Module --------------------------------
701713
- module: golang
702714
#metricsets:

x-pack/metricbeat/module/gcp/_meta/config.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,3 +82,15 @@
8282
exclude_labels: false
8383
period: 1m
8484
collect_dataproc_user_labels: true
85+
86+
- module: gcp
87+
metricsets:
88+
- vertexai_logs
89+
period: 300s # 5 minutes
90+
project_id: "your-project-id"
91+
table_id: "your-project-id.dataset.id.table_name"
92+
credentials_file_path: "/path/to/service-account.json"
93+
# credentials_json: '{"type": "service_account", ...}'
94+
time_lookback_hours: 1 # How many hours back to look for initial data fetch
95+
96+

0 commit comments

Comments
 (0)