Skip to content

Conversation

@datadog-agent-integrations-bot
Copy link
Contributor

Backport eac1ac7 from #21821.


What does this PR do?

This PR replaces the openjdk images we use in integrations-core for the amazoncorretto jdk images.

Motivation

OpenJDK docker images have been deprecated and our tests have started failing. See here for details.

The Amazon Corretto images are open and free images provided by Amazon with their own jdk version. These images are supposed to be pretty stable as it is the internal jdk version used by Amazon, open to the public.

The alpine version of these images uses a newer version of of alpine for which sbt had a bug. For this reason I am also updating the sbt version to the one that included a bufix for the missing __xstat64 symbol.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

* Replace openjdk images with amazoncorretto docker images

* Use al2 version fro jdk19 that does not use alpine

* Update sbt version to fix __xstat64 missing issue

* Fix logs validation in docker compose launch

* Add curl to the Dockerfile to ensure we can still build the test docker image if the wrapper jar file is removed

(cherry picked from commit eac1ac7)
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines 16 to 20
compose_file = os.path.join(HERE, 'docker', 'docker-compose.yaml')
with docker_run(
compose_file=compose_file,
conditions=[CheckDockerLogs('spark-app-hudi', 'finished: show at script.scala:163')],
conditions=[CheckDockerLogs('spark-app-hudi', 'finished: show at script.scala')],
attempts=5,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Wait condition exits after first Spark show instead of job completion

The readiness gate now looks for the generic substring finished: show at script.scala. CheckDockerLogs matches regexes anywhere in the log output, so this will succeed on the very first show() statement executed in script.scala. The Hudi test script performs several subsequent writes/reads before entering its idle loop, and the e2e test asserts JMX metrics that are produced by those later operations. By returning from docker_run after the first show, the test can start before the remaining Hudi work has finished, introducing race‑condition failures when the metrics are still missing. The original pattern with the explicit line number ensured the check waited for the final show call.

Useful? React with 👍 / 👎.

@codecov
Copy link

codecov bot commented Nov 5, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (7.72.x@044e056). Learn more about missing BASE report.

Additional details and impacted files
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-official
Copy link
Contributor

datadog-official bot commented Nov 5, 2025

⚠️ Tests

⚠️ Warnings

❄️ 2 New flaky tests detected

test_autodiscovery_collect_all_metrics from test_discovery.py (Datadog)
Needed at least 1 candidates for 'postgresql.function.calls', got 0
Expected:
        MetricStub(name='postgresql.function.calls', type=None, value=None, tags=['database_hostname:stubbed.hostname', 'database_instance:stubbed.hostname', 'db:dogs_nofunc', 'dd.internal.resource:database_instance:stubbed.hostname', 'foo:bar', 'function:dummy_function', 'port:5432', 'postgresql_version:10.23', 'replication_role:master', 'schema:public'], hostname=None, device=None, flush_first_value=None)
Difference to closest:
        Expected name: postgresql.function.calls
        Found postgresql.connections
        Expected tag function:dummy_function
        Found function:None
        Expected tag schema:public
        Found schema:None
...
test_analyze_progress from test_progress_stats.py (Datadog)
Needed exactly 1 candidates for 'postgresql.analyze.sample_blks_total', got 0
Expected:
        MetricStub(name='postgresql.analyze.sample_blks_total', type=None, value=None, tags=['database_hostname:stubbed.hostname', 'database_instance:stubbed.hostname', 'db:datadog_test', 'dd.internal.resource:database_instance:stubbed.hostname', 'foo:bar', 'phase:acquiring sample rows', 'port:5432', 'postgresql_version:17.6', 'replication_role:master', 'table:test_part1'], hostname=None, device=None, flush_first_value=None)
Difference to closest:
        Expected tag phase:acquiring sample rows
        Found phase:computing statistics

Similar submitted:
Score   Most similar
0.99    MetricStub(name='postgresql.analyze.sample_blks_total', type=0, value=3.0, tags=['database_hostname:stubbed.hostname', 'database_instance:stubbed.hostname', 'db:datadog_test', 'dd.internal.resource:database_instance:stubbed.hostname', 'foo:bar', 'phase:computing statistics', 'port:5432', 'postgresql_version:17.6', 'replication_role:master', 'table:test_part1'], hostname='stubbed.hostname', device=None, flush_first_value=False)
...

ℹ️ Info

🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 90c0fe9 | Docs | Datadog PR Page | Was this helpful? Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants