-
Notifications
You must be signed in to change notification settings - Fork 1.5k
[Backport 7.72.x] Replace openjdk images with amazoncorretto docker images #21841
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 7.72.x
Are you sure you want to change the base?
Conversation
* Replace openjdk images with amazoncorretto docker images * Use al2 version fro jdk19 that does not use alpine * Update sbt version to fix __xstat64 missing issue * Fix logs validation in docker compose launch * Add curl to the Dockerfile to ensure we can still build the test docker image if the wrapper jar file is removed (cherry picked from commit eac1ac7)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| compose_file = os.path.join(HERE, 'docker', 'docker-compose.yaml') | ||
| with docker_run( | ||
| compose_file=compose_file, | ||
| conditions=[CheckDockerLogs('spark-app-hudi', 'finished: show at script.scala:163')], | ||
| conditions=[CheckDockerLogs('spark-app-hudi', 'finished: show at script.scala')], | ||
| attempts=5, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wait condition exits after first Spark show instead of job completion
The readiness gate now looks for the generic substring finished: show at script.scala. CheckDockerLogs matches regexes anywhere in the log output, so this will succeed on the very first show() statement executed in script.scala. The Hudi test script performs several subsequent writes/reads before entering its idle loop, and the e2e test asserts JMX metrics that are produced by those later operations. By returning from docker_run after the first show, the test can start before the remaining Hudi work has finished, introducing race‑condition failures when the metrics are still missing. The original pattern with the explicit line number ensured the check waited for the final show call.
Useful? React with 👍 / 👎.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files🚀 New features to boost your workflow:
|
|
Backport eac1ac7 from #21821.
What does this PR do?
This PR replaces the openjdk images we use in
integrations-corefor theamazoncorrettojdk images.Motivation
OpenJDK docker images have been deprecated and our tests have started failing. See here for details.
The Amazon Corretto images are open and free images provided by Amazon with their own jdk version. These images are supposed to be pretty stable as it is the internal jdk version used by Amazon, open to the public.
The alpine version of these images uses a newer version of of alpine for which
sbthad a bug. For this reason I am also updating thesbtversion to the one that included a bufix for the missing__xstat64symbol.Review checklist (to be filled by reviewers)
qa/skip-qalabel if the PR doesn't need to be tested during QA.backport/<branch-name>label to the PR and it will automatically open a backport PR once this one is merged