[Bug]: Error starting container when using DIND in Kubernetes pod #11139
Description
Module
Core
Testcontainers version
1.21.3
Using the latest Testcontainers version?
Yes
Host OS
Ubuntu
Host Arch
x86
Docker version
Docker version 20.10.15, build fd82621What happened?
I am trying to run a container in a SpringBootTest using TestContainers as part of a Jenkins pipeline that spins up a Kubernetes pod in GCP.
My build/test is running in a Gradle container (built on Ubuntu), and it has /var/run and /var/lib/docker mounted from a docker-dind container.
When the test runs, it seems to start normally and attempt to pull and run the container:
11:01:32 2025-10-31T11:01:31.915+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:31.915Z","severity":"INFO","message":"Testcontainers version: 1.21.3","process":{"thread":{"name":"Test worker"}}}
11:01:34 2025-10-31T11:01:34.609+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:34.609Z","severity":"INFO","message":"Found Docker environment with local Unix socket (unix:///var/run/docker.sock)","process":{"thread":{"name":"Test worker"}}}
11:01:34 2025-10-31T11:01:34.620+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:34.620Z","severity":"INFO","message":"Docker host IP address is localhost","process":{"thread":{"name":"Test worker"}}}
11:01:34 2025-10-31T11:01:34.641+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:34.640Z","severity":"INFO","message":"Connected to docker: \n Server Version: 20.10.15\n API Version: 1.41\n Operating System: Alpine Linux v3.15 (containerized)\n Total Memory: 32102 MB","process":{"thread":{"name":"Test worker"}}}
11:01:34 2025-10-31T11:01:34.718+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:34.718Z","severity":"WARN","message":"Image name testcontainers/ryuk:0.12.0 was substituted by configuration to europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0. This approach is deprecated and will be removed in the future","process":{"thread":{"name":"Test worker"}}}
11:01:34 2025-10-31T11:01:34.719+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:34.718Z","severity":"INFO","message":"Using europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0 as a substitute image for testcontainers/ryuk:0.12.0 (using image substitutor: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor'))","process":{"thread":{"name":"Test worker"}}}
11:01:35 2025-10-31T11:01:34.818+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:34.818Z","severity":"INFO","message":"Pulling docker image: europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0. Please be patient; this may take some time but only needs to be done once.","process":{"thread":{"name":"Test worker"}}}
11:01:37 2025-10-31T11:01:36.974+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:36.973Z","severity":"INFO","message":"Starting to pull image","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37 2025-10-31T11:01:37.016+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:37.016Z","severity":"INFO","message":"Pulling image layers: 0 pending, 0 downloaded, 0 extracted, (0 bytes/0 bytes)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37 2025-10-31T11:01:37.282+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:37.282Z","severity":"INFO","message":"Pulling image layers: 2 pending, 1 downloaded, 0 extracted, (43 KB/? MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37 2025-10-31T11:01:37.295+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:37.295Z","severity":"INFO","message":"Pulling image layers: 1 pending, 2 downloaded, 0 extracted, (70 KB/? MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37 2025-10-31T11:01:37.337+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:37.336Z","severity":"INFO","message":"Pulling image layers: 0 pending, 3 downloaded, 0 extracted, (93 KB/6 MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37 2025-10-31T11:01:37.463+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:37.462Z","severity":"INFO","message":"Pulling image layers: 0 pending, 3 downloaded, 1 extracted, (3 MB/6 MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37 2025-10-31T11:01:37.591+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:37.591Z","severity":"INFO","message":"Pulling image layers: 0 pending, 3 downloaded, 2 extracted, (4 MB/6 MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:37 2025-10-31T11:01:37.722+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:37.722Z","severity":"INFO","message":"Pulling image layers: 0 pending, 3 downloaded, 3 extracted, (6 MB/6 MB)","process":{"thread":{"name":"docker-java-stream-1588146861"}}}
11:01:38 2025-10-31T11:01:37.739+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:37.739Z","severity":"INFO","message":"Image europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0 pull took PT2.920436798S","process":{"thread":{"name":"Test worker"}}}
11:01:38 2025-10-31T11:01:37.816+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:37.815Z","severity":"INFO","message":"Creating container for image: europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0","process":{"thread":{"name":"Test worker"}}}
then something breaks:
11:01:39 2025-10-31T11:01:39.553+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:39.553Z","severity":"INFO","message":"Container europe-docker.pkg.dev/<repo>/testcontainers-ryuk:0.11.0 is starting: 3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2","process":{"thread":{"name":"Test worker"}}}
11:01:40 2025-10-31T11:01:40.311+0000 [DEBUG] [TestEventLogger] {"timestamp":"2025-10-31T11:01:40.309Z","severity":"ERROR","message":"Could not start container","process":{"thread":{"name":"Test worker"}},"error":{"type":"com.github.dockerjava.api.exception.NotFoundException","message":"Status 404: {\"message\":\"No such container: 3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2\"}\n","stack_trace":"com.github.dockerjava.api.exception.NotFoundException: Status 404: {\"message\":\"No such container: 3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2\"}
On the DIND side:
ERROR 2025-10-31T11:01:39.658837495Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.658598613Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
ERROR 2025-10-31T11:01:39.658887882Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.658651357Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
ERROR 2025-10-31T11:01:39.658895372Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.658661442Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
ERROR 2025-10-31T11:01:39.659318815Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.659018953Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2 pid=415 runtime=io.containerd.runc.v2
ERROR 2025-10-31T11:01:39.813347912Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.813119378Z" level=info msg="shim disconnected" id=3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2
ERROR 2025-10-31T11:01:39.813381129Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.813166233Z" level=info msg="ignoring event" container=3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
ERROR 2025-10-31T11:01:39.813398233Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.813207171Z" level=warning msg="cleaning up after shim disconnected" id=3417b89ad3646c6068e1cbd65ec774ae242b5d4e29a565795c0765dfa49414e2 namespace=moby
ERROR 2025-10-31T11:01:39.813401862Z [resource.labels.containerName: docker] time="2025-10-31T11:01:39.813222671Z" level=info msg="cleaning up dead shim"