Serialization of a started container so its class can be recreated (eg. in another thread).

[PieterJanVdb]

I've just implemented some functionality whereby we provision a pool of running postgresql testcontainers in the global setup of vitest so that each worker (fork in this case) has its own running, singleton database. Because the global setup and the test suites itself run in different contexts, I had to create some kind of serializable description of the running testcontainers:

export async function createContainerDescription(
  db: StartedTestContainer,
  ports: number[],
): Promise<TestContainerDescription> {
  const client = await getContainerRuntimeClient();
  const container = client.container.getById(db.getId());

  const inspectResult = await client.container.inspect(container);

  return {
    id: container.id,
    inspectResult,
    name: db.getName(),
    host: db.getHost(),
    ports: ports.map((port) => db.getMappedPort(port)),
  };
}

We could then make use of this 'description' inside a test suite, by first recreating a StartedGenericContainer and then a StartedPostgreSqlContainer with the serializable description:

export async function getContainerFromDescription(description: TestContainerDescription) {
  const mappedPort = description.ports[0];
  assert(mappedPort);

  const boundPorts = new BoundPorts();
  boundPorts.setBinding(5432, mappedPort);

  const client = await getContainerRuntimeClient();

  const genericContainer = new StartedGenericContainer(
    client.container.getById(description.id),
    description.host,
    description.inspectResult,
    boundPorts,
    description.name,
    WAIT_STRATEGY,
    false,
  );

  return new StartedPostgreSqlContainer(
    genericContainer,
    CONFIG.name,
    CONFIG.user,
    CONFIG.password,
  );
}

This all works fine expect for the somewhat hacky way of creating the boundPorts and manually calling inspect on the container instance, but I was wondering if it would be a good idea to include some functionality in the library to achieve this more easily without having to manually recreate all required arguments to the container constructors.

[cristianrgreco]

Hi @PieterJanVdb, in what way is this different from the withReuse functionality? https://node.testcontainers.org/features/containers/#reusing-a-container

Essentially if you start a container with withReuse(), and from another process try and start the same container spec (same image, labels, etc), your startedGenericContainer reference points to the same underlying container as the first one. To have a container per worker I guess you could add a label like process.env.VITEST_WORKER_ID to the container in combination with withReuse().

Unless I've misunderstood something, even more simply, if you use vitest's setupFiles, and just start a new PG container, you will essentially get one container per forked worker process - so unless I'm mistaken your use case works out of the box.

[PieterJanVdb]

I tried it with withReuse, but we execute migrations after the container has started, and with withReuse I couldn't very easily figure out if the container is already running (with the migrations already having been executed on it), without doing a bunch of additional custom stuff, unless I'm missing something. Though it's not like I couldn't just run the migrations again, it's just somewhat of a performance hit.

[PieterJanVdb]

But I can see how this isn't really a desirable feature if it's mostly covered by withReuse

[cristianrgreco]

The issue is now more related to synchronising DB migrations. Which library are you using for applying migrations? I would expect them to lock tables where appropriate, and not re-apply migrations which have been applied, so it shouldn't matter for your use case if you have a single PG container, and 10 workers trying to apply migrations

[cristianrgreco]

I'd also recommend another pattern - for optimal performance, ideally you'd want a single PG container for all your tests. There's technically no reason why you need multiple PG containers, unless you're mutating data and your tests rely on the state of the DB - but this is an antipattern. You could restructure your tests to create their own unique, throwaway tables within the same PG container, and then you don't have to worry about shared state across tests, and you could even run your tests in parallel. Of course it's not always this simple, perhaps the cost of running the migration scripts is so large that creating tables is even more a performance degradation

[PieterJanVdb]

The library executes the migration just fine with locks etc, but due to some reason we have to call out to the bin of the library from node (via child_process), so I just tried to minimize that due to the slowness of it.

Oh and additionally, we have to sleep for about a second after starting the postgresql container before we can execute migrations on it, but that might just be an issue on our end, so that's an additional hit we wanted to avoid.

However, I think I can get around this entire issue by extending the PostgreSqlContainer and implementing the containerStarted lifecycle hook and only executing the migrations and creating the snapshot when reuse != false