integration-race: bump timeout from 20 to 30 minutes

[pohly]

There were still a few jobs runs were some tests (most recently: test/integration/scheduler_perf/misc) timed out. We could split that up a bit more, but as integration testing with race detection isn't something that needs to complete quickly it's simpler to raise the timeout.

/assign @BenTheElder

[BenTheElder]

TODO: set the job-level timeout in case this hangs, right now it's the 2h default we have config-wide

[pohly]

Sorry, I'm not following. Are you suggesting to increase the job timeout?

The job passes in ~70min pretty consistently: https://testgrid.k8s.io/sig-testing-canaries#integration-race-master&graph-metrics=test-duration-minutes

The problem is that scheduler_perf/misc and to a lesser extend scheduler_perf.affinity are close to the 20min per-directory limit, which leads to rare flakes.

[pohly]

I might have understood what you meant: I can reduce the job limit to e.g 90min safely based on how long it takes in practice. Then if each individual test times out after 30min, we abort after 90min instead of 120min.

Doesn't look like a significant change, though?

Hmm, how do I actually set the job-level timeout? https://docs.prow.k8s.io/docs/jobs/ doesn't mention it.

I see

    decorate: true
    decoration_config:
      timeout: 5h

but where is that documented?

https://docs.prow.k8s.io/docs/components/pod-utilities/ mentions "decorate: true" and links to https://docs.prow.k8s.io/docs/components/deprecated/plank/

Sorry, I digress. Let's just use copy-and-paste... done.

[BenTheElder]

Prow could do with revamped docs amongst other things, sig testing are really light on maintainers there at the moment.

But yes, decoration_config timeout is it.

[pohly]

So is the PR okay now? I already reduced the timeout to 90 minutes.

[BenTheElder]

I would suggest placing it much closer to the intended timeout of the workload, but the PR is fine to merge.

[pohly]

Closer would be something like this:

    decoration_config:
      timeout: 90m
    spec:
      containers:
      - image: us-central1-docker.pkg.dev/k8s-staging-test-infra/images/kubekins-e2e:v20250925-95b5a2c7a5-master
        command:
        - runner.sh
        env:
        - name: KUBE_TIMEOUT
          value: "-timeout=30m"
        - name: KUBE_RACE
          value: "-race"

IMHO that's not "close enough" to make the connection. Some comments would have been better.

It's also an unusual place for decoration_config compared to other jobs. Remember that at some point something besides timeout might need to be configured there.

I prefer to keep it as is and don't want to delay further to add comments.

/hold cancel

[BenTheElder]

/lgtm
/approve
/hold

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder, pohly

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• config/OWNERS [BenTheElder]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

@pohly: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

• key integration.yaml using file config/jobs/kubernetes/sig-testing/integration.yaml

In response to this:

There were still a few jobs runs were some tests (most recently: test/integration/scheduler_perf/misc) timed out. We could split that up a bit more, but as integration testing with race detection isn't something that needs to complete quickly it's simpler to raise the timeout.

/assign @BenTheElder

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.