Don't write to runfiles

[jadenPete]

py_binary targets create virtual environments in runfiles. This can be problematic in remote execution contexts (we're seeing errors in Buildfarm with certain sandboxing options enabled) because runfiles are often read-only on those environments. This is a safety measure—action inputs are sometimes symbolically linked or shared between actions to reduce unnecessary I/O, and a py_binary can be the executable of an action.

This PR updates py/private/run.tmpl.sh to create the virtual environment in a /tmp subdirectory. This directory is randomly generated, which should minimize the likelihood of actions stepping on each other's toes. If more separation is needed, each action can get its own /tmp directory with --incompatible_sandbox_hermetic_tmp,

---

Changes are visible to end-users: no

I don't think users should rely on the virtual environment being in one place or another, but if they are, this does constitute a visible change.

• Searched for relevant documentation and updated as needed: N/A
• Breaking change (forces users to change their own code or config): N/A
• Suggested release notes appear below: N/A

Test plan

• Covered by existing test cases
• New test cases added
• Manual testing; please provide instructions so we can reproduce:
  • This isn't reproducable, but we've tested this change extensively in our monorepo with thousands of rules_py-backed actions

[CLAassistant]

All committers have signed the CLA.

[aspect-workflows]

Test

Buildkite build #383 is running...

[arrdem]

Hey @jadenPete, thanks for the PR.

As-is this isn't a change I can consider. You've introduced least very much not obviously correct if not breaking change or at to how imports are processed, and the scope of impact from changing where the dynamically built venvs get laid out too is impossible to assess. We have some users who rely on the current behavior, and there are a number of potential downsides to generating a copy of the venv per process instance as this does.

It seems to me that the new py_venv_* machinery likely addresses the problem of allowing for reentrant binaries better than simply solving the copy collision problem.

I'd be willing to integrate a version of this change which made the destination dir controlled either by an environment variable at runtime, a feature flag at build time or both. But I don't think unilaterally changing it is a great idea.