Skip to content

Commit 0aed62c

Browse files
authored
fix(pypex): spill to a param file for PyPex when necessary (#617)
Binaries with large numbers of source inputs (i.e. large transitive dependency trees) can lead to the argument vector's size exceeding the OS' limits. Spilling to a param file when the argument list grows long avoids this issue. For smaller argument lists, skipping the param file generation is an option that results in slightly less build work (one less file to create). This change is an alternate implementation of https://github.com/aspect-build/rules_py/pull/543/files. Unlike that change, this change uses the dedicated APIs for solving this problem — `ctx.actions.args.use_param_file` on the Bazel side, `fromfile_prefix_char` on the ArgumentParser side. --- ### Changes are visible to end-users: yes - Searched for relevant documentation and updated as needed: yes (no docs reference this case) - Breaking change (forces users to change their own code or config): no * only builds that would have failed previously have their argv spilled to the param file * builds that worked because their argv were under-limit when passed in-line are unchanged in behaviour; they still pass their args inlinei - Suggested release notes appear below: no ### Test plan - Covered by existing test cases - Manual testing: my real application (cannot be shared sadly) fixes #542
1 parent 88716fa commit 0aed62c

File tree

2 files changed

+4
-1
lines changed

2 files changed

+4
-1
lines changed

py/private/py_pex_binary.bzl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,9 @@ def _py_python_pex_impl(ctx):
7676

7777
args = ctx.actions.args()
7878

79+
args.use_param_file(param_file_arg = "@%s")
80+
args.set_param_file_format("multiline")
81+
7982
# Copy workspace name here to prevent ctx
8083
# being transferred to the execution phase.
8184
workspace_name = str(ctx.workspace_name)

py/tools/pex/main.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ def __call__(self, parser, namespace, value, option_str=None):
2626
)
2727
self.default.append(tuple(components))
2828

29-
parser = ArgumentParser()
29+
parser = ArgumentParser(fromfile_prefix_chars='@')
3030

3131
parser.add_argument(
3232
"-o",

0 commit comments

Comments
 (0)