Skip to content

Commit 22691e6

Browse files
authored
fix: extremely wide foreach on argo breaking (#2659)
1 parent d8715ce commit 22691e6

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

metaflow/plugins/argo/argo_workflows.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2136,14 +2136,16 @@ def _container_templates(self):
21362136
foreach_step,
21372137
)
21382138
)
2139+
# NOTE: input-paths might be extremely lengthy so we dump these to disk instead of passing them directly to the cmd
2140+
step_cmds.append("echo %s >> /tmp/mf-input-paths" % input_paths)
21392141
step = [
21402142
"step",
21412143
node.name,
21422144
"--run-id %s" % run_id,
21432145
"--task-id %s" % task_id,
21442146
"--retry-count %s" % retry_count,
21452147
"--max-user-code-retries %d" % user_code_retries,
2146-
"--input-paths %s" % input_paths,
2148+
"--input-paths-filename /tmp/mf-input-paths",
21472149
]
21482150
if node.parallel_step:
21492151
step.append(

0 commit comments

Comments
 (0)