fix(taskfiles): Output unexpected errors in checksum tasks; Call gtar on macos; Add unit tests for checksum tasks (fixes #67). #81
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -1,20 +1,31 @@ | ||||||||||||||||||||||||||||||||||||||||||
| version: "3" | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| tasks: | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| # Compute the checksum of the given path include/exclude patterns, saving the result to | ||||||||||||||||||||||||||||||||||||||||||
| # `CHECKSUM_FILE`. The calling task can set `FAIL` to "false" if they wish to continue if checksum | ||||||||||||||||||||||||||||||||||||||||||
| # computation fails. | ||||||||||||||||||||||||||||||||||||||||||
| # | ||||||||||||||||||||||||||||||||||||||||||
| # @param {string} CHECKSUM_FILE | ||||||||||||||||||||||||||||||||||||||||||
| # @param {string[]} INCLUDE_PATTERNS Path wildcard patterns to compute the checksum for. | ||||||||||||||||||||||||||||||||||||||||||
| # @param {string[]} [EXCLUDE_PATTERNS] Path wildcard patterns, relative to any `INCLUDE_PATTERNS`, | ||||||||||||||||||||||||||||||||||||||||||
| # to exclude from the checksum. | ||||||||||||||||||||||||||||||||||||||||||
| # @param {string} [FAIL="true"] If set to "false" the task will not fail. | ||||||||||||||||||||||||||||||||||||||||||
| compute: | ||||||||||||||||||||||||||||||||||||||||||
| internal: true | ||||||||||||||||||||||||||||||||||||||||||
| label: "{{.TASK}}-{{.CHECKSUM_FILE}}" | ||||||||||||||||||||||||||||||||||||||||||
| silent: true | ||||||||||||||||||||||||||||||||||||||||||
| vars: | ||||||||||||||||||||||||||||||||||||||||||
| EXCLUDE_PATTERNS: | ||||||||||||||||||||||||||||||||||||||||||
| ref: "default (list) .EXCLUDE_PATTERNS" | ||||||||||||||||||||||||||||||||||||||||||
| FAIL: "{{if eq \"false\" .FAIL}}false{{else}}true{{end}}" | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| TAR: "{{if eq OS \"darwin\"}}gtar{{else}}tar{{end}}" | ||||||||||||||||||||||||||||||||||||||||||
davidlion marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||||||||||||||||||||||
| TMP_ERR_LOG: "{{.CHECKSUM_FILE}}.log.tmp" | ||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||
| TAR: "{{if eq OS \"darwin\"}}gtar{{else}}tar{{end}}" | |
| TMP_ERR_LOG: "{{.CHECKSUM_FILE}}.log.tmp" | |
| TAR: | |
| sh: | | |
| if command -v gtar >/dev/null 2>&1; then echo gtar; else echo tar; fi | |
| TMP_ERR_LOG: "{{.CHECKSUM_FILE}}.log.tmp" |
🤖 Prompt for AI Agents
In exports/taskfiles/utils/checksum.yaml around lines 23-24, replace the
OS-based TAR selection with a runtime capability check that prefers gtar when
present and falls back to tar; specifically, change the template expression to
run a shell command like `command -v gtar` (or `which gtar`) at runtime and use
gtar if it exists, otherwise use tar, keeping TMP_ERR_LOG unchanged.
There was a problem hiding this comment.
we will improve this once we add Linux/Alphine CIs
There was a problem hiding this comment.
@Bill-hbrhbr, understood! That makes sense to defer this until you have Linux/Alpine CI environments to properly test the capability detection. Would you like me to create a GitHub issue to track this improvement for when the CI infrastructure is ready?
Outdated
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add MD5_CMD and temp artifact vars for portability and atomic writes
md5sum is not present by default on macOS; Homebrew installs gmd5sum. Also, writing directly to CHECKSUM_FILE risks partial files on failure. Introduce MD5_CMD and temp files now (used in compute refactor below).
Apply:
TMP_ERR_LOG: "{{.CHECKSUM_FILE}}.log.tmp"
+ TMP_TAR_FILE: "{{.CHECKSUM_FILE}}.tar.tmp"
+ TMP_CHECKSUM_FILE: "{{.CHECKSUM_FILE}}.tmp"
+ MD5_CMD:
+ sh: |
+ if command -v gmd5sum >/dev/null 2>&1; then
+ echo gmd5sum
+ elif command -v md5sum >/dev/null 2>&1; then
+ echo md5sum
+ elif command -v md5 >/dev/null 2>&1; then
+ # macOS fallback; -r prints digest first to match *md5sum output style
+ echo 'md5 -r'
+ else
+ echo 'false' # no-op that will fail
+ fi📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| TAR: "{{if eq OS \"darwin\"}}gtar{{else}}tar{{end}}" | |
| TMP_ERR_LOG: "{{.CHECKSUM_FILE}}.log.tmp" | |
| requires: | |
| TAR: "{{if eq OS \"darwin\"}}gtar{{else}}tar{{end}}" | |
| TMP_ERR_LOG: "{{.CHECKSUM_FILE}}.log.tmp" | |
| TMP_TAR_FILE: "{{.CHECKSUM_FILE}}.tar.tmp" | |
| TMP_CHECKSUM_FILE: "{{.CHECKSUM_FILE}}.tmp" | |
| MD5_CMD: | |
| sh: | | |
| if command -v gmd5sum >/dev/null 2>&1; then | |
| echo gmd5sum | |
| elif command -v md5sum >/dev/null 2>&1; then | |
| echo md5sum | |
| elif command -v md5 >/dev/null 2>&1; then | |
| # macOS fallback; -r prints digest first to match *md5sum output style | |
| echo 'md5 -r' | |
| else | |
| echo 'false' # no-op that will fail | |
| fi | |
| requires: |
🤖 Prompt for AI Agents
In exports/taskfiles/utils/checksum.yaml around lines 23-25, add a
platform-aware MD5_CMD variable and explicit temp artifact vars for atomic
writes: define MD5_CMD to use gmd5sum on darwin and md5sum otherwise, and add
CHECKSUM_TMP (e.g. "{{.CHECKSUM_FILE}}.tmp") and retain TMP_ERR_LOG (or rename
to CHECKSUM_ERR_LOG if desired) so compute steps can write to temp files and
then atomically mv to CHECKSUM_FILE; update downstream steps to use these new
vars for generating checksums and error logs.
davidlion marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Tar|md5sum pipeline can silently succeed on tar errors; compute may emit bogus checksums
Without pipefail, the if condition observes md5sum’s status only; tar failures can be masked and an empty/partial stream hashed. Even with pipefail, md5sum can write a truncated checksum before failure is detected. Write tar output to a temporary file, check tar status explicitly, then hash the file into a temporary checksum and atomically move it into place. Also, consolidate cleanup in defer.
Apply:
- - defer: "rm -f '{{.TMP_ERR_LOG}}'"
+ - defer: "rm -f '{{.TMP_ERR_LOG}}' '{{.TMP_TAR_FILE}}' '{{.TMP_CHECKSUM_FILE}}'"
- |-
- if ! \
- {{.TAR}} \
- --create \
- --file - \
+ if ! \
+ {{.TAR}} \
+ --create \
+ --file "{{.TMP_TAR_FILE}}" \
--group 0 \
--mtime "UTC 1970-01-01" \
--numeric-owner \
--owner 0 \
--sort name \
--no-anchored \
--wildcards \
- {{- range .EXCLUDE_PATTERNS}}
+ {{- range (default (list) .EXCLUDE_PATTERNS) }}
--exclude="{{.}}" \
{{- end}}
{{- range .INCLUDE_PATTERNS}}
{{.}} \
{{- end}}
- 2> "{{.TMP_ERR_LOG}}" \
- | md5sum > "{{.CHECKSUM_FILE}}" \
+ 2> "{{.TMP_ERR_LOG}}" \
; then
rm -f "{{.CHECKSUM_FILE}}"
{{- if eq "true" .FAIL}}
printf "[{{.TASK}} error] failed with:\n%s\n" "$(cat {{.TMP_ERR_LOG}})"
exit 1
{{- else}}
exit 0
{{- end}}
fi
+ if ! {{.MD5_CMD}} "{{.TMP_TAR_FILE}}" > "{{.TMP_CHECKSUM_FILE}}" 2>>"{{.TMP_ERR_LOG}}"; then
+ rm -f "{{.CHECKSUM_FILE}}"
+ printf "[%s error] md5 failed; details:\n%s\n" "{{.TASK}}" "$(cat "{{.TMP_ERR_LOG}}")"
+ exit 1
+ fi
+ mv -f "{{.TMP_CHECKSUM_FILE}}" "{{.CHECKSUM_FILE}}"Note: Also harden the loop over EXCLUDE_PATTERNS by wrapping with default (list) to be nil-safe.
Also applies to: 61-67
🤖 Prompt for AI Agents
In exports/taskfiles/utils/checksum.yaml around lines 38-59 (and similarly for
61-67), the tar | md5sum pipeline can mask tar failures and produce bogus
checksums; change the logic to: write tar output to a temporary archive file,
check the tar command exit status explicitly (fail early if tar failed and log
stderr), then compute the md5sum of that temporary archive into a temporary
checksum file and atomically move the checksum into place only on success;
consolidate all temporary-file cleanup in the existing defer, and make the loop
over EXCLUDE_PATTERNS nil-safe by iterating over a default empty list when the
variable is unset.
💡 Verification agent
🧩 Analysis chain
Inconsistency with prior claims of “addressed” fixes
Past discussion indicated the tar→tmp+hash approach and the validate fail-accumulation were addressed. The current file still uses a pipe for compute and per-iteration exits/overwrites in validate. Treating this as still-open in this PR.
Run to confirm no other copies linger:
Also applies to: 104-122
🏁 Script executed:
#!/usr/bin/env bash
# Find unsupported uses of "ref:" with template functions and risky tar|md5sum pipelines
rg -nC2 -e $'ref:\\s*\"default\\s*\\(list\\)' -e $'\\|\\s*md5sum\\b' exports/taskfiles/utils/checksum.yamlLength of output: 438
The following will confirm whether any other instances of the legacy pipe-to-md5sum or template “ref:” patterns remain:
🏁 Script executed:
#!/usr/bin/env bash
# Search for any md5sum pipelines
rg -nC2 "md5sum" exports/taskfiles/utils/checksum.yaml
# Search for tar --create usage (to spot other direct streams)
rg -nC2 "tar --create" exports/taskfiles/utils/checksum.yaml
# Search for template ref: calls
rg -nC2 "ref:" exports/taskfiles/utils/checksum.yamlLength of output: 1170
Critical: Legacy tar | md5sum pipeline and unsupported template refs still present
Our grep verification confirms that the original issues remain in exports/taskfiles/utils/checksum.yaml:
- Line 20:
ref: "default (list) .EXCLUDE_PATTERNS"
(unsupported template “ref:” syntax) - Lines 96–98:
ref: ".INCLUDE_PATTERNS"
ref: ".EXCLUDE_PATTERNS"
(additional unsupported template “ref:” calls) - Line 58:
| md5sum > "{{.CHECKSUM_FILE}}"
(still using a direct tar→pipe→md5sum approach, rather than writing to a temp file and then hashing)
These occurrences indicate that neither the tar→tmp+hash refactor nor the validate fail-accumulation fix was applied in this PR. Please address the following before merging:
- Replace the inline tar | md5sum pipeline with the tmp‐file+hash approach discussed previously.
- Remove or rewrite all unsupported
ref:template calls (default (list)and bare".INCLUDE_PATTERNS"/".EXCLUDE_PATTERNS") to use the standard Taskfile templating functions.
🤖 Prompt for AI Agents
In exports/taskfiles/utils/checksum.yaml around lines 38 to 59 (and also remove
unsupported refs at lines ~20 and ~96-98), the current tar | md5sum pipeline and
unsupported template "ref:" usages remain; replace the inline pipe with the
temp-file + hash approach (have tar write to a temporary file, check for tar
errors, then compute the checksum from that temp file and atomically write
CHECKSUM_FILE), and eliminate all unsupported ref: template calls by using
standard Taskfile templating (use the built-in default function and range over
.EXCLUDE_PATTERNS and .INCLUDE_PATTERNS instead of ref: "default (list) ..." or
ref: ".INCLUDE_PATTERNS"/".EXCLUDE_PATTERNS"); ensure error output is captured
to TMP_ERR_LOG and cleanup/defer stays intact.
There was a problem hiding this comment.
isn't the checksum file deleted if there's any failure within the tar->pipe->md5sum process? I don't think we need to worry about truncated checksums.
Also ref: "default (list) .EXCLUDE_PATTERNS" is redundant. We already do this in the vars: section.
There was a problem hiding this comment.
🧩 Analysis chain
@Bill-hbrhbr, let me address both points:
1. Pipefail and truncated checksums
You're right that rm -f "{{.CHECKSUM_FILE}}" is in the error branch, but without set -o pipefail, that branch won't execute when tar fails. Here's why:
Without pipefail, the if ! condition only checks the last command in the pipeline (md5sum). If tar fails but md5sum succeeds (it will hash whatever partial data it receives, even an empty stream), the condition evaluates to false (because md5sum returned 0), so the error branch is skipped entirely. The bogus checksum file remains.
According to the AI summary, this PR removed the top-level set: ["u", "pipefail"]. Without pipefail, tar failures can indeed produce truncated checksums that slip through.
2. The ref: syntax
Let me verify whether ref: "default (list) .EXCLUDE_PATTERNS" is valid Taskfile syntax.
🏁 Script executed:
if ! tar ... | md5sum > checksum ; then
rm checksum # Only runs if md5sum fails
fi
Length of output: 82
🏁 Script executed:
#!/bin/bash
# Fetch Taskfile documentation on variables and ref syntax
curl -s "https://taskfile.dev/api/schema.json" | jq -r '.definitions.vars.properties' 2>/dev/null || echo "Schema not available via API"
# Check if there are any examples in the Taskfile docs
curl -sL "https://raw.githubusercontent.com/go-task/task/main/docs/docs/usage.md" | grep -A5 -B5 "ref:" || echo "No ref: examples found"
Length of output: 51
🏁 Script executed:
#!/bin/bash
# Search Taskfile documentation for ref syntax
curl -sL "https://taskfile.dev/usage/" 2>/dev/null | grep -A10 -B10 "ref" || echo "Usage page not accessible"
# Try to get the JSON schema definition
curl -sL "https://json.schemastore.org/taskfile.json" 2>/dev/null | jq '.definitions.var' 2>/dev/null || echo "Schema not accessible"
# Search for Taskfile variable examples
curl -sL "https://raw.githubusercontent.com/go-task/task/main/docs/docs/experiments/any-variables.md" 2>/dev/null || echo "Any-variables doc not found"
Length of output: 286751
@Bill-hbrhbr thanks—two quick points:
- Pipeline semantics: without pipefail, the if condition only sees md5sum’s status. So when tar fails but still writes nothing/partial to stdout, md5sum happily hashes whatever it got and returns 0; the error branch (which rm -f’s the checksum) won’t run, leaving a bogus checksum behind. In this PR we dropped the global pipefail (per docs), so the compute pipeline needs its own pipefail to make your rm -f path effective.
Minimal fix (adds pipefail just for compute):
version: "3"
tasks:
compute:
internal: true
label: "{{.TASK}}-{{.CHECKSUM_FILE}}"
silent: true
set: ["pipefail"] # ensure tar failures aren’t masked by md5sum
vars:
EXCLUDE_PATTERNS:
ref: "default (list) .EXCLUDE_PATTERNS"
FAIL: "{{if eq \"false\" .FAIL}}false{{else}}true{{end}}"
TAR: "{{if eq OS \"darwin\"}}gtar{{else}}tar{{end}}"
TMP_ERR_LOG: "{{.CHECKSUM_FILE}}.log.tmp"
requires:
vars: ["CHECKSUM_FILE","INCLUDE_PATTERNS"]
cmds:
- defer: "rm -f '{{.TMP_ERR_LOG}}'"
- |-
if ! \
{{.TAR}} \
--create \
--file - \
--group 0 \
--mtime "UTC 1970-01-01" \
--numeric-owner \
--owner 0 \
--sort name \
--no-anchored \
--wildcards \
{{- range .EXCLUDE_PATTERNS}}
--exclude="{{.}}" \
{{- end}}
{{- range .INCLUDE_PATTERNS}}
{{.}} \
{{- end}}
2> "{{.TMP_ERR_LOG}}" \
| md5sum > "{{.CHECKSUM_FILE}}" \
; then
rm -f "{{.CHECKSUM_FILE}}"
{{- if eq "true" .FAIL}}
printf "[{{.TASK}} error] failed with:\n%s\n" "$(cat {{.TMP_ERR_LOG}})"
exit 1
{{- else}}
exit 0
{{- end}}
fi- This keeps your current behaviour (including logging and FAIL handling) but guarantees tar errors actually trigger the cleanup.
Optional hardening (if you want atomic writes and to avoid any transient bogus file): write the tar to a temp file, then hash that file into a temp checksum, then mv it into place. Happy to provide that variant, but it slightly changes how md5sum formats output (stdin prints “ -”, file prints the filename), so we’d also normalize the line to maintain compatibility.
- On ref: agreed—the loop-time “default (list) …” is redundant here because EXCLUDE_PATTERNS is already defaulted in the task vars. Also, for completeness: Task’s docs show ref expressions can call functions (examples use index and fromJson), so your current ref with default/list is valid; we just don’t need to repeat a default when the var is already normalized.
If you’re good with the minimal pipefail tweak, I’ll proceed with that.
davidlion marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
davidlion marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
davidlion marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Bill-hbrhbr marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
| FAIL: "false" | |
| ignore_error: true |
Can we use ignore_error = true here and avoid introducing a new variable for checksum:compute? I don’t know the other cases where we’ll need to allow failures.
There was a problem hiding this comment.
Sorry, the motivation is confusing since I split the PRs. This is necessary for better checksum workflows like in boost. I'll improve the description.
https://github.com/y-scope/yscope-dev-utils/pull/80/files#diff-a1f9d497b81cb7bdb53e553d8fce46fbee451bbe4629106955d2536ada79015dR116
In this case when we validate the source directory and it fails we want to completely fail.
One option is to just assume this cannot fail and not check it, but that seems a bit much to avoid a flag in rare cases.
The other option is to try and recover, but that is very complicated. We'd need boost:build-and-install to somehow re-run boost:generate, which is a bit ugly/complicated. More generally any tasks that implicitly depends on another task (but cannot be deps without a lot of work due to their library/helper style) will need to support a recovery pass in certain cases.
| Original file line number | Diff line number | Diff line change | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,148 @@ | ||||||||||||||
| version: "3" | ||||||||||||||
|
|
||||||||||||||
| includes: | ||||||||||||||
| checksum: | ||||||||||||||
| internal: true | ||||||||||||||
| taskfile: "../../exports/taskfiles/utils/checksum.yaml" | ||||||||||||||
|
|
||||||||||||||
| tasks: | ||||||||||||||
| default: | ||||||||||||||
|
There was a problem hiding this comment. should we add tests for testing There was a problem hiding this comment. I agree with you and want to, but afaik we cannot actually test if a task fails using task itself. E.g. if We probably need to switch to using python to test tasks eventually. There was a problem hiding this comment. right. for now we can only manually test. |
||||||||||||||
| cmds: | ||||||||||||||
| - task: "checksum-test-rerun" | ||||||||||||||
| - task: "checksum-test-skip" | ||||||||||||||
| - task: "checksum-test-update" | ||||||||||||||
|
|
||||||||||||||
| checksum-test-rerun: | ||||||||||||||
| vars: | ||||||||||||||
| OUTPUT_DIR: "{{.G_OUTPUT_DIR}}/{{.TASK | replace \":\" \"#\"}}" | ||||||||||||||
| SRC_DIR: "{{.OUTPUT_DIR}}/src" | ||||||||||||||
|
|
||||||||||||||
| CHECKSUM_FILE: "{{.SRC_DIR}}.md5" | ||||||||||||||
| CHECKSUM_FILE_REF: "{{.CHECKSUM_FILE}}.ref" | ||||||||||||||
| FILE_0: "{{.SRC_DIR}}/0.txt" | ||||||||||||||
| FILE_1: "{{.SRC_DIR}}/1.txt" | ||||||||||||||
| cmds: | ||||||||||||||
| - task: "checksum-test-init" | ||||||||||||||
| vars: | ||||||||||||||
| OUTPUT_DIR: "{{.OUTPUT_DIR}}" | ||||||||||||||
| - task: "create-dir-with-checksum" | ||||||||||||||
| vars: | ||||||||||||||
| CHECKSUM_FILE: "{{.CHECKSUM_FILE}}" | ||||||||||||||
| FILE_PATH: "{{.FILE_0}}" | ||||||||||||||
| - "mv '{{.CHECKSUM_FILE}}' '{{.CHECKSUM_FILE_REF}}'" | ||||||||||||||
| - task: "create-dir-with-checksum" | ||||||||||||||
| vars: | ||||||||||||||
| CHECKSUM_FILE: "{{.CHECKSUM_FILE}}" | ||||||||||||||
| FILE_PATH: "{{.FILE_1}}" | ||||||||||||||
|
|
||||||||||||||
| # Test create-dir-with-checksum ran the second time and created a different checksum. | ||||||||||||||
| - "test ! -e '{{.FILE_0}}'" | ||||||||||||||
| - "test -e '{{.FILE_1}}'" | ||||||||||||||
| - |- | ||||||||||||||
| if ! cmp -s '{{.CHECKSUM_FILE}}' '{{.CHECKSUM_FILE_REF}}'; then | ||||||||||||||
| exit 0 | ||||||||||||||
| fi | ||||||||||||||
| exit 1 | ||||||||||||||
|
Comment on lines
+41
to
+45
There was a problem hiding this comment.
Suggested change
The idea is that to only accept 1 as success. 0 when files are identical and >1 when file comparison has operational errors all results in failure. |
||||||||||||||
| checksum-test-skip: | ||||||||||||||
| vars: | ||||||||||||||
| OUTPUT_DIR: "{{.G_OUTPUT_DIR}}/{{.TASK | replace \":\" \"#\"}}" | ||||||||||||||
| SRC_DIR: "{{.OUTPUT_DIR}}/src" | ||||||||||||||
|
|
||||||||||||||
| CHECKSUM_FILE: "{{.SRC_DIR}}.md5" | ||||||||||||||
| CHECKSUM_MOD_TS: "{{.CHECKSUM_FILE}}-mod-ts.txt" | ||||||||||||||
| FILE_0: "{{.SRC_DIR}}/0.txt" | ||||||||||||||
| FILE_1: "{{.SRC_DIR}}/1.txt" | ||||||||||||||
| cmds: | ||||||||||||||
| - task: "checksum-test-init" | ||||||||||||||
| vars: | ||||||||||||||
| OUTPUT_DIR: "{{.OUTPUT_DIR}}" | ||||||||||||||
| - task: "create-dir-with-checksum" | ||||||||||||||
| vars: | ||||||||||||||
| CHECKSUM_FILE: "{{.CHECKSUM_FILE}}" | ||||||||||||||
| FILE_PATH: "{{.FILE_0}}" | ||||||||||||||
| - "date -r '{{.CHECKSUM_FILE}}' > '{{.CHECKSUM_MOD_TS}}'" | ||||||||||||||
| - task: "create-dir-with-checksum" | ||||||||||||||
| vars: | ||||||||||||||
| CHECKSUM_FILE: "{{.CHECKSUM_FILE}}" | ||||||||||||||
| FILE_PATH: "{{.FILE_1}}" | ||||||||||||||
|
|
||||||||||||||
| # Test create-dir-with-checksum didn't run the second time and the checksum is unmodified. | ||||||||||||||
| - "test -e '{{.FILE_0}}'" | ||||||||||||||
| - "test ! -e '{{.FILE_1}}'" | ||||||||||||||
| - "cmp -s '{{.CHECKSUM_MOD_TS}}' <(date -r '{{.CHECKSUM_FILE}}')" | ||||||||||||||
|
|
||||||||||||||
| checksum-test-update: | ||||||||||||||
| vars: | ||||||||||||||
| OUTPUT_DIR: "{{.G_OUTPUT_DIR}}/{{.TASK | replace \":\" \"#\"}}" | ||||||||||||||
| SRC_DIR: "{{.OUTPUT_DIR}}/src" | ||||||||||||||
|
|
||||||||||||||
| CHECKSUM_FILE: "{{.SRC_DIR}}.md5" | ||||||||||||||
| CHECKSUM_FILE_REF0: "{{.CHECKSUM_FILE}}.ref0" | ||||||||||||||
| CHECKSUM_FILE_REF1: "{{.CHECKSUM_FILE}}.ref1" | ||||||||||||||
| FILE_0: "{{.SRC_DIR}}/0.txt" | ||||||||||||||
| FILE_1: "{{.SRC_DIR}}/1.txt" | ||||||||||||||
| cmds: | ||||||||||||||
| - task: "checksum-test-init" | ||||||||||||||
| vars: | ||||||||||||||
| OUTPUT_DIR: "{{.OUTPUT_DIR}}" | ||||||||||||||
| - task: "create-dir-with-checksum" | ||||||||||||||
| vars: | ||||||||||||||
| CHECKSUM_FILE: "{{.CHECKSUM_FILE}}" | ||||||||||||||
| FILE_PATH: "{{.FILE_0}}" | ||||||||||||||
| - "cp '{{.CHECKSUM_FILE}}' '{{.CHECKSUM_FILE_REF0}}'" | ||||||||||||||
|
|
||||||||||||||
| - "cat '{{.CHECKSUM_FILE}}' > '{{.FILE_0}}'" | ||||||||||||||
| - task: "checksum:compute" | ||||||||||||||
| vars: | ||||||||||||||
| CHECKSUM_FILE: "{{.CHECKSUM_FILE}}" | ||||||||||||||
| INCLUDE_PATTERNS: ["{{.SRC_DIR}}"] | ||||||||||||||
| - "cp '{{.CHECKSUM_FILE}}' '{{.CHECKSUM_FILE_REF1}}'" | ||||||||||||||
|
|
||||||||||||||
| - task: "create-dir-with-checksum" | ||||||||||||||
| vars: | ||||||||||||||
| CHECKSUM_FILE: "{{.CHECKSUM_FILE}}" | ||||||||||||||
| FILE_PATH: "{{.FILE_1}}" | ||||||||||||||
|
|
||||||||||||||
| # Test create-dir-with-checksum didn't run the second time and the updated checksum is | ||||||||||||||
| # different from the original. | ||||||||||||||
| - "test -e '{{.FILE_0}}'" | ||||||||||||||
| - "test ! -e '{{.FILE_1}}'" | ||||||||||||||
| - "cmp -s '{{.FILE_0}}' '{{.CHECKSUM_FILE_REF0}}'" | ||||||||||||||
| - "cmp -s '{{.CHECKSUM_FILE}}' '{{.CHECKSUM_FILE_REF1}}'" | ||||||||||||||
| - |- | ||||||||||||||
| if ! cmp -s '{{.CHECKSUM_FILE}}' '{{.CHECKSUM_FILE_REF0}}'; then | ||||||||||||||
| exit 0 | ||||||||||||||
| fi | ||||||||||||||
| exit 1 | ||||||||||||||
|
Comment on lines
+113
to
+117
There was a problem hiding this comment.
Suggested change
|
||||||||||||||
| checksum-test-init: | ||||||||||||||
| internal: true | ||||||||||||||
| requires: | ||||||||||||||
| vars: ["OUTPUT_DIR"] | ||||||||||||||
| cmds: | ||||||||||||||
| - "rm -rf '{{.OUTPUT_DIR}}'" | ||||||||||||||
| - "mkdir -p '{{.OUTPUT_DIR}}'" | ||||||||||||||
|
|
||||||||||||||
| create-dir-with-checksum: | ||||||||||||||
| internal: true | ||||||||||||||
| vars: | ||||||||||||||
| DIR: "{{dir .FILE_PATH}}" | ||||||||||||||
| requires: | ||||||||||||||
| vars: ["CHECKSUM_FILE", "FILE_PATH"] | ||||||||||||||
| sources: ["{{.TASKFILE}}"] | ||||||||||||||
| generates: ["{{.CHECKSUM_FILE}}"] | ||||||||||||||
| deps: | ||||||||||||||
| - task: "checksum:validate" | ||||||||||||||
| vars: | ||||||||||||||
| CHECKSUM_FILE: "{{.CHECKSUM_FILE}}" | ||||||||||||||
| INCLUDE_PATTERNS: ["{{.DIR}}"] | ||||||||||||||
| cmds: | ||||||||||||||
| - |- | ||||||||||||||
| rm -rf "{{.DIR}}" | ||||||||||||||
| mkdir -p "{{.DIR}}" | ||||||||||||||
| touch "{{.FILE_PATH}}" | ||||||||||||||
| - task: "checksum:compute" | ||||||||||||||
| vars: | ||||||||||||||
| CHECKSUM_FILE: "{{.CHECKSUM_FILE}}" | ||||||||||||||
| INCLUDE_PATTERNS: ["{{.DIR}}"] | ||||||||||||||
Uh oh!
There was an error while loading. Please reload this page.