testmark-bash

run: each full-road case sources testcase.bash by its absolute path, instead of carrying its text in the case's script.

commit 477bcbbdcb2e39c5ca504c6084cec3732451a21a · parent 0bd1c47577 · browse this tree
Ximianth, 2026-10-25 15:28:15 UTC

Simpler, and cheaper: bash reads a regular file in blocks, where it reads script text from a pipe a byte at a time.
The path is made absolute when the library is sourced, since a case runs in the test script's directory, not the caller's;
examples/test-todos-files.sh, run by make from the repo root, sources the library by a relative path, and fails without that.

diff --git a/README.md b/README.md
index 435921a..f7b1d0b 100644
--- a/README.md
+++ b/README.md
@@ -78,6 +78,7 @@ Each case runs it in a fresh bash, so it's a clean shell:
 - `set -euo pipefail` is on.  A failing command stops the case, and the failure shows up in its output.
   Bash's own error messages there are labeled with the function's name, nothing more, so they read the same wherever the test runs.
 - The working directory is the test script's, so a relative path like `./mytool` works, wherever the test is run from.
+  `$TESTCASE_DIR` holds that directory too, for after a `cd`.
 - It sees exported variables (like `PATH`), but no other variables, and no other functions.
   If it calls helpers, pass them to `testmarktest_provide helper1 helper2` first.
 
@@ -138,12 +139,42 @@ Prefer this over grepping for a fragment of the output:
 an exact check can't pass by accident, it shows the reader what the output really looks like, and regen can write it.
 
 
+More than one hunk in or out
+----------------------------
+
+When a case needs several hunks, or gives several outputs, make its rule with `testmarktest_rule` instead.
+Its function gets nothing on stdin; it reads and writes hunks itself, with `testcase_*` functions.
+[`examples/test-todos-files.sh`](examples/test-todos-files.sh) runs `todos.sh` on two files at once:
+
+```bash
+exercise_todos_files() {
+	testcase_withtmpdir
+	testcase_read_myhunk mine.txt >mine.txt
+	testcase_read_anyhunk shared/theirs.txt >theirs.txt
+	"$TESTCASE_DIR/todos.sh" mine.txt theirs.txt | testcase_output output
+}
+
+testmarktest_rule '*/mine.txt' exercise_todos_files
+testmarktest_run todos-files.md
+```
+
+- `testcase_read_myhunk NAME` prints the case's hunk `CASE/NAME`.
+- `testcase_read_anyhunk NAME` prints any hunk in the doc, by its whole name: for hunks many cases share.
+- `testcase_output NAME` checks what it reads against `CASE/NAME`.  Call it once per output.
+- `testcase_withtmpdir` moves the case into a new temp dir, removed when the case ends.
+  It's the only one that makes files.  `$TESTCASE_DIR` still holds the test script's directory, for reaching the tool, like above.
+
+Anything else the function prints, or a nonzero exit, has no hunk to go in, so it fails the case.
+The fixture for a case that works is just its outputs.
+
+
 Functions
 ---------
 
 | function | does |
 | --- | --- |
 | `testmarktest_rule_basic GLOB FN` | makes each hunk matching GLOB a case, exercised by FN, with the hunk on stdin, and checked against the case's `output` hunk |
+| `testmarktest_rule GLOB FN` | makes each hunk matching GLOB a case, exercised by FN, which reads hunks and gives outputs with the `testcase_*` functions |
 | `testmarktest_provide FN...` | sends more functions into every case, for exercise functions that call them |
 | `testmarktest_run DOC...` | runs every case in the docs, reports each, and writes the docs under `TESTMARK_REGEN` |
 
diff --git a/lib/testmark/run.bash b/lib/testmark/run.bash
index 533c1e4..888cea2 100644
--- a/lib/testmark/run.bash
+++ b/lib/testmark/run.bash
@@ -13,8 +13,9 @@ _testmarktest_fdbase=10
 # The directory of the script being run, absolute; empty if bash isn't running a script file.
 # Set when the library is sourced, so a later `cd` can't move it.
 _testmarktest_dir=
-# The text of testcase.bash, which every case of testmarktest_rule gets.  Status 1 is the end of the file.
-IFS= read -r -d '' _testmarktest_testcase <"${BASH_SOURCE[0]%/*}/testcase.bash" || :
+# testcase.bash's absolute path, for each case of testmarktest_rule to source.  Absolute now, so neither a later `cd` nor the case's own directory can move it.
+_testmarktest_testcase=${BASH_SOURCE[0]%/*}/testcase.bash
+[[ $_testmarktest_testcase == /* ]] || _testmarktest_testcase=$PWD/$_testmarktest_testcase
 
 ## Sets _testmarktest_dir to the directory of script $1, if it's a file.
 _testmarktest_init() {
@@ -176,7 +177,7 @@ _testmarktest_script() {
 	printf 'set -euo pipefail\n'
 	printf 'TESTCASE_DIR=%q\n' "$_testmarktest_dir"
 	if (($# == 2)); then
-		printf '%s\n' "$_testmarktest_testcase"
+		printf 'source %q\n' "$_testmarktest_testcase"
 		printf '_testcase_case=%q\n' "$2"
 		# Hunks arrive on fds 8 and 9 as raw bytes: $(<...) reads them in blocks and parses nothing, where script text is read a byte at a time.
 		printf '%s\n' '_testcase_index=$(</dev/fd/8)' '_testcase_blob=$(</dev/fd/9)' '_testcase_blob=${_testcase_blob%.}' 'exec 8<&- 9<&- 3>&1' '{'
diff --git a/lib/testmark/testcase.bash b/lib/testmark/testcase.bash
index 8d3288c..c1b8649 100644
--- a/lib/testmark/testcase.bash
+++ b/lib/testmark/testcase.bash
@@ -1,5 +1,5 @@
 ## testcase.bash: the functions a case function of testmarktest_rule calls, to read hunks and give its outputs.
-## Not sourced by testmark.bash: the runner sends this file's text into each case's own bash.
+## Not sourced by testmark.bash: each case of testmarktest_rule sources it, in the case's own bash, so these functions exist only there.
 ## They run under the case's `set -euo pipefail`, but may be called where errexit is ignored, so they check every status themselves.
 ## Their contract is documented with testmarktest_rule, in testmark.bash.
 # shellcheck disable=SC2154 # _testcase_case, _testcase_index, and _testcase_blob are set by the case's script