testmark-bash

examples: todos.sh, its fixtures in todos.md, and a table-driven test that make test runs. And a sizable increase in the readme.

commit 9a69562881b0ad3e6e462ee2cae03c54b7f0958c · parent c972b45c76 · browse this tree
Ximianth, 2026-10-24 01:14:28 UTC

The test shows the patterns users should copy: one case per hunk name pair,
normalizing the temp dir out of the output before an exact check,
feeding the check with < <(...) so it runs in this shell,
and testmark_write at the end so TESTMARK_REGEN=1 can update the fixtures.

diff --git a/Makefile b/Makefile
index a8af5cd..72d5216 100644
--- a/Makefile
+++ b/Makefile
@@ -24,8 +24,9 @@ fmt: ## Format source in place.
 .PHONY: check
 check: ## Verify-only and fast; run by the pre-commit hook on every commit.
 	shfmt -d .
-	shellcheck -x -S warning lib/*.bash lib/*/*.bash tests/*.sh
+	shellcheck -x -S warning lib/*.bash lib/*/*.bash tests/*.sh examples/*.sh
 
 .PHONY: test
 test: ## Run all tests.
 	tests/run.sh
+	examples/test-todos.sh
diff --git a/README.md b/README.md
index ced3572..0cb2538 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,147 @@
-PLACEHOLDER
-=========
+testmark-bash
+=============
+
+testmark-bash reads, checks, and patches [testmark](https://testmark.dev/) documents, from bash.
+
+Testmark is data in markdown: a fenced code block, named by a comment line just above it.
+That makes a markdown file a good home for test fixtures:
+the prose around them explains them, they render nicely anywhere markdown does, and a program can rewrite them.
+
+So with this library, a bash test can check a tool's output against a block in a markdown file;
+and when the output changes on purpose, rewrite the block, and leave you a `git diff` to review.
+
+```bash
+source path/to/lib/testmark.bash
+
+testmark_parse doc fixtures.md
+testmark_check doc greeting/output < <(mytool greet world)
+testmark_write doc
+```
+
+It needs bash 4.4 or later.
+
+
+The workflow
+------------
+
+Write the input, and an empty block for the output to go in:
+
+```
+	[testmark]:# (basic/input)
+	```text
+	# TODO: handle spaces in names
+	```
+
+	[testmark]:# (basic/output)
+	```text
+	```
+```
+
+Run the test.  It fails, and shows a diff of what your tool printed against the empty block.
+
+Run it again with `TESTMARK_REGEN=1` set.  The output gets written into the block.
+
+Read it with `git diff`.  If it's right, commit it.
+
+Later, when the tool's output changes on purpose, it's the same: regen, read the diff, commit.
+The diff is the review.
+
+
+A whole example
+---------------
+
+[`examples/`](examples/) has a small tool, `todos.sh`, a fixtures doc for it, `todos.md`, and this test, which `make test` runs:
+
+```bash
+#!/bin/bash
+## Checks todos.sh against every case in todos.md.
+## Run with TESTMARK_REGEN=1 to update todos.md to match what todos.sh does now; then review the change with `git diff`.
+set -euo pipefail
+cd "$(dirname "$0")"
+source ../lib/testmark.bash
+
+work=$(mktemp -d)
+trap 'rm -rf "$work"' EXIT
+
+testmark_parse doc todos.md
+mapfile -t names < <(testmark_names doc)
+fails=0
+for name in "${names[@]}"; do
+	[[ $name == */input ]] || continue
+	case=${name%/input}
+	testmark_body doc "$case/input" >"$work/notes.txt"
+	if testmark_check doc "$case/output" < <(./todos.sh "$work/notes.txt" | sed "s|$work|WORK|g"); then
+		echo "PASS  $case"
+	else
+		echo "FAIL  $case"
+		fails=$((fails + 1))
+	fi
+done
+testmark_write doc
+((fails == 0))
+```
+
+What to copy from it:
+
+- **One case per name.**  Hunks named `CASE/input` and `CASE/output` make a table;
+  adding a case is adding two blocks to the doc, with no change to the test.
+- **Normalize, then check exactly.**  Anything that changes from run to run, like a temp dir, gets replaced before the check (here, with `sed`),
+  so the check can stay exact.
+  Prefer this over grepping for a fragment of the output:
+  an exact check can't pass by accident, it shows the reader what the output really looks like, and regen can write it.
+- **`< <(...)` into the check.**  Not a pipe; see the rules below.
+- **`testmark_write` at the end.**  That's what makes `TESTMARK_REGEN=1` write anything.
+  With nothing to write, it does nothing, so call it unconditionally.
+
+A check doesn't see the tool's exit status.
+When it matters, put it in the output: `< <(mytool || echo "exit: $?")` adds a line only when the tool fails.
+
+
+Rules to know
+-------------
+
+**State changes happen in the shell that parsed.**
+`testmark_check`, `testmark_patch`, and `testmark_write` refuse to run in a subshell, where what they stage would be lost.
+So feed a check with `testmark_check doc NAME < <(cmd)`, not `cmd | testmark_check doc NAME`;
+and don't call these inside `$(...)`, or inside a test helper that runs its command there.
+
+**Output must be empty, or end in a newline, with no CRLF.**
+A hunk's body can't hold anything else, so `testmark_check` rejects it outright, rather than failing forever.
+Normalize it first.
+
+**A check never creates a hunk.**
+Regen rewrites the body of a hunk that's there; a missing one is an error in both modes.
+That's why the workflow starts with an empty block.
+
+
+Functions
+---------
+
+| function | does |
+| --- | --- |
+| `testmark_parse HANDLE FILE` | parses FILE, and names it HANDLE for the other calls |
+| `testmark_names HANDLE` | prints the hunk names, one per line, in document order |
+| `testmark_body HANDLE NAME` | prints a hunk's body, exactly |
+| `testmark_info HANDLE NAME` | prints a hunk's info string (`text` in ```` ```text ````) |
+| `testmark_check HANDLE NAME <OUTPUT` | compares OUTPUT to the body: returns 1 and shows a diff on a mismatch; or under `TESTMARK_REGEN`, stages OUTPUT as the new body |
+| `testmark_patch HANDLE NAME [INFO] <BODY` | stages a new body, and optionally a new info string |
+| `testmark_write HANDLE` | writes what's staged into the file, in place |
+
+Errors come back as a nonzero status, with a `testmark: ...` line on stderr.
+The doc comments in [`lib/testmark.bash`](lib/testmark.bash) are the full reference.
+
+
+Requirements
+------------
+
+Bash 4.4 or later.
+Older bashes lack things the library is built on: `local -`, for one, which lets it set shell options for itself and put yours back, arrived in 4.4.
+[docs/hacking/bash-versions.md](docs/hacking/bash-versions.md) has more.
+On an older bash, sourcing prints `testmark: needs bash 4.4 or later; this is 3.2.57(1)-release` and returns 1.
+(That's macOS's `/bin/bash`.  Use a newer bash from a package manager.)
+
+Nothing else, except `diff`, to show a mismatch.
+The library doesn't depend on your shell options or locale; it sets its own on every call, and puts yours back.
 
 
 Contributing
@@ -10,6 +152,8 @@ Run `make` to see all commands.
 Before committing anything, run `make all`.
 There should be no warning messages.
 
+How the library works, and the bash pitfalls it steers around, are in [docs/hacking/](docs/hacking/).
+
 
 License
 -------
diff --git a/examples/test-todos.sh b/examples/test-todos.sh
new file mode 100755
index 0000000..f8f19b9
--- /dev/null
+++ b/examples/test-todos.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+## Checks todos.sh against every case in todos.md.
+## Run with TESTMARK_REGEN=1 to update todos.md to match what todos.sh does now; then review the change with `git diff`.
+set -euo pipefail
+cd "$(dirname "$0")"
+source ../lib/testmark.bash
+
+work=$(mktemp -d)
+trap 'rm -rf "$work"' EXIT
+
+testmark_parse doc todos.md
+mapfile -t names < <(testmark_names doc)
+fails=0
+for name in "${names[@]}"; do
+	[[ $name == */input ]] || continue
+	case=${name%/input}
+	testmark_body doc "$case/input" >"$work/notes.txt"
+	if testmark_check doc "$case/output" < <(./todos.sh "$work/notes.txt" | sed "s|$work|WORK|g"); then
+		echo "PASS  $case"
+	else
+		echo "FAIL  $case"
+		fails=$((fails + 1))
+	fi
+done
+testmark_write doc
+((fails == 0))
diff --git a/examples/todos.md b/examples/todos.md
new file mode 100644
index 0000000..2d04c3c
--- /dev/null
+++ b/examples/todos.md
@@ -0,0 +1,59 @@
+todos.sh, by example
+====================
+
+`todos.sh` lists the lines of files that mention TODO, as `FILE:LINE:TEXT`.
+
+Each case below is a pair of hunks:
+`CASE/input` is a file's content, and `CASE/output` is what `todos.sh` prints for that file.
+`test-todos.sh` runs every case, with the file in a temporary directory whose path it replaces with `WORK`.
+
+
+### the basics
+
+[testmark]:# (basic/input)
+```bash
+#!/bin/bash
+# TODO: handle spaces in names
+echo "$1"
+# TODO: and newlines
+```
+
+Lines are numbered from one.
+
+[testmark]:# (basic/output)
+```text
+WORK/notes.txt:2:# TODO: handle spaces in names
+WORK/notes.txt:4:# TODO: and newlines
+```
+
+
+### lines come out as they are
+
+Leading whitespace and backslashes are kept.
+
+[testmark]:# (verbatim/input)
+```text
+notes:
+	TODO: indented with a tab
+  \TODO: with a backslash
+```
+
+[testmark]:# (verbatim/output)
+```text
+WORK/notes.txt:2:	TODO: indented with a tab
+WORK/notes.txt:3:  \TODO: with a backslash
+```
+
+
+### nothing to do
+
+No TODOs, no output: the output hunk is empty.
+
+[testmark]:# (none/input)
+```text
+all done here
+```
+
+[testmark]:# (none/output)
+```text
+```
diff --git a/examples/todos.sh b/examples/todos.sh
new file mode 100755
index 0000000..ac0a8eb
--- /dev/null
+++ b/examples/todos.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+## todos.sh lists the lines of files that mention TODO, as "FILE:LINE:TEXT".
+set -euo pipefail
+for file in "$@"; do
+	n=0
+	while IFS= read -r line; do
+		n=$((n + 1))
+		if [[ $line == *TODO* ]]; then
+			printf '%s:%d:%s\n' "$file" "$n" "$line"
+		fi
+	done <"$file"
+done