Going it alone: the primitives🔗

The runner (testmarktest_rule_basic and testmarktest_run) is built on a handful of primitives, and you can call them yourself. Reach for them when the runner doesn't fit: when you want to read testmark docs as plain data, or rewrite them, or check output in a shape the runner doesn't have.

They give you more control, but also more to get right. If the runner fits, use it.

The functions🔗

Every primitive takes a handle: a name you pick when you parse a doc, and use for that doc in every other call.

function	does
testmark_parse HANDLE FILE	parses FILE, and names it HANDLE for the other calls
testmark_names HANDLE	prints the hunk names, one per line, in document order
testmark_body HANDLE NAME	prints a hunk's body, exactly
testmark_info HANDLE NAME	prints a hunk's info string (text in ```text)
testmark_check HANDLE NAME <OUTPUT	compares OUTPUT to the body: returns 1 and shows a diff on a mismatch; or under TESTMARK_REGEN, stages OUTPUT as the new body
testmark_patch HANDLE NAME [INFO] <BODY	stages a new body, and optionally a new info string
testmark_write HANDLE	writes what's staged into the file, in place

Errors come back as a nonzero status, with a testmark: ... line on stderr. The doc comments in lib/testmark.bash are the full reference.

Reading and writing docs as data🔗

Read a hunk:

testmark_parse doc settings.md
greeting=$(testmark_body doc greeting)

Replace a hunk's body, then write the file:

testmark_patch doc greeting < <(echo "hello again")
testmark_write doc

Patches are staged until testmark_write, which writes them all at once, and then parses the file again.

A test loop by hand🔗

examples/test-todos-alone.sh does by hand what the runner does for examples/todos.md:

#!/bin/bash
set -euo pipefail
cd "$(dirname "${BASH_SOURCE[0]}")"
source ../lib/testmark.bash

testmark_parse doc todos.md
mapfile -t names < <(testmark_names doc)
fails=0
for name in "${names[@]}"; do
	case $name in
	*/input)
		case=${name%/input}
		if testmark_check doc "$case/output" < <(testmark_body doc "$name" | ./todos.sh /dev/stdin); then
			echo "PASS  $case"
		else
			echo "FAIL  $case"
			fails=$((fails + 1))
		fi
		;;
	esac
done
testmark_write doc
((fails == 0))

The case statement picks out the hunks this test cares about. It's the shape these loops tend to grow into: a doc often holds several kinds of hunks, and each kind gets its own branch.

Everything the runner did for you is now yours to get right:

• testmark_write at the end. That's what makes TESTMARK_REGEN=1 write anything. Without it, regen stages everything, then throws it away, without a word. With nothing staged, it does nothing, so call it unconditionally.
• Counting failures. Run every case, and exit nonzero if any failed. A bare check under set -e stops at the first mismatch, and you only see one.
• The exit status. A check doesn't see it. Put it in the output: < <(mytool || echo "exit: $?") adds a line only when the tool fails.
• Clean shells. The runner runs each case in a fresh bash; here, everything shares one shell, and one case's leftovers can reach the next.

Rules to know🔗

state changes happen in the shell that parsed🔗

testmark_check, testmark_patch, and testmark_write refuse to run in a subshell, where what they stage would be lost. So feed a check with testmark_check doc NAME < <(cmd), not cmd | testmark_check doc NAME; and don't call these inside $(...), or inside a test helper that runs its command there.

end output with a newline, and don't use CRLF🔗

Hunk bodies are whole lines, and the parser reads CRLF line endings as LF, so a body always ends in a newline (unless it's empty), and never holds a CRLF. Output that doesn't fit that could never match, and regen couldn't fix it either: the next parse would read back something different. So testmark_check refuses such output with an error, rather than letting the test fail on every run. Fix it in the command you're checking, with | tr -d '\r', or by printing the last newline.

the hunk must already exist🔗

Nothing adds hunks. testmark_check and testmark_patch need the hunk to be there, in regen too, and testmark_write only rewrites the bodies of hunks that are there. Start with an empty block, and let regen fill it.