specification of the file formats.
commit 61ee4d179271efca4bdd4754a428d39b2bc1ea14 · parent 2169820719
· browse this tree
warpfork, 2026-10-18 23:09:37 UTC
diff --git a/docs/hacking/FORMAT.md b/docs/hacking/FORMAT.md new file mode 100644 index 0000000..bae83d0 --- /dev/null +++ b/docs/hacking/FORMAT.md @@ -0,0 +1,194 @@ +the Patchette file formats +========================== + +This document is the single source of truth for patchette's on-disk format: +both the content of the files, and how they're arranged on the filesystem. + +This description aims to be simple and implementation-agnostic: we hope +a correct implementation of export and replay can be written from this document alone, in any language. + + +Goals +----- + +A quick recap on what we want from these files: + +- **Lossless roundtrip of git commits.** + - All git commit metadata must be persisted. Messages, authorship, dates, everything. + - All git commit relationships must be persisted. This includes merge commits. + - The patch files replayed on to the same base commit in a repo should result in + the same branch and same SHA of its tip as was stored. +- **Reviewable evolution.** + - The layout is plain text files designed to diff when the patches change, + and stay unaffected whenever possible. The diffs should be meaningful. + - To this end: export is deterministic, and must depend only on the commits + (and never on any git configuration, attributes, or other distracting chaff). +- **Editable by hand.** + - While patchette is meant to allow the users to work within repo when making changes, + we also absolute want to support the edit of patches directly, with ordinary tools, too. + Therefore the formats must be easy to read, modify, and must also be easy to heal. + + +Directory Layout +---------------- + +A *patches dir* contains one directory per tracked branch, +and a series of patch files and one control file in each one. +This can look roughly like the following: + +``` + <patches-dir>/ + topic¦fancy/ + STACK + fix-the-frobnicator.patch + fix-the-frobnicator-2.patch + merge-sprocket-work.patch + ... + docs-polish/ + STACK + woggle-the-borogrove.patch + ... +``` + +### branch directory names + +Directory names are derived from the branch name, with every `/` replaced by `¦` (U+00A6 BROKEN BAR). +(The directory name is a convenience; the `ref` line inside the `STACK` file is the more authoritative value.) + +### patch file names + +Patch file names are derived from commit subjects. + +A sluggifcation is used: the subject is transformed lowercase; +every run of non-alphanumerics becomes one `-`; +dashes trimmed at both ends; +the result is truncated to 40 chars (re-trimmed); +and may never be empty (fallback: `patch`). +Collisions within one series take `-2`, `-3`, ... +in series order. + +Patch file names may "change" if the "same" branch is exported +from a git repo after a rebase or other edits that change the commit subject line. +A reworded subject will likely show up as a file rename +(as git itself would describe it, when tracking the patch files, based on similarity heuristics -- probably; +depending on how much else changed during the rebase). +This may feel a little dissatisfying, but it's unclear what other behavior could be better without inventing concepts of identity that git just doesn't have. + + +The STACK file +-------------- + +The `STACK` file contains the metadata about a branch +(its name, the commit in the upstream it was based on, etc) +and the ordering of commits within it. + +It uses a simple line-oriented format which looks like the following: + +``` +ref refs/heads/topic/fancy +base 5b0130cfd2153bcdb4a6d7ec356e7d805018afc9 +tip cc261dc9f0b878a78e2b80aeb60de397205f61ad +series +fix-the-frobnicator +fix-the-frobnicator-2 +[...addntl commit names...] +``` + +Several pieces of `key value` format data start the file, +then a literal `series` line denotes the end of that section, +and then one patch file name is per line, until EOF. + +Of the key value entries: + +- `ref`: is the full ref name of the tracked branch. +- `base`: the commit SHA of the upstream base this stack was last wrangled against. + (If rebuilding a branch in a fresh clone of the upstream repo from the patch files, + this is the commit that will be checked out before beginning to replay the patches.) +- `tip`: the commit SHA that's expected when this stack is applied to the base commit. + This is saved when a branch is exported from a repo into patch files. + It's optional: when present, it's useful for verification that rebuilding a branch from patches worked correctly. + It may also be disregarded in some workflows: e.g., when hand-editing patch files, + there's no expectable value for tip until the branch has been built. + +In the `series` section: + +- each line is a patch name. These match `^[a-z0-9-]+$`, + and each names a `<slug>.patch` file in the same directory. + +The order of patch names is topological. +While parent information is also stored in the individual patch files, +and thus an overall toposort could be recomputed by reading all the patch files, +we insist that the patch list in the `STACK` file must already be a valid toposort; +tools may (and should) error if it is not. +Duplicate patch names; patch names that refer to no file; and `*.patch` files with no mention +in the series section are all errors. + + +The patch file +-------------- + +Each `*.patch` file looks like the following: + +``` +parents merge-sprocket-work +author Ada Downstream <ada@downstream.example> 1700000480 +0000 +committer Ada Downstream <ada@downstream.example> 1700000480 +0000 +change-id kpqxwvuzyntsmrlokqjihvgzfyexwdcu + +---- +> Teach gizmo to hum +> +> The hum is load-bearing. +> Do not ask why. +---- + +diff --git a/frobnicator.txt b/frobnicator.txt +[...typical diff format...] +``` + +Commit structure and metadata are at the top; +the commit message follows (with substantial demarcations, since it can be freetext); +the diff follows that, and continues to the end of file. + +In detail: + +- **Commit graph structure**: + - `parents <name>...` -- one or more references, space-separated. + - Each `<name>` is either `BASE` (the STACK's base commit) or the name of another patch in the STACK. +- **Commit metadata**: + - The commit's raw headers, verbatim, in the same order as git handles them... modulo the `tree` and `parent` (since we handle that data in other ways). + - Each line is `key value` separated by a space. + - Yes, everything git handles, verbatim. + - So this can include multi-line headers -- e.g. `gpgsig`, `mergetag` -- continuation lines for these begin with a space. + - And other headers unknown to Patchette (e.g. `jj`'s `change-id` rides through this way). + - This block ends at the first empty line. + (Git headers cannot contain an empty line, so this is unambiguous.) +- **Commit message**: + - Flanked by literal `----` lines. + - Each line of body between the `----` boundary markers carries a prefix. + (This ensures that commit messages containing a line of dashes cannot produce a fuss.) + - Two prefix styles are supported: {`> `+line} and {`\t`+line}. + - In both styles, a line with no content skips all whitespace (meaning a `>` is alone, with no trailing space, and a `\t` is skipped entirely). + (This helps the format be produce fewer warnings about trailing whitespace under typical conditions.) + - Note that trailing whitespace in content lines is permitted. + (If patch files are edited by users, their editors may mangle this; + however, there's little we can do about that.) + - One blank line is expected after the closing `----` flank. + (This is mostly just for visual consistency.) +- **Diff section**: + - Continues for the rest of the file until EOF. + - May be empty. (Empty commits are valid.) + - This is the commit's changes vs its *first* parent. + +### diff generation + +Diff generation is: + +``` +git diff-tree -r -p --full-index --binary --no-renames --diff-algorithm=patience <parent1> <commit> +``` + +...and must be run in an environment where no configuration or attributes can reach the diff machinery -- e.g., no global/system/repo-local config, and no gitattributes from any checkout. + +- `--no-renames`: because rename detection changes representation when content drifts, + which manufactures phantom diffs in the patches dir; delete+add is bigger but stable.