the Patchette file formats🔗

This document is the single source of truth for patchette's on-disk format: both the content of the files, and how they're arranged on the filesystem.

This description aims to be simple and implementation-agnostic: we hope a correct implementation of export and replay can be written from this document alone, in any language.

Goals🔗

A quick recap on what we want from these files:

• Lossless roundtrip of git commits.
  • All git commit metadata must be persisted. Messages, authorship, dates, everything.
  • All git commit relationships must be persisted. This includes merge commits.
  • The patch files replayed on to the same base commit in a repo should result in the same branch and same SHA of its tip as was stored.
• Reviewable evolution.
  • The layout is plain text files designed to diff when the patches change, and stay unaffected whenever possible. The diffs should be meaningful.
  • To this end: export is deterministic, and must depend only on the commits (and never on any git configuration, attributes, or other distracting chaff).
• Editable by hand.
  • While patchette is meant to allow the users to work within repo when making changes, we also absolute want to support the edit of patches directly, with ordinary tools, too. Therefore the formats must be easy to read, modify, and must also be easy to heal.

Directory Layout🔗

A patches dir contains one directory per tracked branch, and a series of patch files and one control file in each one. This can look roughly like the following:

  <patches-dir>/
    topic¦fancy/
      STACK
      fix-the-frobnicator.patch
      fix-the-frobnicator-2.patch
      merge-sprocket-work.patch
      ...
    docs-polish/
      STACK
      woggle-the-borogrove.patch
      ...

branch directory names🔗

Directory names are derived from the branch name, with every / replaced by ¦ (U+00A6 BROKEN BAR). (The directory name is a convenience; the ref line inside the STACK file is the more authoritative value.)

patch file names🔗

Patch file names are derived from commit subjects.

A sluggifcation is used: the subject is transformed lowercase; every run of non-alphanumerics becomes one -; dashes trimmed at both ends; the result is truncated to 40 chars (re-trimmed); and may never be empty (fallback: patch). Collisions within one series take -2, -3, ... in series order.

Patch file names may "change" if the "same" branch is exported from a git repo after a rebase or other edits that change the commit subject line. A reworded subject will likely show up as a file rename (as git itself would describe it, when tracking the patch files, based on similarity heuristics -- probably; depending on how much else changed during the rebase). This may feel a little dissatisfying, but it's unclear what other behavior could be better without inventing concepts of identity that git just doesn't have.

The STACK file🔗

The STACK file contains the metadata about a branch (its name, the commit in the upstream it was based on, etc) and the ordering of commits within it.

It uses a simple line-oriented format which looks like the following:

ref refs/heads/topic/fancy
base 5b0130cfd2153bcdb4a6d7ec356e7d805018afc9
tip cc261dc9f0b878a78e2b80aeb60de397205f61ad
series
fix-the-frobnicator
fix-the-frobnicator-2
[...addntl commit names...]

Several pieces of key value format data start the file, then a literal series line denotes the end of that section, and then one patch file name is per line, until EOF. Each key appears at most once.

Of the key value entries:

• ref: is the full ref name of the tracked branch.
• base: the commit SHA of the upstream base this stack was last wrangled against. (If rebuilding a branch in a fresh clone of the upstream repo from the patch files, this is the commit that will be checked out before beginning to replay the patches.)
• tip: the commit SHA that's expected when this stack is applied to the base commit. This is saved when a branch is exported from a repo into patch files. It's optional: when present, it's useful for verification that rebuilding a branch from patches worked correctly. It may also be disregarded in some workflows: e.g., when hand-editing patch files, there's no expectable value for tip until the branch has been built.

The base and tip values are full commit IDs in lowercase hex: 40 characters in a SHA-1 repo, 64 in a SHA-256 one. Abbreviations and ref names are errors (they can resolve differently from repo to repo, or over time).

In the series section:

• each line is a patch name. These match ^[a-z0-9-]+$, and each names a <slug>.patch file in the same directory.

The order of patch names is topological. While parent information is also stored in the individual patch files, and thus an overall toposort could be recomputed by reading all the patch files, we insist that the patch list in the STACK file must already be a valid toposort; tools may (and should) error if it is not. Duplicate patch names; patch names that refer to no file; and *.patch files with no mention in the series section are all errors.

The patch file🔗

Each *.patch file looks like the following:

parents merge-sprocket-work
author Ada Downstream <ada@downstream.example> 1700000480 +0000
committer Ada Downstream <ada@downstream.example> 1700000480 +0000
change-id kpqxwvuzyntsmrlokqjihvgzfyexwdcu

----
> Teach gizmo to hum
>
> The hum is load-bearing.
> Do not ask why.
----

diff --git a/frobnicator.txt b/frobnicator.txt
[...typical diff format...]

Commit structure and metadata are at the top; the commit message follows (with substantial demarcations, since it can be freetext); the diff follows that, and continues to the end of file.

In detail:

• Commit graph structure:
  • parents <name>... -- one or more references, space-separated.
  • Each <name> is either BASE (the STACK's base commit) or the name of another patch in the STACK.
• Commit metadata:
  • The commit's raw headers, verbatim, in the same order as git handles them... modulo the tree and parent (since we handle that data in other ways).
  • Each line is key value separated by a space.
  • Yes, everything git handles, verbatim.
    • So this can include multi-line headers -- e.g. gpgsig, mergetag -- continuation lines for these begin with a space.
    • And other headers unknown to Patchette (e.g. jj's change-id rides through this way).
  • This block ends at the first empty line. (Git headers cannot contain an empty line, so this is unambiguous.)
• Commit message:
  • Flanked by literal ---- lines.
  • Directive lines may appear directly after the opening ----, before any message content. These are bare lines (no prefix); any unrecognized line in this position is an error. Currently defined:
    • message-no-final-newline -- declares that the message does not end with a newline character. (Possible at the git object level, though porcelain tools essentially never produce it. But lossless means lossless.)
  • Each line of body between the ---- boundary markers carries a prefix. (This ensures that commit messages containing a line of dashes cannot produce a fuss.)
    • Two prefix styles are supported: {> +line} and {\t+line}.
    • In both styles, a line with no content skips all whitespace (meaning a > is alone, with no trailing space, and a \t is skipped entirely). (This helps the format be produce fewer warnings about trailing whitespace under typical conditions.)
  • Note that trailing whitespace in content lines is permitted. (If patch files are edited by users, their editors may mangle this; however, there's little we can do about that.)
  • One blank line is expected after the closing ---- flank. (This is mostly just for visual consistency.)
• Diff section:
  • Continues for the rest of the file until EOF.
    • May be empty. (Empty commits are valid.)
  • This is the commit's changes vs its first parent.

diff generation🔗

Diff generation is:

git diff-tree -r -p --full-index --binary --no-renames --diff-algorithm=patience <parent1> <commit>

...and must be run in an environment where no configuration or attributes can reach the diff machinery -- e.g., no global/system/repo-local config, and no gitattributes from any checkout.

• --no-renames: because rename detection changes representation when content drifts, which manufactures phantom diffs in the patches dir; delete+add is bigger but stable.