How a Rolling Edition book gets verified, in the open. This is the public, read-only standards repository: every book's checks, the writing standards and the reference encoder, published from Rolling Edition's platform. To suggest a change, open an issue.
| Path | What |
|---|---|
.github/workflows/book-checks.yml |
The reusable workflow every book pull request runs: examples, book manifest, text addressed to a model, the agent skill, concepts, section IDs. examples-scope says which examples run: all (the default) or affected |
.github/workflows/book-commands.yml |
The reusable workflow a terminal book's pull requests run, from its checks.yml's commands job: the stack's pins and licences, the local lab, and each chapter the change could affect, from a fresh start, on a k3s cluster of its own for a k3s stack. scope is all (the default) or affected; record, by hand, records the outputs as the run's artifact. It does nothing for a book with no steps |
.github/workflows/kind-label.yml |
The kind-label check, rerun whenever labels change. A pull request that changes only files a reader doesn't read on the page passes without a kind label: the book's narration files (narration/), its agent skill's procedures (skill.md), its concepts (concepts.toml) and its generated front page, licences, renovate.json and workflows. Most have no kind of change and ride with the next edition; a change to the concepts, once they're served, gets its kind from triage and cuts an edition (see Concepts) |
re_standards/rules.py, data/rules.toml |
When each newer rule of the checks took effect, so a new rule never stops a released book's pull requests (see Rules with dates) |
data/taxonomy.json |
Areas and skills, the catalog's source of truth, anchored to Wikipedia and Wikidata |
data/review-tiers.toml |
The three review tiers, the file classes triage reads, and the rules that flag a serious risk |
re_standards/tape.py, cover.py |
The reference encoder: title tape, edition code, fingerprint, history tape and cover SVG. Anyone can decode a tape or recompute a code with it |
data/cover-glyphs.json |
The letter outlines a cover file draws its text with, since an image can't load a web font. Generated from the fonts the site uses |
re_standards/catalog.py |
Validates the taxonomy and book.toml, and builds the catalog |
re_standards/voice.py, data/writing.toml [adjustable] |
An authored book's voice.toml, beside book.toml: the author's voice ([profile]: a summary, who speaks, a few traits), a [glossary], words and phrases to [avoid], and [overrides] of only the limits the writing standards mark adjustable, within their bounds: lengths, who speaks and the narrative limits that set the tone. Examples that run, versions and numbers, claims and their evidence, citations and accessibility stay as the standards say. re-check manifest checks it, and re-check style holds the book to it. A change to it alone rides with the next edition, as narration's does |
re_standards/concepts.py, data/concepts.toml |
A book's concepts.toml, beside book.toml: the terms it defines, each quoted from its own sentence, in a context, connected by a fixed set of relation types that follow SKOS, with a permanent address for each (see Concepts). re-check concepts checks it; a book without one passes |
re_standards/bookfmt.py, sections.py |
The book format: chapters, sections, examples and their frozen IDs, and sections.json, with three hashes for each chapter opening and section: sha256, of its source; rendered_sha256, of its text as read, with every binding filled in from that edition's recorded outputs and pins; and content_sha256, the same with each version binding left as written. A section changed when its text as read did, so a library update that moves a number an example prints changes it; when only a version number moved, its content didn't, and the change is a quiet one |
re_standards/steps.py, data/commands.toml |
A terminal book's command steps, checks and files (see Terminal books): shell files at the top of examples/, embedded as Python examples are, with `# |
re_standards/stack.py, data/commands.toml |
A terminal book's stack: book.toml's [lab], the versions the author writes in examples/stack.in.toml, and examples/stack.toml, generated from it by re-check stack --pin, every tool's file by URL and SHA-256, every chart's archive and every image by digest, with their licences (see Terminal books) |
re_standards/commands.py |
The one runner for a terminal book's steps and checks, the pull request's, the release's and the Lab's: each chapter from a fresh start, its steps pasted into a bash login shell in a terminal and read through its shell-integration markers, its checks run beside it (see Terminal books) |
data/renovate-stack.json |
What a terminal book's renovate.json adds to every book's: Renovate's managers for the versions in examples/stack.in.toml (k3s, kubectl, helm and k3d at their releases, charts from their repositories, images by tag) and the images its manifests name. A Python book's renovate.json is data/renovate.json alone |
re_standards/k3s.py |
A k3s stack's backend for the runner: k3s installed from the stack's pins, each file checked by its SHA-256, with fixed flags, no registry and a network fence, and each chapter on the clean cluster, restored from a snapshot (see Terminal books) |
re_standards/labscripts.py |
A terminal book's local lab, so anyone can work through it on their own machine: lab/start.sh, chapter.sh, check.sh and stop.sh, generated by re-check lab --write (see Terminal books) |
re_standards/examples.py, harness.py, runtime/Dockerfile |
Runs every example in its pinned runtime and compares its output. Every runner starts it as python -B -P -s in a clean environment with the hash seed and every numeric library's threads fixed ([run] in data/examples.toml), so a run writes no bytecode and prints the same on any machine; runtimes are precompiled. --seeds 0,1 runs each example under each hash seed and fails when any output differs; --fresh builds the venv in a new folder for that run alone |
re_standards/base_image.py, data/runtime-base.toml |
The runtime image's base, python:{version}-slim for each Python a book pins, by digest, and its Debian packages by version. re-check base-image --refresh rewrites them from Docker Hub and Debian's index |
re_standards/affected.py |
Which examples a change could affect, each with why, or every example and why (see Only the examples a change touches) |
templates/book/workflows/, data/readme.toml |
Each book's own check workflows, generated by re-check readme: its pull requests run the examples their change could affect, and once a week, at a time drawn from the book's name ([workflows] weekly_cron), every example runs again on main |
re_standards/resolve.py, data/examples.toml |
Before the runtime is built, reads the pins with a resolver (uv, pinned with this package) for that runtime's Python and platform. Pins that can't be installed together fail the examples check in plain words, naming the conflict, as their own outcome, unsatisfiable: in the --report JSON, with the resolver's message, and as an annotation on the check run. No example runs then, so it's never mistaken for an example's failure |
writing/ |
The writing standards every book follows: rules with IDs, reasons and checks, and annotated exemplars |
re_standards/addressed.py, data/addressed.toml |
Text written to an AI system rather than to readers: invisible characters anywhere in the book, and phrases such as "ignore previous instructions" in prose, HTML comments, link titles, image alt text and book.toml. Code and inline code are exempt from the phrases, so a chapter that quotes an attack puts it in inline code |
data/writing.toml |
Limits and word lists for re-check style, the lint half of the writing standards |
re_standards/spoken.py, data/narration.toml |
The spoken script narration reads, built by code from each chapter, and the shared rules it reads with: templates, common terms, symbols and numbers as words (see Narration) |
data/drafting.toml |
The model bar: the models books may be drafted with. Open models are for recorded outputs only (W-48) |
prompts/ |
The agents' prompts, published so anyone can read what an agent was told |
Run the checks on a book checkout with re-check:
pip install -e . # from this folder
re-check examples --book path/to/book # every example, in its pinned runtime (--backend venv locally); --report FILE writes
# the outcome (passed, failed, unsatisfiable or error) and each example's result
re-check examples --book path/to/book --backend venv --seeds 0,1 --fresh # a draft's gate: each example under two hash
# seeds, in a runtime built in a new folder
re-check base-image --refresh # the runtime image's base digests and package versions, rewritten
re-check examples --book path/to/book --affected --base-ref <base commit> # only the examples a change since the base
# could affect, each with why; --list prints them and runs nothing; --recorded latest
# adds those the latest edition's record says loaded a library whose pin moved
re-check examples --book path/to/book --jobs 2 --seconds report.json # side by side, packed by each one's time in an
# earlier --report; one with no time, or too slow to share within --timeout, alone
re-check manifest --book path/to/book
re-check stack --book path/to/book # a terminal book's [lab] and stack pins; --pin writes examples/stack.toml (needs the internet)
re-check lab --book path/to/book # a terminal book's local lab (lab/*.sh) is up to date; --write writes it
re-check commands --book path/to/book # a terminal book's steps and checks, chapter by chapter from a fresh start; --record
# writes their outputs once two runs of each chapter print the same; --chapter ID
re-check commands --book path/to/book --affected --base-ref <base commit> # only the chapters a change could affect,
# each with why; --recorded REPORT --floor-days 7 against an earlier run instead
re-check stack --book path/to/book --images # the stack's images, each by digest, for k3s to import (needs the internet)
re-check section-ids --book path/to/book --base-ref <base commit> --previous latest
re-check kind-label --labels kind:deps
re-check kind-label --event "$GITHUB_EVENT_PATH" --book . # a pull request's labels, and the files it changes
re-check style --book path/to/book # the lint rules of the writing standards; --limit chapter_words=1500,6000 sets one
# of a book's own limits, --skip W-7 leaves out a rule it doesn't keep; --base-ref
# <base commit> tells a dated rule whether the change makes the next content edition
re-check addressed --book path/to/book # text addressed to an AI system (hidden characters, injected instructions)
re-check concepts --book path/to/book # concepts.toml: each definition quoted from its section, defined once, relations of the
# fixed types; lists a term used before its definition with no link (--strict fails);
# --previous latest: every address the latest edition released still answers
re-check narration --book path/to/book # terms no rule reads, and examples, blocks and tables with no description (warns)
re-check kit --book path/to/book --write --chapter first-id --chapter second-id # the kit a new draft starts
# with: .gitignore, examples/lib (threads and seeds), recordings/recorded.py
# (record into scratch, then copy in), numbered chapter stubs and each
# chapter's own narration file; without --write, what the book lacks of it
re-check wikidata # needs the internet; --write fills Wikidata IDs
python -m re_standards.tape code SLUG EDITION COMMIT_SHAA new rule holds a released book from its next content edition, so it never stops the book's other pull requests on the day it arrives. A rule listed in data/rules.toml took effect on its since, and holds:
- a draft, from the start, as every rule does;
- a pull request that changes the book's text or examples (the
proseandcodefile classes ofdata/review-tiers.toml), since its merge makes the book's next content edition; - every pull request once the book's text or examples have changed after
since, read from the book's own history on the pull request's base: the newest first-parent commit that changed one of those files.
Anywhere else, what it finds is a warning, with when the rule takes hold, and the check passes. A rule that isn't listed holds every book; one that guards readers or their agents is never listed. re-check manifest, style, skill and addressed take --base-ref, the pull request's base, which the reusable workflow passes.
A released book's own workflows are generated, and change only through a pull request a maintainer approves, since a change under .github/ is flagged. So when their templates change, re-check readme warns that a released book's workflows are out of date, and fails only a draft's. The rest of the generated front page fails when it's out of date, and Rolling Edition's agent regenerates it on the pull request's branch.
A book may index the terms it defines in concepts.toml, beside book.toml (writing standards, section 16). A concept is one term with one meaning in one context, and its definition is the book's own sentence, quoted:
context = "search-and-recommendation" # the context of every concept that names none of its own
[[contexts]] # a narrower context of the book's own, only where two meanings meet
id = "graded-relevance"
broader = "search-evaluation" # a taxonomy area or skill, or another context declared here
label = "Graded relevance"
[[concepts]]
id = "ndcg" # with its context, its permanent address: never changed once released
term = "Normalized DCG"
aliases = ["nDCG", "nDCG@k"] # the forms readers search for
definition = "Normalized DCG (nDCG) divides a ranking's DCG by the DCG of the ideal ranking" # its section's words
formula = "nDCG@k = DCG@k / DCG@k of the ideal ranking"
section = "ndcg" # a section's ID, or a chapter's for its opening
example = "ndcg-by-hand" # an example that shows it, run on every release
applies_to = ["scikit-learn"] # what its meaning depends on: each edition gives the version it's pinned at
wikidata = "Q…" # its Wikidata item, where there is one
[concepts.relations]
computed-from = ["dcg"] # a concept here; context/ID for one here in another context
broader = ["ranking-metric"]
close-to = ["https://rollingedition.com/concepts/machine-learning/evaluation-metric"] # another context's, by address
implemented-by = [{ to = "scikit-learn", versions = ">=1.5" }]
[retired] # a released concept the book no longer defines, to the one that replaces it
"normalized-gain" = "ndcg" # an ID in the file's context, or context/ID, on either side- Contexts are namespaces. Ours are the taxonomy's areas and skills, and the narrower contexts books declare, each under one of them. An organization's file is the same format with a
[namespace]table naming its ownidandbase: its concepts are in contexts it declares, never in ours, and it points at ours by their addresses.concepts.loadreads such a file on its own, with no book. - Permanent addresses come from
data/concepts.toml([addresses]):https://rollingedition.com/concepts/{context}/{id}, and an edition's version of it,…/{id}@{edition}. Once an edition is released, each of its concepts' addresses is frozen, as a section's ID is: the concept stays, under the same context and ID, or[retired]sends its address to the concept that replaces it, and a retired address stays retired unless the concept comes back. No context may bevocabulary, the address of our own vocabulary. - Relation types are a small, fixed set, each with its plain meaning, its SKOS property and whether it may go round in a circle, in
data/concepts.toml([relations]): broader and narrower, part-of, measured-from, computed-from, prerequisite-of, contrasts-with, implemented-by (a package or tool, with an optional version range), same-as and close-to (another context's concept, orwikidata:Q…), and, for an organization's concepts only, maps-to and defined-by-organization.
re-check concepts fails a definition or formula that isn't in its section's text word for word (emphasis, links, line breaks and entities aside, as the renderer reads them; bindings as written), a section that doesn't exist, a concept defined twice, an ID or alias that names two concepts in a context, an unknown context, an applies_to the book doesn't pin, a relation of an unknown type or to a concept that doesn't exist, a circle, a pin outside an implemented-by range, a [retired] entry whose successor isn't a concept here, and, with --previous, an address the edition before released that's no longer a concept or retired. --previous latest reads the addresses the latest edition's machine copy lists on rollingedition.ai (only the addresses, never the definitions); a path to an edition's sections.json reads the concepts.json beside it. It lists, without failing, a chapter that uses a term before its defining section with no link to it: whole words only, a term with a capital after its first letter, such as BM25, only as written, never inside a longer concept's name, and never in the defining chapter's opening. --strict fails on those too. Whoever opens a change to concepts.toml needs no kind label. Once the concepts are served, on the book's Glossary page and to assistants, triage gives such a change the content kind and its merge cuts an edition, the book's first concepts.toml included; until then a change to it alone rides with the next edition, as a later skill.md's does.
At each release, concepts.served gives each concept in the shape it's served in: its addresses, the definition as read, the version each package it applies to is pinned at (examples/requirements.txt, or a terminal book's examples/stack.toml), the edition it was first defined in and last changed in, and its provenance: the book, the section and its permanent link, the edition, the commit, and when the check passed. concepts.carry takes each concept's first edition, and the edition its meaning last changed in, from the edition before's records: a concept whose words, relations and packages are the same keeps its changed edition. concepts.edition_record is what the edition stores, the records and its retired addresses, and concepts.addresses the part of it that's public, for the check on a pull request. Its fields map onto JSON-LD as schema.org DefinedTerm and SKOS Concept, each relation onto its type's SKOS property.
A book pull request runs only the examples its change could affect, against the pull request's base, and prints each with why. An example runs when the change touches:
- its own files,
examples/{id}.pyorexamples/{id}.out; - a local module it imports, directly or through another: any
.pyfile beside the examples, such asexamples/lib/, read withast, relative imports included; - a pin in
examples/requirements.txtwhose version moved, for a distribution it imports or one that a distribution it imports depends on, extras included. Import names and requirements come from the runtime's own metadata (top_level.txtandRECORD), which the harness reads inside the runtime (.re_harness.py --metadata), so the runtime is built only when a pin moved; - with
--recorded, a pin for a distribution the example loaded when it last ran, as the harness records it. The record only ever adds examples: it may come from an older run than the base, and it can't see what another program loads.
When it can't tell, every example runs, and it says why:
examples/.python-versionchanged;- a pin was added or removed, since a library can import another only once it's installed;
- anything under
examples/data/changed, since what each example reads isn't recorded; - any other file beside the examples changed, such as a SQL file, a module's data file or
requirements.in; - a distribution whose pin moved, or one that depends on it, has no import names in its metadata, or installs a
.pthfile, which runs at every start of Python; - the runtime's metadata can't be read, or the base commit isn't in the checkout.
An example whose imports can't all be read, through importlib.import_module or __import__ with a computed name, exec of a computed string, runpy, or another program it starts, runs on every change to a pin or a local module. One that imports a name no installed package's metadata names, and that isn't in the standard library, runs on every change to a pin.
What no change shows, such as a new build of the runtime's base image or a new version of these checks, is covered by the weekly run of every example: each book's checks.yml runs them all every week on main, and can be run by hand.
A terminal book's chapters are worked in a shell, step by step. A step is a shell file, examples/{id}.sh, embedded by a paragraph that is only a link to it, and its recorded output is examples/{id}.out, in the book's one ID namespace. The link's label says what it is:
---
chapter_id: first-deployment
setup: [create-shop-namespace] # earlier chapters' steps that bring a fresh start to this chapter's
---
## Deploy the web app {#deploy-web}
[File: web-deployment](../examples/manifests/web-deployment.yaml)
[Step: apply-web](../examples/apply-web.sh)
[Check: web-has-three-replicas](../examples/web-has-three-replicas.sh)A step's first lines may be directives, which Copy and Run leave out (data/commands.toml, [directives]):
| Directive | What it does |
|---|---|
wait |
Polled after the step until it exits 0; never compared. It may be repeated |
mask |
Named masks from [masks]: age, pod-suffix, ip, uid, time, duration, resource-version, and job, a background job's [1] PID |
compare |
exact (after masks, the default), lines (the expected lines, in order) or none |
tab, background |
Run in another tab, such as a port-forward; a background step runs until the chapter ends |
timeout |
The step's limit in seconds, waits included (default 120) |
check, hint |
Make the file a check, run out of band, Passed or Not yet; and say what to try when it doesn't pass |
expect-exit |
The exit code that counts as passing, when it isn't 0 |
A step never runs sleep: it waits for what it needs with wait. A file is something the steps read, such as a manifest, and its ID is its name without the extension.
The step format also holds the writing standards' rules for command chapters (section 15), line by line ([rules]): nothing that prints differently every run without its mask, such as $RANDOM, date or a job's process ID (W-70); nothing that downloads, and no address beyond the machine and the cluster (W-74); and nothing bash 3.2 can't run, since readers' Macs run the local lab with it (W-76). re-check stack refuses an image a manifest under examples/ names that the stack doesn't pin by its tag, one named by digest or as latest, and imagePullPolicy: Always, since a Lab's cluster has no registry.
book.toml says which Lab a book has and what runs beside its steps: [lab] kind = "terminal" (the default, notebook, is the Python Lab) and stack = "k3s", "compose" or "none" (the default). The author writes the versions in examples/stack.in.toml:
[tools]
k3s = "v1.31.4+k3s1"
kubectl = "v1.31.4"
k3d = "v5.7.5" # readers run the stack on their own machines with k3d
[charts.strimzi-kafka-operator]
repo = "https://strimzi.io/charts/"
version = "0.45.0"
licence = "Apache-2.0"
[images."quay.io/strimzi/kafka:0.45.0-kafka-3.9.0"]
licence = "Apache-2.0"re-check stack --pin generates examples/stack.toml from it, never typed: each tool's file for every platform it runs on, by URL and the SHA-256 its project publishes ([stack.tools]); each chart's archive by URL and SHA-256, from its repository's index; each image by digest, from its registry; and every licence. re-check stack checks that stack.toml is in step with stack.in.toml, that every pin is complete and that every licence ships. {{version:k3s}} binds from stack.toml as {{version:pandas}} does from requirements.txt.
Everything a Lab runs reaches readers' sessions, so it must be open source, and Apache-2.0 is preferred. Refused in every stack: the AGPL, the SSPL, the Elastic License, the Business Source License, the Redis Source Available License, Commons Clause and the Confluent Community License. A licence on neither list is refused until it's read and added ([licences]). The check names what to use instead: SeaweedFS for MinIO, OpenSearch for Elasticsearch, Valkey for Redis, OpenTofu for Terraform, OpenBao for Vault, and the rest in [licences] instead.
A book with no Python example needs no Python runtime: re-check examples passes with nothing to build.
A compose stack (stack = "compose") runs services beside the steps, such as a search engine, from examples/compose.yaml, which the author writes. re-check stack holds it to four rules: every service's image is named by a digest stack.toml pins (REF@sha256:…), nothing is built, no port is published beyond loopback (127.0.0.1:9200:9200), and nothing runs privileged, on the host's network, or mounting a path outside the book's examples folder. The fixture tests/fixtures/opensearch-sampler is the services pilot: one OpenSearch node, two chapters. Its pinned digest is a placeholder of zeros until re-check stack --pin reads the registry's, before its first run.
re-check commands runs each chapter alone, from a fresh start: a workspace holding the book's examples/ files (without the steps' own files and outputs) and, for a stack, its cluster; then the chapter's setup; then its steps in reading order. So chapters run in parallel, and a reader can start anywhere.
- A step runs as a reader runs it: pasted whole into a bash login shell in a terminal of 120 by 40, with
NO_COLOR, UTC andC.UTF-8, and nothing of the caller's environment. The shell's prompt marks where each command starts and ends, with its exit code (OSC 133), and its folder (OSC 7), so the runner reads exactly what a step printed, never the prompt or the echo. Terminal codes are taken out, and a carriage return writes over its line, as on screen. - A step passes when each of its commands exits 0 and the last with its
expect-exit. Then its waits are polled, and its output is compared withexamples/{id}.outas its directives say. Its verdict issame,different,ran(nothing compared),errorortimeout. An error or a timeout stops the chapter, since the steps after it build on it. - A check runs beside the terminal, never in it, in the main tab's folder, and is Passed or Not yet. Each check must also not pass before its section's steps run, so it checks something they do.
- Determinism:
--recordruns each chapter twice ([run] runs), each from a fresh start, and writes the outputs only when every compared step printed the same after its masks. - Backends make the fresh start, with the same two calls,
startandstop.localruns plain shell steps, a stack ofnone, in a scratch folder with the network blocked byunshare -rn, as the examples' venv backend is.k3s(re_standards/k3s.py) runs each chapter on a cluster of its own:- Installed from the pins: k3s for linux-amd64 and its airgap images, and the stack's kubectl and helm, each fetched once into a cache (
~/.cache/re-stack, orRE_STACK_CACHE) and checked by its SHA-256. The stack's images come fromre-check stack --images, which pulls each one by its digest and saves it under its tag. k3s imports them as it starts. - Fixed flags (
[k3s]): the node's name, the cluster's and services' addresses, traefik and servicelb off, no default registry endpoint, and every registry mirrored to a closed port. An image the stack doesn't pin fails at once with ImagePullBackOff, as it would in a reader's Lab. - Nothing reaches the network: the steps run as a user of their own, and the fence lets that user's processes, and the pods, reach only the machine and the cluster.
- The clean cluster: once k3s is Ready, its data dir is kept. Before each chapter, k3s stops, the data dir is restored and k3s starts again, so every chapter starts from the same cluster, its certificates and system pods included. Each chapter's peak memory, k3s and its pods together, is in the report.
composebrings a compose stack's services up from the workspace'scompose.yamlwithdocker compose up --wait --pull never, so an image the stack doesn't pin fails at once, and takes them down with their volumes after each chapter.
- Installed from the pins: k3s for linux-amd64 and its airgap images, and the stack's kubectl and helm, each fetched once into a cache (
- Which chapters run: with
--affected, a chapter runs when its steps, checks, setup or embedded files change (itsinputs_sha256insteps.json), when one of its recorded outputs changes, or when it's new. Every chapter runs when the stack's pins change (their digest), when the code that runs the steps changes, or when another file underexamples/changes, since any step may read it. Against an earlier run's record (--recorded), a chapter that didn't pass runs again, and with--floor-daysso does one whose last passing run is older than that.
Everything a terminal book teaches can be worked through free, on your own machine. re-check lab --write generates the book's local lab from book.toml, its steps and checks and examples/stack.toml, never typed. The scripts are POSIX sh, for Linux (x86-64 or Arm) and macOS on Apple silicon:
| Script | What it does |
|---|---|
lab/start.sh |
A workspace with the book's files, under ~/.rolling-edition/{slug} (or $RE_LAB_HOME), and for a k3s stack the cluster in Docker through k3d, every tool fetched for your platform and checked by its pinned SHA-256, and k3s's image by digest; for a compose stack its services in Docker, from compose.yaml. . ~/.rolling-edition/{slug}/env.sh then opens the workspace in your terminal |
lab/chapter.sh CHAPTER_ID |
A fresh start for that chapter: the workspace (and the cluster) made again, then the chapter's setup steps, each one's waits polled |
lab/check.sh ID ... |
Each check, or every check of a chapter, from where you are in the workspace: Passed, or Not yet with its hint |
lab/stop.sh |
The cluster deleted; your files stay |
You run each step yourself, copied from the page. A chapter's setup never names a step that runs in the background, since a chapter's start is what steps leave once they've ended.
Narration reads a book's text word for word, and never its code, output or tables. re_standards/spoken.py builds each chapter's spoken script by code, never by a model, from the parsed chapter with every binding filled in: one line per recorded paragraph, list item, heading or element, each with its kind and the section it belongs to.
- Headings, prose, quotes, list items and working rules are read as written, links by their text.
- Inline code is read through the lexicon, or by its parts:
lib.store.product_textis "lib dot store dot product text". - Numbers, years, decimals, signs, symbols and edition codes are read as words: 0.615 is "zero point six one five", (Chen et al., 2022) is "twenty twenty-two", 2026.10.6 is "twenty twenty-six point ten point six". A number a binding fills in is a count, never a year.
- Each example, fenced block and table is read as the description the book writes for it, or announced where it has none ("Example: inverted index by hand. Its code and output are on the page.").
- A chapter opens with its number and title, the book's title and the disclosure, and ends with its number. It never names the edition, so a chapter whose words didn't change keeps its recording from edition to edition. The whole-book file opens and closes with credits, which name the edition.
Every word is the edition's text or a template in data/narration.toml, whose rules_version a recorded paragraph's key includes, so a change to how the rules read means recording again.
A book's narration lives in narration/, beside its chapters, so adding or editing a description never changes a section's hash:
| File | What |
|---|---|
narration/lexicon.toml |
The book's own readings: [terms], exact and case-sensitive, such as "nDCG@10" = "N D C G at ten"; [words], whole words in any case, such as the words of an example's ID. They add to the shared terms, and win where both have one |
narration/{chapter_id}.toml |
The chapter's descriptions: [examples.{example_id}], and [[blocks]] and [[tables]], each with section, the section's ID, or the chapter's for its opening text, and starts, the start of its first line as written. blocks covers every fenced block that isn't an example, formulas included. Each has say, which may use {{out:…}} and {{version:…}} |
[examples.ndcg-for-one-query]
say = "The code takes BM25's top ten for clear shower curtain, adds up each grade divided by a log discount, does the same for the best possible top ten, and prints their ratio: {{out:ndcg-for-one-query:ndcg@10}}."
[[blocks]]
section = "score-one-ranking"
starts = "DCG@10 = sum over positions"
say = "D C G at ten is the sum, over positions one to ten, of each grade divided by log base two of the position plus one."
[[tables]]
section = "check-stemming-on-test"
starts = "| Measure |"
say = "A table of the mean difference, interval and p-value of each change this chapter tested."re-check narration lists, per chapter, every term no rule reads (a word that still holds a digit or a character such as _, / or <, or is written in capitals or camelCase, and inline code named whole), every example, block and table with no description, every description whose example, section or first line no longer matches, and bindings that don't resolve. A chapter's own readings can sit in its narration/{chapter_id}.toml, as [terms] and [words], so writers working on chapters in parallel never edit one shared file; they join the book's lexicon, narration/lexicon.toml wins, and two chapters that read a term differently are listed. It warns, and exits 0 unless --strict. --script prints the spoken script instead, for a chapter with --chapter or the whole book, with --edition for the whole book's credits.
Code: MIT. The writing standards in writing/ and the agents' prompts in prompts/: CC BY-NC 4.0, with our added permission: Reuse our books. re-check licences writes both files from data/readme.toml's [licences], which names every licence the books' generated files and the platform use.
data/mark-outlines.json and data/cover-glyphs.json hold letter outlines from two typefaces, each under the SIL Open Font License 1.1: Literata (licence), and IBM Plex Mono (licence), whose reserved name the outlines don't take. The other texts in templates/licenses/, and the Contributor Covenant in templates/community/, belong to their publishers: templates/fetch.py copies them verbatim.