On a Thursday in October, an underwriter opens a recommendation that says approve for a building whose roof age is missing. That is wrong under rule R-ROOF-02, and she flags it. The team's first question is the obvious one: which prompt produced this?
Nobody can answer it with certainty. The prompt lives in the application's configuration, and three people edited it that month. One shortened the system message. One fixed a typo in the output format. One changed the temperature. The run log holds the model's answer and a timestamp. It does not hold the prompt. So the team cannot tell whether the wrong answer came from the shortened text, from the temperature, or from something else, and they cannot run the same prompt again to find out.
In the example, this story has a happier ending, because the prompt was versioned. Version 1.3.0 had removed the "never guess" sentence. The record of the run named the version and its hash. The on-call engineer rolled back to 1.2.0 in one step on 23 October, and the fix shipped as 1.4.0 six days later. This page explains the machinery that made that possible.
A prompt version, in plain words#
A prompt version is an immutable, numbered state of a prompt: its exact text, its variables and its parameters, stored once and never edited. Tiny example: uw-recommend version 1.3.0 is one file; a change to it becomes 1.4.0, a second file.
A prompt registry is the store that holds every version of a prompt and a pointer to the one production uses now. Tiny example: in the example, the folder prompts/uw-recommend/ with five version files and active.json.
A prompt hash is a fingerprint of a version's exact content: a short string that changes if any character changes. Tiny example: version 1.4.0 has the hash beginning c6bb1887fb1a. The example computes it with SHA-256 over the version's content keys.
Model parameters are the settings sent with the text that change how the model answers: which model, the temperature, the maximum length of the answer. Tiny example: {"model": "stand-in-1", "temperature": 0, "max_output_tokens": 600}. They belong to the version, because the same text with a different temperature is a different prompt in practice.
A changelog is the human explanation of each version: what changed and why. Tiny example: 1.4.0: "The never-guess sentence is back. One worked example added. Static text first, dynamic text last."
For a backend engineer. A prompt registry is a database migration folder crossed with an artifact repository. Like migrations, versions are numbered and never edited after they ship; a fix is a new migration. Like a Maven repository, a released version is immutable and identified by coordinates and a checksum. The comparison breaks on testing: a migration either applies or fails, and a compiler checks code before release. Prompt text has no compiler. Its only check is running it on cases, which the testing chapter covers.
The registry of the example#
Here is the registry as the example ships it. Each line is a version, its hash prefix, whether the stored hash still matches the content, the version's name and its changelog.
cd examples/uw-prompts
python3 registry.py
prompt uw-recommend · active 1.4.0
1.0.0 4c84e35b9710 hash ok one string: First version. Everything in one user message; the roof description is pasted in as it is; no output contract.
1.1.0 c75060857176 hash ok roles: Policy, process rule and output contract move to the system message. The submission stays in the user message.
1.2.0 ed19d6f64cbb hash ok data marked: Untrusted fields are escaped and wrapped in untrusted_data blocks; a rule says that text is information. New rule: never guess a missing value.
1.3.0 2dca80e656a9 hash ok trimmed: Shorter and cheaper: wording tightened. The never-guess sentence was removed with the other text.
1.4.0 c6bb1887fb1a hash ok restored and ordered: The never-guess sentence is back. One worked example added. Static text first, dynamic text last: the case line moved to the user message.
Five versions tell the story of this site. 1.0.0 pasted everything into one message. 1.1.0 introduced roles. 1.2.0 marked and escaped untrusted data and added the never-guess rule. 1.3.0 trimmed the text and lost that rule. 1.4.0 restored it and reordered the prompt so that static text comes first.
Next to the versions sits the pointer, active.json. It names the version production reads and keeps every move as an event:
| Date | Event | From | To | By role |
|---|---|---|---|---|
| 2026-09-02 | promote | none | 1.0.0 | release manager |
| 2026-09-16 | promote | 1.0.0 | 1.1.0 | release manager |
| 2026-10-02 | promote | 1.1.0 | 1.2.0 | release manager |
| 2026-10-21 | promote | 1.2.0 | 1.3.0 | release manager |
| 2026-10-23 | rollback | 1.3.0 | 1.2.0 | on-call engineer |
| 2026-10-29 | promote | 1.2.0 | 1.4.0 | release manager |
All dates and roles are synthetic. Two design choices are visible. Promoting and rolling back write only the pointer; no version file is ever touched. And the rollback was one step, by a role on call, without editing any text: return to the version that was active before.
Diagram: The prompt registry · immutable versions, a pointer that moves, a record that names both
Read left to right. An author stores a new version, which is hashed and never edited. A release role moves the pointer. Every run reads the active version and records its number and hash. Click a box to see it in the October story.
- 1 · Author
- A change: Any edit to text or parameters is stored as a new version with a changelog.
- 2 · The registry
- Version files: One file per version, written once. create_version refuses to overwrite.
- Content hash: SHA-256 of the content keys. load refuses a file whose content no longer matches.
- 3 · Release
- Active pointer: active.json names the version production reads and logs every move. No version file is touched.
- 4 · Every run
- Run record: Each run records prompt_version, prompt_hash, params and policy_version, kept apart.
Immutable, and checked#
Immutability is easy to promise and easy to break by accident, so the example enforces it in code, twice. The short script below, run in a scratch copy of the example, tries both ways of changing a stored version. It uses registry.create_version and registry.load.
import json, registry
from pathlib import Path
doc = registry.load("1.4.0")
print("1.4.0 hash:", doc["sha256"][:12], "| params:", doc["params"])
# 1. A change is never written over an existing version.
try:
registry.create_version({k: v for k, v in doc.items() if k != "sha256"})
except registry.VersionExists as err:
print("refused:", err)
# 2. Someone edits the stored file in place: one word in the never-guess sentence.
path = Path("prompts/uw-recommend/1.4.0.json")
original = path.read_text()
path.write_text(original.replace("Never guess a missing value.", "Avoid guessing a missing value."))
try:
registry.load("1.4.0")
except registry.HashMismatch as err:
print("refused:", err)
finally:
path.write_text(original)
1.4.0 hash: c6bb1887fb1a | params: {'model': 'stand-in-1', 'temperature': 0, 'max_output_tokens': 600}
refused: uw-recommend 1.4.0 exists. Versions are immutable: store the change as a new version.
refused: uw-recommend 1.4.0: stored c6bb1887fb1a, content gives d4f00af621a0
The first refusal is the registry's rule: a version is written once. The second is the safety net for everything outside the registry's control: a hand edit, a bad merge, a deploy script that rewrites files. One changed word gives a different hash, and load refuses to serve the file. A prompt that production cannot load is a loud failure. A prompt that silently changed under its old name is the Thursday story from the opening.
What the hash covers is a design decision. The example hashes the content keys (id, version, params, variables, untrusted, messages) and not the description (changelog, date, author_role). Fixing a typo in the changelog does not change the prompt. Changing the temperature does.
Common confusion. A version number and a hash answer different questions. The number is a name people use ("roll back to 1.2.0"), and it says something about the kind of change. The hash is the proof of identity: two runs with the same hash saw the same instructions, whatever anyone named them. Record both.
Provenance: what every run records#
A version helps only if each run says which version it used. Provenance here means the record of where an answer came from: the prompt, the parameters, the policy, the evidence, the model. The example's run record carries, among other fields:
{
"submission_id": "SUB-1003",
"run_id": "run-3182",
"attempt_number": 1,
"trace_id": "f9c03bea4a7d959733fff4943960ec66",
"policy_version": "2026.2",
"prompt_id": "uw-recommend",
"prompt_version": "1.4.0",
"prompt_hash": "c6bb1887fb1a79f439790ebb88604d0b3c64c95cc9ed78e623031195d854ebb9",
"model": "stand-in-1 (scripted stand-in)",
"params": {"model": "stand-in-1", "temperature": 0, "max_output_tokens": 600},
"evidence_ids": ["EV-ROOF-1003"],
"human_decision": null
}
These are the provenance fields of the record agent_sim.py, function run, wrote for UW-003 under 1.4.0 (a selection of its keys). The insurance lab shows a complete record.
Keep the identifiers apart, because each one answers a different question. prompt_version and prompt_hash say which instructions. policy_version says which rules: the policy fixture can change independently of the prompt, and an answer that was right under 2025.4 can be wrong under 2026.2 (case UW-007). run_id names one execution for one submission, and attempt_number names a retry inside it. evidence_ids name what the agent looked at. human_decision records what the person did.
With that record, the Thursday question has an answer in one query: every run with prompt_version 1.3.0 and a missing roof age.
Reproducibility, and its limits#
Reproducibility is the ability to run the same prompt on the same input again and get the same result. Tiny example: running UW-003 under 1.3.0 today gives the same wrong answer as on 21 October.
The example has it completely, because its model is a scripted stand-in: same version, same case, same output, every time. The model here is a scripted stand-in, so these results show how the mechanism works, not how good any model is. With a real model, versioning gives you something weaker and still valuable:
- Exact inputs. You can send exactly the prompt that ran: same text, same parameters, same evidence. That much is fully reproducible, and it is what you need to investigate.
- Not identical outputs. A real model can answer the same prompt differently on two calls. Anthropic's migration guide notes that temperature 0 never guaranteed identical outputs (see the portability chapter's facts). So "reproduce" means re-running the case several times and looking at the distribution, which is what evaluation does.
- Not the same model, unless you pinned it. A model name that is an alias can point to a newer snapshot next month. The version's
paramsshould name a pinned model version, and the portability chapter covers pinning.
Versioning cannot make a model deterministic. It makes the question "what changed?" answerable, which is most of what an incident needs.
Compare two versions in the lab below. Pick 1.2.0 and 1.3.0 to see the trim that broke UW-003, then 1.3.0 and 1.4.0 to see the fix.
| Version | Name | What changed | Cases passing (of 18) | Estimated tokens on UW-101 |
|---|---|---|---|---|
1.0.0 | one string | Everything in one user message; roof note raw; no output contract | 0 | 392 |
1.1.0 | roles | Policy, process and contract in the system message | 8 | 538 |
1.2.0 | data marked | Untrusted fields escaped and marked; never-guess rule added | 18 | 661 |
1.3.0 | trimmed | Shorter; the never-guess sentence removed | 11 | 530 |
1.4.0 | restored and ordered | Never-guess back; one worked example; static first | 18 | 778 |
Pass counts are at the model step, from the stand-in; token counts are estimates by tokens.py (illustrative).
What this is not. Versioning tells you which prompt ran. Deciding whether a new version is better, with graders and repeated trials, is evaluation: eval.sammuthu.com. Recording the run as logs, traces and metrics and keeping them searchable is observability: observe.sammuthu.com. Rolling a version out safely is this site's production chapter.
The same idea for a coding agent. A repository's instruction file is a prompt that lives in git, so it already has versions and hashes: the commit. What teams often miss is provenance on the other side. A pull request opened by a coding agent should say which instruction file commit, which agent configuration and which model produced it, the same way the insurer's run record namesprompt_versionandprompt_hash. Then "why did the agent do this?" has an answer. See diff-to-deploy.sammuthu.com for how that metadata travels with a change.
Deeper: what the version number should mean for a prompt
The example uses three-part numbers, and teams borrow semantic versioning's vocabulary. For prompts, a workable reading is:
| Part | Bump when | Example |
|---|---|---|
| Major | The output contract changes in a way consumers must handle | A new field the cockpit must display; a recommendation value removed |
| Minor | Behavior is meant to change, contract unchanged | 1.2.0 added marking and the never-guess rule; 1.4.0 added an example |
| Patch | Behavior is meant to stay the same | A typo, a reworded sentence |
The trap is the patch. With prompts, "meant to stay the same" is a hope, not a property: 1.3.0 was a cleanup that the author would have called a patch, and it broke seven cases. Treat every bump, patch included, as needing the full golden suite. The number tells people what you intended; the tests tell you what happened.
Deeper: what belongs in a version, and what does not
In a version: every message template, the variable list with trust levels, the model parameters including a pinned model id, the tool definitions the prompt relies on (their descriptions are prompt text; see tools), and the output schema.
Not in a version, but recorded per run: the policy version (it changes on its own schedule and has its own owner), the evidence, the case data, and the provider's actual model snapshot if the provider reports one.
Outside both: secrets and credentials. A registry is read by many people and copied into logs; it must never hold a key.
One question teams argue about is whether tool definitions belong in the prompt version or in a separate versioned contract. If tools are shared by several prompts, version them separately and record both versions per run. If they exist only for this prompt, keep them in the version so that one hash covers everything the model read.
Deeper: a JSON file in git, or a prompt management product?
The example's registry is a folder of JSON files with a hash check, which in a real repository means git plus a check in CI. That is enough for many teams: review happens in pull requests, history is git's, and deploys pin a commit.
Prompt management products add a UI for non-engineers, per-environment labels, and links between versions and traces. The frameworks chapter describes what several of them do, read on their own documentation today, and also records one provider deprecating its hosted prompt objects in favor of keeping prompts in application code. Whatever the store, the same properties make it a registry: versions are immutable, each has a content hash, the active pointer moves without editing a version, and every run records the version and hash it used.
Checklist: can you say which prompt produced this answer?#
| Question | Good answer |
|---|---|
| Is every change to prompt text stored as a new version? | Yes; the store refuses to overwrite |
| Does each version carry a content hash that is checked on load? | Yes; an edited file is refused |
| Do parameters, including a pinned model id, travel with the text? | Yes, inside the hashed content |
Does every run record prompt_version, prompt_hash and policy_version? | Yes, apart from each other |
| Is rollback one step that moves only the pointer? | Yes, by a role on call |
| Does each version have a changelog that says why? | Yes |
In one breath#
Treat a prompt like a release: every change becomes a new, immutable version with a content hash, the parameters travel inside it, and a pointer, not an edit, decides which version production uses. Each run records the prompt version and hash next to the policy version and the other identifiers, so "which text produced this answer?" has an answer, and rollback is one move of the pointer. Versioning makes inputs reproducible; it cannot make a real model's outputs identical, which is why changes are judged by tests.
A version you cannot evaluate is a version you cannot trust. 1.3.0 was stored, hashed and recorded perfectly, and it still shipped a regression. The next chapter is about testing prompts: the cases every version must pass before it moves the pointer.