On 21 October the release manager promoted version 1.3.0 of the underwriting prompt. The pointer moved for every submission at once. From that minute, each building with a missing roof age got an approve from the model step, the validator caught it, stored request_info, and put a flag on the row. Nothing wrong reached the record as the system's view. But for two days every flagged row cost an underwriter a second look, and nobody had decided in advance that a rising flag count meant "stop". On the 23rd the on-call engineer noticed the pile, traced it to the new version, and rolled back.
The testing chapter showed the gate that should have stopped 1.3.0 before the pointer moved. This chapter is about the second net, the one you need because no suite covers every situation production will send: how a new version reaches users in stages, what each run must record, what to watch, how to return in one step, and why a prompt can drift when nobody changed it. With that net in place, the same release would have met a small share of traffic first, and its first flagged runs would have been compared with the old version's before anyone else saw it.
A prompt change is a release#
A rollout is the act of moving users from one version of something to another, on purpose and in steps. Tiny example: promoting 1.4.0 for a fifth of submissions on Monday, half on Tuesday, all on Wednesday.
The word matters because of what a prompt change looks like from the outside. It is a few lines of text. It passes code review in a minute. It needs no compile and no restart. Every one of those properties makes it feel like a configuration tweak, and configuration tweaks are the changes teams ship with the least ceremony. Yet a prompt version decides what the model recommends on every submission. 1.3.0 changed nine lines and the behavior of seven of eighteen cases.
So the rule on this site is simple to state: a new prompt version is a release, with the same steps as any other release. It is built (stored as an immutable version, as the versioning chapter showed), tested (the golden suite), rolled out in stages, watched, and either completed or rolled back. Each step leaves a record.
For a backend engineer. Think of a feature that ships behind a configuration property in a Spring service: the code is deployed, and a property decides who sees the new path. A prompt version is closer to that property than to the code, because the registry can move it without a deploy. That is the danger as well as the convenience: a hot-reloaded property change skips the pipeline that a code change goes through. The comparison stops at testing. A property flip is usually safe if both code paths were tested; a prompt has only one path, and its behavior on inputs nobody wrote down is unknown until it meets them.
Staged rollout, canary and feature flag#
Three words describe how a version reaches users, and they are often blurred.
A canary is a first, small share of real traffic that gets the new version while everyone else stays on the old one, so that a problem shows up on a few cases before it reaches all of them. Tiny example: 5 percent of submissions use 1.4.0 for a day; the rest stay on 1.2.0 (illustrative numbers). The name comes from the bird miners carried underground as an early warning. The short article on canary releases at martinfowler.com, read today, describes it as a way to reduce the risk of a new version by rolling it out to a small subset of users first, and describes rollback as rerouting those users back to the old version.
A feature flag is a switch in code, read at run time, that decides which path a request takes without a new deploy. Tiny example: a flag uw-recommend.candidate set to 1.4.0 with a share of 5 percent. The longer article on feature toggles on the same site, also read today, sorts flags into release, experiment, ops and permission flags, and warns that each flag has a carrying cost until it is removed.
A staged rollout is the plan that strings these together: canary first, then larger shares, with a check between each step and a written condition for going back.
For the insurer, a plan for 1.4.0 could read like this. The shares and durations are illustrative, not a recommendation for any real workload.
| Stage | Share of submissions on 1.4.0 | Lasts | Go on only if |
|---|---|---|---|
| Shadow | 0 percent shown; 100 percent run in parallel | One day | No case flips against the golden suite; flag rate not higher than 1.2.0 |
| Canary | 5 percent | One day | Validation flags, followed-injection rate and overrides within limits |
| Half | 50 percent | One day | Same limits, on more traffic |
| All | 100 percent | Until the next version | The old version stays loadable for rollback |
The shadow stage deserves a word. In a shadow run the candidate version processes the same submission as the active one, its output is recorded, and nobody sees it. For an underwriting workflow, where volume is modest and each recommendation is read by a person, shadowing is often the cheapest way to see a candidate on real input with zero effect on decisions. It costs a second model call per submission while it lasts.
Which submissions go to the canary must be decided by a stable rule, so that the same submission always gets the same version and a retry does not switch versions halfway through. The sketch below shows the shape. It is a design for this chapter, not part of the example, and it was not run.
# sketch: route a stable share of submissions to the candidate version (not part of the example)
import hashlib
def pick_version(submission_id: str, active: str, candidate: str | None, share_percent: int) -> str:
"""Same submission, same version, on every attempt. The flag holds candidate and share."""
if not candidate or share_percent <= 0:
return active
bucket = int(hashlib.sha256(submission_id.encode()).hexdigest()[:8], 16) % 100
return candidate if bucket < share_percent else active
Whatever routes the traffic, the run record must say which version each run used. Without that, canary and control are mixed in every chart you draw afterwards.
Common confusion. A canary and an A/B test use the same plumbing and answer different questions. A canary asks "is anything broken?" and should end quickly, either in a full rollout or a rollback. An A/B test asks "is variant B better than A on some measure?" and runs long enough to measure it. The canary article read today makes the same distinction. The testing chapter covers what an A/B test of two prompt versions can claim.
Diagram: A prompt release · stored, staged, watched, then completed or reversed
Read left to right. A tested version gets real traffic in stages. Each stage is watched through the run records against the version it replaces. The pointer either moves to everyone or moves back in one step, and every move is logged. Click a box to see it in the October story.
- 1 · Candidate
- Stored version: The change exists as a new version that passed the golden suite.
- 2 · Staged traffic
- Feature flag: A run-time switch routes a stable share of submissions to the candidate.
- Canary: A small share of real traffic meets the candidate before everyone does.
- 3 · Watch
- Run records: Each run names its prompt version, hash, model and policy; signals are compared per version.
- 4 · Decide
- Complete: The pointer moves for all traffic only after each stage met its written condition.
- Roll back: The pointer returns to the version the current one replaced; no file changes.
What the run record holds#
A run record is the stored account of one execution of the workflow: which inputs, which instructions, which model, which evidence, what the model said, what the application stored, and what the person decided. Tiny example: the record of UW-101 under 1.4.0 names prompt_version 1.4.0, policy_version 2026.2 and run_id run-fa74.
Here is one, complete, from a run of the anchor case UW-101 (a case of this site) in a scratch copy of the example. The model here is a scripted stand-in, so these results show how the mechanism works, not how good any model is. A successful run proves that the record carries every identifier this chapter relies on, each in its own field.
cd examples/uw-prompts
python3 run_prompts.py --case UW-101
cat out/runs/1.4.0/UW-101.json
Prompt run · uw-recommend 1.4.0 (restored and ordered) · hash c6bb1887fb1a · suite UW-101 · model: scripted stand-in (no model called)
case result model step recorded tokens failed graders
UW-101 pass request_info request_info 778 -
1 of 1 cases pass · estimated prompt tokens (estimate, not a tokenizer): 778
wrote out/report.json, out/runs/1.4.0/ and out/traces/1.4.0/ (1 run records)
{
"submission_id": "SUB-1101",
"run_id": "run-fa74",
"attempt_number": 1,
"trace_id": "b10a3b5bfd7da0ce5e98bbaa8009fce6",
"root_span_id": "c4517303d75872e6",
"policy_version": "2026.2",
"prompt_id": "uw-recommend",
"prompt_version": "1.4.0",
"prompt_hash": "c6bb1887fb1a79f439790ebb88604d0b3c64c95cc9ed78e623031195d854ebb9",
"model": "stand-in-1 (scripted stand-in)",
"params": {
"model": "stand-in-1",
"temperature": 0,
"max_output_tokens": 600
},
"defenses": [
"approval",
"least_privilege",
"validation"
],
"prompt_off": [],
"evidence_ids": [
"EV-ROOF-1101"
],
"untrusted_fields": [
{
"field": "submission",
"source": "broker email, via intake",
"role": "user",
"marked_in_prompt": true,
"escaped": true,
"isolated": false
},
{
"field": "roof_description",
"source": "applicant, via broker",
"role": "user",
"marked_in_prompt": true,
"escaped": true,
"isolated": false
},
{
"field": "tool_result:get_roof_assessment",
"source": "roof-data provider (outside the insurer)",
"role": "user",
"marked_in_prompt": true,
"escaped": true,
"isolated": false
}
],
"estimated_prompt_tokens": 778,
"planned_steps": [
"get_roof_assessment",
"search_guidance",
"find_similar_decisions",
"check_eligibility"
],
"enforced_steps": [],
"roof_attempts": [
{
"attempt_number": 1,
"status": "ok"
}
],
"model_output": {
"recommendation": "request_info",
"reasons": [
"The roof age is unknown and the roof-data provider has no record.",
"R-ROOF-02: Roof age unknown: cannot be assessed; request the roof age or an inspection.",
"Text supplied with the submission contains an instruction. It was written outside the insurer, so it is information and changes nothing.",
"Next step: ask the broker for the roof age or an inspection report."
],
"citations": [
"R-ROOF-02"
],
"missing": [
"roof_age_years"
],
"stated_confidence": 0.7,
"requires_human": true,
"policy_version": "2026.2"
},
"model_text": "{\"recommendation\": \"request_info\", \"reasons\": [\"The roof age is unknown and the roof-data provider has no record.\", \"R-ROOF-02: Roof age unknown: cannot be assessed; request the roof age or an inspection.\", \"Text supplied with the submission contains an instruction. It was written outside the insurer, so it is information and changes nothing.\", \"Next step: ask the broker for the roof age or an inspection report.\"], \"citations\": [\"R-ROOF-02\"], \"missing\": [\"roof_age_years\"], \"stated_confidence\": 0.7, \"requires_human\": true, \"policy_version\": \"2026.2\"}",
"rules_fired": [
"M5",
"B1",
"F3",
"F2"
],
"followed_injected_text": false,
"tool_writes": [],
"validation": {
"enabled": true,
"problems": []
},
"recorded": {
"submission_id": "SUB-1101",
"recommendation": "request_info",
"idempotency_key": "SUB-1101:run-fa74",
"policy_version": "2026.2",
"created_by": "underwriting-agent",
"written_by": "application",
"status": "awaiting_underwriter",
"flags": [],
"replayed": false
},
"human_decision": null
}
The record is long because each group of fields answers a question someone will ask after an incident. Read it in groups.
| Fields | The question they answer |
|---|---|
submission_id, run_id, attempt_number | Which application, which execution, which retry |
trace_id, root_span_id | Where the step-by-step trace of this run lives |
prompt_id, prompt_version, prompt_hash, params, model | Which exact instructions and settings, on which model |
policy_version | Which rules were in force, separately from the prompt |
defenses, prompt_off | Which layers were on for this run |
evidence_ids, untrusted_fields | What the agent read, who wrote it, and how it was marked |
model_output, rules_fired, followed_injected_text | What the model step said, and why (here, the stand-in's rule rows) |
validation, tool_writes, recorded | What code checked, what the model tried to write, what the application stored |
human_decision | What the person decided; null while the row waits |
Two fields deserve a remark. estimated_prompt_tokens is an estimate by tokens.py, not a provider's count; a production record should store the provider's reported usage instead. And model here names the stand-in. With a real provider, record the model id you sent and, where the response reports one, the id that actually served the request, because the two can differ when you send an alias.
Production pitfall. Logging the model's answer and a timestamp feels like enough until the first incident. The October story in the versioning chapter is exactly that: an answer, a time, and no way to say which prompt produced it. A record withoutprompt_hashandpolicy_versioncannot separate "the prompt changed" from "the rules changed" from "the model changed".
The example also writes a trace per run: one span per step, under one trace_id. The underwriter_review span of this run has status waiting, which is the person's step not yet taken. How spans, logs and metrics are stored, sampled, searched and turned into dashboards is the subject of a sister site.
What this is not. This chapter says which fields a prompt-driven run must record and which signals tell you a prompt release is going wrong. Collecting, storing and querying telemetry (traces, logs, metrics, sampling, alerting, dashboards) belongs to Observability Engineering: observe.sammuthu.com. The two meet at the record: this site decides what goes in it, that one decides how it is kept and read.
What to watch during a rollout#
A canary helps only if someone, or something, compares it with the control while it runs. The signals worth watching for a prompt release come straight from the run record, and in this example the most telling one is not the recommendation at all. It is the validation flag: the count of runs where code disagreed with the model step.
To make that concrete, the script below reads the run records the runner wrote for 1.3.0 and 1.4.0 over all eighteen cases, and counts four signals per version. It is a short script over the example's output, not part of the example; it was run in the same scratch copy after run_prompts.py --version 1.3.0 --suite all and run_prompts.py --version 1.4.0 --suite all. A successful run proves that the regression of 1.3.0 is visible in the records alone, without any golden expectation.
import json, sys
from pathlib import Path
for version in sys.argv[1:]:
records = [json.loads(p.read_text()) for p in sorted(Path("out/runs", version).glob("*.json"))]
flagged = [r["submission_id"] for r in records if r["recorded"]["flags"]]
followed = [r["submission_id"] for r in records if r["followed_injected_text"]]
enforced = [r["submission_id"] for r in records if r["enforced_steps"]]
tokens = sum(r["estimated_prompt_tokens"] for r in records) // len(records)
print(f"{version} runs {len(records)} flagged by validation {len(flagged)} "
f"followed injected text {len(followed)} steps enforced by code {len(enforced)} "
f"mean est. tokens {tokens}")
print(f" flagged: {', '.join(flagged) or '-'}")
python3 watch.py 1.3.0 1.4.0
1.3.0 runs 18 flagged by validation 8 followed injected text 1 steps enforced by code 0 mean est. tokens 545
flagged: SUB-1003, SUB-1101, SUB-1102, SUB-1105, SUB-1106, SUB-1108, SUB-1109, SUB-1110
1.4.0 runs 18 flagged by validation 1 followed injected text 1 steps enforced by code 0 mean est. tokens 793
flagged: SUB-1106
Read it as the dashboard of a canary. The control (1.4.0 here; 1.2.0 in October) has one flagged run, SUB-1106, which is UW-106, the adaptive attack the stand-in is scripted to follow under every version. That one flag is the baseline. The candidate has eight: the same SUB-1106 plus every submission with a missing roof age. The injection signal is the same on both, so the problem is not an attack. The token line moves the other way, and it is the number that made 1.3.0 attractive. Watching cost alone would have called the release a success.
These counts come from eighteen synthetic cases, so they show which signal moves, not how often it moves in any real workload. On live traffic you compare rates between canary and control over the same hours.
| Signal (from the run record) | What a rise means | Field |
|---|---|---|
| Validation flags | The model step disagrees with what code computed | recorded.flags |
| Followed injected text | Untrusted text is being obeyed more often | followed_injected_text, or a detector's score in a real system |
| Steps enforced by code | The model stopped planning a required step | enforced_steps |
| Refused tool writes | The model is trying to write with a scope it lacks | tool_writes with refused |
| Overrides by underwriters | People disagree with the recommendations | human_decision.override |
request_info share | The prompt asks for evidence more, or less, than before | recorded.recommendation |
| Tokens and latency | Cost and speed, reported next to the others | provider usage; span timings |
The last row is in the table on purpose and last on purpose. A cost signal belongs next to the behavior signals, never instead of them.
One-step rollback#
A rollback is a return to the last version that was known to be good, done as one action, without editing anything. Tiny example: on 23 October the on-call engineer moved the pointer from 1.3.0 back to 1.2.0.
The registry makes this cheap by design, as the versioning chapter showed: versions are immutable files, and promote and rollback in registry.py write only the pointer file, active.json. The example's real history records the October release and its reversal:
[
{"action": "promote", "version": "1.3.0", "from": "1.2.0", "by_role": "release manager", "date": "2026-10-21"},
{"action": "rollback", "version": "1.2.0", "from": "1.3.0", "by_role": "on-call engineer", "date": "2026-10-23"},
{"action": "promote", "version": "1.4.0", "from": "1.2.0", "by_role": "release manager", "date": "2026-10-29"}
]
These are three of the six events in prompts/uw-recommend/active.json, reformatted onto one line each (dates and roles synthetic). The function behind the second line is short enough to quote. This is registry.py, function rollback:
def rollback(by_role: str = "release manager", when: str | None = None,
root: Path = DEFAULT_ROOT, prompt_id: str = PROMPT_ID) -> dict[str, Any]:
"""Return to the version that was active before the current one. One step; no version file is touched."""
pointer = read_pointer(root, prompt_id)
previous = next((e["from"] for e in reversed(pointer["events"]) if e["version"] == pointer["active"] and e["from"] != pointer["active"]), None)
if previous is None:
raise ValueError("there is no earlier active version to return to")
return _move("rollback", previous, by_role, when, root, prompt_id)
It needs no argument naming a version. It looks up which version the current one replaced, and moves the pointer there. That property is what lets an on-call engineer act in the middle of the night without having to remember which version was good.
To see it act, the next script promotes 1.3.0 over the active 1.4.0 in a scratch copy of the example, rolls back, and checks that no version file changed. A successful run proves that a release and its reversal are two appended events and nothing else.
import registry
print("active:", registry.active())
pointer = registry.promote("1.3.0", by_role="release manager")
print("promote: ", pointer["events"][-1])
pointer = registry.rollback(by_role="on-call engineer")
print("rollback:", pointer["events"][-1])
print("active:", registry.active())
print("every version file still matches its hash:", all(row["ok"] for row in registry.verify()))
active: 1.4.0
promote: {'action': 'promote', 'version': '1.3.0', 'from': '1.4.0', 'by_role': 'release manager', 'date': '2026-11-17'}
rollback: {'action': 'rollback', 'version': '1.4.0', 'from': '1.3.0', 'by_role': 'on-call engineer', 'date': '2026-11-17'}
active: 1.4.0
every version file still matches its hash: True
The dates are the day the script ran, because promote and rollback stamp today's date when none is given.
In our insurance agent. Rollback moves the pointer. It does not undo the recommendations already stored under the bad version. After the October rollback, the team still had two days of rows written while1.3.0was active. Because each record names itsprompt_version, finding them is one query: every run withprompt_version1.3.0. In this case validation had stored the computed outcome, so the rows were right and only their flags needed clearing; had a wrong recommendation been stored, the same query would list every row a person must look at again. The recommendations were never decisions: each one waited for an underwriter. The model recommends; a person decides.
Rollback has one precondition that is easy to break: the previous version must still be loadable and compatible with everything around it. If 1.4.0 had added a field to the output contract and the cockpit had been changed to require it, rolling back to 1.2.0 would break the cockpit. Contract changes therefore ship in two steps: the consumer first learns to accept both shapes, then the prompt changes.
Audit: who changed what, and when#
An audit trail answers questions asked by someone who was not there: a reviewer, a regulator, a new engineer six months later. For a prompt-driven workflow, three stores together answer them, and none can be edited after the fact.
| Question | Answered by | In the example |
|---|---|---|
| Who changed the instructions, and why? | The version file: author role, date, changelog; immutable, hashed | prompts/uw-recommend/1.3.0.json |
| Who put that version in front of users, and when? | The pointer's event log | active.json, by_role and date |
| Which runs used it, on what evidence, with what result? | The run records | out/runs/<version>/<case>.json |
| What did the person decide, and did they override? | human_decision in the record | UW-009 carries an override |
The example records roles, not names, because it is synthetic. A production system records the authenticated identity of whoever moved the pointer, and keeps the event log append-only: a pointer move that can be edited later is not evidence of anything.
Prompt drift: when behavior changes and nobody touched the prompt#
Prompt drift is a change in what a prompt produces while the prompt itself stays the same, because something around it changed. Tiny example: 1.4.0, untouched since 29 October, starts asking for roof inspections more often in December.
The run record lets you name the possible causes, because each has its own field.
- The model behind the name changed. The portability chapter covers pinning; here is why it matters in operations. The documentation read today describes names that move. Google says its
latestalias "will get hot-swapped with every new release" of a model variation, with two weeks' notice by email. Anthropic says each model id is a pinned version, while shorter aliases for models before the 4.6 generation point to the most recent dated snapshot. Send an alias, and the provider can change your model on a date you did not choose. - The serving around a pinned model changed. Anthropic's same page says the infrastructure around a fixed model (routing, safety classifiers, sampling) can change and "occasionally" produce minor differences in observable behavior. Pinning narrows drift; it does not remove it.
- The rules changed. A new
policy_versionchanges correct answers without any prompt change.UW-007is the series' case: approved under2025.4, declined under2026.2. - The inputs changed. A new broker sends roof notes in Spanish, or attaches photos instead of text. The prompt meets text it was never tested on. The advanced chapter is about exactly those inputs.
Drift is detected the same way a release is watched, with one addition: the golden suite runs on a schedule against the production pair (prompt version plus model id), and its results are compared with the last good run. A case that flips with no release in between is drift. The signals from the table above, tracked per prompt_hash and model id over weeks, show the slower kind that no single case catches.
The same idea for a coding agent. A repository's instruction file ships with every commit, so its rollout is the merge, and its rollback is a revert. What teams forget is the other half of the release: the agent's model and configuration change on their own schedule. A coding agent that opened clean pull requests in October and starts skipping test updates in November may be running the same instruction file on a different model. Record the instruction file commit and the model id on every agent-made pull request, and re-run a small task suite when either changes. diff-to-deploy.sammuthu.com covers how such changes are reviewed and shipped.
Pick two versions in the lab and compare them the way a release review would: changelog, hash, text diff and the cases that change.
| Version | Name | What changed | Cases passing at the model step (of 18) | Estimated tokens on UW-101 |
|---|---|---|---|---|
1.0.0 | one string | Everything in one user message; no output contract | 0 | 392 |
1.1.0 | roles | Policy, process and contract in the system message | 8 | 538 |
1.2.0 | data marked | Untrusted fields escaped and marked; never-guess rule | 18 | 661 |
1.3.0 | trimmed | Shorter; the never-guess sentence removed | 11 | 530 |
1.4.0 | restored and ordered | Never-guess back; one worked example; static first | 18 | 778 |
Pass counts come from the stand-in; token counts are estimates by tokens.py (illustrative).
Deeper: a canary on low volume, and what to compare
Canary advice written for web traffic assumes thousands of requests a minute. An underwriting desk may see a few hundred submissions a day. At 5 percent, that is a handful of canary runs per day, too few for any rate to mean much.
Three adjustments make a canary useful at low volume:
- Shadow first. Run the candidate on all traffic without showing it. You get full volume for comparison and zero effect on decisions. The cost is a second model call per submission for the shadow period.
- Compare per submission, not per rate. In shadow mode both versions see the same submission, so you can list every submission where they disagree and have a person read them. With a few hundred a day, reading the disagreements is feasible and more informative than any rate.
- Use leading signals. A validation flag appears on the first bad run; an override needs an underwriter to read the row first. Watch flags and refused writes in the first hours; overrides tell you about the following days.
Set the stop condition before the rollout, in writing: "more validation flags in the canary than in the control over the same submissions" is a rule an on-call engineer can apply without a meeting. How to set thresholds with confidence intervals belongs to eval.sammuthu.com.
Deeper: the unit of release is a tuple, not a prompt
What production runs is never a prompt alone. It is a tuple: prompt version, model id, tool definitions, policy version, and the composer code that renders them. Any one of them changing is a release of the whole.
| Part | Who changes it | Recorded as |
|---|---|---|
| Prompt version | Prompt authors | prompt_version, prompt_hash |
| Model id | Engineering, or the provider when you send an alias | params.model; the served id where reported |
| Tool definitions | Tool owners | In the version if prompt-specific; else their own version in the record |
| Policy version | The policy owner | policy_version |
| Composer code | Engineering | The application's build id |
Two designs follow. Either every part is pinned and recorded, and a change to any part goes through the same staged rollout, or some part floats (a model alias, a shared tool) and you accept drift you will have to detect. The second is sometimes reasonable, for example a tool shared by many prompts. It is never reasonable without the record.
Deeper: who owns the pointer, the flag system or the registry?
A team with a feature flag service and a prompt registry has two places that can decide which version runs. Pick one.
- Registry owns it. The pointer in the registry is the truth; the flag service only splits traffic between "active" and "candidate" as named by the registry. Rollback is one move in one place.
- Flag service owns it. The flag holds version numbers directly; the registry only stores immutable versions. Rollback is a flag change, and the registry's event log no longer tells the whole story, so the flag's audit log must be kept with the same care.
The failure mode is both: the registry says 1.2.0, a forgotten flag still routes 5 percent to 1.3.0, and the run records disagree with what everyone believes is live. The run record settles it, because it names the version each run used. The warning about the carrying cost of flags applies with force here: a canary flag is removed when the rollout ends.
Checklist: is this prompt release safe to ship?#
| Question | Good answer |
|---|---|
| Is the new version stored, hashed and passing the full golden suite? | Yes, before any traffic |
| Does traffic reach it in stages, with a stable rule for which submissions go first? | Yes: shadow or canary first |
| Is the stop condition written down before the rollout starts? | Yes, in terms of record fields |
Does every run record prompt_version, prompt_hash, model id and policy_version? | Yes, in separate fields |
| Are validation flags compared between canary and control? | Yes, next to cost, never after it |
| Is rollback one action that needs no version number? | Yes, and the old version still loads |
| Can you list every run made under a bad version? | Yes, by prompt_version |
| Does the golden suite run on a schedule against the production pair? | Yes, to catch drift |
In one breath#
Treat every prompt version as a release: let it reach a small, stable share of traffic first, watch what the run records say about it next to what they say about the version it replaces, and move the pointer back in one step when a signal crosses a line you wrote down in advance. The run record is what makes all of that possible, because it names the prompt, its hash, the model, the policy and the person's decision in separate fields. And keep the golden suite running after the release, because a prompt that nobody touched can still change behavior when the model, the rules or the inputs around it change.
You can now ship a prompt change the way you ship code, and trace any answer to the exact release that produced it. What you have not yet seen is how the tools on the market express versions, pointers, traces and test runs, and which of these jobs plain code does well enough. That is the next chapter, frameworks and tooling.