Prompt version control: how to version, review, test and roll back prompts

What is prompt version control?

Prompt version control is the practice of saving every change to a prompt as a recoverable version, with a record of what changed, why, and how it tested. For most teams the answer is Git first: keep prompts as files next to your code, review changes in pull requests, and tie each version to an eval run. Add a prompt registry (Langfuse, MLflow Prompt Registry, Amazon Bedrock Prompt management) when prompts must change at runtime without a deploy, or when non-engineers edit them.

The vendor trend points the same way. OpenAI's deprecations page says "The v1/prompts API and reusable prompt objects are scheduled to shut down" on November 30, 2026, and its migration guide advises: "Move versioning to your repo using git commits, PR review, and tests or evals."

Why do prompts need versions at all?

A prompt is code that runs on a model. Small wording changes shift output, and the effect often shows up days later in a support ticket rather than in the test you ran. Versions give you:

• A diff. When results get worse, you can see exactly which words changed.
• Rollback. You can return to the last good version in minutes instead of rewriting from memory.
• Attribution. You know who changed the prompt, when, and what problem they meant to fix.
• Reproducibility. An output from last month can be traced to the exact prompt and model that produced it.
• Safe experiments. You can try a variant on a branch or a staging label without touching production.

This matters even for one person with a personal prompt library. A prompt you improved three times is worth keeping in all three forms, because the second version may have been the best one.

How do you version prompts in Git?

Store one prompt per file, in plain text, with metadata at the top:

---
title: "Support reply: refund request"
model: "the model you tested with"
temperature: 0.3
tags: [support, refunds]
version: 1.3.0
---

You are a support agent for {{product:Acme Billing}}.
Reply to the customer message below in under {{length:120}} words.
Confirm the refund policy, state the next step, and never promise a date.

Customer message:
{{message}}

Then work the way you do with code:

1. Put prompts in a folder, such as prompts/, with one file per prompt and a clear slug.
2. Change prompts on a branch and open a pull request, so someone reads the diff before it ships.
3. Run your eval set in CI on the changed prompt and post the results on the pull request.
4. Merge and deploy only when the evals pass.
5. Tag releases if several prompts ship together, so you can name a known-good set.

Keep model settings in the same file or the same commit. A prompt that worked at temperature 0.2 on one model is a different artifact from the same words at 0.9 on another.

What should a prompt commit record?

A useful commit message for a prompt answers four questions:

• What changed: "Added a rule to never promise refund dates."
• Why: "Two tickets last week quoted dates the policy doesn't allow."
• Evidence: "Eval set refunds-v4: 46 of 50 pass, up from 41."
• Settings: the model and parameters, if they changed.

Semantic versioning helps when code depends on a prompt. A common convention: bump the major number when inputs or the output format change (callers may break), the minor number when behavior changes, and the patch number for wording fixes.

When do you need a prompt registry instead of Git?

A registry stores prompts outside your code and serves them to your app at runtime. That is the right call when people outside engineering edit prompts, when you want to switch versions without a deploy, or when you need per-environment labels.

• Langfuse: its version control docs say "Each prompt version is automatically assigned a version ID." Labels such as production mark what gets served, and "The latest label points to the most recently created version." The SDK fetches by label or by version, for example langfuse.get_prompt("movie-critic", label="staging").
• MLflow Prompt Registry: the docs say "Prompt versions in MLflow are immutable," each version can carry a commit-style description, and movable aliases such as production point at a version. You load a prompt by URI, such as prompts:/summarization-prompt/2.
• Amazon Bedrock Prompt management: you create a version to take a snapshot of your draft, and versions are numbered from 1.

A practical middle path: keep prompts in Git as the source of truth, and have CI push merged versions to the registry with a staging label. Promote to production after evals pass.

How do prompt versions connect to evals?

A version number only tells you something changed. An eval tells you whether it got better. Anthropic's guide to success criteria and evaluations recommends criteria that are specific and measurable, and says "More questions with slightly lower signal automated grading is better than fewer questions with high-quality human hand-graded evals." It also says "Don't forget to factor in edge cases!"

In practice:

1. Build a fixed eval set per prompt: real inputs, including the edge cases that broke it before.
2. Score every version against the same set, with code checks where possible and a model grader where you must.
3. Record the score with the version, in the commit message or as a registry tag.
4. Add a case every time a bug escapes, so the same failure can't return quietly.

The LLM evals guide covers graders, datasets and LLM-as-judge in depth, and the prompt engineering workflow shows where testing fits in the edit loop.

How do you roll back a bad prompt?

Decide this before you need it.

• In Git: run git revert on the commit that changed the prompt. Git's documentation describes it as a way to "revert the changes that the related patches introduce, and record some new commits that record them," so history stays intact. Then deploy.
• In Langfuse: set the production label back to an earlier version; the SDKs then serve that version by default.
• In MLflow: move the production alias back to the earlier version number.

After any rollback, rerun the eval set on the restored version, since the model or the inputs may have changed since it was last good. Then write down what went wrong and add that case to the evals.

How does this apply to a personal prompt library?

The same habits work at small scale. Keep prompts as Markdown files with {{name:default}} blanks in a Git repo, write a one-line note when you change one, and keep the old version when you try a new idea. Kipt is built on that model: it keeps prompts and Agent Skills as Markdown in a private GitHub repo you own, so prompt changes have the same Git history as any other file in the repo. The AI prompt library guide compares other ways to store prompts, and prompt templates shows how blanks work.