# Version Control for Prompts in a Codebase | api.llmnet.nl

[Skip to content](#lm-inhoud)Network/[NL](/en/versiebeheer-voor-prompts-in-code)EN[Hubhub.llmnet.nlCompare models on task, language, cost and licence.](https://hub.llmnet.nl/en/)[Communitycommunity.llmnet.nlPrompt techniques, patterns and system prompts.](https://community.llmnet.nl/en/)[APIapi.llmnet.nlLLMs in production: rate limits, routing, structured output.](https://api.llmnet.nl/en/)[Consultancyconsultancy.llmnet.nlRolling out AI in an organisation, pilot to production.](https://consultancy.llmnet.nl/en/)[Newsnieuws.llmnet.nlAI developments, explained for the Netherlands.](https://nieuws.llmnet.nl/en/)[Benchmarkbenchmark.llmnet.nlMeasure AI quality yourself, on your own tasks.](https://benchmark.llmnet.nl/en/)[Careersvacatures.llmnet.nlAI roles, salaries and career paths in the Netherlands.](https://vacatures.llmnet.nl/en/)[Learnleren.llmnet.nlAI concepts in plain language, beginner to builder.](https://leren.llmnet.nl/en/)[Guidegids.llmnet.nlRun AI privately on your own Mac, PC, NAS or home server.](https://gids.llmnet.nl/en/)[Directorydirectory.llmnet.nlMapping the AI ecosystem: tools, models, companies.](https://directory.llmnet.nl/en/)[Radarradar.llmnet.nlSignals from X, research and communities for indie developers.](https://radar.llmnet.nl/en/)[Appsapps.llmnet.nlReviews of AI apps and open-source repos, with tips for builders.](https://apps.llmnet.nl/en/)[llmnet.nl — main site](https://llmnet.nl/en/)[](https://x.com/intent/post?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fversiebeheer-voor-prompts-in-code&text=Version%20Control%20for%20Prompts%20in%20a%20Codebase)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fversiebeheer-voor-prompts-in-code)[](https://www.reddit.com/submit?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fversiebeheer-voor-prompts-in-code&title=Version%20Control%20for%20Prompts%20in%20a%20Codebase)[](#)[](https://x.com/intent/post?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fversiebeheer-voor-prompts-in-code&text=Version%20Control%20for%20Prompts%20in%20a%20Codebase)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fversiebeheer-voor-prompts-in-code)[](https://www.reddit.com/submit?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fversiebeheer-voor-prompts-in-code&title=Version%20Control%20for%20Prompts%20in%20a%20Codebase)[](#)

# Version Control for Prompts in a Codebase

By Ivo Donker - August 6, 2026

In many software organizations, the use of Large Language Models (LLMs) begins as an experiment. A developer pastes instruction text as a hardcoded string into an API call and ships the feature. As soon as the system runs in production, this approach proves unsustainable. A minor change in the prompt instructions alters the structure of the response, breaks downstream parsers, or introduces unforeseen errors in edge cases. Managing prompts as loose text fragments without a structured process directly leads to production regressions.

To reliably scale LLM-based functionality, a mature version control process is essential. This article covers the design of prompt version management within software architectures, from storage and review strategies to regression testing, binding to specific model versions, and phased rollout patterns.

## Why a prompt behaves like source code, not configuration or content

It is a common misconception to treat prompts as static application configuration or editable content. A setting such as the maximum number of database connections is deterministic: change the number from 10 to 20, and processing capacity scales predictably. Editorial content on a website alters the presentation but has no impact on the executable logic of the software.

A prompt, on the other hand, defines the runtime behavior of the LLM and dictates how the application transforms data, makes decisions, and generates output. Altering a single word or adding an extra few-shot example in the prompt can radically change the output format. When using [structured output](/en/structured-output) or [function calling](/en/function-calling), the prompt acts as the interface contract definition between unstructured model output and your deterministic software code.

Because modifications to a prompt directly impact the logic and stability of your software, a prompt must undergo the exact same lifecycle as source code: peer reviews, automated regression tests, audit trails, and controlled deployments.

## Storage strategies for prompts

There are four common patterns for prompt storage, each with specific advantages and disadvantages for the development workflow and runtime architecture.

 
 
 Storage Location | 
 Advantages | 
 Disadvantages | 
 Suitability | 
 

 
 
 
 Inline in code | 
 Maximum proximity to execution logic; direct storage in Git; no external dependencies. | 
 Poor readability for lengthy texts; JSON escaping pollutes diffs; inaccessible to non-developers. | 
 Small prototypes and very brief instructions. | 
 

 
 Separate files in the repository | 
 Clean Git diffs; version control tied to application release; clean separation of concerns. | 
 Changes require a new application deployment. | 
 Medium to large enterprise applications with strict CI/CD requirements. | 
 

 
 Database | 
 Configurable via an admin dashboard; updates possible without code deployments. | 
 Decoupled from Git history; risk of inconsistency across environments (dev/prod). | 
 Systems where non-technical domain experts need to adjust live instructions. | 
 

 
 External Prompt Registry | 
 Built-in features for evaluation, version control, dashboards, and A/B testing. | 
 Additional external network call; vendor lock-in; extra dependency in the critical path. | 
 Teams with a strong focus on AI product management that require decoupling from the release cycle. | 
 

 

### Concrete implementation: Separate files in the repository

For most software teams, storing prompts in separate files within the Git repository offers the optimal balance between maintainability, security, and transparency. In this setup, prompts are stored in a dedicated folder (for example prompts/) in a clean format such as Markdown or Text, accompanied by a metadata file (JSON or YAML).

prompts/
└── facturatie/
 └── verwerk_factuur/
 ├── v1.0.0.md
 ├── v1.1.0.md
 └── metadata.json

The file metadata.json defines which version is currently designated as active or default and which variables are required:

{
 "prompt_name": "verwerk_factuur",
 "active_version": "1.1.0",
 "versions": {
 "1.1.0": {
 "file": "v1.1.0.md",
 "target_model": "gpt-4o-2024-08-06",
 "temperature": 0.0,
 "max_tokens": 1000
 }
 }
}

## Version numbering, identity, and content hashes

A robust version control system for prompts combines two forms of identification: human-readable version numbers (SemVer) and cryptographic content hashes.

### Semantic Versioning (SemVer)

Apply the MAJOR.MINOR.PATCH principle to your prompt files:

 
- MAJOR: Changes to the prompt that alter the structure or schema of the expected output (e.g., adding mandatory JSON fields), or that fundamentally modify the prompt's intent.
 
- MINOR: Adding additional examples (few-shot learning), restructuring instructions for higher accuracy without breaking the output interface.
 
- PATCH: Linguistic fixes, typo corrections, or minor nuances that have no measurable impact on the logic.

### Content hashes

In addition to the SemVer tag, the runtime code should compute a SHA-256 hash of the populated or unpopulated prompt template. After all, a version number in the code can inadvertently remain unchanged after an edit, whereas a hash is guaranteed to change with every modification to the string.

import hashlib

def calculate_prompt_hash(template_content: str) -> str:
 return hashlib.sha256(template_content.encode('utf-8')).hexdigest()[:12]

By including both the version name (e.g., verwerk_factuur:1.1.0) and the unique content hash in the metadata for every API call to the LLM, an indisputable trace is created within your observability infrastructure. Consult the article on [observability-and-logging](/en/observability-en-logging) for detailed patterns on capturing these traces.

## The unique pairing: Prompt version x Model version

A common mistake in AI engineering is the assumption that a prompt is a universal, model-agnostic instruction. In practice, the behavior of a prompt is strictly coupled to a specific model version and even to a specific provider snapshot.

A prompt that works optimally on gpt-4o-2024-05-13 can yield deviating or degraded results when the same provider migrates to gpt-4o-2024-08-06 or when the call is routed to an alternative model via [model routing](/en/model-routing). Models differ in their sensitivity to system instructions, their processing of markdown formatting, and their tendency toward verbosity.

Therefore, never record a prompt version in isolation, but always as an indivisible tuple of [Prompt ID + Prompt Version + Model ID + Provider Snapshot + Hyperparameters]. When the underlying model changes, this should be treated as a change in the runtime environment. The existing prompt must be re-evaluated and, if necessary, re-versioned for the new model identity.

## The review process and readable code diffs

To conduct an effective code review on prompt changes, the files must be readable in tools like GitHub, GitLab, or Bitbucket. Direct inline storage of multiline text in languages such as Java or Python often leads to unnecessary string escapes (such as \n or interpolated quotes), which complicates reviews.

### Rules for clean prompt diffs:

 
- Use dedicated Markdown or TXT files: Markdown offers the advantage of structuring sections within the prompt using headings (such as # Context, # Regels, # Voorbeelden), which improves processing for both the human reviewer and the LLM.
 
- Avoid JSON escaping in Git: Store templates as plain text. Use well-known templating engines (such as Jinja2, Mustache, or native string templates) to insert variables at runtime.
 
- Clearly separate instructions and data: Ensure the template has a clear delimiter for dynamic runtime input.

Example of a clean prompt template in prompts/samenvatten/v1.0.0.md:

# Rol
Je bent een gespecialiseerde juridische assistent die Nederlandstalige contracten analyseert.

# Taak
Vat de onderstaande tekst samen in maximaal 3 opsommingstekens. Focus uitsluitend op de financiële aansprakelijkheid.

# Constraints
- Gebruik geen vaktermen zonder korte uitleg.
- Als er geen aansprakelijkheid wordt genoemd, antwoord dan exact met: "Geen aansprakelijkheid vermeld."

# Invoer
{{ contract_tekst }}

## Testing upon release: Regression testing on evaluation properties

Testing a prompt differs fundamentally from traditional unit tests. Because an LLM is probabilistic, a test on an exact string match (such as assert output == "expected") can rarely be applied, except for very strict formats. A robust CI/CD pipeline for prompts centers on regression testing against established test sets.

### Setting up a prompt test set

Maintain a dataset in the repository with representative input examples and expected properties (gold standard dataset). These can include edge cases from production practice, including cases that previously led to bugs.

tests/prompts/facturatie_testset.json
[
 {
 "id": "tc_001",
 "input": {"contract_tekst": "De totale aansprakelijkheid is beperkt tot EUR 10.000."},
 "expected_checks": {
 "contains_keywords": ["10.000", "aansprakelijkheid"],
 "max_words": 50,
 "forbidden_words": ["geen idee"]
 }
 }
]

### Evaluation properties (Assertions)

Instead of exact text matches, use the following methods to determine output validity:

 
- Schema validation: Check using JSON Schema or Pydantic whether the output strictly conforms to the expected format. For more information, see [input-validation-and-output-filtering](/en/invoervalidatie-en-outputfiltering).
 
- Heuristic regression tests: Validate hard constraints such as maximum length, presence of specific entities, absence of forbidden terms, and language detection.
 
- Model-as-a-Judge evaluations: For complex qualitative requirements, deploy a heavier, evaluated model that grades the generated output using a rubric (evaluation matrix) on a scale from 1 to 5 based on criteria such as correctness and tone. If you want to benchmark the results against broader model comparisons, check [benchmark.llmnet.nl](https://benchmark.llmnet.nl/en/).

In your CI pipeline (for example, GitHub Actions), run the test suite on every Pull Request where a prompt file in the directory prompts/ has been modified. If the score on the test set falls below a predefined threshold, merging the PR is blocked.

## Phased rollout of prompt changes

A coded and vetted prompt must be safely deployed to production. Because a prompt update can have unforeseen impacts on real users, a release should never be rolled out to 100% of traffic all at once.

### 1. Feature flags and dynamic routing

Connect prompt version selection in code to a feature flag system (such as LaunchDarkly, Unleash, or an internal configuration service). This decouples the deployment time from the activation time.

def get_prompt_version(user_id: str) -> str:
 if feature_flags.is_enabled("use_prompt_v1_1_0", context={"user_id": user_id}):
 return "1.1.0"
 return "1.0.0"

### 2. Canary releases and A/B testing

Initially route a small percentage (for example, 5%) of production traffic to the new prompt version (v1.1.0) while the vast majority continues running on the stable version (v1.0.0). During this period, compare key metrics: parser error rates, latency, token consumption, and user feedback.

### 3. Instant rollback without deployment

If an increased error rate occurs in production (such as failed JSON parsing), the feature flag can immediately be reverted to the previous version (v1.0.0). Because legacy prompt versions remain in the codebase, there is no need for a rushed hotfix deployment or rolling back the entire application build.

Should the API provider experience outages or elevated error rates during rollout, a strategy for [graceful-degradation-bij-llm-uitval](/en/graceful-degradation-bij-llm-uitval) provides the necessary resilience to ensure service continuity.

## What to log for incident reconstruction

When a user reports an incorrect answer or an error message, it is essential to be able to reconstruct exactly how the LLM arrived at that specific output. Modifying a prompt after the fact without detailed audit logging makes troubleshooting impossible.

For every LLM interaction, save a structured log record containing at least the following fields:

 
- request_id: Unique transaction ID.
 
- prompt_name: The functional prompt name (e.g., analyseer_risico).
 
- prompt_version: The exact SemVer version (e.g., 1.2.0).
 
- prompt_hash: The SHA-256 hash of the compiled template.
 
- model_provider: The API provider (e.g., OpenAI, Anthropic, local).
 
- model_version: The exact model snapshot version (e.g., claude-3-5-sonnet-20241022).
 
- hyperparameters: The runtime settings used (such as temperature, top_p, presence_penalty).
 
- input_variables: The dynamic parameters injected into the prompt (anonymized in compliance with privacy regulations).
 
- raw_response_hash: Hash or reference to the generated output.

By storing this information in a centralized logging facility, you can reproduce the exact state of the application at the specific time of execution in a development environment whenever a production call fails.

## Checklist: Prompt Versioning

Use this practical checklist to evaluate prompt processing within your software team:

 
- [ ] Storage: Prompts are stored in standalone, version-controlled files (e.g., Markdown) within the Git repository, separated from application code.
 
- [ ] Identification: Each prompt has an explicit SemVer number, and the runtime computes a content hash of the template.
 
- [ ] Coupling: In the configuration, each prompt version is explicitly tied to a specific model name and snapshot version.
 
- [ ] Reviews: Prompt changes go through a standard Pull Request process, including peer reviews on readable text diffs.
 
- [ ] CI/CD Testing: Changes are automatically tested against a golden regression test suite. The PR is blocked if validation scores drop.
 
- [ ] Deployment: New prompt versions are rolled out behind feature flags and deployed gradually via canary releases.
 
- [ ] Observability: Every production call logs the prompt ID, prompt version, content hash, and exact model version for complete traceability.

© 2026 llmnet.nl · Ivo Donker