For the first two years of the generative-AI boom, "prompt engineering" was treated as a craft somewhere between copywriting and folklore. You tweaked words in a text box, the output got better, and nobody could quite say why. That was fine for demos. It is a liability in production. When a prompt is the thing standing between a user and a wrong answer that ends up in a contract, a support reply, or a financial summary, it is no longer a phrase — it is code. And code has a discipline.

Across the systems Netatum operates, the teams that ship reliable LLM features are the ones who stopped treating prompts as strings and started treating them as software artifacts: versioned, tested, reviewed, observed, and rolled back. This post is the playbook we use to get there.

Prompts are a dependency, not a decoration

The mental shift that unlocks everything else is this: your prompt is a load-bearing dependency with the same blast radius as a database query or an API contract. A one-word change can move accuracy by ten points. A reordered instruction can flip whether the model calls a tool. If you would not edit a SQL query directly in production without a review and a test, you should not do it to a prompt either.

Once you accept that, the engineering requirements fall out naturally:

  • Version control. Prompts live in the repository, not in a Notion doc or a config UI that nobody diffs. Every change has an author, a commit message and a reviewer.
  • Separation of template and data. Instructions are static and reviewed; user content is injected through variables, never concatenated by hand.
  • Tests. A prompt change is validated against a dataset of real cases before it merges.
  • Observability. Every production call logs the resolved prompt, the model, the version and the outcome, so you can trace a bad answer back to the exact text that produced it.

Structure your prompts like modules

Stop writing one giant paragraph. Compose prompts from named, reusable sections the same way you compose a program from functions. A system prompt in our codebases is a rendered template, not a literal string, and it separates the durable contract (role, constraints, output schema) from the volatile inputs.

# prompts/support_triage.v4.yaml
id: support_triage
version: 4
model: claude-opus-4
temperature: 0
system: |
  You are a support triage assistant for {{ product_name }}.
  Classify the ticket and extract fields. Rules:
  - Only use categories from the provided taxonomy.
  - If confidence < 0.6, set category to "needs_human".
  - Never invent an order ID; use null when absent.
output_schema:
  category: string        # one of taxonomy
  order_id: string | null
  confidence: number      # 0..1
inputs:
  - taxonomy
  - ticket_body

This looks mundane, and that is the point. The prompt now has an identity (id and version), a pinned model, an explicit output contract, and declared inputs. It can be loaded, diffed, linted and tested. When something breaks in production, "which prompt, which version" is answerable in one query instead of a Slack archaeology dig.

Test prompts like the flaky functions they are

The objection we hear most is "you can't unit-test a language model, it's non-deterministic." Partly true, and irrelevant. You do not test an LLM the way you test a pure function; you test it the way you test a noisy integration — with a dataset, a scorer and a threshold. Set temperature: 0 where you can, build a golden set of representative and adversarial cases, and gate merges on aggregate scores.

A prompt without an eval set is a hypothesis, not a feature. The eval set is the specification; the prompt is just the current best attempt at satisfying it.

Our regression loop runs on every pull request that touches a prompt file. The shape is simple:

  1. Load the golden dataset for that prompt — typically 50 to 300 labelled cases mined from real traffic.
  2. Run the candidate prompt against every case at temperature 0.
  3. Score outputs with a mix of deterministic checks (schema valid? field extracted? category in taxonomy?) and, where judgement is needed, an LLM-as-judge rubric that is itself pinned and validated.
  4. Fail the build if accuracy drops, if the schema-valid rate falls below 99%, or if any case in the "must never regress" tier flips.

This turns prompt work from vibes into a signal. A change that reads better but scores worse gets caught before a customer sees it. A change that looks risky but lifts the numbers ships with confidence.

Review prompts like a diff, because they are one

Prompt changes go through code review with a checklist tuned to their failure modes. Reviewers look for instruction conflicts, leaked few-shot examples that no longer match the schema, injection surfaces where user text can override the system contract, and silent scope creep where a prompt slowly accretes ten special-case clauses that no eval covers. A useful heuristic: if a clause was added to fix one bad example, there had better be a test case pinning that behaviour, or it will rot.

Guard the boundary between instructions and input

The single most common production incident we see is prompt injection through user-supplied content — a document, a ticket, a web page the agent retrieved. Treat every external string as hostile. Keep it in a clearly delimited region, remind the model that content in that region is data and never instructions, and never let retrieved text sit in the same trusted channel as your system contract. This is the AI-era version of parameterised queries, and the discipline is identical.

Observability closes the loop

Shipping is the start, not the finish. Every production inference should emit the prompt id and version, the model, token counts, latency, and enough of the outcome to grade it later. That log stream is where next month's golden cases come from: real failures, promoted into the eval set, so the same mistake can never silently return. Over a few cycles this compounds into a genuine moat — a test suite that encodes everything your product has learned about its own edge cases.

What this changes in practice

Teams that adopt this discipline stop being afraid of their prompts. They upgrade models without holding their breath, because the eval suite tells them immediately whether behaviour held. They let more people contribute to prompts, because a bad change gets caught by the gate rather than by a customer. And they move faster, not slower — the up-front cost of structure buys back enormous time otherwise spent debugging outputs no one can reproduce.

Prompt engineering did not stop being a craft. It grew up into an engineering practice, with all the boring, wonderful machinery that implies. If your prompts still live in a text box, the highest-leverage thing you can do this quarter is move them into the repo, wrap them in an eval, and never look back.

N
Netatum AI Team
AI Integration & Consulting