Spec-Driven Development for AI Systems: A Whitepaper

Every failed AI project FISTA Solutions has been asked to rescue shared a symptom: nobody could state, in writing, what the system was supposed to do and how anyone would know it was doing it. There were prompts, demos, and enthusiasm. There was no specification. This whitepaper explains spec-driven development for AI systems, why probabilistic components make it indispensable, and how it changes delivery, testing, and governance.

What is spec-driven development?

Spec-driven development is a delivery method in which a precise, reviewable specification of behavior and acceptance criteria is written and agreed before implementation, and implementation is verified against it. It is older than AI. What AI changes is the stakes: when the component you are building is probabilistic, the spec is the only place where correctness is defined at all.

FISTA's foundational explainer is spec-driven development explained, and the field account of using it is spec-driven development in practice. This whitepaper goes deeper on the AI-specific mechanics.

Why do probabilistic systems demand explicit specs?

In deterministic software, behavior can be inferred from code. A reviewer can read a function and know what it does for every input. In an AI system, the central component is a model whose behavior can only be characterized statistically. You cannot read the weights. You can only define what correct means and measure how often the system meets it.

That single fact has four consequences:

1. Correctness must be defined externally. The spec is the definition. Without it, every stakeholder carries a private definition and discovers the differences in production.
2. Testing becomes evaluation. You cannot unit-test a model into correctness; you evaluate it against a dataset derived from the spec.
3. Change becomes regression risk. A new model version or a prompt tweak can silently change behavior across thousands of cases. Only a spec-derived evaluation suite catches it.
4. Governance needs an anchor. Guardrails, approval gates, and audit requirements have to attach to something. They attach to spec clauses.

This is why FISTA describes its outcome as deterministic AI outcomes: not deterministic models, which do not exist, but deterministic definitions of correct behavior and verified compliance with them.

What does an AI specification contain?

A complete AI spec has ten sections. Each is short, concrete, and reviewable by both the domain owner and the engineer.

Section	What it states	Example clause (support triage agent)
Purpose and scope	The job, the boundary, and what is out of scope	Route inbound tickets to one of nine queues; do not reply to customers
Inputs	Sources, formats, freshness, access rules	Ticket subject, body, customer tier, product from CRM; no attachments
Outputs	Schema, allowed values, required fields	Queue ID, confidence score, one-sentence rationale, structured JSON
Business rules	Deterministic constraints that override model judgment	Any ticket mentioning a legal claim routes to Legal regardless of content
Prohibited actions	What the system must never do	Never modify ticket priority; never contact the customer
Confidence and escalation	Thresholds and where uncertain cases go	Below 0.8 confidence, route to human triage queue with rationale
Approval gates	Which actions need a human before taking effect	None for routing; all for any account changes
Data handling	Privacy, retention, residency, redaction	PII redacted before model call; logs retained 90 days in-region
Quality thresholds	Metrics, targets, measurement method	Routing accuracy at or above 95% on the golden set; sampled weekly
Acceptance criteria	Test cases that prove the above	400 labeled tickets covering all queues and edge cases

The tenth section is the one most teams skip and the one that matters most. If you cannot derive test cases from the spec, the spec is incomplete. Guidance on writing these is in how to write acceptance criteria for AI.

How does the spec drive evaluation?

The spec is the source of the evaluation dataset. Each business rule, prohibited action, confidence threshold, and output constraint generates test cases. Together they form the golden dataset against which every model, prompt, and retrieval change is measured. The relationship is mechanical:

• A business rule becomes a set of cases where the rule must fire, and a set where it must not.
• A prohibited action becomes adversarial cases that attempt to elicit it.
• A confidence threshold becomes cases near the boundary that must escalate.
• An output schema becomes structural validation on every response.
• A quality threshold becomes the pass/fail gate in the CI pipeline.

This is why spec-driven teams ship evaluation before they ship automation. The method is elaborated in the AI evaluation and testing whitepaper and in LLM evaluation explained.

How does the spec drive governance?

Governance frameworks ask for human oversight, transparency, logging, risk controls, and documented intended use. In a spec-driven system, each of these is a clause with an owner rather than a policy statement nobody can trace to the code:

Governance requirement	Where it lives in the spec
Intended use and limitations	Purpose and scope
Human oversight	Confidence and escalation; approval gates
Transparency to users	Outputs (rationale field); data handling (notices)
Logging and record keeping	Data handling; audit requirements
Risk controls	Prohibited actions; business rules
Performance monitoring	Quality thresholds and measurement method

When an auditor or regulator asks how the system is controlled, the answer is the spec plus the evaluation results, not a slide. This is how spec-driven development turns the abstractions of the AI governance framework into engineering artifacts.

How does spec-driven delivery change the project?

It moves disagreement forward. In prompt-first projects, stakeholders discover their differing expectations when the demo meets real data. In spec-driven projects, they discover them during discovery, in a document, where resolving them costs a meeting rather than a rebuild. The delivery sequence looks like this:

1. Discovery. Engineers and domain owners extract the real rules, exceptions, and failure costs from the people who do the work today.
2. Specification. The ten sections are drafted, reviewed, and signed by the process owner and the engineering lead.
3. Evaluation dataset. Test cases are built from the spec, labeled by domain experts, and versioned.
4. Implementation. Models, prompts, retrieval, and tools are built and tuned against the dataset.
5. Verification. The system must clear the quality thresholds on the golden set before shadow deployment.
6. Shadow and graduation. The system runs alongside the current process; autonomy is raised as production samples confirm the golden-set results.
7. Handover. The spec, dataset, dashboards, and runbooks are transferred to the owning team.

Step 1 is where forward deployed engineers are decisive. The real rules of a business process live in the heads of the people who run it, and extracting them requires an engineer embedded close enough to ask the third and fourth follow-up question.

What are the common objections, and are they right?

"Specs slow us down." They slow down the start and speed up everything after it. Prompt-first projects feel fast for three weeks and then stall for three months. The honest comparison is time to verified production, not time to demo.

"The requirements will change." They will, and the spec is where the change is recorded, reviewed, and turned into new test cases. Changing a prompt without changing the spec is how systems drift into behavior nobody approved.

"We cannot specify judgment." Correct, and that is the point. Work that cannot be specified is work that should stay with humans, with the AI assisting. The spec exercise sorts delegable work from judgment work honestly. The sorting logic is described in the AI-native enterprise operating model whitepaper.

"The model handles edge cases." Some of them, some of the time. A spec lists the edge cases that matter and puts test cases behind them, so that handling is verified rather than hoped.

What does a spec-driven AI team look like?

Three roles carry the method. The domain owner signs the spec and reviews evaluation results; they own the definition of correct. The AI engineer implements against the spec and owns the evaluation harness. The quality reviewer, often drawn from the domain team, labels datasets and audits production samples. In small organizations one person may wear two hats, but the spec, the dataset, and the sign-off should exist even when the team is two people. Team design guidance is in AI team structure.

How does spec-driven development relate to verification-led engineering?

Spec-driven development defines correct behavior; verification-led engineering is the operating discipline that continuously proves the system still exhibits it. The spec produces the evaluation suite; verification runs it on every change and on production samples, and treats any regression as a blocking defect. The two are halves of one method. FISTA's description of the second half is verification-led engineering.

A worked outline: specifying a contract-review assistant

Consider a legal operations team that wants an assistant to flag non-standard clauses in supplier contracts. A prompt-first approach would paste a contract into a model and ask for risks. A spec-driven approach would produce:

• Scope: identify deviations from the company's playbook for eleven named clause types; do not advise on enforceability.
• Inputs: contract text from the CLM system; the current playbook version; jurisdiction.
• Outputs: per clause, a deviation flag, the playbook position, the contract language, and a severity from a fixed scale.
• Rules: any indemnity or limitation-of-liability deviation is always flagged at the highest severity.
• Prohibited: never suggest alternative language without a human request; never process contracts marked privileged.
• Escalation: unrecognized clause types are surfaced for a human, not classified.
• Quality: recall on deviations at or above the threshold the legal team sets, measured on a labeled set of past contracts.
• Acceptance: the labeled set, covering every clause type and known edge cases, with agreed pass thresholds.

The engineering that follows is constrained, testable, and explainable to the general counsel. The build guide for this system is how to build a contract analysis system.

How FISTA Solutions applies spec-driven development

FISTA Solutions delivers every AI system spec-first. Our forward deployed engineers run discovery inside your business, write the ten-section spec with your domain owners, derive the evaluation dataset, and build governed AI agents that must clear the acceptance criteria before they touch production. The AI enablement practice provides the shared evaluation, gateway, and observability platform that makes the method repeatable across workflows. The approach underpins a record of 150+ projects with 99.9% uptime.

If you have an AI initiative that has a demo but no definition of correct, the fastest fix is a spec. Message FISTA on WhatsApp to scope one, or read how to write an AI spec to start yourself.