AI SLA Checklist

Service level agreements written for conventional software promise uptime and response times. AI systems can meet both while producing wrong answers, drifting after a silent model update, or stalling behind an unstaffed approval queue. This checklist covers the dimensions an AI SLA must add, whether between a vendor and a buyer or between an internal platform team and its consumers. It complements ai sla expectations and the outsourcing contract checklist. This is general guidance, not legal advice.

Who should use this checklist?

Buyers negotiating AI service terms, vendors defining what they will commit to, and platform teams setting internal service levels for AI capabilities.

Are availability and latency defined?

1. Availability target with measurement method, maintenance windows, and exclusions.
2. Latency targets per operation type at defined percentiles, including streaming time to first token where relevant.
3. Dependency handling: how provider outages are treated, and fallback commitments.
4. Capacity commitments for expected and peak volume.

Reference: the LLM production readiness whitepaper.

Is quality expressed as measured thresholds?

Element	Defined?
Agreed evaluation dataset and its ownership
Metrics per category and thresholds
Production sampling method and cadence
Grader calibration and human review role
Reporting of results by category
Remediation commitment on breach
No promises of absolute correctness

Reference: the ai evaluation checklist.

Are oversight response times committed?

1. Approval gate review time by severity and business hours.
2. Exception queue handling time.
3. Staffing responsibility: who staffs the queue and how volume changes are handled.
4. Escalation when queues exceed targets.

Reference: how to build a human review queue.

Are incidents defined and handled?

1. Severity levels including AI-specific incidents: quality regressions, safety events, leakage, provider changes.
2. Response and resolution targets by severity.
3. Containment commitments: autonomy reduction, rollback, disabling.
4. Communication during incidents.
5. Post-incident reports with root cause and prevention.

Reference: the ai incident response checklist.

Are changes governed?

1. Advance notice of model, prompt, retrieval, and configuration changes.
2. Re-evaluation against the agreed dataset before production.
3. Right to defer or roll back changes.
4. Version pinning where feasible; handling of forced provider deprecations.
5. Change log available to the consumer.

Reference: how to build a prompt management system.

Are security and data commitments included?

1. Data use restrictions and provider terms.
2. Security controls and audit rights.
3. Breach notification timelines.
4. Retention and deletion commitments.

Reference: the LLM security checklist.

Is reporting agreed?

1. Cadence and recipients.
2. Content: availability, latency, quality by category, oversight metrics, incidents, changes, cost, drift indicators.
3. Dashboards accessible to the consumer.
4. Review meetings with decisions recorded.

Reference: the ai observability checklist.

Are remedies proportionate?

1. Measurable breach definitions tied to the metrics above.
2. Remedies such as service credits, remediation plans, and termination rights for persistent breaches.
3. Exclusions limited and explicit.
4. Escalation path before remedies apply.

Are roles and dependencies clear?

1. Consumer obligations: data quality, access, staffing of oversight, timely decisions.
2. Shared responsibilities documented.
3. Third-party dependencies and their service levels flowed through.

How should gaps be handled?

An SLA without quality thresholds or change commitments is a software SLA applied to a system it does not fit; renegotiate before signing. Oversight and incident gaps are closed with operational plans. Reporting gaps are closed in the first review cycle.

Worked example: an internal platform SLA for a support assistant

A platform team publishes service levels for a support assistant consumed by three business units. Availability is measured at the gateway with provider outages covered by an automatic fallback model. Latency is committed at the 95th percentile for time to first token. Quality is committed as thresholds by intent on a golden set the business units co-own, with weekly production samples reported by category. The support operations team commits to review approval-gated actions within a defined window during business hours. Prompt and model changes require two days' notice and a passing evaluation run visible to consumers, who may defer adoption for one release. A monthly report covers all of it, including cost per resolved conversation, and a quarterly review adjusts thresholds as the assistant earns autonomy. No one promises the assistant will always be right; everyone can see how right it is.

How FISTA Solutions approaches AI service levels

FISTA Solutions defines service levels for the systems it delivers and operates in these terms: availability and latency with measurement, quality as measured thresholds on agreed datasets with production sampling, oversight response commitments designed with the client, AI-specific incident handling, change notice with re-evaluation, and transparent reporting, without promising outcomes a probabilistic system cannot honor. The AI enablement platform produces the evidence, AI agents are operated against these levels, and forward deployed engineers set them up with your teams. The record behind the approach is 150+ projects with 99.9% uptime.

This checklist is general guidance, not legal advice. To define service levels for an AI system, message FISTA on WhatsApp, or read how to manage ai vendors for the ongoing vendor relationship.