LLM Security Checklist for Production Applications

Conventional application security reviews do not ask whether a PDF can instruct the application, whether a model can be talked into exfiltrating data, or whether a tool call can be induced by a web page. LLM applications require those questions. This checklist covers them with evidence for each item, aligned with the OWASP Top 10 for LLM Applications and with the AI agent security architecture whitepaper. FISTA Solutions applies it to every LLM application and agent delivered through AI enablement and AI agents.

Who should use this checklist?

Application security reviewers, engineering owners of LLM applications and agents, platform teams owning gateways and tool layers, and governance functions signing off on higher-risk systems.

Is prompt injection defended in depth?

1. System instructions are structurally separated from all data; content sources are labeled.
2. Retrieved documents, emails, web content, and tool results are treated as untrusted data.
3. Input and retrieved content are screened for injection patterns where feasible.
4. Defense does not rely on prompt instructions alone.
5. An adversarial injection suite covering every content channel runs in CI and passes.
6. Indirect injection through documents and tool results is specifically tested.

Reference: what is prompt injection and the prompt injection defense checklist.

Is capability bounded?

1. Each agent has its own identity with short-lived, scoped credentials.
2. Tools are least privilege, with read and write separated and consequence levels assigned.
3. No tool can exfiltrate to arbitrary destinations; egress is allowlisted.
4. User-facing actions use delegated, user-scoped authorization.
5. Code, file, and network execution is sandboxed with resource and time limits.
6. Third-party plugins and MCP servers run with minimal credentials and restricted egress.

Reference: how to build tool use for llm agents and ai access control.

Are outputs and actions validated outside the model?

Check	Evidence
Schema validation on tool arguments and outputs	Validation layer tests
Policy rules on allowed tools, targets, values, and rates	Policy configuration and tests
Content checks for sensitive data and prohibited content before outputs leave	Test results
Consequence classification routes gated actions to approval	Gate configuration
Outputs rendered downstream are encoded to prevent injection into browsers or systems	Code review
Deterministic fallbacks on validation failure	Design and tests

Reference: llm output validation and what is a human approval gate.

Is sensitive data protected?

1. Data classes in prompts, retrieval, and outputs are classified with handling rules.
2. Retrieval enforces permissions; unauthorized content never reaches the model.
3. Sensitive fields are redacted or tokenized before third-party processing where policy requires.
4. Outputs and tool calls are checked for leakage.
5. Logs and traces are redacted; access is controlled and logged.
6. Provider terms prohibit training on organizational data; retention and residency are settled.
7. Memory writes, where present, follow explicit policies and are protected from injection.

Reference: ai data leakage prevention and ai data privacy compliance.

Is the supply chain controlled?

1. Models, libraries, plugins, connectors, and MCP servers are inventoried.
2. Versions are pinned; provider model updates trigger re-evaluation.
3. Model artifacts have verified integrity and provenance for self-hosted deployments.
4. Third-party components are reviewed for permissions and behavior.
5. Dependency scanning runs in CI.

Reference: ai supply chain security.

Is usage bounded against denial of wallet?

1. Rate limits per application, user, and tenant are enforced at the gateway.
2. Spend budgets with alerts and hard limits exist.
3. Context, output, retry, and iteration limits are enforced.
4. Anomaly detection on usage patterns is live.

Reference: how to build an llm gateway.

Is the system logged and monitored for security?

1. Every request is logged with identity, prompts and outputs (redacted), tool calls, validation and gate outcomes, and cost.
2. Trajectory-level logging exists for agents.
3. Security telemetry (validation failures, blocked actions, unusual tool sequences, first-seen destinations) flows to the security operations center.
4. Logs are append-only, retained per policy, and access-controlled.
5. An incident playbook covers containment, reconstruction, remediation, and prevention.

Reference: the AI observability whitepaper and the ai incident response checklist.

Has the system been tested adversarially?

1. The adversarial suite (injection, exfiltration, prohibited actions, policy violations) runs in CI.
2. Red-team exercises have been run against high-consequence systems, with findings added to the suite.
3. Penetration testing of surrounding infrastructure and integrations is complete.
4. Permission audits compare granted scopes to specification requirements.

Reference: ai red teaming guide and ai penetration testing.

How should findings be handled?

High-severity findings in injection defense, capability bounding, output validation, or data protection block launch. Others are tracked with owners and dates and revisited at the post-launch review. Every finding becomes a permanent regression test.

How FISTA Solutions uses this checklist

FISTA Solutions builds LLM applications and agents against this checklist as standard: structural separation, least-privilege tools with sandboxing, validation and policy outside the model, redaction and permission-aware retrieval, pinned and reviewed supply chain, gateway-enforced limits, security logging, and adversarial testing in CI. The AI enablement practice delivers the gateway, tool, and logging platform; AI agents are built on it; and forward deployed engineers work with your security team on the review. The record behind the approach is 150+ projects with 99.9% uptime.

To run a security review on an LLM application, message FISTA on WhatsApp, or read securing ai systems for the broader security context.