AI Penetration Testing Scope: What to Test in AI Agents

Security teams commission penetration tests for AI systems and receive reports about TLS configuration and dependency versions. The tests were scoped for a web application, and the component that actually breaks, a model that can be steered by content it reads and that holds credentials to act, went untested. This guide defines the AI penetration testing scope, the scenarios, the evidence, and where fixes belong. It supports the AI security checklist and the AI agent security risks practice. It is general guidance, not a compliance determination.

What does the scope cover?

Area	Scenarios	Control tested
Direct injection	Users attempting to override behavior	Prompt structure, gates
Indirect injection	Instructions planted in documents, tickets, emails, pages, records, tool results	Trust labeling, gates, confirmation on plan change
Tool abuse	Steering the agent to call consequential tools outside specification	Permissions, classification, gates
Privilege escalation	Using one tool's output to gain access through another	Least privilege, delegation scoping
Exfiltration	Moving data out via prompts, retrieval, tools, or logs	Redaction, egress control, DLP policy
Identity	Credential exposure, delegation misuse, revocation gaps	Brokered identities, short-lived credentials
Sandbox and egress	Escaping the environment, reaching non-allowlisted hosts	Sandboxing, network policy
Retrieval poisoning	Planting content that changes future answers or actions	Ingestion controls, source trust
Supply chain	Malicious models, prompts, tools, or MCP servers	Provenance, registry, review
Resilience	Behavior under provider outage; kill switch under load	Fallbacks, tested stop

How are injection tests designed?

For every content type the agent reads, testers plant instructions with three goals: exfiltrate (send data somewhere), act (trigger a consequential tool), and mislead (report false conclusions). Variants include invisible text, metadata, and instructions split across documents. The measure is whether actions stayed within specification, not whether the model "noticed." The attack model is in what is indirect prompt injection.

How are tool abuse and escalation tested?

Testers try to make the agent call each consequential tool without the specified preconditions, chain reads into writes the role should not perform, use one system's output to obtain access to another, and exceed volume limits. A gate that never fires, a tool that accepts arbitrary parameters, or a credential broader than the tool needs are findings. The permission design under test is in how to design tool permissions for AI agents.

How is exfiltration tested?

Every outbound path: prompts to providers (what categories reach which deployment), retrieval (does it return content the user could not see), tool results (whole records versus fields), traces (raw content in logs, who can read them), and agent actions (email, messages, external posts). Controls are those in LLM data loss prevention.

What do identity and sandbox tests target?

Identity: can credentials be extracted from the environment or traces; does delegation scope hold; does revocation take effect within the stated time; are there long-lived secrets. Sandbox: filesystem and process escape, network egress to non-allowlisted hosts, resource exhaustion. The models are in non-human identities for AI agents and AI agent sandboxing.

What does a test engagement look like?

Phase	Activity
Scoping	Agent inventory, permissions, content sources, tools, and autonomy levels shared with testers; rules of engagement for production systems
Reconnaissance	Testers read the agent specification, tool descriptions, and prompt structure; the goal is a test of controls, not of obscurity
Injection campaign	Planted content across every source type, run through the agent in a staging environment mirroring production permissions
Tool and identity testing	Direct attempts against gates, credential brokerage, delegation, and revocation
Environment testing	Sandbox and egress probes from inside the agent's runtime
Reporting and retest	Findings with business impact; fixes at the correct layer; retest and regression capture

Testers need traces. An engagement without trace access produces findings whose root causes are guessed and whose fixes are wrong.

What does the report contain?

Element	Standard
Finding	Reproducible path from attacker-controlled input to business impact
Severity	Based on the agent's real permissions and the data involved
Failed control	Which layer should have stopped it
Fix location	Permissions, gate, sandbox, gateway policy, ingestion; prompt-only fixes are flagged as insufficient
Retest	Confirmed after fix, added to the adversarial regression suite

How does the test fit the lifecycle?

Before first production release at act-with-approval or higher; after material changes to tools, models, permissions, or content sources; annually. Between tests, adversarial cases live in the regression suite per AI regression testing, so the findings do not return quietly.

What are the common mistakes?

1. Web-app scope only.
2. Testing the chat surface, not the agent's tools.
3. Accepting prompt-only fixes.
4. No business-impact framing, so severity is argued.
5. One-off test with no regression capture.
6. Testers without access to traces, so root causes are guessed.

How does FISTA Solutions help?

FISTA Solutions builds every AI agent to be tested this way, with classified tools, gates, sandboxing, brokered identities, and gateway policy, and supports security teams scoping and remediating AI penetration tests through its AI enablement practice, with forward deployed engineers turning findings into platform fixes and regression cases. FISTA has delivered 150+ projects for 50+ companies across 12+ countries.

To scope a test of your agents, message FISTA on WhatsApp, or read AI agent security risks for the continuous practice.