AI Vendor Security Questionnaire: The Questions That Matter

Security questionnaires ask about encryption, access control, and certifications, and AI vendors answer them well. They rarely ask whether the vendor trains on your prompts, which model provider sits behind their product, how they test a new model version before switching, or whether their traces store your customers' data in plain text. Those are the questions that decide AI vendor risk. This guide provides the AI-specific question set, drawing on FISTA Solutions' AI enablement practice. The program it serves is in ai third-party risk management and the full diligence framework in the AI vendor due diligence whitepaper.

What should the questionnaire cover?

Domain	What you need to learn
Data handling	Training use, retention, residency, encryption, deletion, subprocessors
Model governance	Which models, how evaluated, how changed, how monitored, how documented
Security controls	Injection and leakage defenses, access control, secrets, tool permissions
Logging and observability	What is stored, for how long, who can see it, redaction
Incidents	AI incident definitions, history, notification, response
Compliance	Applicable certifications and attestations; regulatory terms available
Explainability and oversight	Support for explanations, human review, and contest
Commercial	Change notice, deprecation, exit, service levels

What questions cover data handling?

• Is customer data, including prompts, uploads, and outputs, used to train, tune, or improve any model? Under what terms can it be? How is opt-out enforced and verified?
• How long are inputs, outputs, and intermediate data retained, by tier? Can retention be set to zero?
• Which regions process and store data? Can residency be fixed?
• Which subprocessors, including model providers, receive customer data? Is the list published and updated with notice?
• How is data deleted on request or termination, and how is deletion evidenced?
• Is data encrypted in transit and at rest, including in vector indexes and logs?

Disqualifying for sensitive uses: training on customer data without a contractual prohibition, no subprocessor list, or retention that cannot be limited. Privacy practice is in ai data privacy compliance.

What questions cover model governance?

• Which models and versions are used, and are they pinned per customer?
• How is a model change evaluated before deployment? Are results shared?
• What notice is given for model changes and deprecations?
• How is production quality monitored, and what thresholds trigger action?
• Is there a model or system card for the product?
• How are fairness and safety tested, and how often?

Disqualifying for decision-influencing uses: no change notice, no evaluation evidence, or no documentation. Model governance expectations are in ai model risk management and documentation in what is a model card.

What questions cover security controls?

• How are instructions separated from customer content to resist prompt injection? What adversarial testing is performed, and how often?
• How is retrieval scoped to each user's permissions? Is it verified at query time?
• What actions can the product take in customer systems, and how are they gated and scoped?
• How are customer credentials and API keys stored and used?
• Has the AI functionality been penetration tested? Are reports available?
• How is multi-tenant isolation enforced in indexes, memory, and caches?

Disqualifying: injection defense described only as prompt instructions, or no permission enforcement in retrieval. Defense standards are in the prompt injection defense checklist and testing in ai penetration testing.

What questions cover logging and incidents?

• What is logged from prompts, outputs, and tool calls? For how long? Who can access it? Is sensitive content redacted?
• Can customers access or export logs for their own audit needs?
• How are AI incidents defined, including leakage, harmful output, and unauthorized actions?
• What is the incident history for the AI functionality?
• What is the notification timeline and content for incidents affecting customer data or decisions?

Disqualifying: raw prompt logging with broad access and no redaction. Incident expectations are in ai incident disclosure.

What questions cover compliance, explainability, and commercial terms?

• Which certifications and attestations cover the AI functionality specifically, not only the hosting?
• Are regulatory terms such as business associate or service provider agreements available, and for which tiers?
• Can the product provide explanations for outputs that influence decisions, and support human review and contest?
• What are the deprecation and exit terms, including data return in usable formats?
• Are quality service levels available where quality is measurable?

Sector terms are in ai and hipaa business associate agreements and contract practice in how to negotiate an ai development contract.

How do you verify answers?

Compare answers with the contract terms for the tier you will use; inspect the configuration options for training, retention, and logging; request certifications, penetration test summaries, and evaluation evidence rather than assurances; test tenant isolation and permission enforcement in a pilot; and confirm subprocessor lists against network egress where feasible. Questionnaires filed without verification protect nobody. Pilot structure is in how to structure an ai pilot agreement.

How should the questionnaire be used over time?

At initial diligence proportionate to tier; at renewal; on material change such as new AI features, new subprocessors, or model changes; and after any incident. Embedded AI features in existing software trigger the questionnaire even without a procurement event. Ongoing management is in how to manage ai vendors.

What does sound use look like?

A healthcare company sends the AI questionnaire to a documentation vendor. Answers show an enterprise tier with training disabled, a business associate agreement available, a published subprocessor list naming the model provider, evaluation on model changes with notice, and redacted logs. The company verifies the configuration in a pilot, signs the agreement for that tier, and reassesses at renewal when the vendor adds a new model provider.

How FISTA Solutions helps with AI vendor assessment

FISTA Solutions helps clients build and run AI-specific vendor questionnaires, verify answers against terms and configuration, test isolation and permission enforcement in pilots, and integrate results into the third-party risk program, and it answers the same questions about its own delivery with evidence. The AI enablement practice leads vendor governance, AI agents are delivered with the controls the questionnaire asks about, and forward deployed engineers embed with client procurement and security teams. The record behind the approach is 150+ projects for 50+ companies.

To ask AI vendors the questions that decide risk, message FISTA on WhatsApp, or read the AI vendor due diligence whitepaper for the full assessment framework.