You're Building on Someone Else's AI
Know What Their Terms Actually Say
Every AI vendor has different data retention policies, training practices, and liability terms. This kit gives you a 60-question scored assessment, a vendor comparison matrix, and pre-researched profiles for the 7 providers your team is actually evaluating.
Version 2.2, updated 19 October 2026. Instant download by email after payment. Licensed to one organization, every employee included. Licence terms. 14-day refund, any reason. How refunds work. Download a sample (PDF), the opening pages of one document.
Why AI Vendor Selection Goes Wrong
Most companies evaluate AI vendors the same way they'd evaluate any SaaS tool: demos, feature comparison, pricing. Security and data handling are an afterthought, or they're delegated to someone who doesn't have a structured way to evaluate them. By the time legal looks at the contract, the engineering team has already committed to a vendor and a timeline.
The stakes are higher than with traditional software. AI vendors often retain prompts and completions by default. Some use your data for model improvement unless you explicitly opt out - and the opt-out process varies by vendor, is often buried in the API documentation, and changes when vendors update their terms. Training on customer data can create GDPR obligations, HIPAA liability, and confidentiality risks that your security review should catch before you sign.
The 12 deal-breaker questions are the ones that matter most. They're the questions that identify vendors whose data practices create liability you can't accept - training on your data without explicit consent, breach notification timelines longer than your regulatory requirements, vendor claims to IP rights over your inputs and outputs. Most vendor evaluations never ask them.
The pre-researched profiles for OpenAI, Anthropic, Google, AWS Bedrock, Azure OpenAI, Cohere, and Mistral save the research that would otherwise take your team days per vendor. Data retention defaults, training policies, BAA availability, compliance certifications - already documented.
Everything You Need to Evaluate AI Vendors
Built by a CISSP/CISM-certified security professional. Not a summary of blog posts.
60-Question Scored Assessment
Six categories: data handling and privacy, security controls, model transparency and control, incident response and reliability, contractual and legal, and AI-specific risk. Each answer scored 0 to 2; 12 critical items are potential deal-breakers if scored 0.
Vendor Comparison Matrix
Side-by-side comparison template for up to 5 vendors. Score, rank, and present findings to leadership with a clear recommendation framework.
7 Pre-Researched Provider Profiles
OpenAI, Anthropic, Google, AWS Bedrock, Azure OpenAI, Cohere, and Mistral. Data retention, training policies, compliance certifications, and contract gotchas - already documented.
DPA Requirements Checklist
45-item checklist covering everything your Data Processing Agreement needs. Subprocessor clauses, data deletion, breach notification, and cross-border transfer requirements.
12 Deal-Breaker Questions
The questions that separate acceptable vendors from liability risks. Training on your data, where your data is processed, breach notification timelines, and who owns AI outputs.
Vendor Evaluation Guide
Step-by-step process for running the assessment: how to gather information, how to score, when to escalate, and how to present findings to leadership and legal.
Built for Teams Choosing AI Providers
If your company is evaluating AI vendors, this kit turns weeks of research into a structured decision.
Engineering Leads
You're evaluating AI vendors for your product. Leadership wants a comparison. This kit gives you the framework to assess security, not just features.
Security Teams
Procurement dropped an AI vendor on your desk for review. The 60-question assessment and DPA checklist give you a structured evaluation instead of ad-hoc Googling.
CTOs & VPs of Engineering
You're choosing between OpenAI, Anthropic, and Google. The pre-researched profiles and comparison matrix give you a defensible decision - not a gut call.
60 questions. 7 provider profiles. One clear decision.
- 60-question scored assessment (6 categories, 12 deal-breakers)
- Vendor comparison matrix template
- 7 pre-researched AI provider profiles
- 45-item DPA requirements checklist
- Vendor evaluation guide with scoring rubric
- Single-user commercial license
Instant download. 5 Word documents (.docx), with the Markdown source of each in a markdown-source folder for Notion, Confluence, or a wiki.
Questions
Is this just for OpenAI?
No. The 60-question assessment framework works for any AI vendor - cloud APIs, self-hosted models, or embedded AI features. The kit includes 7 pre-researched profiles (OpenAI, Anthropic, Google, AWS Bedrock, Azure OpenAI, Cohere, Mistral), but the assessment applies to any provider.
How often should we re-evaluate?
Quarterly for critical vendors (those processing customer data or PII), annually for others. AI vendor terms change frequently - the monitoring section of the evaluation guide sets monthly, quarterly, and annual checks plus triggered reviews.
What if a vendor fails the assessment?
The kit includes a risk acceptance and mitigation path. If a vendor scores poorly on data retention but is otherwise the right technical fit, the DPA requirements checklist shows you what contractual protections to negotiate, and the scoring guide requires a signed risk acceptance and documented compensating controls. Not every failure is a disqualifier - but you should be making that decision consciously, not accidentally.
Does this cover self-hosted and open-source models?
Yes. The 60-question assessment framework applies to any AI provider - cloud API, self-hosted model, or open-source weights. For a self-hosted model, you answer the questions from your own deployment and mark the ones that do not apply N/A. The kit has no separate guidance on model supply chain or weight integrity checks.
We're a small team - who should run this assessment?
The evaluation guide is written for whoever handles security at your company. At an early-stage startup that might be the CTO or a senior engineer. At a larger company it's likely the security team with input from legal. The guide covers gathering information, scoring, escalation, and presenting findings to stakeholders. It does not include a RACI matrix.
David A. Moline, CISSP | CISM
Your AI automation, built by someone who secures DoD systems.
Stop Guessing About AI Vendor Risk
Your AI vendor's terms matter more than their benchmarks. Know exactly what you're agreeing to before you build on their platform.