AI agents · OpenClaw · self-hosting · automation

Quick Answer

How to Audit an AI Vendor for EU AI Act Compliance

Published:

The Short Answer

Ask for four artefacts, not four assurances:

  1. Training-content summary (AI Office template)
  2. Dated model evaluation + adversarial testing records, per model version
  3. Systemic-risk assessment with documented mitigations
  4. Serious-incident reporting procedure with a named owner

Establish role first. Provider or deployer? You may be a provider without knowing it.

Exposure: up to €15M or 3% of global turnover for GPAI provider breaches — including answering a regulator incompletely.

Last verified: August 31, 2026.

Why This Became Urgent

GPAI obligations became enforceable on August 2, 2026. Roughly four weeks later, the EU AI Office issued its first formal requests for information to providers of the most advanced models, covering security/evaluation/monitoring practices and training-content summary compliance. (Detail: first enforcement RFIs.)

The frontier labs will survive this comfortably. The exposed cohort is the middle — companies that fine-tune, wrap or redistribute models into the EU, hold obligations they haven’t mapped, and have no compliance function. They won’t be in the first RFI wave. They’ll be in the one that follows a competitor or NGO complaint.

If you buy AI, your vendors are in that cohort.

Step 1 — Determine Who Is The Provider

This is the question everything else depends on, and it is the one most procurement processes skip.

  • You use a vendor’s hosted model as-is: the vendor is the provider, you are a deployer. Your duties are real but narrower — human oversight, input data relevance, transparency to affected people, log retention.
  • You fine-tune a model and place it on the EU market under your own name: you may be a provider, with the full GPAI obligation set. Substantial modification transfers provider status. Companies acquire this accidentally and routinely.
  • Your vendor fine-tunes an upstream model and sells it to you: your vendor is a provider, and they may not know it either. Ask them directly and note the answer.

Write the determination down with reasoning. That document is itself an artefact.

Step 2 — Demand The Training-Content Summary

Every GPAI provider placing a model on the EU market must publish a sufficiently detailed summary of training content, using the AI Office’s template.

Ask for the URL. Then evaluate it against a simple test: does it describe categories and sources specifically enough that a rightsholder could tell whether their material was likely used? If it says “publicly available web data and licensed datasets” and stops, it is a placeholder.

Observed practice varies widely as of August 2026. OpenAI publishes training-data summaries and describes provenance via Content Credentials, C2PA metadata and SynthID. Anthropic committed to text watermarking applied globally at launch, with a detection API promised. Meta signed the EU Code of Practice on Transparency of AI-Generated Content. Google and Meta both committed in July 2026 to transparency and watermarking tooling.

A vendor with none of this and no timeline is a finding.

Step 3 — Ask For Evidence, Not Policy

Every serious vendor has a published safety framework. Almost none of that is evidence.

Ask for:

  • Model evaluation reports dated and tied to the specific version you will use
  • Adversarial testing / red-team results with methodology and findings, not a summary paragraph
  • The systemic-risk assessment and the mitigations actually implemented
  • Cybersecurity measures protecting model weights and infrastructure

The diagnostic question: “Which document shows the evaluation you ran on version X before release, and what date was it signed?”

Vendors who have done the work answer in days. Vendors who have a policy and no practice answer with a trust-centre link. That distinction is the entire audit.

Step 4 — Test Incident Handling

Serious-incident reporting is an obligation with a clock attached, and clocks require owners.

Ask: who detects, who decides it is reportable, what the internal escalation path is, what the notification timeline to you is, and whether they have ever exercised it.

“We would handle that appropriately” means no process exists.

Step 5 — Get It Into The Contract

Turn the audit into enforceable terms:

  • Compliance warranty covering the specific obligations for their role
  • Notification duty if they receive an RFI or open a formal proceeding touching your use
  • Audit rights on the artefacts above, at reasonable cadence
  • Change notification for material model changes, since substantial modification can shift provider status
  • Indemnity for regulatory penalties arising from their non-compliance

Note the limit honestly: indemnities allocate cost. They do not discharge your own deployer duties. A regulator will not accept “our vendor indemnified us” as a defence to your obligations.

The Realistic Effort Budget

For a single significant AI vendor, this is roughly a half-day of questions and a week of waiting. That is a proportionate cost against a 3%-of-turnover ceiling and a regime whose information-integrity provision punishes disorganised answers as hard as substantive breaches.

Do it once per vendor, refresh annually or on material model change, and keep the artefacts. When your own RFI arrives, the folder is the answer.

Sources