A BigLaw Firm Just Asked How Your AI Contract Analysis Tool Keeps Logs of Its Own Outputs and Who Can Access Them: Answering the Article 12 Record-Keeping Questions

The questionnaire arrived from the IT and risk committee of a Magic Circle law firm. They're evaluating your AI contract analysis tool for rollout across their European practice groups. Everything was going smoothly until the senior IT auditor forwarded a new section:

"Please describe how your AI system automatically generates and retains logs of its operations, in accordance with Article 12 of the EU AI Act. Specify what is logged, retention periods, access controls, and how logs can be made available to regulatory authorities on request."

This is an Article 12 question. It's one of the quieter corners of the EU AI Act's requirements for high-risk AI systems — less talked about than technical documentation or human oversight — but procurement teams at large law firms are increasingly asking for it. Here's how to answer every dimension of it.

---

What Article 12 of the EU AI Act Actually Requires

Article 12 requires that high-risk AI systems be designed with automatic log generation capabilities. The system must automatically record events throughout its operational lifetime — not just when something goes wrong, but as a standard feature of normal operation.

AI systems used for legal document review, contract analysis, and legal research may be classified as high-risk under Annex III depending on the specific use case (particularly if used in judicial or administrative proceedings). Even where a system sits in a gray zone, enterprise law firm buyers are increasingly applying the Article 12 standard to all AI vendor documentation as a best practice requirement.

The logs that Article 12 requires must enable:

1. Reconstruction of the AI system's operation after the fact
2. Identification of situations where the system posed or poses a risk
3. Appropriate traceability of the AI system's operation over its lifetime

The logs must be kept by the provider for the period required under their quality management system (Article 17), and by the deployer for at least six months after operational use ends — unless sectoral law requires longer retention.

---

The Questions Buyers Ask — and How to Answer Them

"What does your system automatically log during each session?"

This is the threshold question. Be specific. A well-architected AI contract analysis tool should log at a minimum:

• Session metadata: User ID (or role), timestamp of session start and end, document IDs processed
• Query or task inputs: The specific analysis request submitted (e.g., "identify all indemnification clauses" or "flag non-standard limitation of liability language")
• Output metadata: A record that an output was generated, the output category, confidence indicators if applicable, and whether the output was reviewed or acted upon
• Human override events: If a reviewer rejected or modified an AI output, that action should be recorded
• Error events: Any system errors, timeouts, model unavailability, or processing failures during the session

A strong answer: "Our system logs all input queries, output event types, session metadata, and user review actions at the application layer. Logs are structured and machine-readable. Full output content is not stored by default in audit logs — it is retained in the document session record, which is linked to the log entry by session ID."

"How long are logs retained?"

Retention periods should be tied to your documented QMS policy. Article 12 does not specify a fixed period, but Article 18 requires providers to retain their technical documentation for ten years after the system is placed on the market or put into service.

For operational logs (as distinct from system documentation), best practice for law firm clients is:

• Provider-side operational logs: Minimum 12 months, configurable to 36 months for enterprise clients
• Deployer-side session records: Minimum 6 months post-session, often 12–24 months in legal sector deployments
• Incident and error logs: Minimum 3 years, to support any post-incident review or regulatory inquiry

State your policy clearly and offer to customize retention for enterprise contracts: "Our standard log retention is 12 months. Enterprise clients may configure retention up to 36 months via [settings/contract term]. Logs are immutable once written — they cannot be modified or deleted by end users."

"Who can access the logs, and what are the access controls?"

Law firm procurement teams ask this because their professional conduct obligations (SRA, Bar rules) require them to understand who inside your organization can see their client-related processing records.

Strong answer: "Audit logs are accessible to: (1) the deploying organization's designated administrators via the management console; (2) our internal security and compliance team for incident investigation, under access policies requiring written justification; (3) regulatory authorities with a lawful basis for access, via a documented request-handling process. Our employees cannot access log content on an ad hoc basis. All internal log access is itself logged with user ID and justification."

"Can logs be made available to competent authorities on request?"

Article 12, read with Article 21 (cooperation with competent authorities), requires that logs be available to market surveillance authorities and, in the context of legal proceedings, to courts and judicial bodies.

Answer: "Yes. Our log access and disclosure process is documented in our compliance policy. We respond to authority requests via our legal department under our Data Requests Policy, which provides for [X-day] response times and requires a written formal request or court order. Logs are exportable in [format: JSON/CSV] for authority review. We will notify affected deployers of any authority log request unless prohibited by the request itself."

"How do logs support traceability of a specific AI-assisted output after the fact?"

This is the audit reconstruction question — can you go back and understand exactly what the AI did in a specific matter? For a law firm that has a client dispute or a professional conduct investigation, this matters enormously.

Answer: "Each AI output is assigned a unique session reference ID that links to: the input document(s) processed, the query submitted, the model version in use at the time, the output event record, and any reviewer actions. Using the session ID, an authorized administrator can reconstruct the sequence of AI operations for that matter within [time period]. We can provide sample log exports demonstrating this traceability on request."

"Are logs protected against tampering or unauthorized modification?"

Answer: "Yes. Audit logs are written to an append-only log store. No application-layer user, including administrators, can modify or delete log entries once written. Log integrity is verified [periodically/continuously] using [checksum/hash verification method]. Log storage is separate from the application database and requires elevated credential access to read."

---

The Intersection With Your Quality Management System

Article 12 doesn't operate in isolation. Your logging architecture should be cross-referenced in your Article 17 Quality Management System (QMS) documentation:

Law firm IT auditors who have reviewed multiple AI vendor questionnaires will cross-check these articles against each other. If your Article 12 logging answer and your Article 17 QMS documentation are inconsistent, expect a follow-up.

---

What Law Firms Are Really Checking For

When a large law firm asks Article 12 questions, they are running a dual-purpose check:

1. EU AI Act compliance due diligence — if they classify this tool as high-risk for any of their use cases, they need to confirm you've met provider obligations before they can deploy it
2. Professional conduct and privilege risk — if an AI tool processes privileged matter documents, they need to know exactly what is retained, by whom, and under what access controls

Your Article 12 answer needs to satisfy both. The technical logging requirements and the confidentiality framing are not in conflict — a well-designed logging system can be both complete and appropriately access-controlled.

---

Bottom Line

Article 12 questions in legaltech procurement questionnaires are a proxy for a deeper question: can we audit what your AI did, if we ever need to? The firms asking this are thinking about client disputes, regulatory investigations, and professional conduct reviews — not just EU AI Act box-ticking. Answer with specificity, and address the log access and confidentiality dimensions in the same breath.

Try Complizo free at complizo.com — paste in your customer's Article 12 questionnaire and get a complete, sourced answer set in minutes.