Skip to content

Get the monthly AI briefing: what matters for your business, in plain English.Subscribe

Menu

Practical GuideAI GovernanceBuilding Capability

What to do after an AI-related data incident

A calm, practical response framework for UK businesses after an AI-related data incident, from the first hour to the lessons learned.

Phil Evans

Co-Founder & CEO

Most AI-related data incidents begin with something ordinary: a document pasted into the wrong tool, an assistant that can see more than it should, a setting nobody checked. What matters is the response in the first few days.

This article sets out a calm, practical response framework for UK businesses, from the first hour to the lessons learned, and explains what makes an AI incident different from other data breaches.

What an AI-related incident can look like

Under UK GDPR, a personal data breach is a breach of security leading to the accidental or unlawful destruction, loss, alteration, unauthorised disclosure of, or access to, personal data. AI can be involved in several ways:

  • Data put into the wrong tool. In 2023, Samsung temporarily restricted staff use of generative AI after employees entered sensitive company information, including source code, into ChatGPT.
  • A flaw in the AI provider’s own service. In March 2023, a bug in ChatGPT exposed some users’ chat histories and payment information. Italy’s data protection authority later fined OpenAI €15 million for several breaches of the GDPR, including €320,000 for failing to notify it of that incident.
  • An assistant turned against its user. In 2025, researchers showed that a crafted email could make Microsoft 365 Copilot send sensitive data to an attacker without the user clicking anything. Microsoft fixed the flaw before any known exploitation.
  • A home-built AI tool that leaks. Apps produced quickly with AI coding tools can go live without proper access controls and expose the data they hold.

Not every one of these is a personal data breach. Samsung’s case involved confidential business information rather than personal data, which raises commercial and contractual questions instead. Equally, staff entering personal data into an unapproved public AI tool may amount to an unauthorised disclosure, depending on the tool’s terms and what happens to the data. The safest approach is to treat it as a potential breach and assess it, rather than assume either way.

Everyday situations that can lead to the same problem

Most AI-related incidents do not start with an attack. They start with someone trying to get their work done. The situations below are illustrative, but each is the kind of thing that can happen in an ordinary working week, through informal workarounds and through approved tools alike.

Informal workarounds

  • The late-night grievance reply. An HR adviser pastes an employee’s grievance letter, including names and details of a health condition, into a free chatbot on their personal phone to help draft a response. Health information is special category data, so the potential risk to the individual is higher.
  • The deadline shortcut. An accountant uploads a client’s bank statements to a consumer AI app to categorise transactions quickly before a filing deadline. The firm has no contract with the app’s provider and does not know where the data is stored or whether it will be used for training.
  • The helpful browser extension. A paralegal installs an AI extension to summarise long documents. Every page they open in the firm’s matter system, including client correspondence, is sent to the extension’s provider.
  • The unofficial note-taker. A manager uses an AI note-taking app on their own phone to record a sensitive performance conversation. The recording and transcript sync to the app’s cloud, outside the organisation’s control, and the employee was not told.

Approved tools and formal processes

  • The assistant that sees too much. An approved AI assistant connected to the shared drive answers a junior employee’s question by quoting a salary spreadsheet. Nobody intended this: the folder permissions had simply been too broad for years, and the assistant made the problem visible.
  • The automatic meeting summary. An approved transcription tool, set up for internal meetings, is still running when a client joins a call. Its summary, containing the client’s personal and financial details, is emailed automatically to every invitee, including an external contractor.
  • The feature nobody switched on. A supplier update adds an AI feature to the firm’s customer relationship system, enabled by default. Customer records start being processed by a third-party model under terms nobody has reviewed.
  • The quick internal build. A team uses an AI coding tool to build a simple client portal. It works, so it goes live, but nobody checks who can see which records.

In each case, the person involved was trying to help. Whether a reportable breach has occurred will depend on the facts, but every one of these should trigger the response framework below.

A response framework

First hour: Contain and preserve

  • Stop the flow. Suspend the account, disconnect the integration, revoke API keys or access tokens, or pause the agent. For an AI agent, switching off its access is usually faster and safer than trying to correct what it is doing.
  • Preserve the evidence. Do not delete chats, prompts or logs, even with good intentions. Export prompt histories and audit logs before they expire, and take screenshots. AI tools often keep records for limited periods.
  • Start an incident log. Note who found the problem, when and what has been done. The time you became aware matters for the 72-hour deadline.
  • Bring in the right people. Usually the incident owner, whoever leads on data protection, IT or security and, where needed, legal advice. If you have cyber insurance, check whether it includes an incident helpline.

First 24 hours: Establish the facts

  • What data, and whose? Identify the categories of data, roughly how many people are affected and whether sensitive data, such as health or financial information, is involved.
  • Where did it go? For data entered into an AI tool, find out which tool and account were used, whether the provider keeps inputs, for how long and where, and whether it can use them to train its models. Business and consumer versions of the same tool can have different terms.
  • Ask the provider. Request in writing that the data is deleted, and ask for confirmation. Keep the correspondence.
  • Assess the risk to people. Consider the likely consequences for the individuals concerned, not just for the business. The ICO’s online self-assessment tool helps you decide whether a breach is reportable.

Within 72 hours: Decide and notify

If the breach is likely to result in a risk to people’s rights and freedoms, you must report it to the ICO without undue delay and within 72 hours of becoming aware of it. The guidelines the ICO points to treat a business as aware once it has a reasonable degree of certainty that a security incident has compromised personal data. You do not need every detail first: UK GDPR allows information to be provided in phases, and the ICO’s online form lets you give basic information and follow up later. Its helpline is 0303 123 1113. Failing to notify a reportable breach can lead to a fine of up to £8.7 million or 2% of global turnover.

If the risk to individuals is high, you must also tell them without undue delay. Explain in plain language what happened, what it means for them and what they can do.

Who you may need to tell

  • The ICO. When: Without undue delay and within 72 hours of becoming aware, if the breach is likely to result in a risk to people. Details can follow in phases. Basis: UK GDPR Article 33.
  • Affected individuals. When: Without undue delay, if the risk to them is high. Basis: UK GDPR Article 34.
  • The controller (if you are a processor). When: Without undue delay after becoming aware. Basis: UK GDPR Article 33(2).
  • Your sector regulator. When: As its rules require. FCA-regulated firms must tell the FCA anything it would reasonably expect notice of. Basis: For example, FCA Principle 11 and SUP 15.3.
  • Clients. When: As your contracts require. Basis: Contract terms and data processing agreements.
  • Your insurer. When: As your policy requires. Basis: Policy notification conditions.
  • Police (Report Fraud). When: If a crime such as fraud or a cyber attack is involved. Basis: Report Fraud, which replaced Action Fraud in England, Wales and Northern Ireland.
  • The NCSC. When: Voluntary, for cyber incidents. The NCSC notes that regulators may take timely engagement with it into account. Basis: GOV.UK report-cyber signposting service.

Two points are easily missed. First, record every breach in your own log, including those you decide not to report, with your reasoning. Second, if affected people are in the EU, EU GDPR notification duties may also apply, so take advice.

Following weeks: Fix and learn

  • Find the root cause. Was it a person, a tool setting, an agent with too much access or a supplier’s failure? Often it is more than one.
  • Fix the control, not just the instance. That might mean moving staff to an approved tool with business terms, narrowing an agent’s permissions, tightening document permissions behind an AI assistant or adding a review step.
  • Be ready for complaints. Since 19 June 2026, under the Data (Use and Access) Act 2025, organisations must have a process for handling data protection complaints and must acknowledge each complaint within 30 days.
  • Talk to staff without blame. People report problems sooner when they are not afraid of the response. Early reporting is what keeps an incident small.

What makes an AI incident different

  • The data may be in someone else’s system. Once information is in a third-party AI service, getting it back or deleted depends on that provider’s terms and cooperation.
  • The evidence can disappear. Prompt histories and agent logs may be short-lived or off by default.
  • The access may be borrowed. An agent or assistant usually acts with a person’s permissions, so the question is not only what the AI did but what that person could reach.
  • It can move fast. An agent can take many actions across several systems in seconds, which is why switching it off comes first.

Preparing before it happens

  1. Add AI to your incident plan. Name the AI tools in use, who can switch each one off and where its logs are kept.
  2. Know your tools. A current register of AI tools, uses and data makes the first 24 hours much faster.
  3. Check your AI contracts. Know each provider’s retention, deletion and breach notification terms before you need them.
  4. Turn on logging. Make sure prompt and activity logs are kept for long enough to investigate an incident.
  5. Tell staff what to do. A single, simple instruction works best: if you think something has gone wrong with an AI tool, stop and report it straight away.
  6. Rehearse it. A short desk exercise based on one of the scenarios above will expose gaps faster than any document review.

An AI-related incident is still a data incident. The law and the timetable are the same; what changes is where the data has gone and how quickly you can find out.

Sources checked October 2026. Notification thresholds, timescales and regulators’ rules may change, and sector rules vary; FCA-regulated firms should note that new operational incident reporting rules take effect on 18 March 2027. Case examples are drawn from public reporting and regulatory decisions and are summarised for illustration.

This article provides general information and does not constitute legal advice. In a live incident, take qualified legal advice promptly, particularly before notifying regulators or affected individuals.

References

UK law and the ICO

Other bodies to notify

Case examples

ShareLinkedInEmail

Keep learning

How-toAI Governance

How to vet an AI vendor before you sign

Six steps to check an AI tool or vendor properly before it touches your data, without turning procurement into a project.

Chris Rigby

Co-Founder & CRO · 14 Oct 2026

Building Capability

Put it into practice

The AI Jargon Buster

Starter PackAI Adoption

The AI Jargon Buster

Twenty words you will hear in every AI conversation, in plain English. What each one actually means, and a way to picture it

Isaac Ward

AI Practitioner · 14 Oct 2026

Free downloadGetting Started

Stay informed

A short monthly briefing on the AI developments that actually matter for your business. No sales pitches.

Monthly. Unsubscribe in one click. We never share your details.