Web scraping to train AI: ICO opens, EDPB tightens

Verifiable fact — In January 2024, the ICO launched a consultation whose response states that legitimate interests is, in practice, the only viable lawful basis to train AI on web‑scraped data. The EDPB (18 December 2024) strictly frames this choice and points to heightened transparency duties (GDPR Article 14). ico.org.uk — edpb.europa.eu.

The case

• United Kingdom — ICO: in its response to the “consultation series on generative AI” (chapter “The lawful basis for web scraping to train generative AI models”), the ICO confirms that “legitimate interests remains the sole available lawful basis” for web scraping to train generative models — provided the three‑part test (purpose/necessity/balancing) is met and transparency (Art. 14 UK GDPR) is ensured. See ICO — Lawful basis for web scraping.
• European Union — EDPB: the “Opinion on AI models: GDPR principles support responsible AI” (18/12/2024) clarifies when and how legitimate interests can be relied upon to develop or use AI models and highlights information duties where data are not collected from the individual (GDPR Art. 14). The EDPB also announces specific guidance on web scraping. See EDPB — Opinion on AI models.
• Applicable law — GDPR: Article 6(1)(f) (legitimate interests), Article 9 (special categories), Article 14 (information when data are not obtained from the data subject). Official text: EUR‑Lex — Regulation (EU) 2016/679.

Legal reasoning

1. ICO position (UK GDPR). The five other bases under Article 6(1) are “unlikely” in the context of scraping for model training; legitimate interests remains the only practicable basis, subject to a strict necessity test (showing no reasonable alternative, e.g., licensed datasets) and a balancing test that recognises the “high‑risk” and “invisible” nature of processing; transparency (Art. 14) is often lacking and undermines the balance. Source: ico.org.uk.
2. EDPB position (GDPR). Legitimate interests is not a “free pass”: it requires a specific purpose and strict necessity check; online publication is neither consent nor a blanket authorisation; full Article 14 notice is due “at the latest within one month” (limited exceptions, notably under Article 89); if Article 9 data may be incidentally collected, both an Article 6 basis and a derogation under Article 9(2) are required. Source: edpb.europa.eu.
3. Luxembourg angle — CNPD. The CNPD reiterates the legitimate‑interest triptych (specified purpose, necessity, documented balancing) and the accountability burden on controllers, including for indirect collection. See the CNPD page “Lawfulness of processing”.

For a primer on the regulation’s obligations, see key GDPR provisions.

Conclusion

Divergence in tone and applicability: the ICO opens a practicable path (legitimate interests as the only realistic basis, under strict conditions), while the EDPB narrows its use, emphasising Article 14 notice and Article 9 limits for model training.

What this changes in practice

• In the EU/Luxembourg, training in‑house models on large‑scale web data remains high‑risk if you cannot demonstrate scraping necessity versus licensed or first‑party alternatives, cannot deliver Article 14 notice on time (or rigorously justify a 14(5)(b) exception with Article 89 analysis), or may capture special categories (Art. 9) without a valid basis.
• In the UK, even if the ICO admits legitimate interests as a default in this context, the evidentiary bar is high: documented necessity, balancing that factors in “high‑risk/invisible processing”, and enhanced transparency (training/source registers, info pages, opt‑outs where possible).
• LU‑UK groups: avoid “exporting” the ICO logic into the EU. In the EU, prioritise licensed datasets, sector partnerships, or “controlled‑data training” architectures with Article 14 notices and access/objection handling (Arts. 15, 21). For operationalisation, explore our AI compliance governance services.

Example — A Luxembourg insurer training a semantic model from public forums: under GDPR, document why licensed data are insufficient; implement Article 14 notices (dedicated pages + webmaster notifications if needed), filter Article 9 categories (detect/mask health, origin, etc.), and offer an objection channel. Under UK GDPR, legitimate interests may still apply but needs a more robust necessity test and innovative transparency (source registers, public notices, rights interfaces).

Frequent pitfalls

1. “Public data = free to use”: false. Mere online publication is neither consent nor a lawful basis (Art. 6).
2. Overlooking Article 14: notice is due “at the latest within one month”, with narrow, justified exceptions.
3. Ignoring Article 9: scraping often captures sensitive data; without an Art. 6 basis and an Art. 9(2) derogation, processing is prohibited.
4. Failure to prove necessity: the ICO expects evidence that no realistic alternative exists (licensed datasets, direct collection).
5. Poor balancing: the “high‑risk” and “invisible” nature weighs heavily; without transparency and safeguards, the balance tilts toward data subjects’ rights.

Official sources

• ICO — The lawful basis for web scraping to train generative AI models (response to the consultation series). ico.org.uk
• EDPB — Opinion on AI models: GDPR principles support responsible AI (18 December 2024). edpb.europa.eu
• GDPR — Consolidated text (Arts. 6, 9, 14). eur-lex.europa.eu
• CNPD Luxembourg — Lawfulness of processing (professionals). cnpd.public.lu

To structure a responsible training project in Luxembourg, see our GDPR in Luxembourg guidance.

Luxgap regulatory expertise article. For personalised guidance on this topic, contact us or configure your online quote.