AI training data provenance: what to ask before you license a dataset

Training data provenance is the documented history of a dataset: where it came from, how it was obtained, under what terms, what was done to it and how it reached you. Before you license a dataset, ask for the record behind each of those five links, not a summary of them. If a licensor cannot show the record, treat the rights as unproven, whatever the contract says.

This guide is for data and research leads who have to sign off on a dataset and then defend that decision later.

What is training data provenance?

Provenance is the chain of evidence that connects a dataset on your servers to the people and organisations who created it. It answers five questions: who made the data and where did it live, how did the licensor get it, what rights attach to it, what was done to it on the way, and how did it arrive with you.

Each question has a different owner and a different document. That is the point: provenance is not one certificate. It is a set of records, each held by whoever did the work at that link.

A useful definition for an internal policy:

Provenance is the ability to show, for every dataset used in training, evaluation or fine-tuning, how it was obtained and under what terms, with a record for each step that a third party could inspect.

Two things are not provenance, although they are often sold as such: a licence agreement on its own is a promise about rights, not proof of them, and a dataset card is only as reliable as the records it was written from.

Why does provenance matter more now?

Three things changed between mid 2025 and mid 2026. A US court separated the lawfulness of training from the lawfulness of acquisition, and the acquisition question cost one company 1.5 billion dollars. The EU AI Act’s obligations for general-purpose model providers took effect, with a mandatory public summary of training content. And a peer-reviewed audit showed how often public dataset licences are wrong.

Bartz v. Anthropic: training and acquisition are separate questions

In June 2025, US District Judge William Alsup ruled that Anthropic’s use of books to train Claude was fair use under US copyright law. In the same order he found that copying and storing more than 7 million pirated books in a “central library” infringed the authors’ copyrights and was not fair use, and he set a trial on damages, according to Reuters. The order included this line, quoted by Reuters: “This order doubts that any accused infringer could ever meet its burden of explaining why downloading source copies from pirate sites that it could have purchased or otherwise accessed lawfully was itself reasonably necessary to any subsequent fair use.”

The case did not reach trial. Anthropic agreed to a 1.5 billion dollar class settlement, which received final approval on 20 July 2026, the largest known settlement of a US copyright case, per Reuters. NPR reported that more than 300,000 writers were involved and that the payout works out at around 3,100 dollars per title.

The lesson for a sourcing lead is narrow: a favourable answer on whether training is a permitted use does not cover you if the copy you trained on was not lawfully obtained. Acquisition is its own link, with its own record.

The EU AI Act: a copyright policy and a public summary

Article 53 of the EU AI Act applies to providers of general-purpose AI models from 2 August 2025. It requires them to keep technical documentation of the training and testing process, to put in place a policy to comply with EU copyright law, including identifying and honouring rights reservations made under Article 4(3) of Directive 2019/790, and to publish a “sufficiently detailed summary” of the content used for training, using a template from the AI Office.

The European Commission’s template FAQ sets out what the summary contains. It has three sections: general information about the model and the modalities of training content; a list of data sources, broken down into publicly available datasets, private datasets, data scraped from online sources, user data and synthetic data; and relevant data processing aspects, including measures taken to respect copyright and to remove illegal content. For scraped data, providers must name the crawlers used, the collection period and the top 10 percent of domains scraped. Models placed on the market before 2 August 2025 have until 2 August 2027 to publish, and the Commission states that enforcement by the AI Office can begin on 2 August 2026, with fines of up to 3 percent of worldwide annual turnover or 15 million euros, whichever is higher.

If you build on or fine-tune a general-purpose model, the FAQ says your summary only needs to cover the data used for the modification, with a reference to the original model’s summary. That means a licensed private dataset you use for fine-tuning is exactly what your own summary will have to describe. Our guide to EU AI Act training data requirements goes through what to ask every supplier so that section is easy to write.

Public dataset licences are often wrong

Provenance problems are not confined to scraped data. The Data Provenance Initiative, described by MIT Sloan, audited more than 1,800 text datasets drawn from 44 widely used collections. Its findings, published in Nature Machine Intelligence, include that more than 70 percent of licences for popular datasets on GitHub and Hugging Face were unspecified, and that 66 percent of the Hugging Face licences the team re-annotated were in a different use category from the author’s original licence, often labelled as more permissive. The authors attribute part of this to contributors mistaking the licence on a code repository for the licence on the data inside it.

So “it is on Hugging Face under an open licence” is a claim to verify, not a fact to rely on.

What record do you need for each link in the chain?

Treat provenance as five links, each with one question and one document that answers it. Origin: who created it and where did it live. Acquisition: how did the licensor get it. Rights: what can lawfully be licensed. Processing: what changed between source and delivery. Delivery: how did it reach you, and what happens at term end.

Link	The question	The record that answers it	Who holds it
Origin	Who created the data, in which system, over what period?	Source record: system of record, date range, creators, volume by type	Data owner
Acquisition	How did the licensor come to hold it?	Acquisition record: ownership, assignment, licence-in, or the owner’s own authority	Licensor
Rights	What can lawfully be licensed for training?	Rights memo: chain of title, third-party terms, consent, restrictions	Licensor, reviewed by your counsel
Processing	What was removed, de-identified, filtered or transformed?	Processing log: method, tests, exclusions, versions	Licensor or de-identification provider
Delivery	How did it reach you, who accessed it, when will it be deleted?	Delivery record and deletion certificate	Both parties

Two of these links deserve a note. For origin, ask for the system, not a category: “support tickets” is a category, while “one ticketing system, 2019 to 2025, written by the company’s own service team” is an origin, and it tells you who the creators were, which drives the consent question later. Acquisition is the link Bartz turned on: if the licensor generated the data in its own operations, the record is short; if it collected, bought or aggregated the data from others, you need those agreements and what they allow.

How do the main data sources compare on provenance?

Four sources cover most training data: scraped web, open corpora, licensed private data and synthetic data. They differ less in quality than in what you can show. Scraped data rarely proves rights or consent. Open corpora often have mis-stated licences. Licensed private data can prove both, if the licensor’s records exist. Synthetic data inherits the provenance of whatever generated it.

Source	Rights shown	Consent shown	Opt-outs handled	Reproducible later
Scraped web	Rarely; relies on fair use or TDM exceptions	Rarely	Only if crawl honoured reservations and you logged it	Partly; the web changes under you
Open corpora	Partly; licences often unspecified or wrong per the audit above	Rarely	Rarely	Yes, if you pin the version
Licensed private data	Yes, if the rights memo and acquisition record exist	Yes, if the memo covers personal data and legal basis	Yes, written into the licence	Yes; a fixed, versioned delivery
Synthetic data	Inherits the rights of the generating model and its seed data	Inherits	Inherits	Yes, if seeds and prompts are kept

Two cautions. “Licensed” is only as good as the licensor’s own chain: a licence from an aggregator who scraped the data moves the problem one step away from you, it does not remove it. And synthetic data is not a provenance shortcut: if the generating model or the seed documents carry unclear rights, those questions travel with the output. Record the generator, its version and the seed data.

How do you check a dataset’s provenance before you license it?

Run a short, fixed procedure for every dataset, however small. Ask for the five records first, under NDA. Read the rights memo before the sample. Check the acquisition story against the origin story. Test the processing claims on the sample. Then write a one-page provenance note and file it with the licence.

1. Sign a mutual NDA before any detail is exchanged. You will need to see contracts, system names and sometimes customer identities to verify rights. None of that should move without an NDA. A licensor who wants to send a sample before the NDA is telling you something about their process.
2. Ask for the five records by name. Source record, acquisition record, rights memo, processing log, delivery plan. Do not accept a deck or a one-paragraph description in place of any of them. Missing records are a finding, not a gap to fill with a warranty.
3. Read the rights memo before the sample. A strong sample with weak rights wastes a week of evaluation. Check that the memo names the owner, lists the third-party terms reviewed, states the legal basis for any personal data, and lists restrictions.
4. Cross-check acquisition against origin. If the origin record says “created by our service team” and the acquisition record says “aggregated from partner sites”, one of them is wrong. Where data came from third parties, confirm the agreements allow onward licensing for AI training.
5. Test the processing claims on the sample. If the log says personal data was removed, search the sample for email addresses, names in signatures and identifiers. If a filter removed third-party content, look for copyright notices and licence headers. A log you cannot reproduce on the sample is a reason to pause.
6. Map the dataset to your own obligations. If you are in scope of the EU AI Act, decide now which section of your training content summary this dataset will sit in and what you will need from the licensor to write it.
7. Write a one-page provenance note and file it. Dataset name and version, the five records and where they are stored, who reviewed them, open questions, and the decision. This is the document you will reach for in two years.

What should you ask a licensor?

Ask questions that have documentary answers. Who created this data, and in which system? How did you come to hold it? Which agreements limit what you can license, and have you read them? Does it contain personal data, and on what basis can it be used for training? What did you remove, and how did you test that? Where are the records?

Use this as a checklist in the first call after the NDA.

Origin

• Which systems did the data come from, over what date range, and what volume by type?
• Who created the content: employees, customers, contractors, the public?

Acquisition

• Did the licensor generate the data itself, or collect it from others? If from others, under which agreements, and do they permit onward licensing for AI training?
• Were any parts purchased, scraped or received under a licence-in? Can we see those terms?

Rights

• Who owns the data, and can the licensor grant training rights? Is there a written chain of title?
• Which third-party terms were reviewed: open-source licences, vendor terms, customer contracts, non-disclosure duties?
• Does the data contain personal data? What is the legal basis for using it, and how was de-identification done and tested?
• Are there export controls, regulatory limits or sector rules that restrict use or destination?
• Has anyone opted out, objected or reserved rights over any part of the data? How was that honoured?

Processing

• What was removed, filtered or transformed between the source system and the delivery? Is there a versioned log?
• Can we run the same checks on the sample?

Delivery and term end

• How will the data be delivered, into what environment, and will access be logged?
• Is there a deletion commitment at term end, with a date and a certificate?
• Will the licensor help us describe this dataset in a regulatory summary?

A licensor who has done the work will answer most of these in writing. One who answers with assurances rather than documents has told you where the risk sits.

What should the licence say about provenance?

The licence should turn the records into obligations. It should warrant the licensor’s rights and name the memo behind the warranty, attach the provenance records as schedules, oblige cooperation with regulatory summaries, forbid re-identification, fix the term and the deletion date, and say what happens if a rights problem surfaces later.

• Rights warranty, tied to a document. The licensor warrants that it holds the rights it grants, and the rights memo is referenced as the basis. A warranty with no record behind it is only as good as the licensor’s balance sheet.
• Provenance records as schedules. Source record, acquisition record, processing log and delivery plan, attached, versioned and dated.
• Permitted uses, stated. Pre-training, fine-tuning, evaluation, safety testing, or a mix. Nothing outside the list.
• No re-identification, no redistribution. No attempt to identify people in the data, and no sharing beyond named staff.
• Cooperation for regulatory summaries. The licensor supplies, on request, what you need to describe the dataset in a training content summary or technical documentation.
• Term and deletion. A defined term, deletion of every copy within a fixed period after it ends, and a signed deletion certificate. Say whether models trained during the term may remain in use; that is a negotiated point, not a default.
• Remedy for a rights defect. Notice, removal of the affected part, and who bears the cost.

Our companion piece on how data owners license company data to AI companies covers the same clauses from the other side of the table.

Where does a written rights memo fit in your records?

A rights memo is a written statement, produced before any licence is drafted, of what can lawfully be licensed and why. It covers six areas: chain of title, third-party terms, personal data and consent, restrictions, use limits, and delivery and deletion. For an AI team it is the rights link in the chain, filed next to the licence and the provenance note.

At Liciva, every deal follows five steps in a fixed order: a mutual NDA, a sample scrubbed of personal data, a rights check, a scoped licence, then delivery and payout. The rights check at step three produces the memo, and its six areas are published rather than described as a vague promise. The memo is written before a licence is drafted, so by the time an AI team sees terms, the rights link already has its document.

That does not replace your own review. Your counsel should read the memo, check its reasoning against the underlying agreements where that matters, and record the conclusion in your provenance note. Data owners who want to understand what the check involves from their side can read the rights check explained.

This article is general information, not legal advice. Rules differ by jurisdiction and change often; check with your counsel before relying on any of it.

Source data you can actually defend

If you need training or evaluation data with provenance you can show, tell us what your model needs. Samples come under NDA, so you can evaluate before you commit. You get one contract with Liciva and a written rights memo your counsel can rely on, covering chain of title, third-party terms, consent and restrictions, before any licence is signed. Start a sourcing request.