Engineering teams version their code carefully and then point it at a folder of documents, a live database table or a spreadsheet somebody maintains by hand. When results change, nobody can say whether the cause was the prompt, the model or the data underneath.
Dataset versioning closes that gap. It does not require elaborate tooling for a small team; it requires deciding that the data an AI feature depends on is part of the system rather than an input that happens to be lying around.
Know which data actually matters
Three kinds need versioning: the evaluation set you judge quality against, the reference material a retrieval system searches, and any examples included in prompts. All three change quietly and all three change behaviour.
Training data matters too where you fine-tune, though most teams touch that rarely. The evaluation set is the one that hurts most when it drifts, because it is the instrument you measure everything else with.
Version the evaluation set like code
Keep it in version control with the application. Every change should be a commit with a reason: a case added because a customer hit it, a case corrected because the expected answer was wrong, a case removed because it tested something no longer relevant.
When a score changes, the first question is whether the set changed. If the set lives in a spreadsheet on a shared drive, that question cannot be answered, and the score stops meaning anything.
Snapshot what retrieval searches
A retrieval corpus updates as documents are added, edited and removed, which means the same question genuinely returns different answers over time. That is usually correct behaviour, and it makes debugging impossible unless you record which version was searched.
At minimum, log the document identifiers and their revision with each request. Better, keep dated snapshots of the corpus so a past result can be reproduced exactly.
When the answer changes and the code did not, the data changed. The teams who cannot prove it spend a week finding out.
Hannah Berg, Lead AI Engineer, Engineered With AI
Record where data came from
For each dataset, note its source, when it was collected, who may use it, any licence or contractual limits and whether it contains personal data. That record answers questions from customers, auditors and your own team a year later.
It also prevents the awkward discovery that a corpus includes material a client provided for one purpose only, which is covered alongside other obligations in handling personal data in AI pipelines.
Keep it proportionate
A small team does not need a dedicated data versioning platform. Git for anything text-based and small, dated snapshots in object storage for anything large, and a manifest file recording which versions a release used will cover most situations.
The manifest is the important part: one file, committed with the code, saying exactly which dataset versions this deployment expects.
Tie versions to results
Every evaluation run should record the model, the prompt version and the dataset versions together. A score without that context cannot be compared against anything, which is how teams end up arguing about whether quality actually improved.
Keeping those results in a simple table over time turns a series of isolated experiments into a history you can reason about, in the same way that prompt versioning and change management does for prompts.
Plan for deletion
Versioning creates copies, and copies of personal data create obligations. If someone asks for their data to be removed, you need to know which snapshots contain it and how to honour that request across all of them.
Decide retention when you start keeping versions rather than when the first request arrives. Snapshots older than a defined period should expire automatically unless there is a stated reason to keep them.
Make restoring a version easy
A snapshot nobody can restore is an archive rather than a version. Write the one command or short script that pulls a named dataset version into a working environment, and use it yourself occasionally, because a restore path that has never been exercised usually has a missing step in it.
The same applies to rolling forward. When a corpus is rebuilt, the job should be repeatable from a recorded source rather than depending on a sequence of manual imports somebody performed once and half remembers.
Can you reproduce last quarter's results?
We will put version control around the data your AI depends on.