Personal data ends up in AI pipelines by accident more than by design. A support ticket contains a name and an address, a document carries a date of birth, a transcript includes a card number the customer read out. All of it flows into prompts, and from there into logs, caches and third-party systems.
Handling personal data in an AI pipeline is mostly about knowing where it accumulates, because the copies nobody planned are the ones with no controls on them.
Prompts and logs are the quiet problem
Careful teams protect their database and then log every prompt in full to a system with broad access and a long retention period. That log is now a copy of the data with weaker protection than the source.
The same applies to caches, evaluation sets built from production traffic, and the spreadsheet somebody exported to investigate an issue. Each is a copy, and copies made informally are the ones that outlive their purpose.
Decide what genuinely needs to leave
A lot of what gets sent is not needed for the task. Summarising a complaint does not require the customer's account number; classifying a ticket rarely requires their address.
- List the fields the task actually needs, then send only those.
- Replace identifiers with tokens you can map back locally, so the model never sees the real value.
- Redact structured items such as card numbers and national identifiers before the call.
- Truncate free text where the tail is unlikely to be relevant and likely to contain more detail than needed.
Minimisation before the call is more reliable than any downstream control, because data that never left cannot be exposed by a misconfigured log.
Ask what would be in a breach notification. If the answer includes your prompt logs, they are part of your data estate and should be protected like it.
Sam Ortiz, Director of Engineering, Engineered With AI
Redaction is imperfect, so layer it
Pattern matching catches structured identifiers reliably and misses names, addresses and anything described rather than formatted. A model-based redaction step catches more and is itself imperfect.
Use both where the risk warrants it, and accept that neither is complete. Where the data is sensitive enough that imperfect redaction is unacceptable, the answer is usually a different architecture rather than a better filter.
Free text is where it hides
Structured fields are easy to control. The problem is the note field where somebody typed a full medical history, or the transcript where a customer volunteered something nobody asked for. Any pipeline consuming free text should assume it contains more than it should.
Check the provider terms before building
Establish whether inputs are retained, for how long, whether they can be used for training, and where processing physically happens. These vary between providers and between tiers of the same provider.
Get the answers in writing before the architecture depends on them. Discovering a residency or retention constraint after building is one of the more expensive ways to learn it, and it usually means changing supplier rather than adjusting configuration.
Retention and deletion have to be real
Where individuals have a right to have their data deleted, that right extends to the copies. If a name sits in six months of prompt logs and an evaluation set, deletion has to reach them.
That is far easier if logs are structured with an identifier you can search, and close to impossible if the data is embedded in free-text prompts with no index. Design for deletion at the start; retrofitting it is the part that genuinely does not work.
Keep the map current
A short document showing what data flows to which provider, what is redacted, where logs live and how long they are kept is worth maintaining. It is what makes an audit answerable and what stops the architecture drifting.
Review it whenever a new feature is added, because that is when a field starts being sent that was not before. This sits alongside access control for AI agents, and the two are usually reviewed together.
Unsure where your data is going?
We will map what leaves your systems and put the right controls around it.