Why does the model need this data?

The first concrete thing a recent client asked me to build was an anonymiser. The client is a US family office managing the wealth of a small group of families, and the automated reports and internal assistants could wait. The team already used Claude every day. The problem was the managing partner's inbox, where clients, vendors, colleagues and friends all write to the same address, and which is full of names and sums nobody wanted leaving the building. They had the order right.

In an artificial intelligence (AI) project that touches sensitive data, the most important decision comes before anyone picks a model: where identifiable data is allowed to live, which systems may touch it, and at what point it stops being sensitive. One question drives all of it. Why does the model need to receive this piece of data?

What is a privacy boundary?

A privacy boundary is a line drawn in the architecture between the systems where personal data may exist and the systems that only ever receive data that has already been treated. The original documents go into a secure processing environment. Inside it, one step finds and replaces whatever points to a person (names, addresses, tax numbers, references to staff), a second step checks the result, and only what passes the check crosses over to the models that will classify or summarise it. The model on the far side does the whole job without ever seeing the originals.

Two terms tend to travel together and do different jobs. A sandbox is an isolated environment in which a program, an agent or a model runs with restricted access to files, network, application programming interfaces (APIs) and permissions; it protects everything else from whatever runs inside. In May, OpenAI explained how it built the Windows sandbox for Codex, its coding agent: the agent writes only inside its workspace, outbound internet access stays blocked until someone approves it, and every process it launches inherits the same restrictions. A secure processing environment works the other way round, because it exists so that sensitive information can be processed without crossing a line set in advance. The sandbox protects the world from the program. The secure environment protects the data from the world. What makes it work is restricted egress, meaning tight limits on outbound connections: the anonymising process takes a file in, hands a file back, and has no route to open a connection to the internet halfway through.

Why isn't the vendor's promise enough?

Security by architecture reduces how much trust a system needs in order to be safe. The most common question about AI tools is whether the vendor trains on your data, and the answer sits in the contract. Even the best answer remains a promise, though: the system receives everything and someone vouches that nothing bad will happen to it. The alternative is to design the system so that certain data never reaches it. The safest sensitive data is the data a model never receives.

European law has asked for this since 2018 under the name data minimisation: the General Data Protection Regulation (GDPR) requires personal data to be "adequate, relevant and limited to what is necessary" for the purpose (Article 5). The principle of least privilege applies the same logic to permissions. Every user, service or agent gets the minimum access its task requires (almost word for word the definition in the glossary of the National Institute of Standards and Technology, the US standards body). An agent that looks things up in a product catalogue has no use for sending email. A process that anonymises documents has no use for a browser.

With agents that take actions, the stakes change in scale. An agent with access to email that reads a document carrying a hidden instruction (what is known as prompt injection) can be talked into sending data out, and the one defence that does not depend on the model behaving well is that it never had the email tool in the first place. None of these principles arrived with AI. Agents have simply made their absence more expensive.

What is the difference between pseudonymisation and anonymisation?

Pseudonymisation replaces identifiers and keeps the key to reverse the swap somewhere else; anonymisation aims to make re-identification reasonably impossible. The difference carries legal weight: pseudonymised data is still personal data for whoever holds the key, and the GDPR applies in full, while truly anonymous data falls outside the regulation. The familiar version is a hospital that sends blood samples to an outside lab under a sample code instead of the patient's name. The lab does all the work without knowing who the person is, and the hospital can always match the result to the right file.

Deleting names rarely makes data anonymous. In 2000, Latanya Sweeney showed that 87% of the US population was probably unique on just three fields: five-digit ZIP code, sex and date of birth. A full UK postcode covers about fifteen properties on average. A job title, a town and an age are enough to point at the only finance director of a company in a small place.

This is moving right now. On 4 September 2025 the Court of Justice of the European Union ruled in case C-413/23 P that pseudonymised data can stop being personal data for a recipient with no reasonable means of re-identifying people, while remaining personal data for whoever holds the key (and without excusing that party from telling people who receives their data). On 8 July 2026 the European Data Protection Board (EDPB) adopted guidelines on anonymisation that build on that ruling and set three tests: no singling out of a person's record, no linking it to other sources, no inferring things about them. The public consultation runs until 30 October, so the final wording may still shift.

The practical reading holds either way. A good pipeline looks at combinations and context, and checking the output matters as much as the transformation itself. Automated detection fails in predictable places: surnames that are also ordinary words, nicknames, email signatures, scanned PDFs. A person reviewing samples stays in the process.

Three boundaries drawn in practice

At the family office, the design settled into three zones. The first works only with public information, such as news and alerts about prospects. The second runs analysis on pseudonymised data, with clients replaced by the codenames the team already used among themselves. Both pass the same test: nothing in them would do harm if it appeared on the front page of the Wall Street Journal tomorrow. The third keeps identified data inside the Microsoft 365 tenant the firm already had, and the Claude subscription is never given the Microsoft connector. The scenario behind the design was a Panama Papers-style leak: clients who have done nothing wrong, and a firm that simply does not want their names in public. The pseudonymiser starts from a dictionary of known entities (people, addresses, companies and trusts) before any generic sweep, because with a handful of families the list of what to protect is known and short.

On this blog, I gave an agent the keys to publish. The interface it uses has its own rules: everything it creates starts as a draft, it can only delete drafts that were never published, it works under a dedicated user with a token kept in an environment variable, and it is capped at 120 requests a minute. Publishing is still a click I make.

ROMEO, the assistant on this site, reads published content only, through a read-only token, and lives in a database separate from the site's. Conversations are anonymous, with no Internet Protocol (IP) address kept, and deleted after 30 days. Each question goes to the model provider, outside the European Union, and the privacy policy says so.

Where does a small business start?

A small or medium-sized business starts with a list of data fields, well before any infrastructure. Take the process you want to hand to AI, write down the fields it would touch, and put the question to each one. Anything without an answer stays inside the boundary or gets replaced before it leaves. Then give each agent its own identity, with minimal permissions and an audit log of what it did, and leave irreversible actions (sending, paying, deleting, publishing) to a person. These choices fit on a whiteboard and hold up in a conversation with whoever looks after data protection.

An AI demo that works takes an afternoon. Something a company will let sit next to its most important data takes a good deal longer, and nearly all of that time goes into deciding what stays on the inside. Who in your business has written that list yet?