A prompt is a text box, and people treat it the way they treat a search engine or a notepad: they type in whatever they know at the moment and paste whatever is on the clipboard. The difference is that the text is processed on an outside provider's server, and from then on the provider's terms decide what happens to it.
Below we go through the routes by which data leaves the organization, the kinds of data that do not belong in an external AI tool, what to look for in the terms of use, and how technical filtering of outbound traffic works. Uncovering unapproved AI use by employees, and the policy that goes with it, are covered in the article on shadow AI.
How data gets out to an AI service
A language model can only work with what it is given. Behind every answer there is a request that carries the user's text, the attached files, and the earlier messages to the provider. Data leaving is simply part of how these tools work, and it happens by several routes:
- Typed and pasted text. Emails, meeting minutes, spreadsheet excerpts, error messages. Pasting is the more common of the two, and it brings along everything that was in the selection.
- Uploaded files. Documents, spreadsheets, screenshots, scanned papers.
- Conversation history. In a longer conversation the model receives the earlier text as context with every new question, and the history stays stored with the provider until the user or a retention rule deletes it.
- Built-in AI features. Summarizing or writing features switched on in an email client, an office suite, or a ticketing system, which send the document's content off for processing.
- Developer tools and APIs. Assistants built into code editors pass the open files and the surrounding code along as context, and a model called with an API key receives whatever the program assembles.
- Agents and connectors. When an AI tool gets access to a mailbox or a file store, it reads the data from there itself, without the user pasting anything.
On the last three routes the user often cannot even see exactly what went out, so rules built only around what people type will fall short.
What counts as prohibited data for AI
“Prohibited” always refers to a specific tool. Data that may be entered into a corporate tool covered by a contract and configured for internal use may not go into a public service used with a personal account. The organization's data classification is the basis for the decision: if a classification scheme exists, the AI rule adds one column to it, and there is no need to invent a new one. How to build such a scheme is covered in the article on data classification.
| Data type | Examples | Why it is sensitive |
|---|---|---|
| Credentials | Passwords, API keys, access tokens, private keys, connection strings | A leaked key can be used immediately, and keys travel unnoticed in logs and configuration files |
| Personal data | Client and employee data, CVs, complaint letters | Processing data that goes to an outside provider requires a legal basis, and where the provider is a processor, also a contract under Article 28 of the GDPR |
| Special category data | Health and biometric data, religious or political beliefs, trade union membership | Under Article 9 of the GDPR, as a general rule they may not be processed, except under narrow exceptions |
| Card data | Card number, expiration date, security code | Any system they are entered into falls within the scope of the PCI DSS standard |
| Trade secrets | Pricing logic, bids, negotiation materials, production recipes | Legal protection depends on the holder taking the steps needed to keep the information secret |
| Material received from third parties | Documentation covered by a non-disclosure agreement, drawings and data received from clients | The contract often explicitly prohibits external processors |
| Technical and security data | Source code, network diagrams, IP address plans, OT environment configurations, vulnerability reports | It gives an attacker a map of the system |
Anonymization often comes up as a solution, but in practice replacing the names is usually pseudonymization. Under the GDPR, pseudonymized data remains personal data if it can be linked back to the person using other information, and a detailed complaint letter can reveal who it is about even without the name.
What to look for in the terms of use
At the same provider, the terms of the consumer and business plans can differ considerably, and the settings change over time. An employee accepts the consumer terms as a private individual when signing up; the organization is not a party to that contract, so on most of the points below it can neither negotiate nor make a central decision. Before approving a tool, answer these questions from the current contract and the data processing documentation:
- Training. Does the provider use the text entered and the files uploaded to improve its models? Is this on by default, and who can turn it off: the user or the organization's administrator?
- Retention. How long are conversations and files stored? Does deletion by the user really delete them, or does a copy remain for a while for abuse monitoring?
- Human access. Can the provider's staff review conversations for quality assurance or abuse investigations?
- Location of processing. In which region is the data processed and stored? If it is outside the European Economic Area, on what legal basis is it transferred?
- Data processing agreement. Can you sign an agreement under Article 28 of the GDPR, and is the list of sub-processors public?
- Sharing. Can a public link be created from a conversation, and can this feature be disabled centrally?
Text, file uploads, browser extensions
The three most common channels each take data out in a different way, so filtering has to handle each of them differently.
Text
Typed or pasted text goes out as an encrypted web request over a TLS connection. On the network you can see which service the user reached, but not the content. To inspect the content, you either have to catch the text on the endpoint before the browser sends it, or decrypt the traffic on a proxy.
File uploads
A document holds more than what shows on the screen. A spreadsheet can contain hidden sheets and columns; a text document can contain tracked changes, comments, and the authors' names in its metadata. The user asks for a summary of one chapter, but the whole file goes out. In screenshots and scanned documents the text travels as an image, so a filter rule written for text only catches it if the filter also runs optical character recognition.
Browser extensions
AI-based extensions (translators, summarizers, writing assistants) typically ask for read access to every page the user visits. Such an extension can send the content of an open internal web application, the email client, or the CRM to its own server. The same goes for desktop applications and code editor extensions, which bypass any protection that relies on the browser.
Technical filtering: monitor, warn, block
Filtering traffic headed to AI services is a narrower use of general data loss prevention (DLP). The filter inspects the outbound text or file and, if it finds sensitive content, acts according to the configured rule. Detection can rely on several methods:
- Pattern matching. Card numbers with a Luhn check (the check digit algorithm weeds out number strings that only happen to look similar), IBANs, Hungarian tax ID numbers, TAJ (social security) numbers, and the distinctive formats of API keys.
- Keywords and dictionaries. Project names, internal code names, a “confidential” marking in the document header.
- Sensitivity labels. If the document has already been classified, the filter reads the label instead of having to analyze the content.
- Fingerprinting and exact match. Recognizes the content of a specific database or document template, even in fragments.
- Machine classification. A trained model estimates whether a text is, for example, a contract or source code.
The decision has three levels. In monitor mode the filter only logs, and the user notices nothing. In warn mode a pop-up tells the user that the text contains sensitive data, and the user can either continue with a justification or go back. In block mode the request does not go out; some solutions instead redact the sensitive part and let the rest through.
Start in monitor mode. The logs from the first few weeks show which rules raise false positives (an order ID can easily look like a card number) and where data is really leaving. Once the rules have been refined, blocking interrupts less work and gives users less reason to look for workarounds.
Where the filter runs (an endpoint agent, a decrypting proxy, or a managed browser) and how it can be bypassed is covered in the article on controlling AI services with technical tools. The logs have a sensitive side as well: the caught cases contain the sensitive data itself, so access to DLP alerts should be granted narrowly. Employees have to be informed in advance about this kind of monitoring, as required by Section 11/A of the Hungarian Labor Code and by the transparency rules of the GDPR.
An approved corporate AI tool as the alternative
Filtering tells people what they must not do, but the employee still has the task to finish. If there is no approved tool for rewording the email or summarizing the spreadsheet, they will do it on their own phone, where none of the controls can see it. For the approved tool, the organization's contract has to give acceptable answers to the questions listed in the section on terms of use. Beyond that, the usual requirements apply:
- sign-in with the corporate identity and multi-factor authentication,
- automatic disabling of a departing employee's account,
- an administrator log of who used what,
- central control over which extensions and connectors are allowed.
For the most sensitive categories of data, where no external processor can be considered at all, a model running on your own infrastructure may be the answer; the article on local language models covers what that requires. An approved tool does not lift every prohibition either. Credentials stay out of the prompt there as well. Processing special category personal data in the approved tool still requires a legal basis resting on one of the exceptions in Article 9 of the GDPR, and large-scale processing or other high-risk cases also call for a prior data protection impact assessment (GDPR Article 35). If the tool answers from internal documents, it should only see the documents the person asking can access. Sensitive information disclosure also appears as a separate risk in the OWASP Top 10 for LLM Applications.
Related service: GenAI data loss prevention →