重要なインサイト
Data loss prevention was built for a world of files and fields. A Social Security number has nine digits in a known format. A credit card number passes a checksum. Write a regular expression, scan the outbound email, block the match. For structured data in documents, it worked well enough for two decades.
AI chat breaks every assumption underneath it. And research published in the past two months shows it breaks one more: some of the riskiest text in the conversation doesn't come from the employee at all.
Why Regex Fails in an AI Chat Window
Sensitive meaning doesn't need sensitive strings. "Draft the memo announcing we're closing the Austin office in Q1" contains no account number, no PII, and nothing a regex would flag. It is also material nonpublic information.
Data gets transformed on the way in. Employees paraphrase, summarize, and paste fragments. A customer list becomes "here are our top accounts and what they pay." The pattern disappears while the exposure stays.
Context decides risk. The same paragraph of source code is fine in the approved enterprise coding assistant and a problem in a personal account on a free tier.
And false positives carry a price. Every unnecessary block teaches an employee to route around the control, usually toward a tool you can't see.
When the Risky Text Is an Instruction
In September, researchers at Zenity Labs published SalesBleed, an attack on Salesforce Agentforce. A hidden prompt sat in a public Web-to-Lead form. When an employee asked the agent to review new leads, the agent read the prompt, queried the Accounts table, and leaked company names and deal sizes through DNS lookups, getting around the platform's URL redaction. Salesforce confirmed a fix in August, before publication.
A week earlier, Palo Alto Networks Unit 42 showed instructions hidden in an HTML comment in a support ticket making an AWS AgentCore Harness agent read plaintext credentials out of process memory. AWS closed the report as informative under its shared responsibility model.
Neither attack contains a string a pattern rule would flag on the way in. Both show up in what the AI did next: which records it read, and where it sent them.
Three Data Sources for Governing AI Conversations
Tool compliance and audit APIs. The major AI providers expose enterprise audit and compliance interfaces for sanctioned workspaces. These show what was said, by which user, in which conversation, with which files. This is the ground truth for the tools you've rolled out.
Browser or endpoint telemetry, where you have it. Sanctioned tools are half the picture. The other half is browser sessions on personal accounts, desktop AI apps, and AI connectors pulling data out of systems like your CRM.
An identity baseline. For each employee, learn what normal AI use looks like given their role, team, tools, and data. A recruiter summarizing résumés is expected. A recruiter exporting the full candidate database into a tool installed yesterday is not. This is the source that cuts false positives.
How a Mature Program Grades Its Responses
Joined on identity, those sources let you grade risk instead of matching strings. A mature program then matches the response to the risk:
Normal use that still needs a record gets logged.
Borderline cases get a short in-the-moment reminder.
Riskier ones get a step-up that asks the user to confirm intent before sensitive data reaches an unapproved destination.
Clear violations, such as regulated health or payment data going into an unsanctioned model, get blocked.
Every decision gets recorded. When your general counsel or a regulator asks whether AI use is governed, the answer is evidence rather than a policy document.
Why AI Data Policies Belong in Plain English
Your legal team writes policy in sentences: "No patient data in external AI tools." Translating that into hundreds of brittle regex rules is where many programs stall. Expect policy to move toward plain language, evaluated against what's known about the person, the tool, and the data.
Pattern matching will keep its job of finding card numbers and Social Security numbers where they shouldn't be. It can't carry an AI data governance program on its own.
Three Questions to Ask About AI Data Governance
Three questions are a good place to start:
- First, of ten prompts your team agrees were risky last week, how many did your current controls flag?
- Second, of ten clearly harmless prompts, how many did they block?
- Third, if a connected AI agent read a thousand customer records and sent them somewhere new, would anything notice, even though no employee typed a word?
The first two numbers tell you whether you're governing conversations or scanning text. The third tells you whether you're watching the agents as well as the people.
We track how sensitive data moves through AI tools and agents, and how teams are governing it. Our AI security resource hub collects our latest research, guides, and analysis in one place.
Get AI Security Resources