Customer and employee data isn't yours to disclose. It was given to your company under terms — a privacy policy, a contract, a law — and none of those terms said and we may forward this to a third-party AI vendor.
There is one narrow exception, if your employer has formally approved an enterprise AI tool for this. It's further down, and it's narrower than most people assume.
The text leaves your network the moment you hit send. It lands on a vendor's servers, where — depending entirely on the product tier and its settings — it may be retained for months, reviewed by a human for safety or quality, or used to train a future model.
You can't audit which of those happened. And there's no unsend: deleting the conversation removes it from your screen, not from every copy that was made along the way.
The legal shape of it is simpler than it sounds. You made a disclosure of personal data to a processor nobody assessed, under no agreement, with no lawful basis recorded. Under GDPR, India's DPDP Act, CCPA and most of what's coming, that's the violation — not whatever happens next.
Wider than most people assume. All of this:
Redaction takes about ten seconds and costs you nothing in answer quality. The model needs the shape of the problem, not the real values.
Write a polite chase email.
Customer: Alex Morgan,
[email protected],
account #A-99231,
84,000 overdue since 14 April.
Write a polite chase email.
Customer: [NAME],
[EMAIL],
account [ID],
[AMOUNT] overdue since [DATE]
— about 3 months late.
You get the same email back. You paste the real values into it yourself, in your own mail client, where they belong.
An enterprise AI tool your company has actually approved for this data. That means a signed data processing agreement, retention and training switched off contractually rather than by a checkbox you ticked, the vendor assessed, and this class of data explicitly in scope.
Two things that are not that exception, and get confused with it constantly:
If you don't know which tier you're on, you're not on the enterprise one. Someone would have made you sign something.
Don't quietly delete the chat and hope. Deleting is fine, but do it and tell whoever owns privacy or security where you work — today, not next week.
Breach notification clocks are measured in hours, and they only start when someone knows. The person who reports their own near-miss is almost never the one who gets in trouble. The person who hid it for a month always is.