Google's Gemini Apps Privacy Hub says a subset of Gemini chats are read by human reviewers. Gemini can also connect to your Google apps through Connected Apps. Put those two side by side and an obvious question follows: what happens to your email once it becomes part of a chat?
I'm working from Google's Gemini Apps Privacy Hub and its Workspace privacy hub. I haven't seen Google's internal sampling or redaction documentation. Where I'm inferring, I'll say so.
What Google's privacy hub says
Google's Gemini Apps Privacy Hub says "a subset of chats are reviewed by human reviewers," including trained reviewers from its service providers. It asks users: "Please don't enter confidential information that you wouldn't want a reviewer to see." It says data reviewed by service providers is disconnected from your account and kept for up to three years. The same page describes Connected Apps, which let Gemini work with your Google apps.
Your inbox becomes conversation text the moment Gemini reads it
My working assumption, as an engineer, is that once a tool returns 4,000 words of email, those words sit in the same transcript as the sentence you typed. If that's right, whatever samples chats for review sees one sequence unless something tags where each part came from.
Keeping retrieved content away from reviewers would take two separate filters. The first strips tool output from the stored transcript before anything is sampled for review, which is ordinary engineering as long as the provenance tags exist. If they don't, you're left guessing which spans of a transcript came from a tool call, and guessing is how a filter quietly fails on some fraction of conversations nobody ever checks. The second filter is much harder. It has to strip retrieved content out of the model's own response, and the response quotes and paraphrases that content freely.
The response is the leak, even when the source isn't
Ask Gemini "what did my landlord say about the deposit" and the answer contains the landlord's name, the dollar amount, the property address, and probably the date the dispute started. Strip the original email from the transcript and the reviewer still reads all of it in the response. A filter that wants to catch this has to recognize paraphrased content from a source it no longer has in front of it. For anyone running this pipeline, this is the part to lose sleep over, because no amount of transcript scrubbing fixes it. The only clean answer is to keep conversations that used connected apps out of the review sample entirely.
"Disconnected from your account" doesn't mean anonymous
Google's privacy hub says data reviewed by service providers is disconnected from your account. Email bodies carry signature blocks, phone numbers, home addresses, order confirmations, the name of a child's school, the last four digits of a bank account. Anyone reading three of those emails knows who you are, account ID or not.
Annotation vendors: who reviews what
Google's privacy hub says its human reviewers include trained reviewers from its service providers, and that the data they review is kept for up to three years. That puts any content in a reviewed chat with a third party whose workforce, workstations and access controls the user never sees. A user who connected their Gmail to a chatbot never agreed to sit at the end of that chain, and couldn't audit it if they wanted to.
Personal Gmail on corporate laptops
For Gemini in Google Workspace business and enterprise editions, Google's Workspace privacy hub says: "Your content is not human reviewed or otherwise used for Generative AI model training outside your domain without permission." I'd still check the current terms against your own agreement.
The real problem is the personal account sitting next to the corporate one. Most organizations have employees who forward a contract to personal Gmail to read on the train, or drop a client spreadsheet into personal Drive because the corporate share was slow that day.
Controls an IT team can put in place this month:
- At the web proxy, add the X-GoogApps-Allowed-Domains header listing your own domains. Google's admin help says this approach blocks sign-in to Google consumer services other than Google Search, which is the point.
- Add a DLP rule that flags or blocks manual forwards of sensitive content to consumer domains like gmail.com, because people will just forward by hand.
- Name consumer AI assistants with connected apps in the acceptable use policy. A policy without a technical control is mostly a hope, but it gives you something to point to when you ask someone to move files back.
Settings to change on a personal account
None of this undoes what already happened. It narrows what happens next.
- Use temporary chats for anything you'd mind a stranger reading. They're still kept with your account for 72 hours.
Five questions for Google
These are the five questions I'd put to Google. The usual statement about taking privacy seriously doesn't count as an answer.
- Is content Gemini retrieves from Gmail or Drive ever included in chats sampled for human review?
- If it is, is that by design, or only when a model response quotes it?
- What redaction runs before a reviewer sees anything? Specifically, does it treat tool output differently from text the user typed?
- Is retrieved content tagged at the point of retrieval, so an exclusion can actually work?
- Can users find and delete reviewed copies, which the privacy hub says are kept for up to three years?
Most people reasonably read "conversation" as the things they typed. A user who agreed to let a contractor read their questions about recipes did not agree to let a contractor read their divorce lawyer's emails.
My position: any assistant with retrieval should treat tool output as its own data class, with its own retention and review rules, excluded from human review by default and tagged at the point of retrieval so the exclusion actually works.
If you've run a vendor review on an AI assistant with connected apps, I'd like to know what the vendor told you about tool output specifically: is retrieved content tagged and excluded from human review, or did the answer only cover what users type?