What Happens to Privacy When a "Safety-First" AI Lab Starts Training on Consumer Chats by Default

On July 8, 2026, a revised privacy policy took effect for one of the most prominent AI labs in the industry. The changes are substantial: biometric identity verification, expanded law-enforcement data sharing, agentic session data flows, and a safety-flag carve-out that overrides user opt-out preferences. This matters to anyone who chose that lab's products specifically because of its privacy positioning. And it matters to us, because we build a product where privacy architecture is structural, not a toggle on a settings page. Here is what actually changed, what it means technically, and where the pressure lands.

Key Takeaways

• The July 8, 2026 policy update added biometric ID verification, "good faith" law-enforcement sharing without a court order, and agentic data-flow provisions on top of the opt-out training defaults introduced in late 2025.
• A safety-flag carve-out means conversations flagged by internal safety systems can still be used for training and review even if a user has fully opted out, with no disclosure of what triggers a flag or notification when one occurs.
• Deletion is forward-looking only: turning off model improvement or deleting a chat stops future use of that data, but does not remove it from models already trained or training runs in progress.
• The pattern across three policy versions (pre-2025, August 2025, July 2026) shows "privacy-forward" branding expanding in scope precisely as the product becomes more agentic and more commercially valuable.
• Opt-out toggles are a UX layer, not a data-architecture guarantee. Provable technical constraints (encryption at rest, zero-knowledge file transfers, short retention windows) are harder to build but harder to quietly re-scope in a terms update.

What Actually Changed on July 8, 2026?

The July 8 privacy policy is the most substantive revision since September 2025. It adds several categories that did not exist before.

First, biometric identity and age verification. Some free and paid consumer users can now be required to submit a government-issued ID and a live selfie before accessing certain features. The third-party processor handling that data is Persona, a verification company backed by Founders Fund, which is also an investor in the lab itself. That conflict-of-interest detail matters, and we will come back to it.

Second, agentic data flows. As AI products move toward tool use, web browsing, and multi-step task execution, the policy now explicitly covers data generated during connected-app and agentic sessions, including third-party sharing rules for those flows.

Third, law-enforcement sharing. The updated policy permits proactive sharing of user conversation data with law enforcement based on an internal "good faith belief" determination, without necessarily requiring a court order.

Fourth, research participation data. New provisions cover data collected when users participate in research, adding another category to the expanding surface area.

How Did the Training Default Change, and When?

The training-default shift did not start in July 2026. It started earlier. In August 2025, the lab announced it would begin training on consumer conversations, where previously it had not used chats for training unless a user explicitly submitted feedback. The new terms took effect October 8, 2025.

New users saw a training-preference choice at signup. Existing users got a pop-up. That pop-up featured a prominent "Accept" button and a much smaller training-permission toggle set to "On" by default, a design choice that multiple outlets flagged as a dark pattern. The data-retention window for users who allow training was extended to five years, up from the prior 30-day window.

Enterprise, education, and government tiers remain excluded from consumer training under separate commercial terms. That boundary has held across all policy versions. The pressure here is specifically on consumer and individual paid users.

Why Does the Safety-Flag Carve-Out Matter More Than the Default Toggle?

Most coverage of the August 2025 change stopped at "the toggle defaults to on." The less-covered, more consequential detail is the safety-flag carve-out. Conversations flagged by internal safety systems can still be used for training even if a user has opted out. The policy does not define what triggers a safety flag. It does not commit to notifying users when a flag occurs.

This means the opt-out is conditional. You can turn training off, and that choice will be respected for conversations that the system does not flag. But the boundary of what "flagged" means is drawn unilaterally, invisibly, and subject to change in a future policy revision without your awareness.

Even the lab's incognito-equivalent mode, which is described as excluding chats from training, still allows flagged content to be reviewed. So the technical guarantee a user receives is not "your data will not be used" but "your data will not be used unless we decide it qualifies under a criterion we define, do not disclose, and can change."

This is a structural point, not a complaint about one company. Opt-out toggles are a UX layer. They exist in the application, not in the data architecture. A settings page can be re-scoped in the next terms update. A cryptographic constraint cannot.

What Does "Deletion" Actually Mean Here?

Forward-looking deletion only. If you turn off model improvement or delete a chat, it will not be used for future training, but your data will still be included in model training runs already in progress or in models that have already been trained. Deletion stops future use. It does not pull data out of a completed or in-progress training run.

This is an honest disclosure on their part, and a common one across the industry. But it is worth stating plainly because many users assume "delete" means "gone." Here, it means "gone from future consideration." The distinction matters if your threat model includes data already absorbed into model weights.

Why Does the Persona Verification Detail Matter?

The biometric verification requirement routes government IDs and live selfies through Persona, a third-party identity verification company. Persona is backed by Peter Thiel's Founders Fund, which is also an investor in the lab requiring the verification. That is a direct equity relationship between the data processor and the data controller's own investor.

"We don't do X ourselves, our vendor does" is a common privacy framing. It shifts the trust surface from one company to two. But when the vendor and the company share investors, the independence implied by the word "third-party" is thinner than the phrase suggests. Vendor selection, equity relationships, and data-handling practices matter as much as the policy text. This is a concrete example of why.

How Does the "Good Faith" Law-Enforcement Clause Work?

The July 2026 policy permits proactive sharing of user conversation data with law enforcement based on an internal "good faith belief" standard. The phrase "good faith belief" is a legal term of art, but it is also a subjective one. It does not require a warrant or court order. It does not require external review before disclosure occurs.

Many cloud services have similar clauses buried in their terms. What makes this one notable is the context: a company that has positioned itself as the privacy-conscious alternative now holds five years of consumer conversation data (for users who allow training) and claims the right to share it proactively based on an internal determination. The surface area of what can be shared and the threshold for sharing it are both larger than they were 18 months ago.

What Does the Independent Scoring Say?

As of mid-2026, the lab earns the highest privacy score among the AI services reviewed by one independent methodology. That score is still only a B-minus (65/100). The gap between "highest among AI services" and "actually good" is real. Consumer training being opt-out, and the safety-flag carve-out sitting outside the opt-out entirely, are both factors in that grade.

A B-minus from the best-scoring player in a category tells you something about the category, not just the player.

What Is the Pattern Across Policy Versions?

Three distinct phases, each expanding the scope of data use:

1. Pre-August 2025: No training on consumer conversations unless the user explicitly submitted feedback. Short retention windows.
2. August 2025 (effective October 2025): Opt-out training by default. Five-year retention for users who allow training. Dark-pattern concerns around the toggle UI.
3. June 2026 (effective July 8, 2026): Biometric identity verification. Agentic data flows. Law-enforcement sharing on a "good faith" basis. Research participation data. Safety-flag override of opt-out.

The trajectory runs in one direction. Each update adds categories, extends retention, or introduces new sharing pathways. None of the updates have narrowed the scope of data use. "Privacy-forward" turns out to be a moving target that tends to expand precisely as a product becomes more agentic and more commercially valuable.

This is not unique to one lab. Separately, users discovered a usage-tracking mechanism embedded inside the product without prominent opt-in disclosure, a revelation that landed hard given three years of public positioning against surveillance-oriented data practices. And a broader terms-of-service review gave the lab a fairness score of 72/100, reflecting tensions between the brand promise and the contractual reality.

What Does This Mean for Competitors and Builders?

If you are building in AI and you have leaned on the "we use the privacy-conscious provider" argument, that argument now requires more qualification than it did in 2024. Enterprise tiers remain carved out, but consumer tiers are operating under a materially different data regime.

For builders, the lesson is structural: policy-layer privacy and architecture-layer privacy are different things. A toggle on a settings page is a promise. Encryption at rest is a constraint. Zero-knowledge file transfer is a mathematical property. These are not interchangeable. The first can be re-scoped in a terms update. The second and third cannot.

We have thought about this distinction a lot while building Selina. Our memory system is encrypted at rest and protected in transit. Memory is NOT end-to-end encrypted, because a slice of each request reaches a frontier provider at inference. We say this plainly because we think stating your limitations builds more trust than a marketing page that omits them. Files and transfers through SelinaSEND are zero-knowledge and end-to-end encrypted. Non-content operational metadata is kept for a short retention window, not zero retention. These are design choices baked into the architecture, not toggles a policy update can quietly flip.

We do not claim this makes us perfect. Memory is adaptive, not a complete transcript. Credits are finite. We have limits, and we publish them. The point is not that we have no constraints. The point is that the constraints are visible, specific, and structural rather than a settings page backed by a policy document that changes annually in one direction.

Why Does Publishing Your Limitations Build More Trust Than Hiding Them?

Because users who discover a limitation themselves feel deceived. Users who read about a limitation in your own documentation feel informed. The emotional difference is enormous. The commercial difference follows.

When a lab spends three years building a brand on being the responsible, privacy-conscious option and then rolls out opt-out training, biometric verification through a conflicted vendor, and proactive law-enforcement sharing in the span of ten months, the brand damage is not from the policies themselves. Many of those policies are defensible in isolation. The damage is from the gap between the positioning and the reality. Users feel misled even if, technically, they were notified.

We made a decision early on: say what we do not do as clearly as what we do. Say "memory is NOT end-to-end encrypted" in the same paragraph where we describe what is encrypted. Say "short retention window" instead of pretending metadata vanishes instantly. This is not self-deprecation. It is a bet that informed users are more durable than impressed ones.

What Should Privacy-Conscious Users Do Right Now?

Check your settings. If you use the consumer tier of any AI product, check whether training on your conversations is enabled by default. If you opted out previously, check whether a new policy revision has introduced carve-outs (safety flags, agentic flows) that sit outside your opt-out. Read the policy diff, not the blog post summary.

Understand what "delete" means. If the product only offers forward-looking deletion, anything you have already shared may persist in trained model weights indefinitely. Adjust your sharing behavior accordingly.

Evaluate architecture, not just policy. Ask whether a product's privacy claims are enforced by cryptographic constraints or by policy text. Policy text changes. Math does not.

Consider where your sensitive workflows live. Enterprise tiers with contractual training exclusions remain materially different from consumer tiers. If your work involves confidential information and you are on a consumer plan, the economics of upgrading (or switching) are worth revisiting given the expanded data use.

Where Does This Leave the "Safety-First" Brand?

Under real pressure, from multiple directions simultaneously. Export-control disputes with the administration, a widely criticized ad campaign, and the privacy policy changes all feed a narrative that the "responsible AI" positioning is being stress-tested on several axes at once.

The market positioning has always carried financial stakes. If the signal that distinguishes one lab from its competitors is trustworthiness, and that signal degrades, the competitive moat narrows. Enterprise customers chose the lab partly because of the privacy story. Consumer users chose it partly because of the safety brand. Both groups are now reading policy updates more carefully than they used to.

None of this means the lab is acting in bad faith. Scaling an AI product is expensive. Training data is valuable. Agentic features require new data flows. These are real engineering and business pressures. But the gap between "we are the privacy-forward option" and "we train on your chats by default, override your opt-out for safety-flagged content, share data with law enforcement on a good-faith basis, and require biometric verification through a conflicted vendor" is a gap that users notice.

The question is not whether the policies are defensible. The question is whether the brand can absorb the distance between where it started and where it is. That is a question every AI company building on trust will eventually face. The answer depends less on marketing and more on architecture.

If you want to see what architecture-first privacy looks like in practice, with the limitations stated alongside the claims: start a free 7-day trial, no card required.

Michael C.

Founder & Principal Engineer, Selina Labs

Michael builds Selina, a privacy-first AI that remembers you across conversations. He ships security-sensitive AI in production — real attacks, real fixes, measured in minutes and dollars — and writes about privacy, security, and LLMs from that seat. Top Rated Plus and expert-verified on Upwork.

• www.upwork.com
• x.com
• medium.com
• selinalabs.io