Grading the Grown-Ups: What the 2026 AI Safety Index Reveals About Privacy and Long-Term User Data

The Future of Life Institute released its 2026 AI Safety Index on July 7, and the top score any frontier lab managed was a C+. That grade belongs to the same class of companies now storing months or years of your conversations, preferences, and personal context in always-on memory systems. If privacy depends on the trustworthiness of the entity holding your data, and the most rigorous independent assessment available says even the best performer barely clears a C+, you should probably read the fine print more carefully than you have been.

Key Takeaways

• The FLI's 2026 index scored nine frontier AI labs across 37 indicators. The highest overall grade was C+ (2.66 out of 4.0). No company scored above C− on existential safety.
• Between early 2024 and mid-2026, the major AI assistants shifted from stateless chatbots to persistent-memory systems, most of them enabled by default, affecting hundreds of millions of users who never changed a setting.
• Voluntary privacy commitments are already being overridden by courts: a federal preservation order forced one provider to retain output logs regardless of user deletion requests, and a separate case alleges another used an existing settings toggle as a backdoor to expand data access without fresh consent.
• Memory poisoning, a class of attack where malicious content is injected into an AI agent's long-term memory, has been demonstrated with success rates as high as 80-99% against current memory architectures.
• The index grades policies and disclosures, not the products you actually use. A passing grade on paper tells you nothing about what happens to your stored memory today. Product-level transparency (audit logs, verifiable deletion, readable memory stores) matters more than an annual report card.

What Did the 2026 AI Safety Index Actually Measure?

The index evaluated nine leading AI companies across 37 indicators grouped into six domains, including governance, transparency, information sharing, and existential safety. Grades were based largely on public policies, published research, company disclosures, and a voluntary survey. Only five of the nine companies completed the survey.

Anthropic earned the top overall score at C+ (2.66). OpenAI followed at C (2.28), Google DeepMind at C (2.01), and Meta at D+ (1.32). Z.ai and Alibaba Cloud landed at D−. xAI, DeepSeek, and Mistral received failing grades.

The existential safety domain was the weakest across the board. No company scored better than C−, and the single best grade in that domain was Anthropic's D+. FLI chair Max Tegmark put it plainly: "AI companies are sprinting toward a cliff."

Why Does a Governance Grade Matter for Your Data?

Because the companies earning C's on governance are the same companies now operating persistent memory systems that store your personal context across sessions. Governance scores reflect whether a lab has clear internal accountability structures, whether it follows through on stated commitments, and whether it shares meaningful information about risks. If those scores are mediocre at best, the privacy policies attached to their memory features deserve proportional skepticism.

The index also found something more concerning: Anthropic, OpenAI, Google DeepMind, and Meta have weakened or eliminated earlier commitments to pause development if their systems approached specified danger thresholds. These are voluntary pledges, made publicly, later quietly softened. If a company will walk back a safety pledge under competitive pressure, the durability of a privacy policy (also voluntary, also under competitive pressure) is a reasonable thing to question.

How Did AI Assistants Go from Stateless to Always-On Memory?

Quickly. Between February 2024 and March 2026, ChatGPT, Claude, and Gemini all transitioned from stateless chatbots to systems that retain long-term personal context, most of them by default.

The pivot point for the largest user base came on April 10, 2025, when OpenAI upgraded its Memory feature so ChatGPT could reference all past conversations, not just items users explicitly asked it to remember. A free-tier rollout followed on June 3, 2025. A tool used by hundreds of millions of people became persistent almost overnight. Most users never changed a setting.

Storage architectures differ in ways that matter. OpenAI and Google use opaque vector-backed memory, while Anthropic chose human-readable markdown files users can open and edit. The distinction is significant: with vector-backed storage, you cannot inspect what the system actually "remembers" about you in a form you can read. With markdown files, you can. That is a product-level transparency decision that no governance index captures.

Google stores Gemini conversations in a consumer account with a default retention period of 18 months, adjustable to 3 or 36 months. By early 2026, Google rebranded its cross-app data flow (Gmail, Drive, Maps) under the name "Personal Intelligence," a label that signals ambition more than restraint.

What Happens When Courts Override Privacy Promises?

They override them. This is not a hypothetical.

On May 13, 2025, OpenAI received a federal court preservation order directing it to retain all output log data that would otherwise be deleted, regardless of user deletion requests or privacy regulation requirements. OpenAI appealed. A district judge affirmed the order on June 26.

Think about what that means for a user who clicked "delete." Their data is still there, held under court order. The privacy policy said one thing. A judge said another. The judge won.

A separate case underscores the pattern. Thele v. Google LLC, filed in the Northern District of California, alleges Google used the existing "Smart Features" toggle as a backdoor to enable Gemini's deeper data access without a clear second consent prompt. As of March 2026, the case is in discovery. The allegation is straightforward: a toggle designed for one purpose was repurposed for another, and users were not meaningfully informed.

These cases illustrate something the FLI index cannot grade. Voluntary commitments are voluntary. They can be revised under competitive pressure or overridden under legal compulsion. The more data a system retains, the larger the surface area for both.

Is Memory Poisoning a Real Threat?

Yes. Security researchers have identified and documented it as a distinct attack class. Memory poisoning writes malicious content directly into an AI agent's long-term memory so it keeps acting on that content in future sessions. Because the memory persists, the attack persists. The user does not need to be re-compromised each session.

Documented attack success rates against LLM agent memory systems run as high as 80-99%. That range is not a typo. The attacks work because most memory systems store vector embeddings in opaque formats the user cannot inspect, and the systems themselves lack integrity checks that would flag injected content as anomalous.

This reframes the "trustworthiness" question. Low governance scores combined with persistent, unauditable memory are a compounding risk. Poor governance means less internal scrutiny. Opaque memory stores mean less external scrutiny. Together, they create conditions where memory poisoning can persist undetected.

Human-readable, user-editable memory (or the option to have no persistent memory at all) is a meaningfully different design choice. It does not eliminate the risk, but it makes the attack surface visible. An attack you can see is an attack you can remove.

What Does the Index Actually Grade, and What Does It Miss?

The index grades policies and disclosures. It does not grade products. As enterprise analysis of the index notes, a C+ governance grade on paper says nothing about what actually happens to an individual's stored memory today. You can have an excellent safety policy document and a memory system that retains everything in an opaque format with no user-facing audit log.

For users and enterprise buyers, the gap between policy and product is where risk lives. Useful transparency would include: verifiable deletion (proof that deleted data is actually gone, not just flagged), audit logs showing what the system remembers about you and when it was accessed, clear documentation of which data reaches third-party infrastructure at inference time, and retention schedules enforced by architecture rather than by policy.

The index also faced methodological challenges. Mistral disputed the framework, arguing it structurally penalizes open-weight release models where the deploying enterprise, not the lab, controls fine-tuning and safety. That is a fair objection. An open-weight model that a hospital fine-tunes for radiology carries different risk dynamics than a closed API endpoint. Grading both on the same rubric compresses distinctions that matter.

What Changed on the Military Front, and Why Does It Matter Here?

From 2024 to 2026, companies including Anthropic, OpenAI, Google DeepMind, and Meta that previously banned military applications gradually reversed course, joining xAI and Mistral in actively seeking defense partnerships. This is relevant to the data question because it signals a broader pattern: stated commitments bending under commercial and geopolitical pressure.

If a "no military use" policy can become a "strategic defense partnerships" policy in two years, a "we delete your data after 30 days" policy can become "we retain your data for model improvement" just as easily. The mechanism is the same. A policy is updated, users are notified in a changelog most will never read, and the default shifts.

Are There Legal Protections for What You Tell an AI?

Fewer than you probably assume. A February 2026 case, United States v. Heppner, established that conversations with AI assistants are not protected by attorney-client privilege and do not qualify as work product. If you discussed a legal matter with your AI assistant thinking that conversation was privileged, it was not.

This matters because persistent memory means the conversation is still there. In a stateless system, the context disappeared after the session. With always-on memory, your discussion of a pending lawsuit or a sensitive business decision is stored, retrievable, and potentially discoverable in litigation. The legal framework has not caught up with the product design.

How Does Selina Handle This Differently?

We built Selina's memory with the assumption that the infrastructure beneath it is not fully trustworthy, because it isn't. A slice of each request reaches a frontier provider at inference time. That is the honest constraint of building on top of frontier models routed per task.

So here is what we do and what we don't claim.

Memory content is encrypted at rest. Memory is NOT end-to-end encrypted. We state that plainly because the alternative (implying a guarantee we cannot make) would be worse than the limitation itself. Files and transfers via SelinaSEND are a different matter: those are zero-knowledge, end-to-end encrypted. The distinction exists because the architecture for file transfer and the architecture for memory-augmented inference have different trust boundaries.

Your account is protected. Content within it is encrypted. Non-content operational metadata is kept for a short retention window, not forever, but also not zero. We do not claim zero retention because that would be inaccurate.

Delete means gone. Actually gone. Not flagged for future deletion. Not retained under a policy that might change. Not sitting in a backup waiting for a court order to surface it. Gone.

Memory is adaptive, not a transcript. We do not store verbatim conversation logs as "memory." The system builds a working model of your preferences and context. It is not perfect. It is not complete. It is useful, and it is yours.

We can't read your content. By design, not by policy.

Why State the Limits?

Because the FLI index, for all its value, demonstrates that policies are only as durable as the incentives behind them. A company can score a C+ today and revise its commitments tomorrow. Architecture is harder to walk back. If your data is encrypted at rest and deletion is immediate and verifiable, the policy layer matters less. Not zero, but less.

We are a small company. We do not have the lobbying budget to fight a preservation order the way a large lab might. What we have instead is an architecture where there is less to preserve. You cannot be compelled to produce data you do not have.

What Should You Actually Do With This Information?

Read the full FLI report. It is 80+ pages and more nuanced than the headline grades suggest. The domain-level breakdowns matter more than the overall letter grade.

Check your memory settings. If you use a major AI assistant, memory is probably on by default. Decide whether you want it on. If you do, understand what format that memory is stored in (opaque vectors vs. readable files) and what retention period applies.

Treat AI conversations as discoverable. After Heppner, assume that anything you tell an AI assistant could surface in litigation. Do not share information with an AI that you would not put in an email to a colleague.

Evaluate products on architecture, not promises. A privacy policy is a promise. Encryption at rest is a fact. Verifiable deletion is a fact. Zero-knowledge file transfer is a fact. Weight your trust accordingly.

The FLI index is valuable precisely because it quantifies how far short the industry falls. A C+ is not a passing grade in any system that matters. The companies earning those grades are the same ones holding years of your personal context in memory systems most users never consciously opted into. That gap between governance capacity and data responsibility is where the risk concentrates.

If you want to try an assistant that states its limits honestly and encrypts your content at rest while being straightforward about what it cannot encrypt: start a free 7-day trial, no card required.

Michael C.

Founder & Principal Engineer, Selina Labs

Michael builds Selina, a privacy-first AI that remembers you across conversations. He ships security-sensitive AI in production — real attacks, real fixes, measured in minutes and dollars — and writes about privacy, security, and LLMs from that seat. Top Rated Plus and expert-verified on Upwork.

• www.upwork.com
• x.com
• medium.com
• selinalabs.io