Embedding Inversion Attack Privacy: Why "We Store Embeddings, Not Text" Is Not a Guarantee

You've heard the pitch. Maybe you've given it yourself. "We don't store your documents. We only store embeddings. The original text is gone." It sounds like a privacy architecture. It is not one. The embedding inversion attack privacy problem is real, it is getting worse, and the compliance story built on top of "just vectors" is thinner than most teams realize. This post walks through the research, the regulatory trajectory, and a bug class we hit in our own stack that made the whole thing concrete for us.

Key Takeaways

• Recent research demonstrates zero-shot inversion of text embeddings across encoders, recovering sensitive information from over 80% of tested documents without needing a custom-trained inversion model.
• Full text reconstruction is not the bar that matters. Attribute inference (diagnosis, nationality, occupation) from embeddings alone exceeds 94% accuracy in some categories, with no reconstruction step required.
• Most proposed defenses (dimension shuffling, linear transforms, moderate differential privacy) have been defeated or destroy utility at the noise levels needed for real protection.
• Embeddings derived from personal data are increasingly likely to be classified as personal data under GDPR and UK GDPR, which means deletion obligations apply, and most vector stores have no clean mechanism to fulfill them.
• The correct framing is not "can someone invert this" but "who do you have to trust, and what happens when that trust boundary breaks."

What Does "We Only Store Embeddings" Actually Mean?

It means the system ran your text through an encoder (a neural network that maps sequences of tokens to fixed-dimensional vectors), stored the resulting float array in a vector database, and discarded (or claims to have discarded) the source text. The vector preserves semantic relationships. That is the entire point: you can do similarity search, retrieval-augmented generation, clustering, classification. The implicit privacy claim is that going backwards, from vector to text, is computationally hard or impossible.

That claim was always more hope than proof. And the research has moved fast enough that "hope" is now the wrong word.

How Do Embedding Inversion Attacks Work?

An inversion attack takes a stored embedding vector and attempts to recover the text (or text-equivalent information) that produced it. Earlier methods, like vec2text, required training a dedicated inversion model per encoder and making a large number of queries to the target embedding API. This was expensive, slow, and encoder-specific. It was still effective enough that the researchers noted vector database platforms were encouraging sensitive data uploads under the premise that "only the text embeddings are shared with third-party services, not the actual text."

The barrier dropped substantially in March 2025. Zhang, Morris, and Shmatikov at Cornell published ZSInvert, a universal zero-shot embedding inversion method. "Zero-shot" means no custom-trained inversion model per encoder. It uses adversarial decoding to work across embedding architectures without the query budget that made earlier attacks expensive to mount. The reconstructed text is not verbatim. It does not need to be.

Does an Attacker Need to Recover the Exact Text?

No. And this is where most engineers' mental model of the threat breaks down. The instinct is to frame inversion as a reconstruction problem: can you get back the original sentence, word for word? If the answer is "not quite," the conclusion is "safe enough." That framing misses the actual risk.

The ZSInvert researchers tested against the Enron email corpus and found that sensitive information leakage exceeded 80% across all encoders tested. The inverted sequences were semantically close, not identical. But "semantically close" for an email containing a diagnosis, a social security number, or a deal term means the sensitive payload leaks even when the surrounding syntax does not reconstruct perfectly.

Their conclusion was blunt: "sharing the embeddings of confidential or sensitive documents with third-party services is equivalent to sharing the documents themselves." They further recommended that data owners "should not store their embeddings in retrieval systems and vector databases unless they fully trust them," and that "any security breach leaking embeddings should be treated as leaking the underlying documents."

That is a direct repudiation of the "we only store embeddings" pitch, from the people who built the tool that breaks it.

What About Attribute Inference Without Any Reconstruction?

Inversion is not the only vector (no pun intended). A separate class of attack skips text reconstruction entirely and extracts sensitive attributes directly from the embedding space. Analysis of this approach shows that attackers can infer nationality, occupation, birthdate, and medical diagnoses via cosine similarity comparisons, without labeled training data, achieving over 94% accuracy on some attribute categories.

Think about what that means for your compliance posture. You stored embeddings instead of text. An attacker (or an insider, or a subpoena) gets access to the vector store. They do not need to reconstruct a single sentence. They probe the embedding space with known reference vectors ("embeddings of text mentioning diabetes," "embeddings of text mentioning bankruptcy") and cluster your stored vectors against them. The attributes fall out. No inversion model required. Just linear algebra.

This is the attack surface most teams are not modeling, because the conversation stays stuck on "can they get the words back."

Can Membership Be Inferred from Similarity Search Alone?

Yes. Even without inversion or attribute extraction, the basic operation of a vector database (similarity search) can leak membership information. A probing attack works like this: an attacker sends probe vectors corresponding to known entities (customer names, specific medical terms, internal project codenames) and observes whether the similarity scores against stored vectors are suspiciously high. This is enough to infer which customers, patients, or subjects have data in the system, without recovering any content at all.

If your vector store is behind an API with no access controls on similarity queries, or if an attacker gains read access to the vectors, this is trivial. It violates privacy regulations. And it requires zero sophistication.

Don't Existing Defenses Solve This?

Not yet, and possibly not soon. The defense landscape is active but immature. Here is where things stand:

Random dimension shuffling. The idea is to permute the dimensions of the embedding vector so an attacker cannot interpret them. Alignment attacks circumvent this by re-learning the permutation from a small number of known text-embedding pairs.

Full-rank linear transforms (including watermark-embedding transforms). These apply a matrix multiplication to the embedding before storage. They fail to meaningfully impede inversion, because the transform is invertible by definition (it is full-rank), and the attacker can recover it given access to the embedding API.

Differential privacy. Adding calibrated noise to embeddings before storage. This is the most theoretically grounded approach. In practice, it only provides meaningful protection when the privacy parameter (epsilon) is set very low, at which point the embeddings lose enough fidelity that retrieval quality degrades to the point of uselessness. You can have privacy or utility. The tradeoff curve is steep and unfriendly.

More recent work, like the SPARSE / Concept-Aware Privacy Mechanisms paper at ICLR 2026, is trying to find better points on that curve. The fact that papers are still being published on how to solve this is itself evidence that the problem is unsolved. If your vendor tells you they have a defense, ask them which one, and then check the literature on whether it has been broken. The answer is usually yes, or "not tested against ZSInvert-class attacks."

Are Embeddings Personal Data Under GDPR?

Increasingly, yes. A UK GDPR legal analysis argues that an embedding is a numerical representation derived from source text, and where that source contains personal data and the embedding can be linked back to an identifiable person (through stored pairing with original chunks, metadata, or re-identification from the vector itself), the definition of personal data is engaged. The ICO's position on anonymisation asks whether re-identification is "reasonably likely," not whether the data merely looks scrambled.

A float array looks scrambled. That does not make it anonymous. Not when an 80%+ sensitive-information recovery rate is published and freely reproducible.

The EU's November 2025 Digital Omnibus proposal would narrow the GDPR definition of personal data around re-identification likelihood, which is a live debate about exactly the anonymization assumption that embedding-based products lean on. Whether that narrowing helps or hurts vendors storing embeddings depends on where the re-identification threshold lands. Given the current research trajectory, I would not bet on "helps."

Why Is Deletion So Hard with Embeddings?

Because there is no clean mapping between a user's data and which vectors it produced. In a relational database, deleting a user's records is a WHERE clause. In a vector store, a single user's data may have been chunked, embedded, and stored as dozens or hundreds of vectors with no consistent key tying them back to the originating user. Metadata linkage is an application-layer concern, not a vector-database primitive. Deletion is structurally hard in a way that most teams discover only when a DSAR (data subject access request) or CCPA deletion request arrives and they realize they cannot fulfill it.

This is not an inversion attack. It is a mundane, certain, auditable compliance gap. Procurement teams should be asking: "Show me how you delete all vectors derived from a specific user's data within 30 days." If the answer involves manual engineering work or "we'd have to re-embed the whole corpus," that is a failed control.

What Should Your Threat Model Actually Look Like?

The researchers' framing is the right one: treat a breach of your vector store as a breach of the underlying documents. Model it that way in your risk register. If that changes your compliance posture, your compliance posture was wrong.

Concretely, this means:

• Embeddings in a vector database need the same access controls, encryption at rest, and audit logging as the source documents.
• Your data processing agreements with vector store providers need to account for the fact that embeddings are (likely) personal data, not anonymized derivatives.
• Deletion workflows need to trace from user identity to every vector derived from that user's data, which means building and maintaining that mapping at the application layer.
• Similarity search endpoints need access controls and rate limiting, because unauthenticated similarity search is a membership inference oracle.

None of this is exotic. It is the same set of controls you would apply to a database full of plaintext documents. The point is that "we store embeddings" does not buy you an exemption from any of it.

What Does This Look Like in Practice? A Bug Class We Hit.

When we were building Selina's memory system, we ran into a bug class that crystallized the whole problem. The naive approach is: take user input, embed it, store the embedding. The slightly less naive approach is: encrypt the text, then embed the encrypted text. This sounds like defense in depth. It is actually useless, because the embedding of encrypted text captures the statistical structure of the ciphertext, not the semantics of the plaintext. Your retrieval breaks. So you do the obvious fix: embed the plaintext (for retrieval), encrypt the plaintext (for storage), store both. Now you have an unencrypted embedding sitting next to encrypted text, and the embedding is the weaker link.

The order-of-operations matters, and it is easy to get wrong. "Encrypt before embed" sounds right but produces garbage retrieval. "Embed then encrypt the embedding" preserves retrieval only if you decrypt before search, which means the decrypted embeddings exist in memory during every query. The attack surface is not in the storage layer. It is in the computation path.

We ended up with a design where memory content is encrypted at rest, but we are honest about the fact that a slice of each request reaches a frontier provider at inference. Selina's memory is not end-to-end encrypted, because the computation has to happen somewhere, and that somewhere is not a fully trusted enclave. Files and transfers via SelinaSEND are zero-knowledge encrypted. Memory is not. We say so because the alternative is pretending, and pretending is the exact failure mode this entire post is about.

How Should You Evaluate a Vendor's "Embeddings Only" Claim?

Ask these questions. Get the answers in writing.

1. Where are the embeddings stored, and who has read access? If the answer is "a managed vector database service," ask which one and what their breach notification history looks like. Treat that access as equivalent to document access.
2. What is your deletion workflow for a specific user's data? Can they trace from a user ID to every vector derived from that user's inputs? If not, they cannot fulfill a DSAR.
3. What embedding model do you use, and have you tested it against published inversion attacks? If they say "our embeddings are not invertible," ask them to cite the paper. If they cite a paper from before March 2025, point them to ZSInvert.
4. Are similarity search endpoints authenticated and rate-limited? If not, they have a membership inference vulnerability that requires no sophistication to exploit.
5. Do your DPAs classify embeddings as personal data or anonymized data? If anonymized, ask them how they reconcile that classification with the ICO's "reasonably likely" re-identification standard given published 80%+ recovery rates.

Most vendors have not thought through these questions. That itself is the answer.

Where Is This Going?

The attacker's cost is dropping. ZSInvert removed the per-encoder training requirement. Attribute inference never needed an inversion model to begin with. Defenses are not keeping pace. Regulatory definitions are tightening toward treating embeddings as personal data. The "embeddings are safe" window, if it ever existed, is closing.

The correct architectural response is not "find a better way to obscure the embeddings." It is to build your trust boundaries as if the embeddings are the documents, because functionally, they are. Encrypt at rest. Control access. Maintain deletion mappings. Be honest about what is and is not end-to-end encrypted. And stop using "we only store embeddings" as a sentence that ends a compliance conversation. It should start one.

If you want an AI assistant that remembers you across conversations and is built around these constraints instead of around pretending they don't exist: start a free 7-day trial, no card required.

Michael C.

Founder & Principal Engineer, Selina Labs

Michael builds Selina, a privacy-first AI that remembers you across conversations. He ships security-sensitive AI in production — real attacks, real fixes, measured in minutes and dollars — and writes about privacy, security, and LLMs from that seat. Top Rated Plus and expert-verified on Upwork.

• www.upwork.com
• x.com
• medium.com
• selinalabs.io