AI
Multi-modal RAG pipelines — ones that ingest scanned documents, product photos, or diagrams alongside text — widen the attack surface of a normal RAG system: there are now more stages (OCR/vision extraction, image embedding, text embedding) and more places sensitive content can leak. Zero-trust architecture, as defined in NIST SP 800-207, doesn't add a perimeter around this — it removes the assumption of a trusted perimeter entirely and requires every request, at every stage, to be explicitly verified.
Applying zero trust to each pipeline stage
1. Pre-retrieval authorization: Access control has to be enforced before a document or image enters a user's context — not as a filter applied to the model's answer afterward. Each piece of content gets a sensitivity label (e.g. Public, Confidential, Restricted) at ingestion, and retrieval queries are evaluated against both the requester's permissions and that label before anything is returned.
2. Service-to-service authentication: The ingestion service, the vision/OCR extraction step, the embedding service, and the retrieval API should each verify the identity of the service calling them — typically via mutual TLS (mTLS) with SPIFFE-style workload identities — rather than trusting a request just because it originated inside the same cloud VPC.
3. Vector database hardening: Encrypt embeddings at rest, scope every query with a per-tenant filter so one customer's retrieval calls can never touch another's vectors, and keep the retrieval service's database credentials read-only — it should query the vector store, never write to or reconfigure it.
# Retrieval enforces authorization BEFORE the vector search runs,
# not as a filter on results afterward
def authorized_retrieve(query, requester, vector_db, top_n=5):
# 1. Resolve what this requester is allowed to see
allowed_tenant = requester.tenant_id
allowed_labels = requester.clearance_labels # e.g. ["Public", "Confidential"]
# 2. Query filter is applied at the database level, not in application code,
# so a bug in retrieval logic can't leak across tenants or classifications
results = vector_db.search(
query,
filter={
"tenant_id": allowed_tenant,
"sensitivity_label": {"$in": allowed_labels}
},
limit=top_n
)
return results| Pipeline Stage | Zero-Trust Control | What It Prevents |
|---|---|---|
| Ingestion (OCR / vision extraction) | Sensitivity labeling at intake | Unlabeled content silently entering the retrieval pool |
| Embedding & vector store | Encryption at rest, per-tenant query filters, read-only service credentials | Cross-tenant leakage, unauthorized writes to the index |
| Retrieval API | Pre-retrieval access check against requester's clearance | Restricted content ever entering the model's context window |
| Inter-service calls | mTLS with SPIFFE-style workload identity | Any internal service being trusted purely by network location |
The common failure mode in RAG security is applying access control as a filter on the model's output — checking after the fact whether the answer contained something the user shouldn't see. By the time that check runs, the restricted content has already entered the context window, which is the actual point of exposure. Zero trust for RAG means moving every one of these checks earlier, to the moment content is requested, not the moment it's returned.
Practical Challenge
Design a query-filter schema for a mock vector database that scopes results by both tenant_id and sensitivity_label, then write a test that confirms a "Confidential"-cleared user cannot retrieve a "Restricted" document even if the raw vector similarity score is highest.
Concept Check
Sources & Further Reading
- NIST Zero Trust Architecture Project Overview — the source of the "never trust, always verify" and continuous policy evaluation principles applied here.
- Secure RAG Pipelines: Prevent Data Leakage with Zero-Trust — Kiteworks — covers pre-retrieval authorization, sensitivity labeling, and vector database hardening for RAG specifically.