Stale ACLs in RAG: When Revoked Access Still Returns Docs

Before tuning a propagation window, confirm the filter fires at all. The access-control documentation (page metadata: ms.date 2026-06-08, touched 2026-08-03) records a period where it did not:

"Before REST API version 2025-11-01-preview, earlier preview versions 2025-05-01-preview and 2025-08-01-preview returned all documents when using a service API key or authorized Entra roles, even if no user token was provided. Applications that didn't validate the presence of a user token could inadvertently expose results to end users if not implemented correctly or following best practices."

That is not drift. That is the whole index returned to any caller holding a service key. From November 2025 the behavior changed: filters now apply even when using only service API keys or Entra authentication, and if the user token is omitted, ACL-protected content is not returned.

The second gate is incremental sync:

"Incremental ACL updates require the 2026-05-01-preview REST API or later. In earlier preview API versions, the system captures ACLs only on the first ingestion of each item. Later permission changes require explicit reindexing."

Two constraints to surface at design time: ACL ingestion requires application permissions and delegated permissions are not supported, and the Azure portal does not support this feature, making every step here REST or SDK work. All of it is preview, not GA, which matters for a control you plan to put in front of a regulator.

Two terms carry this design space. Both are enterprise-search vocabulary rather than vendor-defined ones, with no canonical definition on any Azure page, so settle what you mean before an architecture review rather than during one.

Early binding resolves permissions at index time and materializes identifiers into the index. Retrieval filters on a copy. Fast, self-contained, no runtime dependency on the source system, and drifting by construction.

Late binding stores a pointer and evaluates access against the source system per query. No drift, at the cost of coupling every retrieval to the availability and latency of whatever holds the truth.

Azure AI Search is early binding for the ACL payload, with one late-binding step:

"Query-time enforcement evaluates the caller's Microsoft Entra claims against the permission metadata that's already stored in the index."

"Group expansion occurs at query time through Microsoft Graph."

That second quote kills a common design assumption. Nested group resolution is server-side, and the overview page lists it as a benefit: you do not need to implement nested group resolution, multilevel ACL traversal, or post-query trimming in your application. Teams that flatten group membership into the index themselves are building drift they did not need.

Microsoft also documents its own escape hatch, which reads as a scoping statement rather than a footnote:

"For scenarios that require the full SharePoint permissions model, sensitivity labels, and out-of-the-box security trimming, use a remote SharePoint knowledge source. This approach calls SharePoint directly via the Copilot retrieval API. Governance remains fully in SharePoint, and query results automatically respect all applicable permissions and labels."

Read plainly, and this reading is ours rather than doc text: if you need the real permission model, do not materialize it. Any runbook omitting that sells you the harder option without mentioning the easier one.

Do not let that table imply the SharePoint ACL path carries sensitivity labels. It does not: labels are marked unsupported there and handled by a separate feature, which carries its own preview constraints (single tenant, RBAC auth, REST and SDK only, no Autocomplete or Suggest on Purview-enabled indexes, system-assigned managed identity).

The indexer uses SharePoint change tokens to pick up role assignment additions and removals incrementally, in the same way it picks up content changes. That covers less than the phrase "incremental ACL updates" suggests.

The two "No" rows are where the exposure lives. Revoking a group at the library level is routine SharePoint administration, and it is documented as not picked up automatically on subsequent indexer runs. An operator who has watched an item-level change land on the next indexer run has every reason to expect the library-level change to behave the same way. It does not propagate at all.

POST https://{service}.search.windows.net/indexers/{indexer}/resync?api-version=2026-05-01-preview
Content-Type: application/json

{ "options": ["permissions"] }

POST https://{service}.search.windows.net/indexers/{indexer}/resetdocs?api-version=2026-05-01-preview
Content-Type: application/json

{ "documentKeys": ["doc123", "doc456"] }

One caveat that costs an entire window: to fully succeed, resync requires an indexer run after completion. The call marks work to be done. It does not do the work.

Per-source behavior differs enough to matter:

Content freshness and permission freshness are different clocks with different triggers: a webhook firing on document edit will not fire when someone changes who can read it. Our guide to updating a RAG knowledge base without rebuilding everything covers the content half; this is the permission half, and it needs independent monitoring.

The scheduling documentation gives both ends of the range:

"The amount of time between the start of two consecutive indexer executions. The smallest interval allowed is 5 minutes, and the longest is 1,440 minutes (24 hours). Format it as an XSD 'dayTimeDuration' value... The pattern for this value is: P(nD)(T(nH)(nM))."

The obvious arithmetic, lag equals schedule interval plus indexer run duration, is our derivation and not a vendor statement. Treat it as a floor that holds only when nothing else on this list fires.

The failure-backoff row compounds. One document failing repeatedly, a corrupt PDF or a transient error on a single library, degrades the refresh cadence for everything that indexer touches: a five-minute cadence backs off toward the documented maximum of at least once every 2 hours or 24 hours because of an unrelated content problem, and no security dashboard says so. Best case is small and knowable, worst case is not bounded by documentation, and the difference is invisible unless you instrument it.

Integrated vectorization is where correctly configured ACLs quietly stop existing.

"In all chunked scenarios, every chunk must carry the ACL fields. Permission filters apply per document, so a chunk missing ACL fields can't be returned to the right caller."

"If your skillset uses projectionMode: skipIndexingParentDocuments, indexer field mappings for ACL fields are bypassed. Set ACL fields via indexProjections.mappings on every chunk instead."

Nothing is stripped and nothing errors. The mappings are bypassed, so the fields land empty, and since the parent document is not indexed, the query-time filter runs against chunks whose permission fields are blank. Project all three sources onto every chunk:

{
  "indexProjections": {
    "selectors": [{
      "targetIndexName": "chunks",
      "mappings": [
        { "name": "UserIds",  "source": "/document/metadata_user_ids" },
        { "name": "GroupIds", "source": "/document/metadata_group_ids" },
        { "name": "SharePointSiteUrl", "source": "/document/metadata_spo_site_url" }
      ]
    }]
  }
}

The field name is metadata_spo_site_url, not metadata_sharepoint_site_url, and the docs call the wrong spelling out as a named troubleshooting symptom. A two-index parent plus child pattern needs both field mappings on the parent index and index projections on the chunk index. Every strategy in our document chunking guide inherits this requirement.

Quote both sentences, because the first scopes the second:

"The following indexer features don't support permission inheritance in indexed documents originating from SharePoint. If you use any of these features in a skillset or indexer, document-level permissions aren't included in the indexed content."

None of the four looks like a security feature, which is the problem. The enrichment cache and debug sessions get switched on during development for cost and iteration speed, then survive into a pipeline nobody re-reads. Any of them turns a permission-filtered index into an unfiltered one, and the symptom is documents returning to everyone, not an error.

ADLS Gen2: "Each file or directory can have up to 32 ACL entries permissions."

SharePoint: "SharePoint data source in search supports up to 1,000 permission entries per file. Each entry represents a unique user or group assignment in the item's permission list."

"If an item exceeds these ACL entry limits, permissions beyond the limit might not be enforced at query time."

Might not be enforced: no error, no truncation warning, no distinguishing log line. The documents most likely to exceed these caps are the ones a regulated deployment cares about most, the long-lived, heavily shared files that accumulate individual grants over years. Thirty-two entries is not a generous ceiling for a directory in a mature tenant.

Two semantics change how you model policy. Filters are OR, and there is no deny primitive. Each filter is applied independently and a document is authorized if any filter succeeds, so a user with access through userIds but not through groupIds still gets the document. A SharePoint model leaning on deny or limited-access entries cannot be expressed here: denial has to be handled upstream by not granting. Several principal types are not evaluated at all. Information Management policies are not evaluated, ingested, or honored at query time. Only shareable links scoped to "Specific people" are supported. External and guest users are unsupported, as are Purview labels on this path.

SharePoint site groups are supported from the 2026-05-01-preview, with group IDs carrying an spg: prefix so they stay distinguishable from Entra group object IDs. That path needs a sharePointConnectorAppRegistration block (applicationId, tenantId, federatedCredentialId) on the index plus a SharePointSiteUrl field flagged sharepointSiteUrl: true.

{
  "permissionFilterOption": "enabled",
  "fields": [
    { "name": "UserIds",  "type": "Collection(Edm.String)",
      "permissionFilter": "userIds",  "filterable": true, "retrievable": false },
    { "name": "GroupIds", "type": "Collection(Edm.String)",
      "permissionFilter": "groupIds", "filterable": true, "retrievable": false }
  ]
}

The data source carries indexerPermissionOptions: ["userIds", "groupIds"]. Queries pass the end-user token in x-ms-query-source-authorization, and the client application separately needs Search Index Data Reader or Search Index Data Contributor. If the query token is omitted, only public documents accessible to everyone are returned. retrievable can be flipped from true to false with no index rebuild, which is the documented way to inspect what actually landed in the permission fields and then close it again.

Fail-closed is the design, and it has an availability bill. If ACL evaluation fails, for example because the Graph API is unavailable, the service returns 5xx and does not return a partially filtered result set. That is correct, and it means a Microsoft Graph incident becomes a retrieval outage by design. Put it in the runbook before an on-call engineer decides the search cluster is broken and starts restarting things.

Sites.Selected separates a defensible design from a tenant-wide one. Granting Sites.FullControl.All so a search indexer can read permission metadata is a privilege inversion that will not survive review. The User.Read.All requirement has a documented reason: for list items and ASPX pages the SharePoint REST API returns only the user's email, so the indexer calls Microsoft Graph to resolve each email to an Entra object ID. Client secrets work only for the Microsoft Graph-only document-library row; everything else requires a federated credential. Same principle we argue in role-based access control for AI applications: the privilege a system holds to enforce a policy should never exceed the privilege the policy protects.

Stop estimating and measure. The mechanism is x-ms-enable-elevated-read: true, gated on Search Index Data Contributor or a custom role including Microsoft.Search/searchServices/indexes/contentSecurity/elevatedOperations/read. It works only on Search POST actions, and you cannot perform an elevated read query on a knowledge base retrieve action.

The procedure, in a staging tenant that mirrors production topology:

Permission failures are not what generic RAG hardening catches. LayerRAG-Bench, a single-author arXiv preprint that has not been peer reviewed, spans 8 enterprise domains, 240 tasks, 9 fault scenarios, 2 contract modes and 38,880 live task-level records across nine models, and reports on that unreviewed basis that schema normalization raises schema-drift success from 0.000 to 0.913 while stale evidence, missing tool output, denied permissions and wrong-session context survive the same fix, and that groundedness-only evaluation produces substantial false positives under stale evidence. It reports no leak rate, so read it as directional. The takeaway is structural. The eval suite certifying your retrieval quality will not notice a permission problem.

Retrieval quality and retrieval authorization are separate axes. Our analysis of enterprise deep-search agents and the retrieval bottleneck touches permission-scoped retrieval repeatedly, but from the recall side: whether an agent finds evidence it is entitled to. This is the opposite failure, and the metrics detecting one are blind to the other. Tenant partitioning is a third, separate control, covered in multi-tenant RAG isolation: silo, pool, or bridge.

The same drift exists in self-hosted stacks, minus the documentation. Two things port cleanly.

The selectivity tax on filtered vector search. Where a user is entitled to a small share of the corpus, the permission filter is the most selective predicate in the query, and approximate indexes handle selective filters badly. OpenSearch documents the constraint: because the native library indexes are constructed during indexing, it is not possible to apply a filter on an index and then use this search method, and all filters are applied to the results produced by the ANN search. With post-filtering, because it is performed after the vector search, this approach may return significantly fewer than k results for a restrictive filter. The mitigation is choosing the filter mode deliberately rather than accepting post-filtering as inevitable.

In OpenSearch 3.1 and later, with the Faiss engine and HNSW, the Lucene ACORN filtering optimization is applied during HNSW traversal when memory-optimized search is enabled. From 3.5, when a Faiss efficient-filtered search returns fewer than k results even though more than k documents match the filter, an exact-search fallback guarantees k results, and index.knn.faiss.efficient_filter.disable_exact_search turns it off for latency-sensitive workloads. For a permission filter, disabling it means a user sometimes not seeing documents they are entitled to: the safe direction of the error, but still a recall regression you should choose on purpose. Note also that OpenSearch document-level security is a separate Security-plugin feature from k-NN filtering, and conflating the two produces an architecture where neither does what you assumed.

Take, as a hypothetical rather than a measurement, a user entitled to a fraction of a percent of the corpus. As the entitled subset shrinks, post-filtering returns emptier result sets until an over-restrictive permission filter is indistinguishable from broken search, and teams widen the filter to "fix" it. Tuning mechanics are in our pgvector HNSW tuning guide; the ACL-specific point is that permission predicates sit where approximate search is weakest.

The custom pipeline is a real option with a published shape. Microsoft's engineering blog published a walkthrough in April 2026 for propagating SharePoint document permissions into AI Search and RAG pipelines with a custom Microsoft Graph integration rather than the native indexer: Sites.Selected for least privilege, per-item permission reads, every identity normalized to an Entra object ID, allowed-users and allowed-groups fields materialized into the index, and query-time filtering. It states the tradeoff cleanly, that permission changes in SharePoint are not automatically propagated to downstream systems and a revoked user may still see the document's content in the search index or RAG pipeline until the next ingestion run. Its pitfalls are worth adopting whichever path you take: filtering after retrieval means the data already leaked to the application layer; emails and display names change while GUIDs do not; ignoring group expansion leads to silent overexposure; and assuming real-time sync is wrong, so plan for periodic refresh. No staleness-window number appears there either.

The custom path buys least privilege and full control of refresh cadence, and costs you the change-token machinery, /resync, /resetdocs and server-side group expansion, all of which you now build and operate. The native path gives you those and costs you a preview-stage feature, tenant-wide Graph permissions unless you scope with Sites.Selected, and the unsupported principal list above.

The failure here is rarely a bad filter. It is a correct filter running against a copy of the truth made at a moment nobody wrote down, in a system where three documented behaviors can silently widen the gap between that moment and now. State the propagation window as a measured number with a remediation SLA attached and it stops being a security unknown and becomes an operational parameter, which is the form an auditor can work with. What leaves the boundary after retrieval is a separate control, covered in preventing data leakage in AI applications, and the sector-specific version shows up in healthcare AI attack vectors HIPAA does not cover.

Particula Tech builds and audits permission propagation for on-premise and private-cloud RAG in healthcare, finance, legal and public sector: binding model, sync topology, privilege scoping, chunk-level ACL assertion, and the negative test suite that produces a defensible window in minutes rather than an assurance in adjectives. More in our AI security pillar.