Design an Enterprise Document Search System with Access Control
Your company provides a cloud workspace platform used by enterprise customers to create, edit, and share documents, pages, and comments. The platform needs a unified search engine that allows users to perform full-text queries across all content they have access to.
Because content changes rapidly and enterprise documents contain sensitive data, the search system must ensure strict authorization: a user must never see search result snippets or document titles for content they do not have read permissions for. Permission structures can be complex, involving direct grants, team memberships, and inherited folder permissions.
Design the search, indexing, and authorization pipeline capable of serving sub-100ms queries over billions of documents while keeping search indexes synchronized with real-time content and permission updates.
- Perform full-text search across document titles, body text, and comments with relevance ranking.
- Filter search results by metadata such as author, document type, creation/modified dates, and specific folder scopes.
- Incorporate real-time access control check so search results only show documents the requesting user is currently authorized to view.
- Ingest content additions, edits, deletions, and access permission modifications with low latency.
- P99 search latency under 100ms across 10 billion documents.
- Indexing lag under 5 seconds for document content updates.
- 100% precision on authorization filtering with zero security leaks.
- High availability (99.99%) for the search query path.
10 billion total documents across 100,000 enterprise tenants. Peak search query volume of 20,000 QPS. Content mutation rate (edits/creates) of 5,000 ops/sec. Permission changes occur at 1,000 ops/sec.
- High-level architecture diagram detailing the document ingestion pipeline, search index clusters, and query execution flow.
- Data model for the inverted search index and permission mapping structures.
- Detailed query execution design explaining how authorization/ACL filtering is applied (e.g., pre-filtering vs post-filtering vs early-stage index matching).
- Strategy for handling hot tenants and large-scale permission inheritance updates.
Design articulates a clear strategy for incorporating access control lists (ACLs) into the search query lifecycle (e.g., embedding permission tokens in the inverted index vs executing runtime post-filtering via an authorization service or BitSet filters) and weighs latency vs index update amplification.
Candidate specifies how the search index is sharded (e.g., tenant-based sharding vs document-ID sharding), how query fan-out is controlled, and how hot tenants are isolated to avoid noisy-neighbor performance degradation.
Presents an asynchronous ingestion pipeline using distributed queues (e.g., Kafka) that separates text extraction, tokenization, and index updates while guaranteeing idempotent index writes and near-real-time freshness.
Provides a concrete strategy for handling bulk access permission updates (e.g., modifying permissions on a root folder with 500k documents) without triggering massive write amplification or locking search indexes.
Explains how textual relevance (e.g., BM25) is combined with non-textual ranking signals (recency, document access frequency, user affinity) during multi-stage retrieval and re-ranking.
Every functional requirement in the brief is visibly served by something on the board, and the non-functional targets are addressed rather than ignored.
Components are labelled, data flows are drawn as connections between them, and the direction of each flow is unambiguous.
Follow-up: How does your design handle a permission inheritance cascade—for example, when an enterprise admin changes access permissions on a top-level folder containing 500,000 sub-documents—without overwhelming the ingestion pipeline or causing search stale periods?
Strict ACL enforcement (zero permission leaks across tenants or unauthorized users). P99 search query latency must remain under 100ms. Ingestion lag for newly created or edited documents must be under 5 seconds.
- Views
- 41