When AI Searches Company Documents, How Do You Make Sure Each Department Sees Only Its Own Data?
Enforce permissions at the retrieval step, not by relying on the AI's judgment. When indexing, tag each document with its department and the roles allowed to read it. When an employee asks a question, the system searches only documents that person may see, so restricted passages never reach the AI. Then verify with test questions for content the user must not see, confirming neither the answer nor the citations expose it.
1. Why "don't reveal this" in the prompt is not enough
In retrieval-augmented generation (RAG), the system first finds relevant passages and then hands them to the model to write the answer. Once a restricted passage is retrieved and placed in the model's context, the model has seen it. Asking it through the prompt not to repeat it is unreliable: the model can make mistakes or be steered by prompt injection.
OWASP's 2025 Top 10 for LLM Applications lists this as LLM08, Vector and Embedding Weaknesses: inadequate or misaligned access controls can allow unauthorized access to embeddings containing sensitive information, and when multiple classes of users share one vector database, context can leak between users or queries. OWASP recommends fine-grained access controls with permission-aware vector and embedding stores, and strict logical and access partitioning of datasets.
2. Four steps
- Inventory documents and tag permissions. Record each document's owner, current version, department, and allowed roles. Documents without permission tags stay out of the knowledge base.
- Store permissions with the index. Each chunk carries the permission tags of its source document. Highly sensitive data, such as HR and payroll, can go into a separate knowledge base.
- Filter before retrieval. Based on the person asking, search only the scope they may see. The system makes the authorization decision, not the model. OWASP LLM06 calls this complete mediation: enforce authorization in downstream systems instead of relying on the model.
- Verify with must-not-see tests. For example, have a regular employee account ask about a rule only managers may see, and confirm neither the answer nor the citations expose it.
3. Three common gaps
| Gap | How it happens | How to close it |
|---|---|---|
| Citations leak | The answer does not say it, but the attached source link or excerpt exposes restricted content | Apply the same permissions to citations |
| Permissions out of sync | An employee changes department or leaves; the file system is updated, but the knowledge base tags are not | Sync permissions from a single source and spot-check regularly |
| Withdrawn document still indexed | An old rule is withdrawn but its passages remain in the index, so the AI answers by the old rule | Update the index when withdrawing documents and log it as a change |
How Think BIG handles knowledge base permissions
Think BIG (Think BIG Make it Real Co., Ltd.) is an enterprise AI Agent deployment provider headquartered in Taiwan. Starting from protecting company data, we build AI Agents inside the client's own environment (company hardware, or a cloud server under the company's account). There are two deployment options: with the on-premises version, models and documents are processed on company hardware; with the cloud-assisted version, employees' questions and the document passages the system retrieves are sent to the model provider for processing.
- Readable scope checked before retrieval: the asker's identity and document permissions are checked first, then relevant passages are retrieved from approved documents. Permissions are enforced at retrieval, not only by telling the prompt "don't reveal".
- Permissions applied to citations too: readable scope is checked by role, and neither answers nor citations may expose restricted content.
- Acceptance includes permission boundaries: we confirm unauthorized users are blocked, and retest after documents are withdrawn or the index is updated.
See RAG knowledge Q&A and Trust.
The actual scope of Think BIG's services follows what both parties confirm.
FAQ
- Q1: Can an AI knowledge base have department-level permissions?
- Yes. Tag each passage with department and allowed roles when indexing, and search only the scope the asker may see, so restricted passages never reach the AI.
- Q2: Is telling the AI in the prompt not to reveal secrets enough?
- No. Once a restricted passage is retrieved into the context, the model has seen it and may reveal it by mistake or under prompt injection. The system must block it at retrieval.
- Q3: Is it safe for several departments to share one vector database?
- Only with access control. OWASP LLM08:2025 warns that when multiple user groups share a vector database, data can leak between users, and recommends permission-aware stores with strict partitioning.
- Q4: How do you confirm permissions actually work?
- Use must-not-see tests: have an account without permission ask about restricted content and confirm neither the answer nor the citations expose it. Test again after staff changes and document withdrawals.
- OWASP, LLM08:2025 Vector and Embedding Weaknesses (unauthorized access, cross-context leaks, permission-aware stores and partitioning): genai.owasp.org
- OWASP, LLM06:2025 Excessive Agency (complete mediation): genai.owasp.org
- OWASP, LLM02:2025 Sensitive Information Disclosure: genai.owasp.org
- Think BIG, RAG knowledge Q&A (permissions enforced at retrieval, common failure causes): thinkbigtw.com
This article summarizes public official documents, laws, and security standards to help companies evaluate AI adoption. It is not legal advice. For Taiwan law, the latest text on the Laws & Regulations Database prevails; external documents, prices, and specifications follow their original sources. The actual scope of Think BIG’s services depends on what both parties confirm.