A model does not automatically know your documents.
Retrieval brings relevant material into the model's context when a question arrives. Instead of relying only on what the model learned during training, the application searches a collection and supplies selected passages with the question.
“How long can I keep an expense receipt?” The system first needs the relevant policy, the right version and permission to show it. A confident answer from general knowledge is not sufficient.
Three stages, three different jobs
- Prepare. Clean documents, split them into useful passages and retain metadata such as source, date and access permissions.
- Retrieve. Find passages relevant to the question, with permission checks applied before content reaches the model.
- Answer. Give the model the selected context and ask it to produce a response supported by that evidence.
Similarity is a clue, not proof.
Embeddings represent content as vectors. A vector search can locate passages with related meaning, but a nearby vector is not a guarantee that the passage answers the question. Keyword search, filters and reranking can help, depending on the task.
Keep the source attached
Each chunk should retain an identifier that resolves to its source. A citation is useful only if the cited passage supports the claim. Generating reference numbers alone does not establish correctness.
Missing evidence should change the answer
If the collection does not contain the required information, the system should explain that limitation. Depending on the workflow, it can ask a clarifying question, suggest an authorised source or hand over to a person.
Test retrieval and answering separately.
- Did the search find the necessary evidence?
- Was the user permitted to access it?
- Did the response accurately represent that evidence?
- Did the system recognise an unanswerable question?
Our lab makes two of these decisions visible using a tiny synthetic collection. It uses keyword rules, not vector embeddings or a live language model.
Put it into practice
Inspect one answer from end to end.
Choose one non-sensitive policy and write five questions: two with direct answers, one requiring several passages, one the policy cannot answer, and one a different role must not access.
- Write the expected source passage before testing the system.
- Record the retrieved passages separately from the generated answer.
- Mark unsupported claims and missing citations. A fluent answer alone is not a pass.
A small evaluation sheet with a question, expected evidence, permitted role, observed answer and result.