Start with the code that exists.
The RAG Document Assistant has two entry points. ingest.py reads two text files, splits them into overlapping character chunks and stores them in a local Chroma collection. app.py retrieves passages for a question and sends that context to Azure OpenAI through a Streamlit interface.
This separation keeps document preparation out of the answering loop. It also creates a dependency worth making visible: the answering application needs indexed text before it can provide a document-based answer.
In the reviewed baseline, the application creates the collection if necessary, constructs the Azure client, and asks for two retrieved results when a question arrives. It joins the returned text directly into a prompt. There is no explicit application check for an empty collection or an empty context.
Pull request #1 proposes the change described here. It is available for review and has not been merged as of this article's publication. This is a development project, not a production assurance claim.
Make the application decide when to stop.
A system prompt asks the model to answer only from the supplied context and acknowledge missing information. That is a useful instruction. An empty context is a condition the application can detect before asking the model anything.
| Condition | Proposed behaviour |
|---|---|
| No records in the collection | Show ingestion instructions and stop before retrieval or Azure client construction. |
| One indexed record | Request one result instead of two. |
| No usable retrieved text | Show a clear response in chat history and skip Azure generation. |
| Non-blank passages returned | Continue with the existing context-based answer path. |
The change uses the collection's record count for the first check. A second check examines the returned document text, ignoring missing and blank passages. Azure client construction moves after that check, so an empty-evidence path does not need to initialise the generation client.
Text presence is not evidence quality.
A non-empty passage can still be irrelevant, outdated or misleading. This guard does not detect those conditions. It does not prove an answer is faithful, enforce document permissions or make retrieved content safe from prompt injection. Those are separate design and evaluation problems.
The useful claim is deliberately narrow: when no usable document text is available, this path does not ask Azure to generate an answer.
Test the boundary you can actually observe.
Five offline tests passed for the proposed change. They cover an empty collection; absent, empty and blank retrieval text; a one-record collection; passage content reaching the prompt; and an idle page with no question.
The tests replace Chroma, Streamlit and Azure with test doubles. They can check whether retrieval or generation was called, what context was sent, and whether the explanatory response was retained in chat history. They do not establish real retrieval quality, browser rendering or live Azure integration.
That distinction matters when presenting an engineering portfolio. A green test result is evidence for a specific behaviour. It is not evidence that the whole system is ready for operational use.
The next useful increments
- Attach source identifiers and show citations with answers.
- Make re-ingestion update changed documents predictably.
- Create a small evaluation set with answerable and unanswerable questions.
- Handle missing configuration and service failures clearly.
Put it into practice
Trace the failure path before the happy path.
Review the pull request and run its offline tests from the repository root:
python -m unittest discover -s tests -v- Find the check that prevents retrieval against an empty collection.
- Find the second check that prevents generation without document text.
- Explain why unrelated but non-empty text still passes these checks.
A short review note stating what this guard guarantees, what the tests verify, and what still needs a live integration test.
Continue exploring