The agent kept answering. It had lost its knowledge.

A student built an AI agent for a final-year project. The agent answered technical questions from a folder of manufacturer manuals, stored as PDF files in a cloud document library. The student published the agent, and it worked for a few days. Then, after a period of testing with several questions, the agent stopped syncing with its knowledge base. That is what the student reported in the vendor's community forum.

The agent stayed published. It kept giving answers. But it no longer used the manuals it was built to answer from. The student also reported that the platform did not say why the sync had stopped.

From the outside, it still looked like a working agent.

A user who runs an agent builder on their own server, in Docker, reported on the project's issue tracker that a flow stopped indexing after an upgrade. A web application sent Word files to the flow. The flow split each file into pieces and stored them in a vector database. After the upgrade, the call still returned success, and the server's own history showed the call. But nothing new was indexed, and there was no error. A later version was still broken.

A user who runs an LLM app platform on their own server reported on the project's issue tracker that a published chat app lost part of what its knowledge search found. The search step found pieces of documents with images attached. In the builder's preview, run as the user's own account, the images reached the model. For end users of the published app, they never did. The run succeeded, with no error and no log line. The model wrote its answer from the caption text alone. The user counted runs on their own server over seven days: 19 preview runs, 5 of them with images; 24 end-user runs, none with images.

In each case the agent, or the flow behind it, kept working. Answers came out. Calls returned success. What was missing was the knowledge: a document folder that no longer synced, files that were never indexed, images that never reached the model.

None of the three showed an error. Each person found the gap because they looked for it.

A green tick on a run says that the run ended. It does not say that the agent had its knowledge when it answered. For more on runs that report success and did not do the job, see A green tick is not a finished job.

The third report adds one more lesson: the test passed and production failed. The preview ran as the builder's own account; real users ran as someone else. A test that does not run the way users run can miss the fault.

These checks apply to any tool that lets an agent answer from your documents, ours included. Where EpicStaff answers a check, the answer is under it.

1. See the indexing status of every document

Indexing is the step that splits a document into chunks, short pieces of text, and stores them so that the search can find them. A document that is in the folder but not indexed is not in the agent's knowledge. Find the screen that shows the status of each document. Check it after every upload, not only on the first day.

In EpicStaff: EpicStaff shows an indexing status for each document.

2. Know how a failed document shows, and who would notice

Indexing can fail for one document and work for all the others. Find out what a failed document looks like in your tool. Then decide who looks, and how often. A failure mark that nobody looks at tells nobody anything.

In EpicStaff: A failed document is marked failed. Nothing alerts you; you have to look.

3. Check that the run record shows what the knowledge search returned

When an answer looks wrong, the first question is: what did the search give the agent? If the run record shows only the question and the answer, you cannot tell a bad search from a bad answer. Look for the chunks the search returned, and the file each one came from. For what else a run record should hold, see The agent finished. Nobody can say what it did.

In EpicStaff: For the standard knowledge search, the run record shows the chunks the search returned, with their text, file name and score.

4. Know what the agent says when the search finds nothing

A search can return nothing. The question can be outside the documents, or the index can be empty. Many agents then answer from the model's general training, in the same calm tone. Write instructions that tell the agent to say so when the documents gave it nothing. Then test those instructions with a question that your documents cannot answer.

In EpicStaff: An empty search is visible only if you open the run record. The answer itself looks normal.

5. Know whether the agent runs when its knowledge base is empty, still building or failed

There are three states to ask about: the knowledge base has no documents, it is still indexing, or indexing failed. In each state, does the agent refuse, warn, or answer as usual? Try each state on a test agent, so that you know the answer before a user finds it.

6. Know when each document was last indexed

A sync that stops leaves the index as it was on that day. The source can change after that, and the agent will not know. For a few important documents, compare the date the source last changed with the date the index last took it in. If you cannot see the second date, keep your own record of each re-index.

7. Ask a question only the knowledge base can answer, on a schedule

Pick five questions whose answers are only in your documents: a part number, a clause, a figure from a manual. Write down the right answers. Ask the agent those questions on a fixed day each week, and compare. Ask them the way your users do: through the published agent, with a normal user's account, not only in the builder's preview. The third report above shows why.

People ask an agent because they do not know the answer. That is the point of the agent. It also makes the user the person least able to see that an answer did not come from your documents.

So these checks cannot be left to users. Someone who knows the documents has to own them: the status screen, the run records and the weekly questions. Name that person before the next sync stops.