You cannot test an AI agent like normal code. You can test most of the workflow.

Ask a developer how they know their code works, and they point to the tests. Same input, same output, every time. A test that passed yesterday passes today. If it fails, something changed.

Now put a language model in the middle. Give it the same input, and the answer is a little different on each run. The words change. The order changes. Sometimes the answer is better, sometimes worse. A test that checks for exact text fails for no good reason, and soon the team stops trusting it. Then the team stops running it.

This is a real gap. But it is smaller than it looks. The model is not deterministic. Most of the workflow is.

One team reported that they wanted plain unit tests for their agent workflow. Their framework's documentation showed how to test one way of building a workflow, but not the other. The storage that holds a run's saved state was fixed when the code loaded, so their tests could not swap in a test store. They built wrappers and global switches to get around it. They called these workarounds hacks, and some of them broke tests that ran in parallel.

Notice what they were trying to test. Not the model's writing. They wanted to check the plumbing: that state is saved, that a step gets the right input, that the run goes where it should. That part should be easy to test. When it is not, the tool is in the way, not the model.

If you are new to agents, what an AI agent is gives the short version. In brief: a model decides, tools act, and a workflow ties the steps together.

Take one run and sort every piece into two groups.

Fixed parts. These behave the same way every time, given the same input.

Tool calls. When a step calls a tool with given arguments, the tool does a known thing. You can test it with fake or recorded data. Output shapes. A step should return certain fields with certain types. A list of results is not empty. Routing. If a value is above a limit, the run goes to branch A. If not, it goes to branch B. That is ordinary logic. Data handling. Parsing, mapping, saving state and reading it back. Model parts. These change from run to run.

The text the model writes. Which tool the model picks, when it has a choice. How it summarises, ranks or classifies. In many workflows, the fixed parts are most of the code. Test them like normal code, with exact checks. The model parts need a different kind of test.

For model output, stop asking "is this the same text as last time?" Ask "does this meet the bar?" And write the bar down before you run anything.

Good criteria are concrete. You can answer each one with yes or no:

The answer names the customer's order number. The summary is under 120 words. The label is one of five allowed labels. The reply does not promise a refund. Every claim in the answer appears in the source documents. Some of these you can check with a few lines of code. Others need a person, or a second model that acts as a reviewer. A second model is useful, but it is also not deterministic. Treat its verdict as a signal, not as proof, and check a sample of its verdicts by hand.

Then run each case several times, not once. One pass tells you little. A case that passes nearly every time tells you something real. A case that passes only half the time shows you a weak spot before your users find it.

Here is the tension. For the model parts, you will never get the certainty of a normal test suite. A set of fifty cases does not cover what real users will send. A pass today does not promise a pass after the model provider updates the model.

So testing before release is not enough. You also need to look at real runs after release and compare what happened with what you expected. A run that ends with a green tick can still have done the wrong job — we wrote about that in a green tick is not a finished job. Tests catch the failures you can imagine. Looking at real runs catches the rest.

That is not a reason to skip tests. It is a reason to be honest about what they prove.

Each layer gets its own kind of test and its own moment to run. Start at the top: the first rows cost almost nothing and catch many quiet errors.

What you test	How you test it	When it runs
The shape of every step's output	Write down fields, types and allowed values. Make the step fail when the shape is wrong.	On every change
Your tools	Alone, with fake or recorded data. No model is involved, so the checks are fast and exact.	On every change
The routing	Replace the model with a stub, a stand-in that always gives the same answer. Feed fixed inputs into each branch and check where the run goes.	On every change
The model's output	A small set of real cases with written criteria. Twenty good cases are better than two hundred vague ones. Run each case several times and track how often it passes.	Before each release and after each model change
Real runs	Compare what happened with what you expected. When a real run fails, add it as a new case.	All the time, after release

Put the cheap rows in CI, so they run without anyone starting them.

In EpicStaff, a step can be given a structured output: you define the shape of the result, and the step must return that shape. If it cannot, the step fails instead of passing. You can start a whole flow with your own test input, and test a Python step on its own. A script or CI job can start a flow run through the API with an API key. EpicStaff has no built-in test or evaluation feature. Keep your own set of test inputs with written criteria, and run the whole flow through the API before each change.