Module 4 · Page 4 of 8 · 25 minutes

Verify before you trust

An agent can produce something that looks polished and still be wrong. Verification means checking not only whether the output looks right, but whether it actually works, is correct, and holds up under realistic cases.

Inspection is not the same as verification

When an agent finishes a task, your first instinct may be to look at the result and ask: Does this look like what I wanted?

That's useful, but it's only the beginning. A dashboard can look professional while calculating a metric incorrectly. A form can work for the example you tried but fail when a field is blank. A classification tool can run successfully while using the wrong business rule.

It helps to distinguish two activities:

Activity	Question	Example
Inspect	Does the output look like what I asked for?	The dashboard has the right charts and seems easy to use.
Verify	Does it actually behave correctly?	The totals match the source data and the filters work as intended.

In vibecoding, verification is one of the main ways the human stays in control of the work.

Why verification matters

An agent can produce the wrong result for many different reasons. It might:

misunderstand your task,
misunderstand the context or business rules,
make an incorrect assumption about the data,
write code with a bug, or
build something that works for one case but fails for others.

Importantly, the problem isn't always that the AI “made something up.” Sometimes the code is perfectly valid — it simply implements the wrong rule.

Suppose you asked an agent to calculate customer lifetime value and your company defines revenue as net revenue after returns.

The agent could write flawless code that uses gross sales instead. The program runs. The numbers look plausible. But the answer is still wrong because the agent misunderstood the business definition.

A simple verification framework

When reviewing something the agent built, work through four questions:

Does it look right?
  ↓
Does it work?
  ↓
Is it correct?
  ↓
What could break it?

1. Does it look right?

Start as a user. Is the output organized the way you expected? Are important elements missing? Does the interface make sense? Does the result seem to address the original business problem?

2. Does it work?

Actually use the thing that was built. Click the buttons. Change the filters. Submit the form. Upload the file. Run the workflow from beginning to end.

3. Is it correct?

Check some outputs against answers you already know or can calculate independently. A system that runs without errors isn't necessarily producing the right answer.

4. What could break it?

Try cases that are unusual, incomplete, or near important boundaries. Don't test only the example most likely to work.

Example: verifying a commission calculator

Suppose you ask an agent to build a simple sales commission calculator. The rule is that salespeople earn 5% commission on sales up to $100,000 and 7% on sales above $100,000.

Does it look right?

The page has a place to enter sales and clearly displays the calculated commission.

Does it work?

You enter a sales amount, click Calculate, and the app returns a result.

Is it correct?

Enter a value where you know the answer. For example, at $50,000 in sales, the commission should be $2,500.

What could break it?

What happens with $0 in sales? A blank input? A negative number? Exactly $100,000? $100,001? A very large value?

Notice that you don't need to understand the underlying code to perform these checks. You only need to understand what the system is supposed to do.

Test cases where you know the answer

One of the most useful verification techniques is to create known cases: inputs for which you already know what the correct output should be. For example:

What the agent built	Known case you could test
Commission calculator	Enter a sales amount where you can calculate the commission by hand.
Inventory dashboard	Compare one region's total with the original spreadsheet.
Customer segmentation	Check several customers whose category you can determine yourself.
Expense form	Try one case you know should be accepted and one that should be rejected.

If the system can't pass simple cases with known answers, there's little reason to trust it on cases where the answer is unknown.

Tip You don't have to come up with every known case yourself. You can ask the agent to generate test cases with known answers, or build them independently — in a separate conversation, or using a different dataset — so the tests aren't shaped by the same assumptions that produced the original work.

Example: turn business rules into test cases

Suppose you are automating a tax-filing workflow for employees. You have two useful sources of information:

a document listing the tax rules and filing requirements the workflow must follow, and
interviews with employees describing cases that are especially complicated or time-consuming.

Before trusting the automation, you can use those materials to create a set of known test cases. Some should represent normal situations, while others should capture the difficult cases employees encounter in practice.

Example prompt

Read @tax-rules.md and @employee-interviews.md.

Create a set of test cases for this tax-filing workflow.

Include:
- straightforward cases that test the main rules,
- cases mentioned by employees as difficult or time-consuming,
- combinations of rules that could be easy to mishandle.

For each test case, specify:
1. the input situation,
2. the rule or requirement being tested,
3. the correct expected outcome.

Do not test the workflow yet. First give me the test-case list
so I can review it.

After reviewing the list, you can use the same cases to test the workflow you built:

Example prompt

Now run the approved test cases against the workflow.

For each case, show:
1. the expected result,
2. the actual result,
3. whether it passed or failed.

For any failure, explain the discrepancy before changing the workflow.

Test the edges, not just the happy path

Agents often test the clean, normal case. You should also try inputs that are unusual or near important boundaries, such as missing values, zeros, duplicates, or values exactly at a cutoff.

For example, suppose a dashboard flags sales declines of more than 10%. You would want to test values right around the cutoff:

-9.9%
-10.0%
-10.1%

The first two should not be flagged, while -10.1% should. You should also check other edge cases that could affect the rule, such as missing values or incorrectly formatted percentages.

Example prompt

Identify the most important edge cases for this sales-decline alert.
Test them and report the expected result, actual result, and pass/fail.

Match the verification method to the output

Different kinds of outputs require different checks.

What the agent built	How you might verify it
Calculator	Check several cases by hand or in Excel.
Dashboard	Compare totals and metrics with the source data.
Classification	Inspect examples assigned to each category.
Form	Try valid, invalid, missing, and unusual inputs.
Data analysis	Reproduce a few important numbers independently.
Document summary	Trace important conclusions back to the source.
Workflow	Run it end-to-end on a realistic case.

Good to know The techniques above work well for small projects. As a project grows, developers often turn these checks into repeatable automated tests — unit, integration, regression, and load tests — so the computer reruns them automatically instead of someone checking the same things by hand after every change.

Next · Page 5 of 8

Carry context across sessions

→