Acceptance criteria / 4 min read

The artifact is the test.

A tool call can succeed while the workflow produces the wrong result.

FlakeMap research note / Synthetic example / 8 October 2026

Start at the end

For the PO-204 fixture, the expected outcome is exactly one purchase-order draft preserving the order reference. That is more useful than asking whether each tool returned successfully.

Make the failure observable

The timeout example creates PO-204-A from the retry and PO-204-B from the late original response. Each tool path can look locally successful while the task fails.

Keep the rubric reviewable

Write the criterion in language an engineer and reviewer can both inspect. Attach the artifacts and retain the source fixture. This article describes a synthetic example, not a measured customer outcome.

Order lookupPO-204

Stock checkTimeout injected

PO draftRetry + late response

↳Two paths reach draft creation.Candidate lacks a shared deduplication guard.

Inspect the example