See the denominator
Show 6 of 8 criterion passes rather than an unqualified quality score.
FlakeMap Trust Review / Proposed module 02
Repeat with
intention.
Compare baseline and candidate against the same small, seeded run set. Keep the condition, seed, criterion, and artifact together.
Seed / timeoutv0.8 → v0.9
1011 draft1 draft
1021 draft1 draft
1031 draft1 draft
1041 draft2 drafts
1051 draft1 draft
1061 draft1 draft
1071 draft1 draft
1081 draft2 drafts
Baseline 8/8 · Candidate 6/8 · syntheticDEMO — SYNTHETIC DATA
The question this module answers
Does the task survive repeated runs?
- Input
- A scenario, two versions, and fixed seeds
- Output
- A run matrix with explicit outcomes
Inspect each seed
Open a run to see the tool condition and the exact resulting artifact.
Compare like with like
Keep seeds and conditions aligned between the two versions. Fixture determinism does not promise deterministic external models.
Follow the proposed workflow
A visible path from
input to decision.
01
Choose baseline and candidate
A scenario, two versions, and fixed seeds
Proposed process / synthetic exampleModule boundaries
Useful evidence.
Honest limits.
Is Trust Review live?
This module is available to explore in the synthetic demo. A first pilot would test one scoped module with approved fixtures.
What does the demo execute?
It selects deterministic stub fixtures and changes local mock states. It does not call models, write to tools, import customer traces, or modify CI.
How is value validated?
Compare with the existing eval gate and confirm a meaningful missed regression independently. A named engineering owner must commit a gate or fund a trial before expansion.
A smaller test. A clearer decision.
Start with the workflow
you can’t afford to guess.
Agree on one task criterion, an approved fixture, and a named engineering owner.
Request a replay study