Fathom
Specs for Non Deterministic Systems

Specs for Non Deterministic Systems

Intermediate

How to specify something whose output is different every time. The pixel spec and the acceptance criterion both stop working, and what replaces them is a behaviour contract, a rubric, and a set of examples that doubles as the eval set.

#LessonThe question it answers
01Why a Pixel Spec FailsHow do you specify behaviour when the output is different every time?
02The Behaviour ContractMust, must never, should, may: how do you write one?
03Writing the RubricHow does quality get defined before it can be measured?
04Golden Examples and Counter ExamplesHow does the spec become a dataset?
05The Prompt Is the SpecWho owns it, who reviews it, and how is it versioned?
06The AI Requirements DocumentWhat does the document actually look like now?
07Specifying Tone and StyleHow do you write down something nobody can define?
08Specifying What It Must RefuseHow do you describe the boundary without listing every case?
09Specifying Failure BehaviourWhat should it do when it does not know?
10Acceptance Criteria as Pass RatesHow does definition of done change shape?
11Reviewing a Spec You Cannot Test by ReadingWhat does a good review of this document look like?
12Worked: A Behaviour ContractOne real feature, specified end to end

↑↓ navigate↵ openesc close

drag to pan · scroll to zoom