Where quoted numbers come from, why most of them do not apply to you, and how to build the only benchmark that actually governs a decision, which is your own measurement of your own work over time.
| # | Lesson | The question it answers |
|---|---|---|
| 01 | Where Benchmarks Come From | Why is most of what gets quoted unsourced, and how do you check one? |
| 02 | Stage, Segment and Why Comparison Fails | Why is a seed stage number useless to a Series C company? |
| 03 | Public Model Benchmarks | What does a leaderboard score predict about your product? Usually little |
| 04 | Contamination and Gaming | Why do benchmark scores rise without capability rising? |
| 05 | Your Eval Set Is the Only Benchmark | How do you build the one number that means something to you? |
| 06 | Measuring Drift Across Versions | How do you compare two models on your own work, fairly? |
| 07 | Benchmarks by Sector | The table, every number sourced and dated |
| 08 | Reading a Public Filing | How do you get real figures instead of conference slides? |
| 09 | Building Your Own Baseline | Why is your trailing number the only valid comparison? |
| 10 | Reporting Numbers Honestly | What has to accompany a figure for it to be usable by somebody else? |
| 11 | When Somebody Quotes a Number at You | What are the four questions that settle it? |
| 12 | Worked: Checking a Claim | One widely repeated benchmark, traced to its source |