Research That Changes Decisions

The idea in one minute#

Research earns its cost when a decision would have gone one way and goes the other way because of it. That is a demanding test and most research fails it, not because it was done badly but because no decision was named before it started. A study with no pending decision attached produces a document that is read, agreed with and filed. The fix is procedural rather than methodological: write down the decision and what each possible finding would mean for it, before you talk to anybody, and refuse to start until somebody can tell you what they would do differently.

A picture#

flowchart LR
  D{":i-compass: <b>A pending decision</b><br/><small>named before anything starts</small>"} --> PRE[":i-notebook-pen: <b>What each finding would mean</b>"]
  PRE --> M[":i-search: <b>Method</b><br/><small>chosen to fit the question</small>"]
  M --> F[":i-check: <b>Findings</b>"]
  F --> CH[(":i-stamp: <b>Decision changes or is confirmed</b>")]
  NOD[":i-book-open: <b>No decision attached</b>"] -.-> DECK[":i-triangle-alert: <b>A document everyone agrees with</b><br/><small>and nothing changes</small>"]
  class D,PRE queue
  class M,F compute
  class CH memory
  class NOD neutral
  class DECK warn

The top path and the bottom path use identical methods and produce identical quality. Only one of them was worth doing.

How it really works#

Write the decision first#

Before any study, one short paragraph: what is being decided, who decides it, when, and what the current leading option is. Then, for each plausible finding, what it would imply. If every finding leads to the same action, do not run the study.

This single habit removes most wasted research, and it has a second effect: it makes stakeholders commit in advance, which prevents the familiar outcome where a finding is accepted intellectually and then ignored because it was inconvenient.

Match the method to the question#

The recurring error is using a qualitative method to answer a quantitative question, or the reverse. Interviews tell you what exists and why; they cannot tell you how common it is. Data tells you how common and where; it cannot tell you why.

Sample composition beats sample size#

Who you include decides the answer more than how many you include. Three groups are routinely missing and each one changes conclusions: people who churned, people who evaluated you and chose otherwise, and people who use a workaround instead of any product at all. A study of happy current users will reliably recommend making the product more like itself.

The readout is the deliverable, and it is short#

One page: the decision this affects, the claim, the evidence for the claim with a quote or a number, the confidence, and what would change your mind. Anything longer is read by the person who wrote it.

A finding with no evidence attached is an opinion with better formatting, and a finding without a stated confidence invites everyone to assume whatever confidence suits them.

Ongoing beats episodic#

A research project that happens twice a year produces two moments of insight and ten months of decay. A standing habit, five customer conversations a week and a sampled set of session recordings, produces a continuously refreshed picture and costs less in total. The episodic version exists because it is easier to schedule, not because it works better.

What fills the box in 2026#

Recruiting through Respondent or your own user base. Sessions through Dovetail, Grain or the call tool you have. Replay through FullStory, Hotjar or PostHog. Models are genuinely good at clustering transcripts and surfacing themes you did not ask about, and genuinely dangerous at inventing a theme that reads well. The discipline is that every theme carries a real quote, and a theme without one is deleted.

Sector lens#

Remember this#

• Research earns its cost when it changes a decision. Name the decision before starting.
• If every possible finding leads to the same action, do not run the study.
• Interviews tell you what and why, never how many. Data tells you how many and where, never why.
• Sample composition beats sample size. Churned, lost and workaround users are usually missing.
• The readout is one page with evidence and a stated confidence.
• Continuous contact beats a project twice a year, and costs less.

Try it#

1. Take the last study your team ran. Write down the decision it was meant to change. If you cannot, you have found the pattern.
2. For your next question, write the two or three possible findings and what each would imply. Circulate that before doing the work.
3. Check your last research sample for the three missing groups. Add one churned customer this week.

Check yourself#

1. Why does naming the decision in advance prevent a finding from being accepted and then ignored?
2. You want to know how many users hit a problem. Why are interviews the wrong instrument, and what would you use?
3. A study of current active users recommends extending the product in the direction it already goes. What is wrong with the sample, and what would you add?