The Problem with One Study and Certain Conclusions

Foundation · 12 min

One study is a brick, not a building

Scientific knowledge is not built from a single study; studies accumulate to form a picture. A single study may be limited by a small sample, a short duration, or a particular group. The finding of a four-week study with twenty people is valuable, but for it to become general advice for millions it needs to be repeated and tested in different groups. The certainty you see in headlines is usually absent from the study itself; researchers tend to use cautious phrases such as may, or appears to be associated with.

For mouse experiments and cell studies this warning applies twice over: an effect observed in the laboratory often does not carry over to human nutrition in the same form. If the headline mentions mice, there is a longer road ahead before the result can be generalised to people.

Observational studies versus experiments

In observational studies, researchers watch people's existing habits and look at how they relate to health outcomes; nobody's behaviour is changed. In controlled experiments, participants are divided into groups and one group follows a specific intervention. Observational studies are valuable for seeing long-term relationships in large populations, but they are open to confounding factors: for example, people who eat a certain food may also happen to move more. Experiments are stronger on cause and effect, but they are usually smaller and shorter. The two complement each other; neither replaces the other.

In well-designed experiments, randomly assigning participants to groups makes the groups comparable; observational studies have no such guarantee.

Appearing together is not causing

Correlation means two things change together; causation means one brings about the other. An observation that coffee drinkers get a certain disease less often does not prove that coffee protects; coffee drinkers may share other characteristics too. A large share of the misunderstandings in nutrition news comes from mixing up these two concepts. When you read the phrase was found to be associated with, simply remembering that this does not mean caused is a powerful filter on its own.

Keep the possibility of reverse causation in mind too: people who gave up a food sometimes gave it up because they were already ill. Interpreting a visible association on its own is like watching a film from the middle.

How evidence gets stronger

When many studies examining the same question have accumulated, researchers bring them together in systematic reviews and meta-analyses; in other words, they look at the whole wall rather than single bricks. Institutional nutrition guidance is also built on this accumulation. A practical ordering is therefore: a single study is a clue, repeated studies are a pattern, and reviews and guidance are the most reliable kind of summary. You can find the source evaluation questions in the source types lesson.

This ordering gives you a practical reading strategy: when you see a striking claim, your first question can be whether the finding has made its way into reviews and guidance. If it has not, the claim is still at the single-brick stage and can go on hold.