How to Read a Health News Headline More Carefully

A headline has a dozen words to make you click. A study has dozens of pages of conditions, limits and caveats. Something has to be left out, and it is usually the part that would tell you whether the news applies to you.

Here is a short checklist, tried out on a made-up headline. It takes a few minutes and needs no scientific training.

The headline

“A handful of zorberries a day slashes memory loss risk by 40%, study finds”

There is no such fruit as a zorberry. A made-up food keeps the example from being mistaken for a claim about a real one.

Suppose the article underneath says this: researchers gave 600 adults aged over 65 in one city a questionnaire about diet and a memory test at a single visit. Those who reported eating zorberries every day were 40% less likely to score below a certain level. A zorberry growers’ association paid for the study.

The checklist

1. Was it a study in people?

The US National Center for Complementary and Integrative Health (NCCIH) puts this question first in its checklist for health news stories. Results from animals or laboratory work may not apply to human health until studies in people examine them. Our example: yes, people. Not every headline passes this check.

2. What kind of study was it?

Did researchers give something to one group and not to another, or did they observe what people already do? The US National Cancer Institute (NCI) explains that studies comparing groups of people can show an association but cannot, on their own, prove that a behaviour or substance causes a disease. The link might be due to chance, or to some other factor. Our example: observation only, at a single point in time. People who eat zorberries may differ from others in many ways. And since diet and memory were measured on the same day, we cannot even tell which came first. More on this in association and causation.

3. How many people, for how long, and who?

The NCCIH checklist asks whether enough people were studied, since small studies give weaker results; whether the study lasted long enough; and whether the participants were like you in age, sex or other respects. Our example: 600 people, no follow-up at all, all over 65 and from one city. It says little about a 40-year-old elsewhere.

4. Does the verb match the evidence?

“Slash”, “prevent” and “cause” are words about cause and effect. “Linked to” and “associated with” are words about a pattern. Our example: “slash” claims far more than a questionnaire can show. An accurate version would be “people who ate zorberries scored better on a memory test in one survey”.

5. Forty per cent of what?

A percentage with no starting point is a relative figure. Faced with a “ten times more likely” headline, Cancer Research UK asks “ten times what?” In a 2013 article on media stories it explains that you need the underlying absolute numbers, because when the starting risk is very small even a large relative change may make little absolute difference. Our example: suppose 10 in 100 people who did not eat zorberries scored low, against 6 in 100 of those who did. That is “40% less likely”, and also “4 fewer people in every 100”. The second version is easier to judge. See relative and absolute risk.

6. What was actually measured?

The NCCIH asks whether the difference was meaningful: a small difference may interest scientists and still matter little to health or quality of life. Our example: the headline says “memory loss”. The study measured one score on one test on one day. Those are not the same thing.

7. Is this the first report?

One study rarely proves anything, the NCCIH says. The NCI makes the same point from the other side: scientists gain confidence when many studies show a similar association and there is a plausible biological explanation. Our example: the article mentions no other research. Treat the finding as an early observation.

8. Who paid, and does the story mention limits?

The NCCIH advises more caution when the funder could profit from the result, and asks whether the story discusses the study’s limitations. Funding does not make a finding wrong. It is a reason to wait for independent confirmation. Our example: funded by a growers’ association; no limitations mentioned.

The scorecard

A fair summary would be: “In one small survey, older adults who ate zorberries were less likely to score low on a memory test; this shows a pattern and no more.” That is less exciting than the headline, and it is also not nothing. Early observations are how research questions begin.

A shorter version for busy days

When there is no time for eight checks, three do most of the work:

1. People or not?
2. Observed or tested?
3. What are the absolute numbers?

If the article cannot answer these, the headline has not earned a change in your habits, or a share.

What good reporting looks like

Careful headlines exist, and they are worth noticing. Signs of a careful story: it names the journal and the type of study, gives absolute numbers alongside percentages, quotes someone who was not involved in the research, states the limits, and says how the finding fits with earlier work. Cancer Research UK gives a real case in which digging into the research behind an alarming headline showed how small the absolute numbers were. Sense about Science, a UK charity, points to further tricks to watch for, such as choosing an extreme value to make a story more dramatic.

Before you act on a story

News about a single study is rarely a reason to change a medicine, a treatment or an established habit. Guidance from health bodies changes when evidence accumulates, not with each morning’s headlines. The NCCIH’s checklist ends with a reminder to talk to a health care provider before trying an approach you have read about, however promising it sounds.

If a story worries you or seems relevant to your own health, save the link and bring it to your next appointment. The same habits help when a claim reaches you through a feed instead of a newspaper: see finding reliable health information online.