Key takeaways
- Check design: human, controlled, randomised and blinded studies are strongest.
- Size, duration and a precise description of the material tested matter.
- Look for meaningful outcomes decided in advance and full reporting.
- Replication by others is essential before drawing conclusions.
Not all studies are equally reliable. This article gives you a practical checklist for judging the quality of a mushroom study, whether you’re reading the paper or a news story about it, with real examples from studies discussed in our Library.
1. Study design
Was it in cells, animals or people? For human studies:
- Was there a control or placebo group?
- Were participants randomly assigned to groups?
- Was it blinded, so that neither participants nor researchers knew who received what?
See In Vitro, Animal and Human Studies.
2. Size and duration
Very small or short studies are more likely to produce chance findings, and their results often shrink or disappear in larger trials. Look at how many participants were involved, how many in each group, and how long the study ran. Many mushroom trials have had 15 to 25 people per group and run for a few weeks.
3. Dropouts
How many participants dropped out, and why? If many people leave a study, especially from one group, the results can be distorted. Good studies report dropouts and analyse results in a way that accounts for them. In one Lion’s Mane trial, for example, four participants dropped out because of side effects, which is itself useful information.
4. What was tested
A high-quality study describes its material precisely: species, part used, preparation, amount and, ideally, analysis of what it contained. Vague descriptions such as “mushroom extract” make results hard to interpret or repeat. For Reishi, many studies didn’t confirm which Ganoderma species they used. See Ganoderma Species Explained.
5. What was measured, and when was it decided?
Did the study measure outcomes that matter directly, or markers such as blood tests? See Correlation, Mechanism and Causation.
Crucially, were the measures decided before the study began? Good trials are pre-registered on public databases such as ClinicalTrials.gov, stating in advance what they will measure. That prevents researchers measuring many things and reporting only those that happen to show a difference. You can often find the registration number in the paper and compare what was planned with what was reported.
6. Statistics and reporting
- Are all results reported, including those that showed no effect?
- How many outcomes were measured? If a study measures many things, some will show differences by chance.
- How big is the effect? “Statistically significant” means unlikely to be pure chance, not large or important.
- What are the confidence intervals? A confidence interval gives the range within which the true effect probably lies. A wide interval means great uncertainty. For example, a Cochrane review of Reishi found participants taking it were 1.67 times as likely to report side effects, but the confidence interval ran from 0.86 to 3.24: from slightly less likely to more than three times as likely. Because that range includes 1 (no difference), the result is uncertain.
- Relative or absolute? “Halved the risk” sounds dramatic, but halving a 2% risk to 1% is a small absolute change.
7. Limitations and conflicts
Good studies acknowledge their limitations and declare funding and conflicts of interest. Industry funding isn’t automatically a problem, but independent replication matters more when the funder sells the product.
8. Replication
Has anyone else reproduced the result, ideally an independent group? Single studies, however promising, are rarely conclusive. Systematic reviews help by bringing studies together.
9. Publication bias
Studies with positive results are more likely to be published, publicised and cited than studies that found nothing. So the published literature can look more favourable than the full picture. Null results, such as the cycling studies that found no effect of Cordyceps, deserve as much attention as positive ones. See Cordyceps Research.
Quick checklist
- Human, controlled, randomised, blinded?
- Large and long enough? Dropouts reported?
- Material clearly described?
- Meaningful outcomes, pre-registered in advance?
- All results reported, with effect sizes and confidence intervals?
- Limitations and funding declared?
- Replicated independently?