PUBLICATION: I Don’t Know What to Believe: Making sense of science stories
Below is an excerpt from Sense About Science’s publication, “I Don’t Know What to Believe: Making sense of science stories” [PDF].
- Science has a system for assessing the quality of research before it is published. This system is called peer review.
- Peer review means that other scientific experts in the field check research papers for validity, significance and originality – and for clarity.
- Editors of scientific journals draw on a large pool of suitable experts to scrutinise papers before deciding whether to publish them.
- Many of the research claims you read in newspapers and magazines, find on the internet, or hear on television and the radio are not published in a peer-reviewed journal.
- Some of this research may turn out to be good but much of it is flawed or incomplete. Many reported findings, such as claims about “wonder cures” and “new dangers”, never come to anything.
- Unpublished research is no help to anyone. Scientists can’t repeat or use it and as a society we can’t base decisions about our public safety – or our family’s health for example – on work that has a high chance of being flawed.
- So, no matter how exciting or compelling new scientific or medical research is, you must always ask… Is it peer reviewed? If not, why not? If it is peer reviewed, you can look for more information on what other scientists say about it, the size and approach of the study and whether it is part of a body of evidence pointing towards the same conclusions.
A Short Explanation of Peer Review
When a researcher, or team of researchers, finishes a stage of work, they usually write a paper presenting their methods, findings and conclusions. They then send the paper to a scientific journal to be considered for publication.
If the journal’s editor thinks it is suitable for their journal they send the paper to other scientists who research and publish in the same field asking them to:
- Comment on its validity – are the research results credible; are the design and methodology appropriate?
- Judge the significance - is it an important finding?
- Determine its originality - are the results new? Does the paper refer properly to work done by others?
- Give an opinion as to whether the paper should be published, improved or rejected (usually to be submitted elsewhere). This process is called peer review. The scientists (peers) assessing the papers are called referees or reviewers.
Scientists never draw firm conclusions from just one paper or set of results. They consider the contribution it makes in the context of other work and their own experience. It usually takes more than one research paper for results to be seen as good evidence or accepted as a public truth.
