Home > PJ (current issue) > Broad Spectrum | Search

PJ Online homeThe Pharmaceutical Journal
Vol 274 No 7341 p333
19 March 2005

This article
Reprint   Photocopy

PDF 40K, Acrobat Reader

Comment

Peer review — a filter, not a barrier, in the search for scientific truth

By John Clements

John Clements is the science secretary, Royal Pharmaceutical Society

Does the public understand how advances in scientific knowledge are made? How does the public evaluate the constant stream of scientific discoveries (or “discoveries”) reported in the press and media? Is the public aware of the peer review process? And why should this be important?

The organisation Sense About Science set out to answer these questions and in 2004 it published “Peer review and the acceptance of new scientific ideas”, which should be essential reading for all working in research — and in the media.

Since the time of Francis Bacon, the processes of experimentation, discovery, hypothesis building, testing, re-evaluating and peer review have been fundamental to advances in scientific knowledge. SAS’s report reminds us of what we understand by the peer review process and its central role in the advancement of scientific knowledge. In addition, it addresses the question of the public’s perception of scientific discoveries, as gleaned from the media. It points out that the public is generally unaware of the peer review process and so has no basis on which to judge the validity of the various claims made in the press for scientific advances or discoveries.

Some of the recent issues include the suggested link between MMR vaccine and the development of autism in children, the use mobile telephones and the potential for brain damage, the possible health risks from foods containing genetically modified crops, and the implications of producing human clones. These are straightforward questions for the public but often the answers are complex or incomplete.

Often there are misunderstandings by the public about the current state of knowledge and these misperceptions can have important implications when they relate to public health and environmental matters. For example, it is reported that over half the public believes the medical profession to be divided over the health risks of MMR whereas it is more or less united that there is no risk.

Undoubtedly, there is now a greater “hunger” by the public for scientific knowledge, and this is to be welcomed. There is also greater pressure than ever on scientists to publicise their discoveries. Just occasionally, scientists may be tempted to consider their findings so important as to bypass the peer review process. Going straight to the public may be seen to underline the importance of the claim. It was a pharmacist working at the University of Sunderland who, in 2002, claimed to have identified some children whose autism may have resulted from the MMR vaccine. The results were never published in a scientific journal. Nevertheless, they were cited 41 times in newspaper articles and thereby gained credence in the public eye. Unfortunately, it may take many years to change the public’s perception, once established.

Prominence given to early, unsubstantiated findings causes confusion in the public mind. Furthermore, it reduces the effectiveness of public health information when it is needed. There is some evidence that the public wishes to see more care taken in evaluating information before there is widespread reporting. In a survey of the public’s reaction to media coverage of the alleged link between MMR vaccine and autism, 48 per cent thought that the media should have waited for more information before reporting on the possible link.

Peer review, although not perfect, offers a process with checks and balances and may detect plagiarism and falsification. Cases of fabrication or fraud are, fortunately, rare. Claims for the “discovery” of two new chemical elements from a laboratory in California were subsequently admitted to be false. A scientist at Bell Laboratories was exposed as constructing fraudulent data that were included in 25 papers he had published in the area of nanotechnology.

The media frenzy can be intense. In December 2002 a US company announced the birth of the first cloned human. The international media devoted front pages to the story for two or three weeks, based simply on the company statement and in the absence of corroboration. There was no baby.

It is important that discoveries that affect our health are made known so that we can all benefit. But this is the dilemma. How soon does a research finding become fact? Clearly there is a need for early independent corroboration and analysis. The greater the significance of the research finding, the more important it is that the research is on a sound footing, as peer-reviewed work. Proper review and independent corroboration take time and resources. There is also a certain impatience to “know the answer”. But the costs of acting on flawed findings are potentially high and may damage public confidence in science and scientists.

There are good examples where substantial findings, properly replicated, can change ideas and clinical practice. Research in 1983 showed Helicobacter pylori could be linked to gastritis and peptic ulcers, contrary to prevailing opinion. Once substantiated, this resulted in new treatments of peptic ulcer and gastritis and lower rates of stomach cancer.

What can be done? We, as scientists, understand the importance of peer review and the need for other workers to corroborate initial findings. Peer review also involves placing qualifications and limitations on the conclusions. There is a tension between the exact findings, as published in a peer-review journal, and their interpretation in the media. Important qualifying language and caveats are often omitted in the popular press.

A part of our remit should be to make the peer review process known more widely by journalists, press officers and the public at large. We need to raise the public profile of peer review and seek to explain that the standards of evidence on which research claims are made can be highly variable. Also, peer review should be recognised as a filtering device, not a barrier, in the search for scientific truth.

Scientists should ensure that they have input into the content of press releases to ensure their accuracy, and monitor news stories on their research. The public needs a scientific interpretation of the data to make them meaningful. For example, the publication by the Department of Health of specific energy absorption rates for mobile telephones, unaccompanied by any interpretation, served no apparent purpose.

There is a need to encourage more dialogue between scientist and journalists. Last year the Royal Pharmaceutical Society was involved with Sense About Science in an event “The voice of young scientists in the media”. This brought together young scientists and science correspondents working in the media and promoted a mutual understanding of their work. A tangible outcome was the increased willingness expressed by the scientists to engage with journalists and publicise their findings. Such events deserve support.

Back to Top


©The Pharmaceutical Journal