|
Frank Leach, medicines information pharmacist at North West Medicines
Information Centre, Central Liverpool Primary Care Trust,
and Brian Faragher, senior lecturer in statistics and research methods
at the University of Manchester
|
There is good reason to suspect that most health professionals have
a weak grasp of statistics. A formal study of the critical appraisal
skills
of pharmacists in the US, for example, found that most did poorly in
the statistics section. Several years ago, the editor of the BMJ expressed
the view that 99.9 per cent of his fellow professionals had a “shaky
knowledge” of statistics. These deficiencies may contribute to
the poor standards of many clinical trial reports, as highlighted by
the authors of the Consolidated Standards of Reporting Trials (CONSORT)
statements. Their conclusion, that “inadequate reporting borders
on unethical practice when biased results receive false credibility”,
is especially relevant to patient safety. Much of this concern has a
statistical dimension.
A curious amalgam
Statistics has been defined as a “curious amalgam of mathematics,
logic and judgement”. Its approach to the analysis of problems
often seems to run contrary to instinct and it is richly endowed with
cumbersome formulae and a seemingly endless list of curiously named tests.
Its intrusion into clinical research is necessitated by the phenomenon
of random variation, which renders the response of patients to treatment
interventions both variable and unpredictable. In the context of clinical
trials, it provides an estimate of a treatment effect and qualifies that
estimate by a measure of its reliability. In skilled hands, it has allowed
populations of patients to benefit from experience gained with representative
samples of fellow sufferers and it remains an essential component of
medical research. Nevertheless, cynicism concerning the role of statistics
and the contribution of statisticians to medical research is not uncommon.
This must be weighed against evidence that misinterpretation of valid
data or uncritical acceptance of faulty data can lead to patients being
denied beneficial treatments or being treated with ineffective drugs,
sometimes with potentially fatal consequences.
Excessive emphasis on the outcome of significance tests, with a dichotomous
approach to interpretation of data, is one of the key current problems
of clinical trial data analysis. Trials with insufficient subject numbers
are likely to produce inconclusive results. To classify these as “negative” in
outcome is erroneous and hazardous. The conventional use of 95 per cent
confidence intervals (CIs) is exactly equivalent to use of the arbitrary
5 per cent significance level, but it provides much more useful information.
An understanding of the use and misuse of CIs is one of the most useful
skills in critical appraisal of data.
In spite of the contributions of official bodies, such as the National
Institute for Clinical Excellence, the range and variety of issues surrounding
drug treatment demand informed input at local level, a contribution which,
we would argue, pharmacists are singularly well-placed to meet. To fulfil
this role, their training needs, particularly in the areas of critical
appraisal and statistics, might benefit from review. Emphasise theory and practice
We believe that statistical teaching, if and when provided on undergraduate
pharmacy courses, should emphasise both theory and practice. At postgraduate
level, the emphasis should be on achieving an understanding of basic
statistical concepts, especially within the context of clinical trial
data. Both critical appraisal and statistics have, for many years,
been regular themes in the UK Medicines Information training course
and a number of bulletins on elementary statistics are now available
on the UKMi website at www.ukmi.nhs.uk.
Pharmacists who have regular involvement with trial data evaluation
and medicines selection policies are likely to find collaboration
with a
medical statistician mutually advantageous. Disclaimer The views expressed in this article are those of the authors
and do not necessarily reflect those of either Central Liverpool PCT
or the University of Manchester.
An information pharmacist’s
perspective
Frank Leach writes: My first encounters with statistical analysis
date from my years as a research student, when “number
crunching” formed an essential part of my endeavours.
Data from seemingly innumerable insulin assays would be fed
into a primitive calculator and the all-important P value,
usually by courtesy of the Student’s t-test, would be
anxiously awaited. When P was less than 0.05, thus signifying “statistical
significance”, there would be rejoicing; when greater
than 0.05, an equal measure of disappointment. The statistically
enlightened will hasten to condemn this dichotomous (“yes” or “no”)
approach to data analysis.
During three subsequent decades, working within the NHS Regional
Medicines Information Service, I have become increasingly involved
in teaching “critical appraisal” skills to health
professionals (typically pharmacists and clinicians). This
pleasant diversion, together with membership of several ethics
and medicines management committees, has imposed frequent (and
sometimes urgent) familiarity with statistics and engendered
a personal fascination with its complexities. It is not, I
hasten to add, a subject which I find at all easy. The opportunity
to work collaboratively with a medical statistician, initially
under the auspices of the North West Multicentre Research Ethics
Committee, has proved especially rewarding and has already
prompted a number of collaborative ventures.
In my experience, many health professionals (including pharmacists),
whom I have met in the course of my work, have problems in
understanding basic statistical concepts. This learning need
is likely to be a significant obstacle to those routinely involved
with critical appraisal of clinical trial data. |
A medical statistician’s
perspective
Brian Faragher writes: As a medical statistician with dual
teaching/practitioner responsibilities, I have several concerns
relating to the misuse of statistics in clinical trials. The
interpretation of clinical trial outcome is too often based
solely on significance testing, with corresponding neglect,
or misinterpretation, of confidence intervals. I also think
that there is a misplaced emphasis on “cook-book” analysis,
ie, the uncritical application of tests to data without due
consideration of the nature of the target data or an understanding
of the theory underpinning the tests.
This problem has probably been exacerbated by the wide availability
of computer-based analytical packages. Many investigators seek
statistical assistance at too late a stage and then seek to
improve statistical significance levels by means of data manipulation
or dredging. They should, ideally, consult statisticians at
an early stage of trial planning.
In my experience, the NHS, in contrast to the pharmaceutical
industry, does not facilitate early intervention by statisticians.
Some are now employed within NHS trust research and development
departments but most remain primarily academic in orientation.
More thought needs to be given to the creation of support statistical posts
within the NHS. I envisage that these would have a number of key functions,
for example, provision of instruction in basic statistics, advice on the selection
of appropriate trial designs, assistance with research supervision and performance
of data analyses, especially with complex data.
Offering support facilities is, in my view, more important than attempting
to train health professionals to specialist levels of statistical competence. |
|
The
Pharmaceutical Journal