The Pharmaceutical Journal Vol 264 No 7082 p220-222
February 5, 2000 Articles

Pharmacy on the web: quality information or information overload?

By Michelle Wake, MA, ALA, and Linda Lisgarten, ALA, MIInfSci

It can be difficult to distinguish between good and poor quality information on the world wide web and pharmacists need to be aware of the criteria for evaluating such information. This article describes how to find information on the web and outlines the criteria that should be used to evaluate it

Being able to assess the quality of information obtained from the world wide web is increasingly important for pharmacists, not just for their own information but also for helping patients who have carried out, or wish to carry out, a web search. In December, 1998, research showed that 10.6 million adults in Britain had used the internet at least once in 1998 and that 73 per cent of these had accessed the web during the previous four weeks.¹ Another survey, by Which? Online, reported that health web sites were among the most popular² and the National Library of Medicine in the United States discovered that one third of the searches carried out on its free MEDLINE service PubMed were carried out by the general public, not health professionals.³ The web contains a great deal of health related material from companies, charities, governments, research institutions and private individuals.
Locating information on the web can be difficult, but it can be even more difficult to distinguish between good and poor quality information, and, unfortunately, there have now been numerous reports of dubious health material on the web. For example, a study in the British Medical Journal found that only four out of 41 sites searched for information on child fever provided the correct guidelines on diagnosis and care.⁴ Another survey found that 40 per cent of web pages sampled on the subject of Ewing's sarcoma were from non-reviewed sources and, of these, 6 per cent gave incorrect information.⁵ There is even evidence to show that a patient has become ill because of poor web information.⁶ It is also possible for web information to do harm even when it is accurate: for example, if patients misinterpret material designed for health professionals or read pages out of context.⁷ As yet there is no evidence that the quality of information on the web is any worse or any better than that delivered by other media, and of course the public have always been able to obtain health information through non-official channels. However, it is so easy and cheap to publish on the web that it becomes simpler than in the past for producers of poor information to disseminate their products or their beliefs or both in a form which looks authoritative to the general public.⁸ Although there is no definitive definition of quality and the usefulness of any information depends on users' needs, pharmacy professionals need to be as aware of criteria to evaluate the web, as they are for other "traditional" sources.⁹ These evaluative criteria can help us avoid, or at least limit, information overload.

Why is it difficult to search?

The web is already huge, and growing so fast that we can only estimate its size. A study in Nature estimated that the web contains approximately 800 million pages of information¹⁰ and it has also been estimated that it is doubling in size every year.¹¹ There are search tools, known as engines and subject directories, to help you explore the web, but you can still find yourself with thousands of "pages". For example a search (June 24, 1999) using the engine Altavista (www.altavista.com) found 1,222 pages for raloxifene and 107,920 pages for osteoporosis.
A list of search tools and how to use them can be found at www.searchenginewatch.com or at www.philb.com.
But search tools do have their limitations. To begin with you are not searching the web as it exists at that very moment. You are searching a database that has been created before, whether by machine, in the case of search engines, or by humans, in the case of subject directories. This explains why you sometimes get an error message telling you that a page cannot be found when you have selected it from a list provided by a search tool. The author of the page may have deleted it or moved it months ago, but it takes time for the search tool to find this out. You are not even searching the entire web as it existed when the database was created. Even the largest search engines only cover a small proportion of the web and some have a geographical bias, which is usually North American or European. For example, it has been estimated that the largest engine, Northern Light (www.northernlight.com), only indexes 16 per cent of what is available. Meta-search engines, such as BigHub.com (www.bighub.com), which allow you to search several engines in one go, can increase this, but only to around 42 per cent.¹⁰ The tools differ in the frequency of their updating so that some are more current than others. A few will allow you to carry out more complex searches which give you a better chance of locating what you need quickly but all differ in their search speeds and the amount of help they offer. Whichever search tool you use, it is always worth spending time reading the "help" section. Many provide brief user guides which can be read in 10 minutes and this can save a lot of time and frustration later on. Some search engines are "intelligent" in that they store your search strategy and rerun it against new material added to the engine since your last login. My Yahoo! (www.yahoo.com) is a free intelligent engine.
The important fact to be aware of is that most engines and directories do not evaluate the sources they find, although directories will often give a better description of a site's contents than an engine. But, there is no guarantee of peer review here. They are not checking the credentials of or the content of web sites.

Why is it so difficult to find quality?

No central body controls the internet, so anyone can publish anything they choose, however inaccurate, and although some countries do have legislation that allows prosecution for infringing copyright, libel and decency, because the internet is global these laws can be totally ineffective. For instance, the United Kindom may ban the advertising of licensed prescription drugs to the general public, but in the US it is perfectly legal and so advertisements appear on US web sites which can then be accessed by anyone in the UK. Concerns about such "cross border advertising, promotion, and the sale of medicinal products through the internet" led to the World Health Organisation setting up a group to suggest how countries can act together.¹² But as yet the situation is not, and cannot be, regulated.
Since the mid 1990s, a number of rating and reviewing services have been created, for example, the popular Magellan internet guide (www.mckinley.com/magellan). However, many do not explain how they have evaluated the sources and several studies have doubted that they give a good indication of quality.^13,14 But at least some evaluation has taken place. Unfortunately, the amount of information they cover is less than that of the other search tools mentioned because their creation and upkeep is time consuming. Also, these services are aimed at all users of the internet and thus they may lack information on the more specialised sites of interest to health care professionals.

Virtual libraries

Which search tool you use depends on your needs. If you are looking for a known document, search engines are good, but if you are after general information, subject gateways, also called virtual libraries, are better. These have been created by subject experts and only list those sites which have met high quality standards. A good starting point is the Argus Clearinghouse (www.clearinghouse.net) which evaluates these services.
One of the most comprehensive virtual libraries for pharmacy is Pharmweb (www.pharmweb.net). Founded in 1994 by Dr Anthony D'Emanuele at Manchester university, it covers worldwide sites and provides a range of links including pharmaceutical companies, government information sources, and poisons and drug information.¹⁵ There are also sections aimed at patients. For medicine and health related subjects there is OMNI (http://omni.ac.uk) which is based at the University of Nottingham but is advised by a steering group not just from higher education but also from the Department of Health, the Wellcome Trust and the British Library.
A virtual library set up for the general public is Healthfinder (www.healthfinder.gov). This is a US government site, which does not just index government information but also includes pages from universities, charities, professional societies and commercial companies.
Health on the Net (www.hon.ch) is a European-based gateway that only lists sites that follow its code of conduct.
During 2000–01, the National Health Service will fully open its National Electronic Library for Health (www.nelh.nhs.uk) which is being designed for professionals and the public.
Virtual libraries can provide an excellent starting point for looking at the web but they are still web pages, so, when using them and the sites they list, you still need evaluation criteria.

Efforts to evaluate the web

Numerous groups are involved in developing web quality criteria. In the medical field, OMNI's advisory group for evaluation criteria, the Health on the Net Foundation, the Health Summit Working Group (www.hitiweb.mitretek.org/hswg) and the "DISCERN" project¹⁶ are prominent.
Governments are increasingly aware of the problem. The "G8 ENABLE" project, started in 1994 by the Group of Seven Ministers (now G8 with Russia) and involving Australia, has recommended that "health information quality academies" should be set up at national level, but as part of an international federation, to provide guidance on quality.¹⁷
There are technological developments which have been suggested as tools for evaluating and controlling access to websites. One is to use electronic labels, metadata, to indicate the qualitative contents of a page. The World Wide Web Consortium (www.w3.org/pub/WWW/PICS/) has developed standards for these labels, called PICS (Platform for Internet Content Selection).¹⁸
A prototype for medical information, med-PICS, has been designed by Eysenbach and Diepgen.⁷ Filtering software could then be used to select only those pages with the "right" electronic labels.
However, as noted by the EU Action Plan on the Use of the Internet (www2.echo.lu/legal/en/internet/actplan.html), existing filtering software is not very sophisticated. It has been designed primarily to block offensive material such as pornography and racist matter but, at present, it will also frequently block perfectly acceptable sites. A study in Utah's public libraries discovered that the software prevented access to, among other things, the Bible, the Koran and ‘A Connecticut Yankee in King Arthur's Court'.¹⁹ There is also the issue of censorship. In the future it may be possible for the software to filter for quality using the PICS standards. The Microsoft browser, Internet Explorer, already supports PICS. Unfortunately there is the risk that metadata will be abused, eg, by web authors and others deliberately inserting misleading labels.

Guidelines for evaluating a website

Many of the criteria we would apply to printed materials to decide their quality can also be utilised for electronically "published" material.²⁰ Considerations include:

Who maintains or funds the site? This should be clear. Do you recognise those who maintain or fund the site as reputable and would any sponsors or advertising contracts listed influence its contents in a detrimental fashion?

What is the URL? The Uniform Resource Locator (the address) of a site can tell you its location and the type of organisation or author behind it. Most UK sites end in ".uk". All countries, apart from the US, have such a code. Another part of the address tells you the type/domain of the organisation:

• ac.uk (a British academic institution)
• co.uk (a British company)
• org.uk (a non-profit British organisation)
• gov.uk (a British government body)

Some US domain names are different:

• .edu (an educational organisation)
• .mil (a military body)
• .com (a company)

(Note, however, that addresses ending in ".com" are now increasingly used world-wide, including in the UK.)
The tilde, "~", in a URL is often used for personal pages being hosted by an organisational site. However, personal sites do not always use it and some organisations do have a tilde in their URLs.
Many companies include their names in their URLs but, because it has been very easy to register domain names and because, of course, many companies have the same name, this is not always the case.

Is there a logo on the site? If there is a logo on the site, do you recognise it as belonging to a reputable company? Copyright laws do apply to the web and organisations will usually fiercely defend the use of their logos. It is true that some might slip through but it gives you yet another clue to a site's general standing.

Is an author named? If the author is named and you do not recognise the name, do a search on a reputable bibliographic database such as Medline (a version is available free at www.ncbi.nlm.nih.gov/PubMed), International Pharmaceutical Abstracts, or e-PIC (the new database from the Royal Pharmaceutical Society), to see if the author has ever published in a peer-reviewed journal.

Does the site have a contact name, postal or e-mail address? Even without recognising the name, at least you know someone is willing to take responsibility for the information.

Is the information up to date? A well-designed web page should provide details of when it was created or last updated. But make sure that this is the date you are looking at because some sites automatically display the current date.

Are there any references? You need to know if the "facts" have any scientific basis or are merely opinion.

Has the site won an award? Do not be swayed by any awards won by a site unless you know the reputation of the award giver and the criteria they used to judge the site. It is all very well a site winning an award for innovative use of graphics, but what about the factual content?²¹

Does the site take the responsible precaution of making clear who the site is aimed at? If the site provides health information, does it warn readers that they should contact health professionals if they have any symptoms or worries?

Is there a "help" page? If you have used a virtual library or a reviewing service, use its "help" page to check the evaluation criteria used.

Does the site rank search results? Most search engines will display your results in a ranked order. Use their "help" pages to check how they decide on this order of relevancy.

Does the site state a confidentiality policy if it collects personal information?^22,23 If a site does contain a confidentiality policy, are there any security measures listed to back this up? Any computer linked to a network can be hacked into, but there are some safeguards, such as encryption, which can limit the risk, even if these safeguards are not totally infallible.

The criteria above are concerned primarily with the reliability of information on a website, but once this has been ascertained it is then appropriate to consider more aesthetic elements such as layout, ease of use and access.²⁴ Also some sites charge for access, and speed of access to different sites can vary greatly, depending not only on your own hardware, software and communication links but also on those of the website. The time of day has a very real affect. It is always quicker to use a country's sites early in the morning local time. Remember that lots of images on a web page can look attractive, but they can take a long time to download. (If you feel that the images are not important, that they are there for decoration rather than to explain, you can turn them off using your web browser. In Netscape, select Edit/Preferences/Advanced. In Internet Explorer choose View/Internet/Options/Advanced.)

Conclusion

Searching the web can be very time-consuming, and indeed in many cases printed books and journals are more appropriate, easier, quicker and more reliable. However, the web cannot be ignored. It provides very valuable information to pharmacists and their patients, and although it should not and cannot replace the patient-pharmacist relationship, it can be part of it, if used wisely. Indeed it could be used to encourage greater contact between the general public and health care professionals.²⁵
The first step has to be for pharmacists to familiarise themselves with the web and to be aware of its limitations as well as its possibilities. Projects are already under way to do this, including one by the Unit for Health Services Development, affiliated to the University of London School of Pharmacy, which aims to encourage psychiatric pharmacists' use of the internet.²⁶ More and more pharmacists will have access to the web if the Government's aim to "connect every GP surgery and hospital to NHSnet" by 2002 is met.²⁷
Ultimately, it is impossible and possibly undesirable to regulate what is "published" on the web. However, for it to be a useful tool, and not just a source of information overload, we all need to be aware of evaluation guidelines and be prepared to explain their use to an internet-aware general public.

Michelle Wake is deputy librarian and Linda Lisgarten is chief librarian and head of library and information services at the School of Pharmacy, University of London, 29-39 Brunswick Square London WC1N 1AX. Correspondence to Ms Wake (e-mail library@ulsop.ac.uk)

References