Global review on access to health information in developing countries

Discussion Paper 1. The role of systematic reviews

by Paul Chinnock, Cochrane Collaboration

Systematic reviews are a cornerstone of evidence-based medicine. If the argument for systematic reviews works in the industrialised world, then it is even more true for the developing countries. Where resources are scarce, it is vital that these are invested primarily in safe, effective interventions.

The Cochrane Collaboration has published more than 2000 systematic reviews and over 1500 protocols. It has taken steps to make access to the Cochrane Library online free of charge in many parts of the developing world and its policy is to encourage people in such countries to participate in the writing of reviews. But are Cochrane reviews useful to practitioners in 'the majority world'? Are reviewers' conclusions 'globally relevant'?

It will be several years before the majority of medical conditions and interventions have been addressed in systematic reviews. However, there is a mismatch between the global burden of diseases and the amount of research devoted to these conditions. Conditions of particular interest to resource-rich settings are more frequently addressed by reviewers. This is reflected even in the background discussions, which usually focus on the impact of conditions in the USA and/or Western Europe

Many major concerns in developing nations have yet to be the subject of a review. To take surgical conditions as an example, African surgeons would list amongst their priorities sigmoid volvulus, umbilical hernia, gunshot wounds and gangrene. Many such areas remain unaddressed, despite important improvements, illustrated for example by the list of reviews delivered by the Cochrane Infectious Diseases Group.

Healthcare professionals in resource-poor settings may wonder whether reliance on older, cheaper, lower-tech approaches, has made their practice quite distinct from that of their colleagues in the West. For example, ketamine, which is widely used for anaesthesia in Africa has not been the subject of a Cochrane review.

The usefulness of existing reviews is limited because they rely largely on research in resource-rich settings. Research from developing countries is seldom indexed or published in retrievable media. Cochrane reviewers aim to identify all studies published or unpublished, and in whichever language. Yet it is still possible that important data are missed.

Identified studies are appraised by reviewers and when the inclusion criteria are met (in most cases this requires a trial to be properly randomised) these are included in the analysis. However, RCTs may not be the sole or appropriate source of evidence in resource-poor situations, and these criteria may result in the exclusion of valuable valid evidence.

There are concerns about the 'transferability' of evidence between settings with different resourcing. To give just a few examples, patients in poorly resourced areas typically consult when disease is more advanced, compared with highly resourced settings. , they are more likely to have been self-medicating on 'prescription-only' drugs or on traditional treatments; even when they have presented to a health facility, diagnosis and referral (when needed) will take much longer. Limited resources will affect the delivery of treatment itself and follow-up. Levels of compliance with treatment are also much lower. Similarly, legislation is a healthcare intervention that is important, for example, in the prevention of road traffic injury but enforcing the legislation is usually harder in developing countries.

Potentially, these differences may result in a mismatch in the effectiveness of treatments evaluated in RCTs between highly resourced and poorly resourced settings. Some treatments will be sufficiently robust to retain much of their effectiveness in a resource-poor context; others will not.

Rather than claiming that their conclusions are globally applicable, reviewers could conclude with statements such as, ''There is evidence for the effectiveness of this intervention in the settings where the included studies were conducted and in other settings with similar levels of resources."

Questions

What should we do to make a good product (the systematic review) better? The following questions are offered for consideration.
Titles. How can we get more reviews written on a) priority health problems in the majority world, b) affordable interventions?
Context. Should there be some reviews that specify the context in which the condition is occurring and in which the intervention is delivered?
Background sections. How can we persuade reviewers to look at conditions/ interventions globally and not just as they affect the US and Western Europe?
Search for studies. How can we improve our success rate in finding work done in developing countries?
Inclusion criteria. Are high thresholds in the quality assessment of studies excluding valid useful research conducted in developing nations?
Analysis. How can we persuade reviewers to consider whether heterogeneity can be explained by differences in resourcing and to conduct the appropriate analysis?
Discussion/conclusions/implications must address the issue of where the recommendations apply — everywhere, or just in settings similar to where the research has been done?
Dissemination of the findings of reviews. Is this best done by circulating the reviews themselves, or are reviews a stage in the production of more accessible evidence-based health information materials? WHO's Reproductive Health Library (RHL) presents Cochrane reviews on CD-Rom with summaries, commentaries etc. from a developing world perspective. Clinical Evidence produces synopses of the evidence, integrating Cochrane Reviews into clinical questions, delivered in user-friendly formats and languages. Are more initiatives like these needed?
Research is needed into the impact of systematic reviews on practice in the developing world so far. Specific questions include: What percentage of reviews are relevant to healthcare in low-resource settings?, Is policy being set in different countries, using evidence-based sources?, How many healthcare workers use the Cochrane Library? Has this influenced their practice?, What do they think would make reviews more useful?
How can we involve more people from developing countries in the writing and peer reviewing of systematic reviews?

Paul Chinnock

Acknowledgements

My thanks to Dr Fred Bukachi (Norrlands University Hospital, Sweden) and Dr Luis Cuervo (Clinical Evidence/BMJ Knowledge) for valuable input into the above paper.

This is the first in a series of discussion papers that will be posted on HIF-net at WHO and other email lists as part of the Global Review on Access to Health Information in Developing Countries — see below*. We look forward to hearing your reactions and ideas — please send your responses to hif-net@who.int Your views will be collated and presented at a special session of the Cochrane Colloquium in Ottawa, Canada, on Monday 4 October 2004: http://www.colloquium.info/?PageID=254&pid=80&ItemID=380

[HIF-net at WHO profile: Paul Chinnock taught in Kenya, then edited schoolbooks for developing countries. He retrained as a nutritionist and worked in The Gambia. He was Editor of Africa Health for most of the 1990s, and was also responsible for Medicine Digest, International Diabetes Digest and Caribbean Health. He became Managing Editor of the Injuries Group of the Cochrane Collaboration in September 2002. Paul would like there to be more information for evidence based medicine that reflects the realities of working in resource-poor environments. Paul has also worked as a freelance editor and writer for WHO, DFID, Panos Institute, Healthlink and the International Hospitals Federation.
paul@pchinnock.fslife.co.uk ]

[*The Global Review on Access to Health Information in Developing Countries was launched on 12 July 2004 with the technical and in-kind support of 20 organizations working in health information and knowledge exchange. The Review aims to review what works and what doesn't, lessons learned, and ways forward for the future. ***Further progress on the Review is dependent on funding to cover the costs of coordination, consultation, and networking activities in developing countries***. For details, see http://www.inasp.info/health/globalreview/index.html ]

'HIF-net at WHO': working together to improve access to reliable information for healthcare providers in developing and transitional countries. Send list messages to hif-net@who.int To join the list, send an email to health@inasp.info with name, organization, country, and brief description of professional interests.