ORIGINAL ARTICLES

The Trial Protocol Tool: the PRACTIHC software tool that supported thewriting of protocols for pragmatic randomized controlled trials

Shaun Treweeka,*, Kirsty McCormackb, Edgardo Abalosc, Marion Campbellb, Craig Ramsayb,Merrick Zwarensteind, on behalf of the PRACTIHC collaboration

aTayside Centre for General Practice, University of Dundee, DD2 4BF Dundee, UKbHealth Services Research Unit, University of Aberdeen, Polwarth Bldg, Foresterhill, AB25 2ZD Aberdeen, UK

cCentro Rosarino De Estudios Perinatales (CREP), Pueyrredon 985, (2000) Rosario, ArgentinadInstitute for Clinical Evaluative Sciences and Department of Health Policy, Management and Evaluation,

University of Toronto, 2075 Bayview Ave, Toronto, Ontario M4N 3M5, Canada

Accepted 24 December 2005

Abstract

Objective: To develop a tool that would make it easier for researchers, especially those in low- and middle-income countries, to writeresearch protocols for pragmatic randomized controlled trials.

Study Design and Setting: A series of focus groups was held at the inaugural meeting of the Pragmatic RAndomized Controlled Trialsin Health Care (PRACTIHC) project in 2001 to develop a desired specification for the Trial Protocol Tool. A working group of five indi-viduals from the PRACTIHC group was formed to develop content for the tool.

Results: The Trial Protocol Tool was developed in English and Spanish as a Microsoft Windows HTML help system. A Web-basedversion is also available. This main body of the tool provides information, advice, and resources about the major headings that shouldbe part of every research protocol. Illustrative examples are used throughout and are taken directly from the tool’s protocol library. Addi-tional resources include checklists, programs (e.g., a sample size calculator), and example documents (e.g., patient information leaflets).

Conclusion: The Trial Protocol Tool packages all the key requirements for the development of a research protocol into one resource.We believe that the use of the tool will help researchers to design effective trials and to write high-quality protocols. ! 2006 Elsevier Inc.All rights reserved.

Keywords: Randomized controlled trials; Pragmatic; Study protocols; Software; Protocol resources; Low- and middle-income countries

1. Background

The randomized controlled trial (RCT) is the widely ac-knowledged design of choice for evaluating the effective-ness of health care interventions [1,2]. Pragmatic RCTsare evaluations of health care effectiveness carried out un-der real world conditions, in typical settings and on typicalusers. Effectiveness of the intervention is usually assessedby its impact on simple outcomes of importance to usersof the intervention such as death, major disability, cost,and quality of life. Pragmatic trials are often more relevantto policymakers than efficacy trials because they generallyuse the same criteria of effectiveness as those used by pol-icymakersdnamely, user perceptions, important and

visible outcomes, usual health service planning entities,and typical service limitations.

The first step in running a high-quality pragmatic RCT isto produce a good research protocol. This is true both forobtaining funding to run the trial and for ensuring thatthe trial itself will produce meaningful results. Despitethe existence of guidelines and individual tools to help withthis, these are spread across hundreds of journals, Websites, books, conference proceedings, and software tools.Access to this dispersed information can be a problem,especially in low- and middle-income countries [3]. More-over, inexperienced researchers may not appreciate the fullrange of issues that need to be addressed in a protocol andmay, therefore, fail to look for resources that address theseissues. In this paper, we report the development of a soft-ware tool (known as the Trial Protocol Tool), which pack-ages this information in a single, easily accessible, andpractical software tool. A screenshot of the current versionof the Trial Protocol Tool is shown in Fig. 1.

* Corresponding author. University of Dundee, Tayside Centre forGeneral Practice, Mackenzie Building, Kirsty Semple Way, Dundee DD24AD, United Kingdom.

E-mail address: streweek@mac.com (S. Treweek).

0895-4356/06/$ e see front matter ! 2006 Elsevier Inc. All rights reserved.doi: 10.1016/j.jclinepi.2005.12.019

Journal of Clinical Epidemiology 59 (2006) 1127e1133

The Trial Protocol Tool was developed as an output ofa European Commissionefunded partnership between Eu-ropean Union, African, and Latin American trialists knownas the Pragmatic RAndomized Controlled Trials in HealthCare (PRACTIHC) project (www.practihc.org). The aimof PRACTIHC was to promote the global science of prag-matic RCTs with special emphasis on trials conducted inlow- and middle-income countries. PRACTIHC partnersbased in Africa and Latin America considered the lack ofa single tool to support the production of high-quality pro-tocols to be a significant barrier to meeting this aim. Giventhe focus of PRACTIHC, the examples included in the TrialProtocol Tool are primarily from trials mounted in low- andmiddle-income countries. Further information on the widerPRACTIHC project can be found on the project Web site.

2. Tool specification

A series of focus groups was held at the inaugural meet-ing of the PRACTIHC project (which was held in CapeTown, South Africa, in 2001) to develop a desired specifica-tion for the Trial Protocol Tool. Attendees at the meeting in-cluded experts in trial design, clinical researchers, and trialmanagers. The focus groups were asked to prioritize (a)the information they would like the tool to contain, (b) theresources they would like to be available within the tool,and (c) any desired design features. In addition to these func-tional specifications, attendees were also asked to indicateany information technology constraints, which might befaced in their individual clinical settings given that the tool

would be primarily used in low- or middle-income coun-tries. Discussion of the results of the focus groups was heldin a plenary session at the end of the meeting, and the fol-lowing consensus requirements were identified:

Content

! Sections describing the key elements of a trial pro-tocol covering design, conduct, and analysis

! Within each section, short paragraphs explainingthe key features required with examples of sug-gested text

! Pointers to further information including furtherreading materials and Internet resources

! Library of relevant trial protocols

Resources

! Sample size calculator for individually randomizedtrials

! Sample size calculator for cluster randomized trials! Gantt chart tool to aid the planning of RCTs

Design features

! Be as small as possible! Not require an Internet connection! Be simple to use! Be expandable! Support multiple languages (in particular Englishand Spanish)

! Preferably run on several platforms but must run onWindows 95 and above

Fig. 1. A sample screenshot from the Trial Protocol Tool.

1128 S. Treweek et al. / Journal of Clinical Epidemiology 59 (2006) 1127e1133

3. Content development

A working group of five individuals from the PRAC-TIHC group was formed to develop the Trial ProtocolTool content (E.A., M.C., K.McC., C.R., S.T.). These fiveindividuals represented a range of expertise including trialdesign, trial management, statistics, Information Technol-ogy, and clinical application. All had significant experiencein the design and conduct of RCTs and the development ofsuccessful trial protocols.

There are six main sections in the Trial Protocol Tool:

1. What should be in a good protocol?2. Useful documents3. Protocol Library4. Teaching resources5. Web resources6. Glossary

Fig. 2 shows screenshots of the opening pages of thesesix sections.

3.1. What should be in a good protocol?

This is the main body of the Trial Protocol Tool, and itlists the major headings that should be part of everyresearch protocol.

An Internet search was initially undertaken to inform thedevelopment of the structure of a ‘‘good protocol.’’ A num-ber of protocol ‘‘checklists’’ were identified in the Internetsearch and reviewed within the working group (examples ofwhich can be found in the reference section) [4e6]. Follow-ing discussions, a consensus structure was agreed upon.This was passed to the PRACTIHC management groupfor wider review and final approval. The PRACTIHCmanagement group consists of experts from 10 countriesall experienced in the design and conduct of RCTs (seeAppendix 1). The section headings for ‘‘What should bein a good protocol?’’ are shown in Table 1. Each headinggenerally contains several subheadings. The ‘‘Overviewof RCTs and trial design’’ section, for example, has six sub-headings: Parallel trial, Cluster randomised controlled trial,Multiple arm trial, Factorial clinical trial, Crossover design,and Equivalence design.

Each section was developed to a common structure:

! Introduction! Things to consider when writing a protocol! Illustrative examples (generally several)! Additional resources! Further reading

When drafting the ‘‘introduction’’ and ‘‘things to con-sider’’ sections, the aim was for the text to be instructionalas well as informative. For example, when describing therange of possible trial designs, care was taken to not onlyoutline the features of each design, but to guide the useras to when the use of each design might be appropriate.For example, when describing a crossover design, both

the characteristics of the crossover design and the situationswhere its use might be appropriate are outlined:

‘‘In a crossover design, each participant is randomizedto a sequence of two or more treatments.Crossovertrials produce within participant comparisons,whereas parallel designs produce between participantcomparisons.Crossover trials can be used to investi-gate chronic conditions, such as asthma, where the ob-jective is to investigate the participants’ short termresponse to therapy. The condition must also be stable,so that the circumstances at the beginning of eachperiod are more likely to be the same. Clearly, not allinterventions can be studied in crossover designs. Forexample, comparing surgical procedures or evaluatinglong term outcomes such as 5 year survival where it isimpossible to crossover to another intervention.’’

Illustrative examples are taken directly from ‘‘real-life’’research protocols (which are available in full in a protocollibrary reference section). The user can therefore see howother researchers addressed this part of the protocol andthen, if desired, he or she can go to the full protocol inthe Protocol Library. The library currently contains 12 fullprotocols of which five describe trials done in low- andmiddle-income countries, and three of the protocols arealso available in Spanish (see Appendix 2). Additionalresources include checklists, images, programs, exampledocuments (e.g., patient information leaflets), guidelines,and links to Web sites. Resources that are part of the TrialProtocol Tool are differentiated from Web resources bymeans of an icon before the resource name.

A lead individual from the working group was identifiedfor each section of the protocol. The lead person was de-termined primarily by the content under review (e.g., sta-tistical analysis was led by the statistician, trialmanagement was led by the trial manager). This lead per-son took responsibility for preparing the first draft of thecontent for that section. (For each section, an Internetsearch was also undertaken [by K.McC.] to identify anyrelevant literature and/or resources to inform the develop-ment of the content. Reference lists and bibliographies ofkey documents were also searched. This draft was then cir-culated around the larger working group for comment andrevision. This process was repeated until the workinggroup had no further comments. Once the content was ‘‘fi-nalized’’ by the working group, each section was passed toan editorial subcommittee (made up of three researchers inAberdeen) who assessed all sections for style and consis-tency. Following this, all sections were passed to thePRACTIHC management group for wider review and finalapproval.

3.2. Other sections

The remaining four sections (Teaching resources, Use-ful documents, Web resources, and Glossary) contain text

1129S. Treweek et al. / Journal of Clinical Epidemiology 59 (2006) 1127e1133

and links that support the information given in the Protocolchecklist. The teaching resources are a series of Power-Point presentations that can be used for teaching or self-study. The Glossary is based on the glossary in theCochrane Collaboration Handbook and is used withpermission.

4. Translation of content into Spanish

Following final approval, each section of the ‘‘Whatshould be in a good protocol?’’ component was sent tothe PRACTIHC partner in Argentina (E.A.) for translationinto Spanish. The translation was done by a linguist using

Fig. 2. Screenshots from the six main sections in the Trial Protocol Tool.

1130 S. Treweek et al. / Journal of Clinical Epidemiology 59 (2006) 1127e1133

Trados Workbench (a translation software package) andTagEditor (a translation tool interfacing with Trados Work-bench). After editing, linguistic and stylistic review, thetranslation was done by a second linguist to check forcompleteness, accuracy, style, grammar, terminology, andconsistency. Translations were finally checked by E.A.

5. Resources

A sample size calculator incorporating both individualand cluster randomized trials was developed for inclusionin the Trial Protocol Tool. It uses standard formulas forsample size calculations [7] and provides sample size cal-culations for both the comparison of proportions and thecomparison of means. A basic Gantt chart builder was alsodeveloped. This allows the different stages of the trial to bepresented visually on a monthly timeline from the proposedtrial start date. Both tools were developed using VisualBasic v6.

Additionally, we included as resources a wide range ofchecklists, leaflets, brochures, and articles. In particular,we included checklists and text written by David Sacketton RCTs for ‘‘Clinical epidemiology: how to do clinicalpractice research’’ [8]. All extra resources are included withpermission.

6. Technical information

The Trial Protocol Tool was developed as a MicrosoftWindows HTML help system, the standard help systemused with Windows 95 and above. This choice gives thetool a familiar Windows interface, does not require an In-ternet connection to run, and provides a range of off-the-shelf functionality. The software for running WindowsHTML help, including Internet Explorer upon which it re-lies, is part of the Windows operating system. This meansthat the size of the Trial Protocol Tool is decided by its

content and not the delivery system. The basic HTML helpdevelopment tool, Microsoft HTML Help Workshop, isavailable free from Microsoft’s Web site.

For the content, HTML versions of Word documentswere created using Mozilla 1.4 for Mac OS X, althoughany HTML editor could be used. The HTML documentswere then linked to the help system in HTML Help Work-shop. The HTML documents themselves can contain linksto the full spectrum of electronic document formats andcan, therefore, include not only textual information but alsographics, documents created by other programs, sound,video, links to Web sites, and other programs. Once theHTML documents have been added or updated, the collec-tion is compiled into a single file (a .chm file). This file canthen be distributed by email, from a Web site, or on CD.

If an HTML document contains a link to a pdf or Worddocument, for example, the document is included in thecompiled .chm file. Although convenient, this has a num-ber of disadvantages; in particular, it quickly leads toa large .chm file. To avoid this, such documents are notembedded but kept as external files in the same folderas the compiled .chm file. When the user clicks on thelink the file will open in the program to which it is asso-ciated. For example, clicking a link to a .pdf file willlaunch an instance of Acrobat Reader. However, if a userdoes not have the appropriate program installed, he or shewill not be able to open the file. To minimize this prob-lem, the external files within the Trial Protocol Tool areeither programs themselves or use programs that are freesuch as Acrobat Reader.

The current version (5/18/06) of the tool comprisesa compiled .chm file (520 kB), an external library of proto-col examples (2.6 Mb), a set of teaching resources(3.7 Mb), and a collection of other supporting external ma-terial (8.6 Mb). The .chm file is usable without the externalfiles and meets the PRACTIHC requirement for small size.The tool is, however, much more useful with the externalfiles. The combined size of these resources is large, al-though compression and splitting the download into smallerchunks will make things more manageable. We have alsodeveloped a Web-based version using chm2web distributedby A!K Research Labs (http://chm2web.aklabs.com/) thatis platform independent and works well with only a modestconnection speed.

7. Comment

The first step in running a high-quality trial is to producea good research protocol. Despite the existence of guide-lines and individual tools to help with this, these are spreadacross many diverse locations and different media. TheTrial Protocol Tool presented in this paper packages thisinformation into a single, easily accessible, and practicalresource. It uses a familiar Windows or Web-based inter-face, is available in English and Spanish, and contains thefull spectrum of electronic document formats.

Table 1Section headings for ‘‘What should be in a good protocol?’’

1. Summary2. Introduction3. Overview of RCTs and trial design4. The research question5. RCT population6. Allocation of interventions7. The interventions8. Outcome assessment9. Postrecruitment retention strategies10. Safety monitoring and adverse events11. Data collection and management12. Sample size13. Analysis strategies14. Ethical aspects of RCTs15. RCT management16. Economic evaluation17. Consumer involvement18. Reporting, dissemination, and notification of results19. References

1131S. Treweek et al. / Journal of Clinical Epidemiology 59 (2006) 1127e1133

The Trial Protocol Tool has been evaluated twice by thePRACTIHC group and was used as the core component ofa training course in RCTs held in South Africa in April2004 and 2005. The feedback from the evaluations andthe training courses was overwhelmingly positive, althougha number of small modifications were proposed. The toolhas been modified to take account of this feedback andwe expect this development to continue albeit at a reducedpace now that the PRACTIHC project has formallyfinished.

The PRACTIHC project has a number of other outputsthat have been operationalized as practical tools withinthe Trial Protocol Tool. One important example is the train-ing material on RCTs for teachers and students. This wouldallow a student to read some lecture material on samplesize calculations, then look at the examples within the TrialProtocol Tool before using the calculator to do his or herown sample size calculation. A trial simulator that can beused to explore aspects of the design, conduct, and analysisof RCTs is another example. The intention is to providea tool that will help people to design effective protocolswithout the need for them to do huge amounts of indepen-dent work.

The current Trial Protocol Tool meets most, but not all,of the PRACTIHC group’s original requirements for a pro-tocol tool. Although the core tool (the .chm file) is small,the external resources are not. Separating these into a seriesof smaller chunks would make it easier for individuals withslow modem connections to obtain them, although this willnot be a satisfactory solution for all potential users. Otheroptions we are exploring include breaking down the exter-nal resources into individual documents (e.g., a single pro-tocol) and distributing the Trial Protocol Tool on CD inlow- and middle-income countries. We have also developeda Web-based version, which makes the Trial Protocol Toola platform-independent tool that can be viewed in any mod-ern Web browser.

The Trial Protocol Tool is now being promoted throughthe PRACTIHC network, conferences, and local meetingsand is available for download from the PRACTIHC Website. The Trial Protocol Tool is also discussed with medicaland other health science students and with health care pro-fessionals via the teaching and training commitments ofmembers of the PRACTIHC partnership. We encourage re-searchers to use the tool in their own teaching and researchwork.

8. Conclusion

This paper presents the Trial Protocol Tool, which pack-ages all the key requirements for the development ofa high-quality research protocol into one resource. Webelieve that the use of this centralized resource will helpresearchers, particularly in low- and middle-income coun-tries, to design effective trials and to write high-qualityprotocols.

9. Availability and requirements

Project homepage

http://www.practihc.org.

Operating system

Windows HTML help version works in Windows 95 andabove.Web-based system works with Linux, Mac OS 9/OS X, andWindows.

Other requirements

Windows HTML help requires that Internet Explorer ver-sion 3 or above is installed. The browser-based version re-quires nothing more than a modern browser.

License

PRACTIHC is happy to have its products used at nocharge in free educational programs. However, when tuitionis charged, PRACTIHC’s royalty fee is 10% of gross re-ceipts. If you are planning to use the Trial Protocol Toolwithin a for-profit organization in any way, please contactus to discuss a donation to our work.

Acknowledgment

The development of the Trial Protocol Tool was sup-ported by the European Commission’s 5th FrameworkINCO programme, contract ICA4-CT-2001-10019. TheHealth Services Research Unit, University of Aberdeen,UK, is funded by the Chief Scientist Office of the ScottishExecutive Health Department. The views expressed arethose of the authors and not necessarily those of thefunding body.

Appendix 1

The PRACTIHC management group

CoordinatorDr Andy OxmanNorwegian Knowledge Centre for the Health ServicesNorway

Partners (in alphabetical order)Dr Edgardo AbalosCentro Rosarino De Estudios PerinatalesArgentina

Prof Marion CampbellUniversity of AberdeenUK

Prof Julie CliffUniversity of Eduardo MondlaneMozambique

(Continued)

1132 S. Treweek et al. / Journal of Clinical Epidemiology 59 (2006) 1127e1133

Appendix 1continued

Dr Susanne DoepfmerUniversity of LuebackGermany

Dr Lelia DuleyUniversity of LeedsUK

Prof Ubaldo FarnotHospital Gineco-Obstetrico ‘‘America Arias’’Cuba

Dr Carl LombardMedical Research CouncilSouth Africa

Dr Juan Manuel LozanoJaveriana UniversityColombia

Dr Cecilia Stalsby LundborgNordic School of Public HealthSweden

Prof Godfrey WoelkUniversity of ZimbabweZimbabwe

Appendix 2

The 12 full protocols in the Protocol Library

1. WHO calcium supplementation trial for the prevention of pre-eclampsia among low calcium intake women1,2

2. Latin American Trial on Caesarean Section (Perinatal care trial)1,2

3. Corticosteroid randomization after significant head injury (CRASH trial)4. International neonatal immunization study (INIS trial)5. International subaracnoid aneurysm trial (ISAT)6. Magnesium sulfate for treatment of pre-eclampsia: a trial to evaluate

the effects on women and their babies (Magpie trial)1,2

7. Rational Prescribing in Primary Care (RaPP-trial). A randomized trialof a tailored intervention to improve prescribing of antihypertensiveand cholesterol-lowering drugs in general practice

8. Multicenter RCT of symptomatic vs. intensive bisphosphonate therapyfor Paget’s disease (PRISM trial)

9. A cluster randomized control trial to evaluate the effectiveness ofa community-based program to improve tuberculosis control in theagricultural sector of the Winelands Health District, South Africa(WELLINGTON trial)1

10. Knee arthroplasty trial (KAT trial)11. Randomized placebo-controlled trial of daily oral vitamin D and

calcium for the secondary prevention of osteoporosis-related fracturesin the elderly (RECORD trial)

12. Syphilis in Pregnancy: A Health Systems Intervention Trial(SYTRUST trial)1

1 This study was done in at least one low- or middle-income country.2 This protocol is also available in Spanish.

References

[1] Altman DG. Better reporting of randomised controlled trials: theCONSORT statement. BMJ 1996;313:570e1.

[2] Pocock SJ. Clinical trials: a practical approach. Chichester: Wiley;1996.

[3] Smith R. Closing the digital divide. BMJ 2003;326:238.[4] ICHHarmonisedTripartiteGuideline:Guideline forGoodClinical Practice

E6(R1) (Current Step 4 Version dated June 10, 1996). Available at http://www.ich.org/cache/compo/276-254-1.html. Accessed March 2003.

[5] HIV Prevention Trials Network. HPTN protocol template. Availableat http://www.hptn.org/network_information/network_templates.htm.Accessed March 2003.

[6] Centers for Disease Control and Prevention. Developing a protocol:a guide for CDC investigators. Available at http://www.cdc.gov/niosh/nas/mining/pdfs/Protocol%Checklist.pdf. Accessed March 2003.

[7] Campbell MJ, Julious SA, Altman DG. Estimating sample sizes forbinary, ordered categorical and continuous outcomes in two groupcomparisons. BMJ 1995;311:1145e8.

[8] Haynes RB, Sackett DL, Guyatt GH, Tugwell P, editors. Clinical epi-demiology: how to do clinical practice research. 3rd ed. Philadelphia:Lippincott Williams & Wilkins; 2005.

1133S. Treweek et al. / Journal of Clinical Epidemiology 59 (2006) 1127e1133