The next steps towards responsible animal-based research

PDF hosted at the Radboud Repository of the Radboud University

Nijmegen

The following full text is a publisher's version.

For additional information about this publication click this link.

http://hdl.handle.net/2066/169267

Please be advised that this information was generated on 2022-07-24 and may be subject to

change.

http://hdl.handle.net/2066/169267

EVALUATION OF STRATEGIES TO IMPROVE SCIENTIFIC QUALITY ANDRESPONSIBLE ANIMAL USE IN RESEARCH

JUDITH A.K.R. VAN LUIJK

The next steps towards responsibleanimal-based research

Evaluation of strategies to improve scientific quality andresponsible animal use in research

Judith A.K.R. van Luijk

This thesis was financially supported by Executive Board Radboud University and Board of Directors Radboudumc.

The printing of this thesis was financially supported by the Willy van Heumenfonds

The next steps towards responsible animal-based researchEvaluation of strategies to improve scientific quality and responsible animal usein researchAuteur: Judith A.K.R. van LuijkOntwerp: Proefschriftontwerp.nlDrukwerk: Proefschriftmaken.nlISBN: 978-94-6295-610-0

© 2017 Judith A.K.R. van LuijkNo part of this thesis may be produced in any form or by any means withoutwritten permission of the author or of the publisher holding the copyright of thepublished articles.

The next steps towards responsibleanimal-based research


Proefschriftter verkrijging van de graad van doctoraan de Radboud Universiteit Nijmegen

op gezag van de rector magnificusprof. dr. J.H.J.M. van Krieken,

volgens besluit van het college van decanenin het openbaar te verdedigen

op woensdag 24 mei 2017om 10.30 uur precies

door

Antonia Katharina Romelia van Luijkgeboren op 16 april 1982

te Oeffelt

Promotoren:Prof dr. M. Ritskes-HoitingaProf dr. R.J.P.M. Scholten (UMCU)Prof dr. G.J. van der Wilt

Copromotoren:Dr. H.B. van der Worp (UMCU)Dr. ir. P.P.A.M. Leenaars

Manuscriptcommissie:Prof. dr. H.A.E. ZwartProf. dr. T. Nevalainen (University of Eastern Finland, Finland)Prof. dr. P. Vergara (Universidad Autónoma de Barcelona, Spanje)

The next steps towards responsible animal-based research


Chapter 1 General introduction Evaluation of strategies to improve scientific quality and promote responsible animal use in research 9

Chapter 2 Assessing the Search for Information on Three Rs Methods, and their Subsequent Implementation: A National Survey among Scientists in The Netherlands 15

Chapter 3 Assessing the application of the 3Rs: A survey among animal welfare officers in The Netherlands 43

Chapter 4 Outcomes of a Dutch workshop on improvements for the 3Rs in daily practice 63

Chapter 5 The protective effect of meniscus allograft transplantation on articular cartilage: A systematic review of animal studies 77

Chapter 6 Systematic Reviews of Animal Studies; Missing Link in Translational Research? 111

Chapter 7 Effects of database selection and language restriction in systematic reviews and meta-analyses of animal studies 139

Chapter 8 General discussion How to improve responsible use of animals in

research; About 3Rs and SRs 161

English Summary 174 Nederlandse Samenvatting 178 Dankwoord 182 Curriculum vitae 186

Table of content

CHAPTER 1General introductionEvaluation of strategies to improve scientifi c quality and promote responsible animal use in research

Judith van LuijkMarlies LeenaarsMerel Ritskes-Hoitinga

Chapter 1

12

General introduction

Background

Animals are being used in fundamental and applied research, to advance health and safety for both humans and animals. The use of animals in research has always been controversial, this is why the Universities Federation for Animal Welfare in the UK initiated a study in 1954 for more humane techniques in laboratory animal experimentation. This study led to the 3R-principles (replacement, reduction and refinement), which were first introduced by William Russell and Rex Burch in their publication The Principles of Humane Experimental Technique in the late-1950s [1]. Russell and Burch argue that good science and animal welfare are intricately intertwined, for example chronic stress in research animals can result in inaccurate and misleading experimental data. The 3Rs are defined as: Replacement; avoid or replace the use of animals wherever possible, Reduction; employ strategies that will result in fewer animals being used and which are consistent with sound experimental design and Refinement; modify husbandry or experimental procedures to minimize pain and distress. Later this definition was expanded to also encompass the improvement of welfare (e.g. by providing enrichment). Laboratory animal science is the scientific field focussing on improving scientific quality and animal welfare. The 3Rs have become the central theme to implement this concept. Over the last decades the 3R principles have become incorporated in many animal research policies and legislations worldwide, as well as funding programs and education. However, despite all efforts, practice has shown that successful implementations of the 3Rs and thereby improving scientific quality and animal welfare is not as straight forward as it might seem [2].The aim of this thesis is to contribute to responsible use of animals in research by providing strategies to improve scientific quality and animal welfare in animal-based research. First the state of affairs on implementation of the 3Rs in the Netherlands was studied. This evaluation was used to generate new ideas on how to improve quality of animal- based research. This initiated further investigation of a new upcoming methodology in animal based research namely systematic reviews of animal studies. Therefore this thesis consists of two parts: part one focuses on the 3Rs and part two on systematic reviews.

Thesis outline part 1

In order to get insights into the current situation in the Netherlands, the Netherlands Organisation for Health Research and Development (ZonMW) [3] funded the project: “3Rs search and implementation – a national survey”. The results of this study are


13

presented in the first part of this thesis. The aim of the national survey was to gain insights into the professional view on the 3R principles from researchers, animal welfare officers and animal ethics committee members. Chapter 2 of this thesis presents the survey outcomes of the first group: the researcher, which is a very important group as they are primary responsible for the implementation of 3R methods in their research. Chapter 3 describes the survey outcomes of the animal welfare officers and animal ethics committee members. These two groups are primary responsible for the ethical evaluation of animal based research including adequate implementation of 3R methods. Chapter 4 presents the outcomes of the follow-up 3Rs workshop after the surveys. Respondents from all 3 groups were asked to participate in this one-day workshop to discuss how 3Rs search and implementation in the Netherlands can be improved.

Shift in research focus

Over the course of the research for this thesis we have found that improving quality and welfare in animal-based research requires more than ‘just’ stimulating better implementation of the 3Rs. Issues discussed in Chapter 4 such as poor experimental design and unpublished negative or neutral research results are not unique for pre-clinical research. Similar issues have existed - or still exist - in medical research. In medical research, systematic reviews of clinical trials have contributed significantly to raising awareness for these issues and have stimulated improvement as the need for guidelines on design and reporting of trials became apparent. The systematic review methodology is a systematic, thorough and transparent way of collecting, appraising and analysing all available studies and is a method to summarize and synthesise evidence in a most objective and reproducible way.A systematic review aims to be a transparent summary of all available evidence on a certain topic. The methodology specifically aims to produce a complete overview and reduce possible bias in the review process to a minimum. An overview of available evidence is useful in the reduction of unnecessary duplication of animal studies and provide evidence-based input for the design of new animal studies (including refinement methods) and clinical trials. Another pivotal step in a systematic review is providing transparency on the validity of the available evidence, by assessing the methodological quality of the included studies. Therefore, the conduct of systematic reviews of animal studies, seems a logical and natural road to follow when aiming to promote responsible animal use in research.

Chapter 1

14

Thesis outline part 2

The second part of this thesis focuses on the methodology that is currently used for systematic reviews of animal studies, specifically on the steps of searching for studies (completeness) and risk of bias assessment (quality). First Chapter 5 is an example of a systematic review of animal studies. This review aims to appraise and summarize all available evidence of animal studies on the protective effect of meniscus allograft transplantation on articular cartilage. Chapter 6 aims to evaluate the use of a risk of bias assessment in systematic reviews of animal studies, including an evaluation of what type of items are being scored and what the actual quality of these items was in the included primary studies. Chapter 7 aims to provide empirical evidence on how choices in the search strategy (database selection and language restriction) might affect the outcome of a systematic review or meta-analysis. The last chapter (Chapter 8) provides an overview of the main findings of the previous chapters, and their implications in the domain of animal based research. The final chapter also provides recommendations for further applications of the systematic review methodology in animal-based research, in order to promote responsible use of animals.


15

References

1. Russell, W.M.S. and R.L. Burch, The

Principles of Humane Experimental

Technique. 1959, London, UK: methuen.

2. Leenaars, M., et al., Assessing the

search for and implementation of the

three Rs: A survey among scientists. ATLA

Alternatives to Laboratory Animals, 2009.

37(3): p. 297-303.

3. ZonMW. October 2015; Available from:

http://www.zonmw.nl/en/.

CHAPTER 2Assessing the search for information on three Rs methods, and their subsequent implementation: A national survey among scientists in the Netherlands

Judith van LuijkYvonne CuijpersLilian van der VaartMarlies LeenaarsMerel Ritskes-Hoitinga

ATLA 39(5), 429-447, 2011

Chapter 2

18

Abstract

A local survey conducted among scientists into the current practice of searching for Replacement, Reduction and Refinement has highlighted the gap between the statutory requirement to apply 3R methods and the lack of criteria for a 3R search. To verify these findings on a national level, we conducted a survey among scientists throughout the Netherlands. Due to the low response rate the results give an impression of opinions rather than being representative for the Netherlands. The findings of both surveys complement each other and indicate that there is room for improvement. Scientists perceive searching for 3R information in literature as a difficult task, and specific 3R search skills and knowledge of 3R databases are limited. Rather than using a literature search, researchers obtain 3R information through personal communication, so that published 3R possibilities often remain unfound and unused. A solution may be to move beyond the search for the 3Rs and choose another approach. A method that seems most appropriate is systematic reviews (SR). It provides insight into the necessity of new animal studies, as well as optimal implementation of available data and the prevention of unnecessary animal use in the future.

Introduction

Dutch legislation requires that available Replacement, Reduction and Refinement [1, 2] (Three Rs) alternatives must be used. Before starting a new animal study, researchers are obliged to obtain approval from an Animal Ethics Committee (AEC). In the AEC application, researchers need to state whether Three Rs alternatives exist for this particular study, what evidence they have found for this, and whether these 3R alternatives will be applied. There are currently no criteria determining how this evidence should be presented, nor strict guidelines for how this evidence should be searched for and evaluated. It is therefore not known how scientists conduct their searches for Three Rs alternatives, what type of information sources they use, whether the search leads to useful information and whether Three Rs alternatives are successfully implemented in the study design.In 2009, Leenaars et al. published a survey among scientists of the Radboud University Medical Centre. The results show that there is room for improvement in searching more effectively for the Three Rs. For instance, searching for the Three Rs is not a structural part of a research process and search skills for Three Rs alternatives are limited, as is knowledge of specialised Three Rs databases. Respondents indicated dissatisfaction with the availability and accessibility of information regarding the Three Rs. No budget for a Three Rs search is allocated and time spent on a Three Rs search is rather limited

3R suRvey among ReseaRcheRs

19

[3]. The current questionnaire was held among scientists in the Netherlands in order to examine whether the local results were more generally found on a national level as well, and if so, what opportunities the respondents see for improving the search and implementation of the Three Rs.

Materials and Methods

A survey was conducted among researchers in the Netherlands (Federation of Laboratory Animal Science Associations (FELASA) category C scientists). The survey was descriptive in nature and designed to systematically categorise the problems that scientists experience when searching for the 3Rs. The questionnaire used in this study is an extended version of the local questionnaire by Leenaars et al. [3]. Questions were added based on in-depth face-to-face interviews with five researchers, two members of the AEC and one Animal Welfare Officer AWO. A closed set of answers to each question ensured that respondents were consistent in the way they answered. There was room for comments in questions for which the set of answers was not exhaustive. Multiple answers were allowed in inventory questions, for example about which resources were used. The complete questionnaire can be found in the appendix. The results of the survey were separated from the background and contact details of the respondent to safeguard the anonymity of the respondents and to exclude possible bias. The results were analysed per question.

Results

This results section is a selection of the full data overview in the Appendix.

RespondentsFifty-two scientists from 20 different organisations responded to the questionnaire. As it is not known how many scientists in total are currently involved in animal experimentation in the Netherlands, nor how many scientists were approached for this survey, since the researchers could only be approached indirectly via the AWOs. Therefore a percentage of the response cannot be calculated. Most respondents (32 in total) are employed by 9 different universities and academic medical centres. All respondents were familiar with the term 3Rs and were able to provide a description of the meaning.

Chapter 2

20

Use of information sourcesThe 3R information sources that scientists (N=52) evaluated as being ‘high’ contributors to 3R knowledge in general were: the researcher’s own experience in conducting animal experiments (76.9%), the FELASA category C course (71.2%), the researcher’s own experience with research applications (57.7%), and personal communication with the AWO (61.5%) or with animal caretakers and technicians (53.8%). Personal communications with other researchers in animal-based research were also evaluated as ‘high’ contributors (48.1%). The majority of respondents (73%) indicated that updating oneself with 3R literature scores as a ‘low’ contribution. Researchers were asked which information sources they use in order to gain knowledge on the 3R possibilities for a specific research topic and how much these sources actually contribute to the implementation of 3Rs in their work. The information sources considered as ‘high’ contributors were: personal communication with colleague researchers (69.2%), information from their own earlier work (61.5%), earlier work done by the researcher’s own research group (61.5%), personal communication with the AWO (55.8%) or with animal caretakers and technicians (50.0%), and the feedback of the AEC (48.1%). Literature searches and outsourcing of literature searches were not considered important contributors. When the scientists (N=52) were asked which databases they knew, respondents answered that they were familiar with PubMed (96.2%), Google (86.5%) and Web of Science (59.6%). The sources Embase (21.2%), Agricola (15.4%), Toxnet (15.4%) and NCA (15.4%) were known by a relatively small group. Only a handful of respondents had heard of NC3Rs (7.7%), Altweb (5.8%), Zebet (3.8%) and Scopus (3.8%). The online information sources AWIC, Norina, FRAME and Go3R were known to one and the same respondent. UCCAAI, AVAR and Altbib were not known to any of the respondents. For cases where databases were actually used for the 3R search (N=39), we asked how frequently they were used. When searching for possibilities on the 3Rs in Pubmed, 30 respondents (76.9%) indicated that they used it either frequently or always. For cases where Embase was used for the 3R search (N=10), 20% indicated that they used it frequently. Frequent use of Web of Science was 21.7% (N=23), for Google 54.3% (N=35) and Agricola 14.3% (N=7). Online information sources specialised on the 3Rs were hardly known and thus barely used.

Search activitiesWhen respondents were asked if they conducted a literature search specifically on the 3Rs for their research, 41 respondents (78.8%) answered ‘no’, whereas 11 (21.2%) did conduct such a specific literature search. Of those not conducting a specific 3R literature search (N=41), 39.0% indicated that it did seem useful to conduct a literature search on the 3Rs, the other 61.0% indicated that it did not seem useful.


21

When the 41 respondents who did not conduct a 3R literature search were also asked why not, they selected one of the following pre-set answers: I do not think this will lead to new information (29.3%), I have never thought about this (17.1%), it is not part of my project (7.3%), I do not know how this will improve my research (4.9%), I do not know how to search for the 3Rs (4.9%), it does not have priority (4.9%), or it will delay my research (2.4%). The respondents writing AEC applications were asked how many hours they spent on a 3R search. This question was answered by 46 out of 52 respondents (88.5). For each AEC application, respondents spent an average of 3.3 hours on a search for 3R possibilities. The majority (93.5%) did not outsource the 3R search. Table 1 presents the results of the question that asked whether scientists perform a specific literature study on the 3Rs in relation to each phase of research, and so, how much time is then spent on this process. Nearly all respondents (98.1%) indicated that they do not include the search for 3R possibilities in their budgets.

Issues with searching Researchers were asked how they evaluate their skills in searching databases for the 3Rs (N=41). Most researchers (56.1%) indicated that they have sufficient skills for searching for the 3Rs in these databases, whereas 12.2% rated this as insufficient. The other 31.7% answered neutrally.When evaluating satisfaction with the availability of information on the 3Rs, 26.9% (N=52) was dissatisfied, 51.9% was neutral and 21.2% was satisfied. About the same results were obtained when respondents were asked about the accessibility of the 3R information (30.8% dissatisfied, 48.1% neutral and 21.2% satisfied).

Implementation of 3R knowledge in practiceWhen asked whether they think 3R possibilities are often missed, scientists indicated that information on the 3R possibilities is missed; this was ‘sometimes to regularly’ for 63.5% (N=52) of the respondents and ‘often to always’ for to 9.6%. The 3R possibilities that are known to the respondents are ‘often to always’ implemented, as indicated by 80.8%. Respondents were asked in which phases of research they are involved, and how they evaluate the importance of each ‘R’ per phase (N=52). Most researchers were involved in writing AEC applications (88.5%), study plans or ‘work protocol’ (92.3%), and conducting the animal experiments (82.7%). During AEC applications (N=46), 52.2% found that Replacement plays a role, with 95.7% indicating Reduction, and 87.0% Refinement. Those involved in study plans (N=48) indicated that Refinement is most relevant (81.3%), followed by Reduction (54.2%) and Replacement (14.6%). During the practical execution (N=43), Refinement is considered most important

Chapter 2

22

(83.7%), followed by Reduction (46.5%) and Replacement (14.0%). The attention for Replacement appears to be low in all phases of research.

SolutionsScientists were asked to give reasons for not implementing 3R possibilities in their own research (N=52). Frequently mentioned answers by respondents were: ‘lack of time, budget and knowledge’. Scientists were then asked to estimate the degree of implementation of each of the 3Rs in own research if given unlimited time and budget. Twenty-eight respondents (53.8%) thought that the implementation for Replacement with unlimited resources would be ‘none to very limited’, followed by 23.1% who thought this would be ‘reasonable’ and for 23.1% this would be ‘relatively good to good’. For Reduction the scores are as follows: 34.6% ‘none to very limited’, 28.8% ‘reasonable’ and 36.5% ‘relatively good to good’. For Refinement: 13.5% ‘none to very limited’, 30.8% ‘reasonable’ and 55.8% ‘relatively good to good’. The answers to the question ‘If you could start a new initiative on how to improve optimal use of existing 3R knowledge, what would be your point of focus?’ were pooled. The respondents were also asked to give each item a priority score from 1 (most important) to 4 (least important). The item ‘more openness between organisations on how animal experiments are conducted’ received the highest score, followed by ‘support at the level of the research department’ and ‘better (accessibility of) information systems and databases on 3R knowledge’. In fourth place came ‘improved facilitation of 3R knowledge exchange inside and outside the organisation’, and in fifth place ‘systematic reviews of existing literature’.

Discussion

A broad range of opinions from scientists working in different types of organisations has been obtained in this survey. As mentioned above, the results of the questionnaire cannot be interpreted as generally representative for all researchers conducting animal experiments in the Netherlands, but it does give valuable insight into how researchers from different backgrounds experience and evaluate the current 3R information search and how it can be improved.The 3R principles of Replacement, Reduction and Refinement of animal studies are a well-known concept. In practice however the 3Rs cannot be seen as three equal matters. For instance, Replacement gets the most attention during funding applications and AEC applications but less so during the other phases. This is understandable, since Replacement usually involves multiple steps (e.g. including international multicentre validation studies) and multiple groups/organisations (e.g. involvement of legislative bodies). The choice


23

for animal-based research can be made if Replacement methods are not suitable or available. This then explains why Reduction and Refinement get the most attention in the other research phases. From this perspective it is therefore useful to treat Replacement separately from the other two Rs, since it usually covers a much broader perspective.Updating oneself with 3R literature is a low contributor to general 3R knowledge and a specific literature search on the 3Rs is not usually conducted. Even though updating oneself with the latest literature is one of the prerequisites of good science [4], this seems not to be common practice for 3R literature. The majority of respondents neither reserves budget for a 3R literature nor conducts it. The minority that does search for the 3Rs, searches for an average of only 3.3 hours. The low use of literature may also be caused by the fact that it is complex and ineffective to do a Pubmed search with a specific focus on the 3Rs. Moreover, the specialised 3R databases and websites are hardly known and, when used, difficult to search due to differences in content, quality and search profiles [3]. This corroborates the outcome of the previous local questionnaire [3] and underlines the low priority given to the 3R search throughout a variety of organisations. There is clearly not yet sufficient pressure from legal authorities, funding bodies, editorial boards or AECs for scientists to perform a proper 3R literature search.In general, the relevance of the implementation of the 3Rs for laboratory animal welfare and the quality of science [5, 6] is supported by the majority of researchers. According to the majority of the respondents, known 3R possibilities are implemented most of the time, although a smaller majority also thought that information on the 3R possibilities is sometimes missed or that known 3R possibilities remain unused. This is worrying because the law requires implementation of 3R methods whenever possible. This may be related to the fact that the search and implementation of the 3Rs are not seen as easy tasks. This corroborates our earlier findings and our own practical experience [3]. It is the researcher’s own personal contacts and experience that are especially considered high contributors to obtaining 3R knowledge. The situation whereby professionals consult other people rather than literature or databases is not specific to laboratory animal science however; it is also seen in information-seeking behaviour of engineers and public health practitioners, for the simple reason that they are easily accessible [7, 8]. People seeking information act in such a way that they minimise the effort spent on gaining information, and do not use the quality of information as the main criterion (the ‘principle of least effort’ in information science).If given unlimited time and budget, about half of the respondents thought that especially the implementation for Refinement would be improved in their own research. Half of the respondents working at institutes where legally required tests are performed feel (very) strongly stimulated by the licence holder to implement the 3Rs, even though one third state that it is difficult to implement the 3Rs for such tests. This is a remarkably high figure, since legal guidelines make it virtually impossible to make fast 3R changes and thus to implement the 3Rs. Here there appears to be

Chapter 2

24

greater motivation for implementing the 3Rs, due to the sometimes harsh nature of legally required tests.Respondents were asked what focus points they would wish to have if starting a programme for more optimal use of 3R knowledge. The top 3 was as follows: 1) more openness between organisations, 2) better access to 3R information and databases as well as support at the research department, and 3) better knowledge exchange. The range in wishes illustrates the complexity of the subject, and there does not appear to be a single solution or a single strategy for improving the use of existing 3R knowledge. More openness in the exchange of information between institutes is considered the most important factor for better use of 3R knowledge. This requires that there be a more direct manner for researchers to make contact and communicate with one another, e.g. through a central (protected) database. While time and money are perceived as obstacles for implementing the 3Rs, they score low on the list of stimuli. It seems that money is a problem, but not a solution. This makes sense, since money will not provide knowledge and expertise for setting up an animal experiment that fully incorporates 3R knowledge, whereas collaboration with experts will increase knowledge and expertise. Because current procedures need to be adapted, new strategies need to be formulated and put into practice. Here we see opportunities for Systematic Reviews, on which we elaborate further below. Only a few respondents indicated that a 3R service would be their focus point of choice for more optimal use of 3R methods; we therefore suggest another strategy for a more successful search and implementation of the 3Rs. Prior to planning and executing new research, it is good science to perform a critical review of existing knowledge, in order to determine whether the current hypothesis is really novel and to become familiar with how previous similar studies were conducted [9]. This knowledge synthesis can be reached by following the path of systematic reviews (SR) [6]. It is worthwhile taking as an example the medical field, which made this successful change about 20 years ago. Evidence-based medicine based on systematic reviews is now considered the highest form of scientific evidence. Although SRs are now commonly performed for clinical studies, and have been for decades, this is surprisingly not yet the case for animal-based research; despite the fact that animal studies are used to predict the effect size of new therapies as well as their possible side effects in humans [10]. The advantage of introducing SRs into the field of animal-based research is that SR requires the involvement of specialists in several scientific disciplines, which allows for integrated and critical knowledge synthesis of ‘all’ available data and evidence. SRs will improve scientific quality, prevent unnecessary animal use and improve translational value [10]. Due to the critical analysis of all available knowledge, this will also ‘automatically’ lead to finding and implementing the 3Rs [6, 11, 12].


25

Concluding remarksVarious studies describing strategies to improve 3R implementation have been published over the last decade [13-20]. This survey shows there is still much room for improvement. While huge efforts have been made in developing 3R databases, they are hardly known or used and appear to be ineffective, which seems a waste of resources. Local 3R experts may offer a solution, since the local/personal network (knowledge gained via oral communication) is considered the main information source. The possibility of increasing the implementation of the 3Rs by facilitating local knowledge exchange through personal contacts is an important issue but also an opportunity. Because the 3Rs differ so much in nature, it is very difficult to determine common strategies for implementing all the 3Rs. Ideal implementation of the 3Rs will only occur when the 3Rs are an integral part of scientific practice. It may therefore be more fruitful to talk about gaining knowledge for responsible experimental design and animal use, instead of gaining 3R knowledge. This brings us back to the roots of the Three Rs: humane science, as expressed by Russell and Burch [1]. We believe that introducing the SR methodology from the medical field into animal-based research will lead to improvements in quality of science, animal welfare and implementation of the 3Rs, and enable genuine progress in humane science [4, 12]. This systematic approach will help in the identification, selection and analysis of all relevant published studies. An SR can tackle the above-mentioned inconsistencies and difficulties in the Three R search strategies. It does this by providing guidelines for formulating a search question, how and where to search for studies and how to process this literature into a review in an interdisciplinary team, naturally triggering personal contacts and networking in the process [11]. Systematic reviews have major potential for improving the quality of animal studies, compliance with laws and regulations, and most importantly preventing unnecessary animal use and improving animal welfare. We therefore propose that we move beyond the 3Rs towards systematic reviews as the leading concept.

Acknowledgments

This project was funded by the ZonMW programme ‘Dierproeven begrensd II’, project number 114000092. The authors would like to thank the researchers who completed the questionnaire and the animal welfare officers in the Netherlands for their active contribution in approaching and stimulating researchers within their institutes to participate in the survey.

Chapter 2

26

References




2. Tweede-Kamer, Wet op de Dierproeven.

1977, De Staatssecretaris van

Volksgezondheid en Milieuhygiëne,

Hendriks.





37(3): p. 297-303.

4. Hooijmans, C.R., M. Leenaars, and

M. Ritskes-Hoitinga, A gold standard

publication checklist to improve the

quality of animal studies, to fully integrate

the three Rs, and to make systematic

reviews more feasible. ATLA Alternatives

to Laboratory Animals, 2010. 38(2): p.

167-182.

5. Ritskes-Hoitinga, M. and L.B. Gravesen.

Refinement benefits animal welfare and

quality of science. NC3Rs #6 March 2006

(Invited articles) http://www.nc3rs.org.uk

2006.

6. Forni, M., Laboratory animal science: A

resource to improve the quality of science.

Veterinary Research Communications,

2007. 31(SUPPL. 1): p. 43-47.

7. Hertzum, M. and A.M. Pejterson,

The information-seeking practices of

engineers: searching for documents as

well as for people. Information Processing

and Management, 2000. 36: p. 761-778.

8. Revere, D., et al., Understanding the

information needs of public health

practictioners: a literature review to

inform design of an interactive digital

knowledge management system. Journal

of Biomedical Informatics 2007. 40: p.

410-421.

9. Gaines Das, R., et al., Planning for

reduction. Altern Lab Anim, 2009. 37(1):

p. 27-32.

10. Pound, P., et al., Where is the evidence

that animal research benefits humans?

British Medical Journal, 2004. 328(7438):

p. 514-517.

11. Egger, M., G.D. Smith, and D.G. Altman,

Systematic Reviews in Health care. 2001:

BMJ publishing Group.

12. Hooijmans, C.R., et al., Enhancing

search efficiency by means of a search

filter for finding all studies on animal

experimentation in PubMed. Laboratory

Animals, 2010. 44(3): p. 170-175.

13. Schiffelers, M.J., et al., Factors stimulating

or obstructing the implementation of the

3Rs in the regulatory process. Altex, 2007.

24(4): p. 271-8.

14. Lloyd, M.H., B.W. Foden, and S.E.

Wolfensohn, Refinement: Promoting the

three Rs in practice. Laboratory Animals,

2008. 42(3): p. 284-293.

15. Howard, B., M. Hudson, and R. Preziosi,

More is less: reducing animal use by

raising awareness of the principles of

efficient study design and analysis. Altern

Lab Anim, 2009. 37(1): p. 33-42.

16. de Boo, J. and C. Hendriksen, Reduction

strategies in animal research: a review

of scientific approaches at the intra-

experimental, supra-experimental and

extra-experimental levels. Altern Lab

Anim, 2005. 33(4): p. 369-77.


27

17. Klein, H.J. and K.A. Bayne, Establishing

a culture of care, conscience,

and responsibility: addressing the

improvement of scientific discovery and

animal welfare through science-based

performance standards. Ilar J, 2007.

48(1): p. 3-11.

18. Smaje, L.H., et al., Advancing refinement

of laboratory animal use. Laboratory

Animals, 1998. 32(2): p. 137-142.

19. Reed, B.T. and M. Jennings, Promoting

consideration of the ethical aspects of

animal use and implementation of the

3Rs. AATEX Special Issue, 2008. 14(Proc.

6th World Congress on Alternatives &

Animal Use in the Life Sciences, August

21-25, 2007, Tokyo, Japan): p. 131-135.

20. Bronstad, A. and A.G. Berg, The role of

organizational culture in compliance with

the principles of the 3Rs. Lab Anim (NY),

2011. 40(1): p. 22-6.

Chapter 2

28

Table 1 In what phase of research do you conduct a literature study specifically on the 3Rs and how much time (on average) do you then spend on this activity? (N=11, more answers per person possible)

No. of respondents No. of hours spent

Funding application 5 1, 8, 4, 12, 1

AEC application 10 120, 3, 1, 2, 2, 3, 1, 1, 1, 1

Study plan 4 120, 2, 1, 1

Practical conducting the study 2 40, 1

Reporting/publication 3 24, 8, 1

Other: Starting phase of a project 1 20


29

Chapter 2

30

Appendix chapter 2:

Translated from Dutch to Englisch

Part 1: Background researchers

1.1 Name organization

Excluded for privacy reasons.

1.2 Gender

Male Female

23 29

1.3 Year of Birth

Excluded for privacy reasons. Mean age 37,73

1.4 What type of organization do you work for?

University 32

Industry 4

Knowledge institute 11

Contract Research Organization 5

Other, namely. 0

1.5 What is the main purpose of the animal experiments?

Humane medicine 44

Veterinary medicine 13

Field Biology 1

Environmental research/ecosystem 2

1.6 Did you ever or how often do you execute animal experiments for the following purposes?

Purpose Never Sometimes Regularly Often Always

Development, production, control or calibration of reagents, medicine, vaccines and/or diagnostics 25 8 8 3 8

Toxicological research 34 9 1 5 3

Diagnostics 39 10 1 0 2

Education and training 31 18 2 1 0

Fundamental research 4 7 8 12 21


31

1.7 How many years of experience in animal experimentation do you have?

Years of experience Respondents

0-2 years 9

3-5 years 15

6-10 years 9

11-15 years 6

> 16 years 13

1.8 Did you participate in the course on laboratory animal science (Required by Dutch law) and if so in which year?

LAS course Respondents

No 1

No, I obtained exemption by the ministry in 1986 9

Yes, year: 42 (Years are excluded for privacy reasons)

Part 2: View on the 3R principles

2.1 Are you acquainted with the 3Rs?

Yes No

52 0

2.2 Can you give a description of your perception of the 3Rs?

All respondents gave a more or less similar description of the 3Rs which was in compliance with the definition by Russell and Burch.

2.3 To what extend do you (dis)agree with following statements?

Fully disagree Disagree Neutral Agree Fully

agree

3R implementation is important for animal welfare 0 1 0 9 42

Existing 3R possibilities are optimally applied 1 7 27 16 1

3R implementation is of benefit to the animal, not to the researcher 9 32 4 6 1

Optimal implementation of the 3Rs is important in my job 1 2 9 29 11

3R implementation will lead to higher appreciation by journals 7 11 32 1 1

3R implementation will increase research costs 3 21 14 14 0

3R implementation needs to be rejected because of the necessity to compare results with earlier findings 18 19 15 0 0

3R possibilities often remain unused 1 7 15 25 4

The obligation for 3R implementation increases bureau-cracy 5 16 11 18 2

Chapter 2

32

Better animal welfare leads to better experimental results 0 3 15 15 19

Finding the 3Rs is simple 7 18 14 13 0

3R implementation is easy 6 17 18 10 1

Application of the 3Rs slows down innovation 16 22 11 2 1

Part 3: Information sources

3.1 To what extent did the following information sources contribute to your 3R-knowledge in general?

Not applicable

Low1 2 3 4

High5

Education and training 6 11 7 12 12 4

Laboratory Animal Science course (Felasa cat. C course) 8 3 0 4 23 14

My own experience with conducting animal experiments 0 1 2 9 28 12

The experience with 3R possibilities within my research team 2 6 7 16 17 4

My own experience with research application 2 4 4 12 22 8

Conferences/workshops/symposia 8 14 14 9 6 1

Updating oneself with 3R literature 9 16 12 9 5 1

By conducting 3R research 14 14 6 8 7 3

Through the network within my own field of research 3 7 7 19 15 1

By consulting contacts in the laboratory animal science network:- AEC members

6 12 9 10 12 3

- AWO 1 2 5 12 23 9

- Researchers 2 3 5 17 23 2

- Animal care staff and technicians 2 6 6 10 21 7

3.2 In case you are working on a specific research topic: which sources provide you with the necessary information on 3R possibilities? Can you please specify for each source to what extent is has been helpful in the process of optimal implementation of 3R possibilities in your research?

Not applicable

Low1 2 3 4

High5

Information from my own previous work 8 4 3 5 25 7

Previous work by my own research group 2 4 1 13 26 6

Consulting: ...AEC members

7 9 13 12 9 2

...AWOs 2 1 4 16 20 9

...other researchers (colleagues) 1 2 5 8 28 8

...Animal care staff and technicians 2 6 8 10 18 8

...bio statistician 6 9 16 9 9 3


33

...Replacement expert 15 13 11 8 4 1

A search in scientific databases, search engines, 3R databases or websites

11 10 12 5 11 3

Outsourcing a literature search 37 8 2 1 3 1

Consultation within the own network of professional asso-ciations (not specified)

41 6 3 2 0 0

Use of online internet forums. (not specified) 39 5 5 3 0 0

Feedback from the AEC 8 7 8 4 19 6

Other. (not specified) 45 5 1 1 0 0

3.2.1 You have indicated that you sometimes outsource a literature search. To whom do you outsource? (N=4)

Information specialists 1

A specialized search service 1

Student assistants /students 6

PhD students 3

Other, namely: (I don’t outsource (2)/Do it myself, I misin-terpreted the question /not applicable) 4

3.3 There are numerous databases, websites and search engines for a 3R search. Can you indicate in the list below which ones you know?

Pubmed 50

Google 45

Web of Science 31

Embase 11

Agricola 8

TOXNET 8

NCA 8

NC3R 4

Altweb 3

ZEBET 2

Other, namely. Scopus (2) 2

AWIC 1

Norinia 1

FRAME 1

GO3R 1

UCCAAI 0

AVAR 0

Altbib 0

Chapter 2

34

3.3.1 In case you use these databases, websites and search engines for your 3R search, please indicate for each database the intensity of consultation.

Never1 2 3 4

Always5

PubMed 3 2 4 9 21

Embase 2 2 4 2

Web of Science 6 8 4 4 1

Google 5 3 8 13 6

Agricola 5 1 1

AWIC 1

Norina 1

UCCAAI

AVAR

TOXNET 2 3 1 1

Altbib

Altweb 2 1

NCA 3 2 1 1

FRAME 1

GO3R 1

NC3R 3 1

ZEBET 1 1

Other, namely. Scopus 1 1

3.3.2 How do you evaluate your own 3R search skills to search in these databases and websites?

Very insufficient1 2 3 4

Very sufficient5

3 2 13 13 10

3.4 Is there within your organization a form of 3R support?

Respondents

No 15

Yes, other, namely (specified below) 15

Yes, a replacement expert 9

Yes, a 3R research department 8

Yes, an information specialist for a literature search 6

Other, namely. “an AWO” 7

Yes, meetings for exchanging 3R knowledge 5

Yes, a service centre for 3R advice 4

Other, namely. “The AEC, Central service for laboratory animals or AEC members” 5

Other, namely. “The laboratory animal science department” 3

Other, namely. “A platform for alternatives within the organization” 1

Other, namely. “an expert within the field” 1

Other, namely. “I don’t know” 1


35

3.4.1 Do you, and if yes to what extent, use this support? (N=38)

Never Sometimes Regularly Often Always

13 11 6 5 3

3.5 To what extent does your professional environment stimulate you to implement 3R possibilities in your research?

Not1 2 3 4

Very much5

1 8 20 15 8

3.5.1 Can you elaborate on your previous answer?

Previous answer Comment

5 3R Research is (part of) my job/interest/experienceProfessional context/merges with other interests:- fewer/better animal studies result in faster/cheaper researchfocus of Animal Ethics Committee (AEC)3Rs are of interest/focus point of the client or organization

4 Optimization of research and reduction of costsInput on experiments by: AEC, AWO animal care staff and technicians.(not so much by my supervisor, more interested in the results)Implementation 3Rs merges with other interests e.g. new research opportunitiesFocus of the organization/AEC/animal care staff and techniciansOwn interest/experience

3 Dutch magazine for applied laboratory animal science (Biotechniek)Research question is the main focusHave to because of the AECIts more expected then stimulated, when stimulation is defined as positive motivationMost of our research are standard procedures and thus already optimized, otherwise consult the AWOIn the veterinary context: we need to learn more from our daily practice next to experimental situations, needs more attention from editors of veterinary journalsSome research is mainly guided by regulatory agencies. This makes the implementation of 3Rs not possible and a search thus un-use full.Focus of the research department/AECOwn interest/elaboration with colleges

2 No stimulation within our group, it is more seen as a burdenI am the most experienced worker

1 Not part of a group, an thus not directly simulated. However do collaborate and elaborate with my supervisor, other groups, the AWO and AEC.

3.6 Evaluation of information sources

Very

dissatisfied Dissatisfied Neutral Satisfied Very satisfied

Are you satisfied with the availability of information on replacement, reduction and refinement? 0 14 27 10 1

Are you satisfied with the accessibility of this infor-mation? 2 14 25 10 1

How satisfied are you with the effort needed seen in relation to the output of a 3R search? 3 17 23 8 1

Chapter 2

36

3.7 Space for additional comments on your experience with accumulating 3R knowledge.

Selection of the most relevant comments:- Refinement comes mainly from the Animal Welfare Officer, whereas Replacement and Reduction arise from the own

research group and own experience- This depends on what area of research you are working in- I do not use information from others in this area- Except from feedback from persons, I never use information sources on the 3Rs, and did not know they existed- It has no priority- For my type of studies I cannot use e.g. computer models as Replacement, so here I do not need extra knowledge.

For Reduction more knowledge would be advisable, but I have too little experience. The Refinement I apply is based on experience, and not on information sources on the 3Rs

- My information comes from AEC members, animal technicians and animal welfare officers and experienced researchers.- The culture at my department reflects an attitude that experience in itself suffices. But, as a young researcher I feel partly

responsible for new 3R developments. So I have tried to do 3R searches, however, it appears to be very complex to find specific 3R information. Refinement can always be applied, but Replacement and Reduction is more difficult.

- It is difficult to find a good 3R method for the purpose of a study, certainly from literature. Mostly this process works better through discussions with researchers in another field or AEC members.

- Within my department the 3R efforts arise from discussions with colleagues on Refinement and Reduction. There is a lack in (accessibility to) good 3R sites, and especially a lack on how 3Rs can be applied within a specific field of research. The AEC gives good feedback, which leads to discussion on the planned execution of studies.

Part 4: 3R knowledge in practice

4.1 In which phases of research are you involved? And can you specify to which of the Rs you contribute in each phase?

Phase Replacement Reduction Refinement

Funding application 27 17 24 13

AEC application 46 24 44 40

Study protocol 48 7 26 39

Practical execution 43 6 20 36

Writing report /publication 49 6 12 14

Other moment, namely:- monitoring of the experimental

part of the test- autopsy on dropouts

2 0 0 2

4.2 How much time do you spend on a 3R search for each AEC application?

0 h 4

1 h 27

2 h 7

3 h 2

4 h 2

5 h 1

6 h 1

Other namely. (16 h /72 h) 2


37

4.3 Do you outsource the search for 3R possibilities? Please specify.

No 43

Yes, to- AWO- knowledge group- experienced researchers or self

3

4.4 Who reviews your application for 3R possibilities? (open question)

AEC 20 respondents

AWO 19

Colleague 8

Scientific committee/staff/supervisor 8

Nobody 5

Animal-care staff/technician 3

4.5 How do you experience the AEC’s advice on your planned animal experiment?

Never Rarely Sometimes Often Always

Help to improve quality of research 7 6 22 9 2

Help to improve animal welfare 1 7 15 18 5

Necessary for AEC approval 2 4 8 12 20

Interference 8 17 18 3 0

Necessary bureaucracy 7 6 15 15 3

Unnecessary bureaucracy 13 13 15 5 0

4.6 Do you conduct a specific literature search, for 3R possibilities in your research?

Yes No

11 41

Chapter 2

38

4.6.1 What is the reason for not executing a 3R literature search?

I don’t think this will reveal new information. 12

I have never thought about it. 7

It is not part of my project. 3

I don’t know how it will improve my research. 2

I don’t know how to search for the 3Rs. 2

It has no priority. 2

It will delay my research. 1

Other namely:- it is not useful (1 respondent), - there are no 3R possibilities for the type of studies I am doing (1), - I already work on the 3Rs in a different way (5 respon-

dents: a pilot study; 3R study is part of the entire literature study; in the project proposal a clear description of which part of the 3Rs that is examined is presented; 3Rs are an integral part of the research, so all aspects are always included; I rely on experience of colleagues that have already evaluated the 3R possibilities)

- We use standardised studies and protocols for which the-re are no 3R possibilities, or which make a new literature study before every study superfluous (3)

- It is not my own research, I am merely a spectator (1)- Required expertise for us is very specific (1)

12

4.6.1.1 Do you think this is useful? Yes (N=16) Comments:

- It could lead to new information, ideas and developments (5 respondents), - New information may lead to an improved protocol and/or better animal welfare (3)- Not tried it before (1)- In case an animal study is not necessary then you should not do it (1), - In case I encounter problems I give feedback and try to indicate alternatives (1), - Particularly want to learn more on Reduction (experimental design) (1)

4.6.1.2 Do you think this is useful? No (N=25) Comments:

- It takes too much time- I don’t think this will lead to new information (4 respondents)- There are no good alternatives for our type of research- A pilot study serves this purpose - Our group has a large 3R experience already- When producing anti sera Reduction is not relevant, Replacement not possible and animal caretakers practice Refinement- I know the 3R possibilities and get advice from a statistician- Study design is dictated by legal requirements (2x)- Some experiments cannot be performed differently (2x)- Required expertise for our studies is very specific- Once yearly review of toxicity study setup seems sufficient to me- In our project proposal it is clearly described which part of the 3Rs is being examined- I always do a power analysis on the most important outcome- I already include this in the study design- I get the information in a different way- For my research there is no alternative and I already use the minimum nr of animals per group- The study design is fixed at the moment of the AEC application


39

4.6.2 In which research phase do you conduct the 3R literature search? And what is the time length of your search?

Respondents Time (Hours)

Funding application 5 1, 8, 4, 12, 1

AEC application 10 120, 3, 1, 2, 2, 3, 1, 1, 1, 1

Study protocol 4 120, 2, 1, 1

Practical execution 2 40, 1,

Writing report /publication 3 24, 8, 1

Other moment, namely: Starting phase of a project 1 20

4.6.3 As a researcher, what are your practical possibilities (in time, budget, skills, knowledge) to find ways to optimize 3R implementation in research?

None Very limited Limited High

Replacement 7 18 15 7

Reduction 1 6 28 8

Refinement 1 7 18 14

4.6.3.1 Space for additional comments:

Reduction with my own data the others from literature

I have no limitations

3R search goes hand in hand with cost/effectiveness research

Essential part of my work.

I don’t do my own research, however I do elaborate with researchers and animal care staff and technicians

Replacement is in my field of research not an option

Right statistics can reduce the number of animals

Refinement can be difficult because in some situations it requires more animals to reduce the amount of suffering per

animal. More animals vs. less suffering related to the purpose of the experiment is a difficult balance.

We use an animal-PET scanner and thus is replacement not an option.

Refinement and Reduction is discussed during the writing and design process of the study

4.7 What is your estimation of the potential for implementing 3R possibilities in case time and budget are unlimited?

None Very low Low High Very high

Replacement 11 17 12 7 5

Reduction 5 13 15 13 6

Refinement 4 3 16 17 12

Chapter 2

40

4.7.1 Space for additional comments:

You can never be sure if authors from other studies have the same quality or moral standards as you have. How will the experiment change if you change something in the set-up? What have the authors of the other studies possibly left out?

I have done everything possible, more time or money would not have resulted in more 3R implementation

It is hard to say how field research can effectively replace experimental animal-based research

Replacement and Reduction are not an option in my field of research. Refinement could be an option e.g. developing an animal model with reduced suffering

No own research/experience

Implementation of more alternatives needs to by established by modification of legislation not by optimizing methodology

Replacement is not an option in my field of research. Reduction can be improved by collaboration with other research groups and consulting experienced animal technicians. Refinement search for optimal dose for post operative analgesia

4.8 Does your AEC (also) advice on legally required animal experiments?

Yes No

29 23

4.8.1 To what extent do you think legislative demands (e.g. European Pharmacopoeia) inhibits 3R implementation in animal experimentation? Legislative requirements make it...

Very difficult1 2 3 4

Absolutely not difficult5

4 6 15 1 3

4.8.2 Despite these regulatory demands, to what extent does your license holder stimulate you to fully implement 3R possibilities in your animal experiments?

Not at all1 2 3 4

Very stimulating5

1 5 9 12 2

4.9 Do you include the 3R search in your research budget?

Yes No

1 (not specified by respondent)

51

4.10 How often do the following situations occur?

Never Sometimes Regularly Often Always Don’t know

How often do you think that 3R information is missed and thus not applied? 5 16 17 4 1 9

Do you apply all the known 3R possibilities in your research? 1 2 6 28 14 1


41

4.11 When known replacement, reduction or refinement methods were not fully applied, what was the cause for this?

This is not an issue/does not happen 17 (respondents)

Practical or Scientific reasons 15

Time/money (resources) 15

Legislation 7

Carelessness/unfamiliarity 5

4.12 In case optimal implementation of the 3Rs in your research has been applied, what has made this possible?

Good support and collaboration 15 (respondents)

Own motivation and experience 15

Scientific of practical considerations 8

Time and money 3

4.13 Room for additional comments to the use of 3R knowledge in practice:

Balance between science and practice can be difficult. E.g. the need to reproduce the animal experiment for validation and publication, but not receiving the permission of an AEC to duplicate experiments.

This survey is extremely focused on clinical (human) and fundamental research. Hard to translate some questions to the agricultural or veterinary setting.

Research can dependent on many factors such as the willingness of the animals to co-operate in training

I believe that the implementation of the 3Rs depends on motivation and creativity of the researcher and his attitude/respect towards the lab. animal.

It is suggested that many 3R possibilities are already available, however in practice this is not the reality. There is a lot of 3R research being conducted, the pace however is low. 3R implementations can interfere with the research and thus interfere with answering the research question. The theory is good, however in practice not always feasible.

An issue that I am currently struggling with: Previous research has indicated that more post operative analgesia is needed, however this introduces variation in the new test groups compared to the previous groups. Still I think and feel the need that the extra analgesia should be admitted and that this will result in refinement of the research.

I think that there is much 3R info but that this is hard to reach and thus not used. Additionally legislation is to strict on older (non-3R) methods.

I think that researchers need to be informed where to find information. This makes it easier to evaluate what has been done and what is available within a certain field of research. Prior to writing a new application.

Chapter 2

42

Part 5: Wishes

5.1 Suppose you could start a new project for gaining a more optimal use of existing 3R knowl-edge, what would be your focus points?

More openness between organizations about animal experimental practices. 26

Better and accessible information systems and databases with 3R literature. 19

Support at a level of a research department (3R research and development). 19

Well facilitated 3R knowledge exchange between individuals within and between organisations. 17

Courses to refresh and update 3R knowledge. 13

Systematic reviews of excising literature 10

External expert on alternatives that can be consulted, just like a bio-statistician. 10

A 3R literature search service for you specific research. 8

More focus on 3Rs in education 7

Funding – Providing budget for conducting a literature search by the researcher. 7

Encourage 3R assessment before funding applications. 6

Other, namely:(A) funding for 3R research and development (B) quicker validation and acceptance of replacing in-vitro methods by regulatory bodies (C) professional 3V support during planning phase (D) periodically refreshment/update courses(E) More lenient legislation

5

5.2 Please prioritize the following statements from High priority (4 points) to Low priority (1 point)

Priority high low

4 3 2 1 Score

More openness between organizations about animal experimental practices. 14 4 5 3 81

Support at a level of a research department (3R research and development). 8 5 4 2 57

Better and accessible information systems and databases with 3R literature. 5 6 8 0 54

Well facilitated 3R knowledge exchange between individuals within and between organisations. 4 9 2 2 49

Systematic reviews of existing literature. 4 4 2 0 32

Courses to refresh and update 3R knowledge. 0 3 8 2 27

External expert on alternatives that can be consulted just like a bio-statistician. 2 4 3 1 27

A 3R literature search service for you specific research. 3 2 1 2 22

More focus on 3Rs in education 0 6 0 1 19

Funding – Providing budget for conducting a literature search by the researcher. 3 0 2 2 18

Encourage 3R assessment before funding applications. 2 0 2 2 14

Other, namely: 2(B/E)

1(A)

1(D)

1 (C)

14


43

Part 6: Evaluation of this questionnaire

6.1 Room for additional comments (any comments on this survey or important topics you have missed):

Selection of comments for privacy reasons:- A lot depends on the discussion what information is needed before we can go to human trials. - Although I am in favour of openness, the increasing violence of activists leads to the opposite.- New multimedia, such as the Journal of Visualised experiments or even YouTube could contribute to the 3Rs to demon-

strate practical (im)possibilities of new research techniques and make this easily accessible to a large audience. - This questionnaire has raised my 3R awareness again. I find it important to increase 3R knowledge and that this deserves

attention. This research and questionnaire have certainly contributed.- It is my experience that there is a focus on the 3Rs in the FELASA category C course and AEC applications. However,

besides this, there is not a lot of attention for the 3Rs. I have never had questions from a journal on the execution of the animal experiments. As long as a local AEC has approved it, then that is ok. I have never heard of funding bodies asking about the planned execution of animal experiments. It would be good to inform researchers where to find 3R information, also in FELASA category C courses. Publication of 3R experience should be stimulated.

6.2 Would you like to be informed about the outcome of this survey?

Yes No

36 16

6.3 Can we contact you for additional information/comments?

Yes No

32 20

6.4 Would you be willing to participate in the follow-up study? If yes, can we contact you?

Yes No

20 32

CHAPTER 3Assessing the application of the 3Rs: A survey among animal welfare offi cers in the NetherlandsAssessing the application of the 3Rs: A survey among animal welfare offi cers in the Netherlands

Judith van LuijkYvonne CuijpersLilian van der VaartTineke Coenen de RooMarlies LeenaarsMerel Ritskes-Hoitinga

Laboratory Animals 47(3) 210-219, 2013

Chapter 3

46

Abstract

Implementation of the 3Rs in animal studies is a legal requirement in many countries. In the Netherlands, Animal Welfare Officers (AWOs) are appointed to monitor the welfare of laboratory animals. As part of this task, AWOs give advice to researchers and can therefore have an influential role in implementing 3Rs methods in research. A national survey was conducted to gain more insight into how Dutch AWOs obtain and apply 3Rs information in their daily work. Nearly half of the AWO population filled out the questionnaire (15/32; a response rate of 46.9%). Two-thirds of the respondents pointed out that finding 3Rs information is not an easy task and more than half of the respondents believed that information on possibilities to implement the 3Rs is regularly being missed. Respondents indicated that most 3Rs information is obtained from direct colleagues and other AWOs. Special online 3Rs databases are rarely used. All responding AWOs feel that they contribute to Refinement (15/15), nearly one-third of the respondents feel they contribute to Reduction (4/15), and one AWO feels he/she contributes to Replacement (1/15). According to the respondents, a better exchange of knowledge can contribute to a more successful 3Rs implementation. How this knowledge exchange can best be established and facilitated needs further exploration. To that end, the authors make suggestions for a 3Rs-integrated evidence-based approach.

Introduction

As in many other countries, Dutch law only permits animal experiments if available 3Rs methods have been considered and, if feasible, implemented. This means that, if possible, experiments have to be performed without animals (Replacement), with fewer animals (Reduction) and/or with less pain/distress for the animals (Refinement) [1]. Nowadays, also improved welfare, for example through cage enrichment, is considered part of refinement. Information about and expert knowledge of the 3Rs principles are necessary for effectively applying these principles in research. To facilitate the retrieval of information regarding the possibilities to implement the 3Rs in a specific research field/study , a lot of effort has been put into the development of specific 3Rs databases [2, 3] and of guidelines on how to search for 3Rs information [4-7]. An earlier survey by Leenaars et al. revealed that, despite all these developments, there is still much room for improvement in the way scientists currently retrieve information about the 3Rs (from databases) [8]. From this survey, it was concluded that searching for the 3Rs is not considered an integral part of the research process (and thus not funded), knowledge of 3Rs databases is minimal, and that search skills in general are limited.

3R suRvey among aWos

47

Scientists, however, are not the only persons playing a role in implementing the 3Rs. In the Netherlands, each licence holder (a legal or natural person possessing a licence to conduct animal experiments at their institution) has to appoint an Animal Welfare Officer (AWO). The task of this officer is to monitor the welfare of laboratory animals before, during and after experiments (Dutch Experiments on Animals Act, in Dutch; Wet op de Dierproeven, Article 14) [9, 10]. In general, the role of an AWO is comparable with the work of a FELASA category D officer [11]. The AWOs have a pivotal role in ensuring the proper conduct of animal experiments. They are in direct contact with scientists designing animal studies as well as with the animal care staff and technicians, who actually handle the animals and carry out the biotechnical procedures (such as drug administration or operations). The AWOs are required by law to give their advice about laboratory animal science-related topics in all research protocols that are assessed by the Animal Ethics Committees, including advice on implementation of 3Rs information. The AWOs can therefore play a crucial role in influencing the quality of the design and conduct of animal experiments as well as in safeguarding the implementation of the 3Rs. At present, we do not know how AWOs gain their knowledge on 3Rs possibilities, how this knowledge is implemented and whether obstacles and/or possibilities exist for improvement of 3Rsimplementation. In order to answer these questions, a survey was designed and sent out to all AWOs in the Netherlands. Similar surveys were sent out to scientists who are involved in animal-based research [12] and members of Animal Ethics Committees (unpublished observations).

Materials and methods

From April to June 2009, a national survey was conducted among AWOs in the Netherlands, in order to study their views on the use and implementation of 3Rs knowledge. An online questionnaire was developed and distributed among all Dutch AWOs.

Questionnaire design The outline of the questionnaire was developed by the second author (YC) and was based on: 1) a previously conducted survey among researchers involved in animal-based research at the Radboud University Nijmegen Medical Centre [8] and 2) in-depth semi-structured interviews with five researchers, two Animal Ethics Committee (AEC) members and one AWO. The survey was descriptive in nature and included both qualitative and quantitative questions. Three AWOs, a communication expert and a knowledge management specialist assisted in optimising the questionnaire. The questionnaire was tested by three AWOs and adjusted according to their feedback.

Chapter 3

48

The use of closed-ended questions ensured that respondents were consistent in their answers. There was room to give additional comments to questions, in case a respondent would not consider the provided set of answers exhaustive. Some questions allowed for multiple answers, e.g. on information sources. The language of the original questionnaire was Dutch; an English translation of the complete questionnaire can be requested from the authors.

Questionnaire distributionA link to the online questionnaire was distributed among all AWOs in the Netherlands, with the help of the AWO-group of the Dutch organisation for Laboratory Animal Science (Nederlandse Vereniging voor Proefdierkunde; NVP). The Dutch inspectorate sent out a letter to all licence holders in the Netherlands, asking them to stimulate participation in this survey within their institutes.

Data analysesAll answers given by the respondents were in Dutch. Despite the efforts of the authors, some misinterpretations and/or small translation errors cannot be ruled out. To safeguard anonymity and to exclude potential bias, the survey data were disconnected from the respondents’ background and contact details. The results were analysed per question. The closed-ended multiple answer questions, the yes/no questions and the questions with scaled answers were analysed through counting frequencies in Excel. The answers to open questions were listed and categorised by inductive analysis. The data were analysed by the first (JvL) and second author (YC).

Results

ResponseAt the time of the survey, the Dutch professional association of AWOs consisted of 42 members, of whom 32 were actually appointed as an AWO by a licence holder. Fifteen AWOs filled out the questionnaire (response rate 15/32 = 46.9%). The affiliations of the responding AWOs were as follows: universities (6/15), knowledge institutes (3/15), industry (3/15), Contract Research Organisations (CRO) (2/15) and the government (1/15).

Views on the 3Rs principlesQuestion 2.1, table 1. A large majority of the respondents agreed with the statements ‘3Rs implementation is important for animal welfare’, ‘Optimal implementation of the 3Rs is important in my job’ and ‘Better animal welfare leads to better experimental data’. A vast majority disagreed with the statements that ‘the 3Rs benefit animal welfare, but not the researchers’, ‘3Rs implementation needs to be rejected because


49

of the necessity to compare results with earlier findings’ and that ‘application of the 3Rs will slow down innovation’. Neutral responses or a wider diversity of opinion were found with respect to the other statements: ‘Existing 3Rs possibilities are optimally applied’, ‘3Rs implementation will lead to higher appreciation by journals’, ‘3Rs implementation will increase research costs’, ‘3Rs possibilities often remain unused’, ‘The obligation of 3Rs implementation increases bureaucracy’, ‘Finding information on 3Rs methods is simple’ and ‘Implementation of 3Rs methods is easy’.

General 3R information sources Question 3.1, table 2. The information sources considered to contribute the most to general 3Rsknowledge are ‘own experience as an AWO’, ‘consulting other AWOs’, ‘the postgraduate training to become an AWO’, ‘conferences, workshops and symposia’ and ‘consulting animal care staff and technicians’. Large differences of opinion are seen among AWOs concerning the contribution of their ‘academic education’, of ‘consulting AEC members’ and of ‘conducting 3Rs Research’. Information sources evaluated as low contributors are: ‘consulting researchers’ and ‘updating oneself with literature on the 3Rs’.

Information sources for 3Rs information requests Question 3.2, table 3. ‘Own knowledge and experience’, ‘Other AWOs’, ‘Consulting other members of the Dutch AWO organisation’ and ‘Animal care staff and technicians’ are the most frequently consulted information sources when AWOs are requested to provide 3Rs information to scientists. ‘Searching in scientific and/or 3Rs databases’, ‘AEC members’, ‘Outsourcing a literature search’ and ‘Consulting within an organisation or online forum’ are the least consulted information sources or are considered not relevant (chosen answer: ‘not applicable’) according to the majority of the respondents. The vast majority of the respondents answered ’indifferent’ to the option: ‘Consulting researchers’.

Acquaintance with and use of databases for 3Rs searchQuestion 3.3.Participants were asked which databases, websites and search engines for finding information on 3Rs methods they were familiar with. The best known databases, websites or search engines were: PubMed [13] by 15/15, Google [14] by 14/15, NCA [15] by 12/15, NC3Rs [16] by 10/15, Agricola [17] and FRAME [18] by 8/15, Altweb [19] by 7/15 and ZEBET [20] by 5/15. Web of Science [21], AWIC [22] and Norina [23] were known by 4 of the 15 respondents and 3 of the 15 respondents were familiar with Embase [24] and Toxnet [25]. Two respondents were familiar with GO3R [26] and one respondent knew the website Altbib [27]. None of the respondents indicated to be familiar with UCCAAI [28] or AVAR [29]. Three respondents used the option to add extra online 3Rs information sources. The sources they added were: CompMed [30], Laboratory Animals [31] and FELASA [32].

Chapter 3

50

Question 3.4. When respondents were asked which databases, websites and search engines they most frequently used to find relevant 3Rs literature (score 4 or 5, where 5 is ‘very often’), the majority answered: PubMed (12/15) and Google (10/15).Question 3.5. Nearly half (7/15) of the respondents considered their own skills to search for information on 3Rs methods in online databases, websites and search engines to be insufficient. Another group of 7 respondents answered ‘indifferent’ to the question about the sufficiency of their search skills. Only one respondent believed to have sufficient search skills for retrieving information on relevant 3Rs methods.

Evaluation of online 3Rs information sourcesQuestion 3.8, 3.9 and 3.10. Forty percent (6/15) of the AWOs were dissatisfied (score 4 ‘much’ or 5 ’very much‘) with the availability of 3Rs information and another 6/15 answered ‘neutral’ (score 3). A small majority (8/15) was dissatisfied (score ‘much’ or ‘very much’) with the accessibility of information and 5/15 answered ‘neutral’ (score 3). Nearly half (7/15) of the respondents were dissatisfied (score 4 or 5) with the balance between search effort required and retrieved 3Rsinformation.

Experience with 3Rs advice Question 4.1. As the answers presented in table 4 show, responding AWOs mainly advise researchers on Refinement methods: 8/15 always give advice on humane endpoints and 7/15 always give advice on anaesthesia/analgesia. Least advice is given on Replacement methods: 9/15 never have advised on the use of computer simulations and only 7/15 sometimes advise on the use of human biomaterial.Preferences and possibilities regarding giving advice to the AEC Question 4.4, table 5. For most given topics, the scores for whether AWOs would like to give more advice to the AEC on that particular topic and for whether this is already possible in the current practice were largely similar. However, there was one exception: ‘The advice to the AEC on the assessment of the effort put into searching, finding and implementing 3Rs as demonstrated by researchers’. Of the respondents, 12/15 claimed that they would like to advise the AEC more on this topic, while 4/15 of the respondents indicate this is currently already possible.

AWOs’ influence on the 3RsQuestion 4.10, figure 1. All 15 respondents consider their influence on the implementation of Refinement to be ‘high’. The influence on the implementation of Reduction is considered ‘high’ by 4/15 respondents, ‘medium’ by 7/15 and ‘low’ by 4/15 respondents. Influence on Replacement is considered ‘low’ by 11/15 respondents and ‘medium’ by 3/15 respondents. One respondent indicated that his/her influence on Replacement is ‘high’.


51

Factors inhibiting implementation of 3Rs methods Question 4.12. The majority of the respondents (8/15) answered that information on 3Rs methods is ‘regularly’ missed in an information search. Six respondents think that this information is missed ‘often to always’, and one respondent thinks that this information is missed ‘sometimes’. Question 4.13. The frequency of not implementing potentially suitable 3Rs methods is: ‘sometimes’ according to 4/15 respondents, ‘regularly’ according to 7 and ‘often’ according to 3 respondents.Question 4.13. AWOs were asked to elaborate on the possible reasons for missing information on 3Rs possibilities. They were not specifically questioned about their own role in this matter nor about the role of the researcher or the AEC member. Frequently mentioned reasons for missing relevant 3Rs information were: ‘unaware of or unfamiliar with the possibilities’, ‘lack of knowledge on how and where to search’, and ‘lack of interest or priority’.Question 4.14. A similar question followed concerning the possible reasons for not implementing known 3Rs possibilities. Frequently mentioned reasons were: ‘lack of time, resources and knowledge on how and where to search for 3Rs information’, ‘the necessity to compare results with earlier findings’, ‘difficulties with “prescriptive legislation” (legally required animal testing)’ and/or ‘difficulties with accessibility of 3Rs information’.

Stimulating factors for 3Rs implementation according to AWOsQuestion 4.15. AWOs were asked to elaborate on what they regard as stimulating factors for successful implementation of 3Rs methods. Eight of the 15 respondents mentioned in their answer ‘the positive attitude/willingness of the researcher towards the 3Rs’ as a stimulating factor. Other frequently given answers were: ‘advice from AWOs and AECs’ (5/15), ‘good cooperation and preparation’ (4/15) and ‘enthusiastic and motivated animal care staff and technicians’ (3/15) . Also ‘support from management’ (2/15) and ‘sufficient time and experience’ (2/15) were mentioned as stimulating factors.Question 4.16. Respondents were given the opportunity to give additional comments on their personal experience with 3Rs information in practice. Some of the individual comments were: ‘Researchers should know the added value of implementing the 3Rs principles’, ‘More publicity and exchange of information is needed’, ‘The researcher is responsible for the implementation of the 3Rs principles’, ‘There is insufficient knowledge about experimental design among researchers, AWOs, AEC members, editors and referees; this needs to be improved by training and supervision/quality control’.

Ways to facilitate the optimal use of current knowledge on the 3RsQuestion 5.1, table 6. The four most selected items that were believed to contribute to a better implementation of 3Rs methods were: ‘Support at the level of a research department’ (10/15), ‘A 3Rs literature search service for your specific research’ (9/15) ‘Well facilitated 3Rs knowledge exchange among individuals, both within and between

Chapter 3

52

organisations’ (8/15), and ‘Better accessible information systems and databases with 3Rs literature’(5/15). Six individual respondents selected the option ‘other’ and added the following comments: “Each organisation should appoint an expert on alternatives, at the same regulatory and organisational level as an AWO”, “Funding of small projects without bureaucracy”, “The first question on the research plan (AEC form) should be: Why an animal experiment? What did you do and which sources did you consult to optimally apply the 3Rs in your animal experiment?”, “ONE information system, not 17!!”, “One national research centre, funded by users, that executes literature studies as well as practical 3Rs research and development (data sharing, surveys, common goals and joining forces, advice, publications)” and “Information exchange to encourage sharing”.

Data not presentedAnswers to questions 3.6, 3.7, 3.11, 4.2, 4.3, 4,5, 4.9 and 4.11 are not presented for one or more of the following reasons: the answers to these question were too ambiguous for meaningful interpretation, the question did go into too much detail for the scope of this manuscript, and /or the question is too specific for the Dutch situation. For these reasons, the answers to these questions were considered to be of less relevance for an international audience and are therefore not shown and discussed.

Discussion

In this survey, the views and perceived influence of Dutch AWOs on the implementation of the 3Rs have been investigated. The AWOs perceive the search for 3Rs information as a difficult task and acknowledge that 3Rs possibilities are sometimes missed and, as a consequence, not implemented in research. Given that the main task of an AWO is to monitor the welfare of the laboratory animals before, during and after experiments [9], it is not surprising that nearly all respondents agree that implementing the 3Rs is important in their job. In practice, they advise most frequently on Refinement matters and also consider their influence on the implementation of Refinement methods to be the highest, compared to Reduction and Replacement.According to the survey by van Luijk et al. (2011), scientific researchers consider AWOs and colleague researchers to be the most important sources for obtaining 3Rs information [12]. In line with this result, AWOs should acquaint themselves with 3Rsinformation sources and advise researchers on where to find relevant 3Rs information. Responding AWOs know more online 3Rs information sources than the responding AEC members and researchers. On average, AWOs were familiar with 6.9 sources, whereas AEC members were familiar with 3.7 sources (unpublished data) and researchers were familiar with an average of 3.4 online information sources [12]. Nevertheless, searching these


53

online sources remains a difficult task as there are over 100 different databases and it is almost impossible to perform an effective and adequate search in all of these [8]. This could imply that specifically searching for 3Rs information is not the most fruitful way for accumulating relevant 3Rs information [8, 12]. Strengthening personal communication between researchers, AWOs and other experts seems a better way to go, as this is already perceived as an important source for 3Rs information. This might be achieved by forming 3Rsexpert groups, which would include an AWO, a statistician and possibly a Replacement expert. However, relying solely on personal communication may introduce the risk of information remaining local or becoming outdated. Instead of collecting 3Rs information in separate databases or websites, it would be more useful to have this information incorporated in scientific papers. We would suggest to not address the 3Rs as a separate part per experiment, but incorporate them more as best practice in the broad endeavour to find answers to a research question. For example by conducting a comprehensive search as seen in the Systematic Review (SR) methodology. A comprehensive search is a thorough and transparent way of accumulating and all available relevant publications. De Vries et al conducted a comprehensive search in PubMed, EMBASE and 3R databases to produce an overview of the possibilities of using tissue-engineered constructs as a replacement of laboratory animals. Most relevant information was found in the PubMed and EMBASE search (238 primary articles) compared to the 3R databases search where 6 relevant primary studies were found that did not come up in the PubMed or EMBASE search[33]. One should note that the search strategy for 3R databases is more difficult to design as these databases usually do not have the option to search for thesaurus terms and are often less structured [8]. Collecting and combining all available evidence helps to make ethically and scientifically sound choices when designing a new line of animal-based research e.g. on the choice for the most appropriate animal model [34, 35]. Additionally, a more transparent search process can assist AWOs and AEC members in advising researchers, as it provides them with better insights into how and where researchers have searched for information. According to the answers to question 4.4 of this questionnaire, this type of insight is highly desirable, but hardly achievable in current practice. The conduct of a comprehensive search requires the participation of a team of experts such as a librarian or information specialist, a laboratory animal scientist and an expert in the field. The inclusion of these multiple fields of expertise can have a positive influence on the personal communication and thus implementation of 3R methods. The Systematic Review methodology is common practice in the field of clinical research [36, 37]. Even though animal studies often form the basis for clinical research, SRs of animal experiments are still very scarce [35, 38]. SRs of animal experiments need consideration as they have the potential to improve the scientific quality of animal experiments, to make decision-making (e.g. choice for animal model and study design) more transparent, to lead to reduction by preventing unnecessary duplication of animal experiments and to improve animal welfare [34, 39, 40].

Chapter 3

54

A weakness of this survey is that the main focus was on the 3Rs as a whole instead of on each individual R separately. A few respondents commented that they would have liked to specify their answers per R. Unfortunately this was sometimes not possible due to the design of the questionnaire and formulation of the questions. This may have lead to ambiguity in the answers and thus may have weakened the results. On the basis of the results of this questionnaire, it can be concluded that future surveys on the 3Rs should address Replacement separately from Reduction and Refinement.AWOs already make an important contribution to the implementation of Refinement methods in animal-based research. In order to enhance the quality of animal-based research and welfare of laboratory animals, other strategies, next to and in compliance with the 3Rs principles, need to be developed such as facilitating personal communication related to 3R methodologies and compressive search strategies for retrieving written 3R information.Without underestimating the value or the importance of the 3Rs, one could say that a specific 3Rs literature search may not be the most effective way to retrieve information for 3Rs implementation. Instead, combining multidisciplinary expert collaboration and synthesis of scientific evidence may be a more fruitful way forward and should therefore be considered and explored.

Acknowledgments

This project was funded by ZonMW ‘dierproeven begrensd II’ (project number 114000092), which is gratefully acknowledged. The authors would like to thank the respondents, the board of the organisation for AWOs in the Netherlands and Rita From of the Dutch inspectorate for their help in distributing and promoting this survey. Also many thanks to Dr. Rob de Vries, Caroline van Oostveen and Radboud in’to Languages for their assistance in revising the manuscript.


55




2. Hakkinen, P.J. and D.K. Green,

Alternatives to animal testing: Information

resources via the internet and world wide

web. Toxicology, 2002. 173(1-2): p. 3-11.

3. Smith, A.J. and T. Allen, The use of

databases, information centres and

guidelines when planning research that

may involve animals. Animal Welfare,

2005. 14(4): p. 347-359.

4. AWIC. Animal Welfare Information Center.

[cited 2011 September]; Available from:

http://awic.nal.usda.gov.

5. Bottril, K., Information: Needs for the

future. ATLA, 2002. 30 (Supplement 2): p.

145-149.

6. CCAC. Three Rs Search Guide by

the Canadian Council on Animal Care

(CCAC). [cited 2012 August]; Available

from: http://searchguide.ccac.ca/.

7. IMPI, Searching for 3Rs Information -

Published Literature Sources. 2002.





37(3): p. 297-303.

9. Tweede Kamer, Wet op de Dierproeven.

1977, De Staatssecretaris van

Volksgezondheid en Milieuhygiëne,

Hendriks.

10. EBRA. Animal experimentation legislations

in The Netherlands. [cited 2012 August];

Available from: http://www.ebra.org/

netherlands+1_16.htm.

11. Berge, E., et al., FELASA guidelines for

education of specialists in laboratory

animal science (Category D). Report of the

Federation of Laboratory Animal Science

Associations Working Group on Education

of Specialists (Category D) accepted by

the FELASA Board of Management. Lab

Anim, 1999. 33(1): p. 1-15.

12. van Luijk, J., et al., Assessing the search

for information on Three Rs methods,

and their subsequent implementation:

a national survey among scientists in

the Netherlands. Altern Lab Anim, 2011.

39(5): p. 429-47.

13. PubMed. [cited 2012 August]; Available

from: www.ncbi.nlm.nih.gov/pubmed/.

14. Google. Google Web Search and Google

Scholar. [cited 2012 August]; Available

from: http://www.google.com.

15. NCA. Nationaal Centrum Alternatieven

voor dierproeven. [cited 2012 August];

Available from: http://www.nca-nl.org/.

16. NC3Rs. National Centre for the

Replacement, Refinement and Reduction

of Animals in Research. [cited 2012

August]; Available from: http://www.

nc3rs.org.uk/.

17. AGRICOLA. AGRICOLA and Veterinary

Science Journals. [cited 2012 August];

Available from: http://usain.org/

AGRICOLA/vetscience.html.

18. FRAME. Fund for the Replacement of

Animals in Medical Experiments. [cited

2012 August]; Available from: http://www.

frame.org.uk/.

19. Altweb. Alternatives to Animal Testing.

[cited 2012 August]; Available from:

http://altweb.jhsph.edu/.

References

Chapter 3

56

20. ZEBET. database on alternatives to

animal experiments on the internet. [cited

2012 August]; Available from: http://www.

bfr.bund.de/en/zebet_database_on_

alternatives_to_animal_experiments_on_

the_internet__animalt_zebet_-1508.html.

21. Web of Science. Web of knowledge.


http://apps.webofknowledge.com/.

22. AWIC. Animal Welfare Information Centre.


http://awic.nal.usda.gov/.

23. Norina. A Norwegian Inventory of

Alternatives. [cited 2012 August];

Available from: http://oslovet.norecopa.

no/NORINA/.

24. EMBASE. [cited 2012 August]; Available

from: http://www.embase.com/.

25. TOXNET. Toxicology Data Network.


http://toxnet.nlm.nih.gov/.

26. Go3R. Semantic internet search enginge

for alternative methods to animal testing.


http://www.go3r.org/.

27. Altbib. Bibliography on Alternatives to

Animal Testing. [cited 2012 August];

Available from: http://toxnet.nlm.nih.gov/

altbib.html.

28. UCCAAI. UCDavis Centre for Animal

Alternatives Information. [cited 2012

August]; Available from: http://lib.ucdavis.

edu/dept/animalalternatives/.

29. AVAR. Alternatives in Education

Database. [cited 2012 August]; Available

from: Website no longer available.

30. CompMed. E-mail list by the American

Association for Laboratory Animal Science

(AALAS). [cited 2012 August]; Available

from: http://www.aalas.org/online_

resources/listserves.aspx#compmed.

31. Laboratory Animals. The Royal Society

of Medicine Press. [cited 2012 August];

Available from: http://la.rsmjournals.

com/.

32. FELASA. Federation of European

Laboratory Animal Science Associatons.


http://www.felasa.eu/.

33. De Vries, R., et al., Potential of tissue

engineering for developing alternatives to

animal experiments: a systematic review. J

Tissue Eng Regen Med, 2013. in press.

34. Hooijmans, C.R., et al., The effects of long-

term omega-3 fatty acid supplementation

on cognition and Alzheimer’s pathology

in animal models of Alzheimer’s disease:

a systematic review and meta-analysis. J

Alzheimers Dis, 2012. 28(1): p. 191-209.








167-182.




37. Cochrane Collaboration. The Cochrane

Collaboration. [cited 2012 September];

Working together to provide the best

evidence for health care]. Available from:

http://www.cochrane.org/.


57




p. 514-517.

39. Wever, K.E., et al., Ischemic

preconditioning in the animal kidney, a

systematic review and meta-analysis.

PLoS One, 2012. 7(2): p. e32296.

40. Sena, E.S., et al., Publication bias in

reports of animal stroke studies leads to

major overstatement of efficacy. PLoS

Biol, 2010. 8(3): p. e1000344.

Chapter 3

58

Table 1 General view on the 3Rs. Question 2.1: To what extent do you agree or disagree with following statements? (N=15)

Fully disagree Disagree

Neutral

Agree

Fullyagree

3Rs implementation is important for animal welfare 0 0 0 1 14

Existing 3Rs possibilities are optimally applied 1 4 6 4 0

3Rs implementation is of benefit to the animal, not to the researcher 11 3 0 1 0

Optimal implementation of the 3Rs is important in my job 0 1 0 5 9

3Rs implementation will lead to higher appreciation by journals 1 3 6 5 0

3Rs implementation will increase research costs 2 7 4 2 0

3Rs implementation needs to be rejected because of the necessity to compare results with earlier findings 9 5 1 0 0

3Rs possibilities often remain unused 1 2 3 9 0

The obligation for 3Rs implementation increases bureau-cracy 2 3 6 4 0

Better animal welfare leads to better experimental results 0 0 1 3 11

Finding information on 3Rs methods is simple 3 7 5 0 0

3Rs implementation is easy 2 6 7 0 0

Application of the 3Rs slows down innovation 8 4 3 0 0

Figure 1 Bar diagram of the level of perceived influence of AWOs on the implementation of the 3Rs.

High

Low

Medium

Refinement

15

10

5

0Replacement Reduction


59

Table 2 Information sources contributed to general 3Rs knowledge. Question 3.1: To what extent have the following information sources contributed to your general 3Rs knowledge?

NA*

Very Little

1 2 3 4

Very Much

5

Academic education 0 5 2 0 6 2

Postdoctoral education to become AWO 0 0 3 2 4 6

Own experience as an AWO 0 0 1 1 10 3

Conferences/workshops/symposia 0 0 1 5 7 2

Keep up with 3Rs literature 0 0 2 9 3 1

Own research on the 3Rs 2 3 3 3 1 3

Personal communication with...

...Researchers 0 1 1 8 4 1

...colleague AWOs 0 0 1 1 10 3

...Animal Ethics Committee members 0 1 4 3 3 4

...Animal technicians and care staff 0 0 3 4 4 4

* Not applicable

Table 3 What sources do you consult to get 3Rs information to help you formulate a specific advice? Survey question 3.2: In the following situation: When a researcher or an AEC member asks your advice on Replacement, Reduction or Refinement matters, what sources do you consult to get this information? Please specify how often you use these sources.

NA*

Very Little

1 2 3 4

Very Much

5

Own knowledge and experience 0 0 2 5 5 3

By consulting...

...Researchers 0 0 2 11 2 0

...Other AWOs 0 0 1 2 9 3

...Animal Ethics Committee members 0 3 5 5 2 0

...Animal technicians and care staff 0 0 3 4 7 1

...Other, namely... 7 3 2 1 1 1

By searching in scientific or 3Rs literature databases 0 4 4 5 1 1

By outsourcing a literature search 9 4 2 0 0 0

Consulting within the organisation for Dutch AWOs 0 3 1 2 8 1

Consulting within an organisation, namely... 7 3 3 1 1 0

Consulting within an online forum, namely... 7 4 1 2 1 0

Other, namely... 12 1 1 1 0 0

* Not applicable

Chapter 3

60

Table 4 AWOs advice on the 3Rs Question 4.1: How often do you advise researchers on the following topics:

Never Seldom Sometimes Often Always

Animal model

The choice whether a research question should be answered with an animal model

2 3 5 5 0

Choice for an specific animal model 1 2 6 6 0

Knowledge and information

Applicability of 3Rs methods from similar previously conducted research

0 4 7 2 2

Possible search activities to retrieve research-specific 3Rs methods

1 5 8 1 0

Pointing out relevant information sources on 3Rs methods 3 6 3 2 1

The possibility to consult others (specialists) 1 2 6 5 1

Replacement

Whether the use of human material is possible 4 3 7 0 1

Possible use of computer simulations 9 3 3 0 0

Reduction

Optimal use of in vitro techniques prior to an animal experiment 1 3 9 0 2

Optimal use/sharing of the experimental animal (e.g. practice chirurgical techniques post mortem)

0 0 8 7 0

If the correct statistical tests are used 0 1 6 6 2

Refinement

If the correct biotechnical procedures have been applied 0 0 3 6 6

If the correct analgesia / anaesthesia is administered 0 0 2 6 7

Correct use/definition of humane endpoints 0 0 0 7 8

Training of animals for better cooperation in the experiment 0 3 5 4 3

Use of cage enrichment 0 0 4 5 6


61

Table 5 Topics on which AWOs would like to advise the AEC more. Question 4.4: On which of the following topics of a research application would you like to advise the AEC more (desirable), and is this possible in the current practice?

Desirable Possible

Animal model

Substantiation of the choice whether a research question should be answered with a animal model

11 11

Substantiation of the choice for a specific animal model 10 12

Knowledge and information

Use of 3Rs methods from similar previously conducted research 11 8

How the search for information on 3Rs methods was conducted 7 5

Demonstrated effort by the researcher to find 3Rs methods 12 4

Which information sources have been consulted 7 5

Which experts have been consulted 8 8

The competences of the personnel carrying out the biotechnical procedures 10 9

Replacement

Whether the use of human material is possible 8 6

Whether the use of computer simulations is possible 7 5

Reduction

Optimal use of in vitro methods prior to animal experiment 8 8

Optimal use/sharing of the experimental animal (e.g. practice chirurgical techniques post mortem)

8 9

Optimal and correct use of statistical tests 8 9

Refinement

If the standard biotechnical procedures are applied 11 13

If the correct analgesia and anaesthesia is administered 12 12

Correct use/definition of humane endpoints 11 13

Training of animals for better cooperation in the experiment 11 13

Social housing of the animals 11 13

Use of cage enrichment 12 13

Chapter 3

62

Table 6 Suggestions and priorities for improving 3Rs use. Survey question:In case you could start a new project for gaining a more optimal use of existing 3Rs knowledge, what would be of your primary focus? (Select a minimum of 1 and a maximum of 4 items)

Number of respondents

Support at a level of a research department (3Rs research and development). 10

A 3Rs literature search service for you specific research. 9

Well-facilitated 3Rs knowledge exchange between individuals within and between organisations. 8

Other * 6

Better and accessible information systems and databases with 3Rs literature. 5

External expert on alternatives that can be consulted, just like a bio-statistician. 4

More openness between organisations about animal experimental practices. 4

Encourage 3Rs assessment before funding applications. 3

Systematic reviews of excising literature 2

Funding – Providing budget for conducting a literature search by the researcher. 2

More focus on 3Rs in education 1

Courses to refresh and update 3Rs knowledge. 1

* comments by individual respondents can be found in the text.


63

CHAPTER 4Outcomes of a Dutch workshop on improvements for the 3Rs in daily practice

CHAPTER 4Outcomes of a Dutch workshop on improvements for the 3Rs in daily practice

Judith van LuijkMarlies LeenaarsAnne Marie van DongenLilian van der VaartMerel Ritskes-Hoitinga

ALTEX 29 (4), 440-443, 2012

Chapter 4

66

Abstract

This article describes the outcome of a workshop that was held to generate new ideas to improve the use of the 3R principles in science. The participants of the workshop represented researchers, Animal Welfare Officers and members of Animal Ethics Committees from various Dutch affiliations, including academia, industry, contract research organizations and knowledge centres. The workshop resulted in six diverse consensus statements, which are presented and discussed in this article. It may be concluded that there is not a straightforward solution to improve the implementation of 3R methods in animal-based research. The authors discuss the concept of a Systematic Review as a possible solution for several items mentioned in the consensus statements and as a method to move beyond the 3Rs.

Introduction

Over the last 50 years, the 3Rs have been the guiding principles in animal-based research [1]. Even though they are incorporated in the legislation of many countries, it is difficult implement the 3Rs principles in science and also to evaluate the effectiveness of implementation [2, 3].In 2009 a survey was conducted in order to investigate how professionals (scientists, Animal Welfare Officers and Animal Ethic Committee members) search for the 3Rs methods. The results showed that searching for 3Rs methods is perceived as a difficult task [4, 5]. These findings were in compliance with earlier findings of a local survey [6]. The majority of the respondents were not familiar with specialized databases and websites for animal science and 3Rs , while there are over 100. Also the proper skills to search in these databases was limited. Most perceived information on the 3Rs comes from direct colleagues and personal communication with other professionals [4, 5]. This is an undesirable situation as information remains local and may thus inhibits the use of new 3Rs innovations.For this reason a workshop was organised to investigate potential strategies to improve the implementation of the 3Rs principles in science. The workshop resulted in six consensus statements which are presented in this article. From the diversity of the statements and outcomes of other (inter)national studies [2, 7-14] it can be concluded that there is not a straight forward solution to improve implementation of the 3Rs. Systematic Reviews (SR) of animal studies may be a suitable approach to solve several of the issues raised. While SRs are common practice in clinical research, this is not the case in animal-based research while similar advantages from this approach can be expected. Moreover SRs appear a valuable tool to study the translational value of animal studies [15-19]. The potential value of SRs of animal studies for both scientific

Dutch workshop on 3r improvements

67

and ethical reasons (including the 3Rs principles) will be discussed in this paper as a way to move us beyond the 3Rs.

Three Rs Workshop

In 2010 the 3R Research Centre (SYRCLE since February 2012) organized in collaboration with Vaart Innovation and Knowledge, a workshop in the framework of a project funded by the Netherlands Organisation for Health Research and Development (ZonMW) : “Three Rs information in practice - national survey of searching, finding and applying the 3Rs” (project number ZonMW 114 000 092). The aim of this study was to gain insights into how Researchers, Animal Welfare Officers (AWO) and members of Animal Ethic Committees (AEC) incorporate the 3Rs principles [1] in their work. This workshop was a follow up of earlier survey studies [4, 5]. The questionnaire results showed that there is room for improvement of 3Rs implementation.Respondents of the questionnaires were asked if they were willing to participate in a follow up study. A total of 36 respondents were approached for participation in the workshop of which 18 participated. Together, the participants represented 8 different organisations including universities, industry, contract research organizations (CRO) and knowledge centres. It was for the first time in the Netherlands that researchers, AWOs and AEC members from different organisations were brought together to discuss the current state of affairs on the 3Rs principles and how to improve the use of these principles in daily practice. The aim of the workshop was to deepen the ideas suggested in the surveys, and to formulate and prioritize required actions for a better exchange of 3R knowledge and a more optimal 3R implementation in practice. Table 1 shows the affiliations of the 18 participants of the workshop. It should be noted that five participants exerted in daily practice two professional roles: some AWOs and researchers also being a member of an AEC. To avoid a possible conflict between two professional roles, those participants were asked to select one main profession during the workshop, e.g. preferably the same one as was taken to fill out the questionnaire. All of them indicated to represent their main profession which was either researcher or AWO. During the introduction of the workshop, a detailed overview of the survey results was presented. Accordingly the workshop participants were asked to comment in plenum. Based on this general discussion, the more detailed discussion topics for the afternoon session were formulated.In order to facilitate more in-depth discussions, the participants were divided into two groups for the afternoon session. The group composition was based on combining various backgrounds and professions, to create two more or less similarly mixed groups. Each group was led by one of the professional discussion leaders of Vaart Innovation and Knowledge Management. In the concluding plenary session, the two

Chapter 4

68

groups presented and discussed their main points, which together were used as the basis to formulate mutual consensus statements. Full agreement by participants to these consensus statements was given at the end of the workshop, and also to a written ZonMW report of the workshop (in Dutch) which was circulated afterwards [20].

Consensus statements

At the end of the workshop the participants had formulated six consensus statements, which are described below. Table 2 provides an overview of these consensus statements and includes suggested key players for achieving each goal mentioned. The key players were both suggested by the workshop participants during the workshop and also added later according to feedback received on the workshop report.

1. Division of the 3Rs and avoiding the use of the term ‘alternatives’ In order to address each R appropriately in its own right it is, according to the

participants, advisable to split the 3Rs into “Replacement” and “Reduction and Refinement” (also referred to as ‘best practice’). Replacement requires a different approach as compared to the application of Reduction and Refinement methods. Replacement methods for instance, require validation and sometimes even legislative involvement in order to be accepted on a national or international scale. By addressing the 3Rs as a whole, the risk arises that no full justice is done to each of the 3Rs.

An additional issue raised, was the use of the term ‘alternatives’ in the context of the communication within the scientific field, as well as with ‘the outside world’ (‘the public at large’).

In practice the term ‘3R alternatives’ is frequently used, but often creates the impression (especially to the public at large) that these only refer to genuine replacements. In practice, Replacement is only a very small - and in some fields even negligible- part of the 3Rs. It was suggested to abolish the term ‘alternatives’ altogether. Clearly, abolishing the term ‘alternatives’ in the scope of Laboratory Animal Science does not seem realistic, since it is a very well established term. However, the image of the ‘3R alternatives’ in this respect requires more attention. The utility and need of the 3Rs should be expressed better in a more scientific context, alongside with the well-known animal welfare arguments. The workshop participants also expressed the feeling that the current image of the word ‘3R alternatives’ is often seen as “dusty”. The field of laboratory animal science with a focus on 3R alternatives is therefore often seen as a service or secondary field of research only. For reasons of promoting proper science as well as animal welfare, it is important to create a clearer and better definition of the 3R’s individually, and the field of laboratory animal science as a whole.


69

2. Awareness for Replacement on a higher level and earlier stage of research, appointment of a Replacement expert.

Optimal implementation of each of the 3Rs requires different expertise. For questions related to Refinement and Reduction methods an AWO and/or a biostatistician can be approached. However, who to consult for Replacement possibilities is not clear according to the results of both surveys [4, 5] and the workshop discussions. The workshop participants suggested the appointment of a local Replacement expert, in a similar manner as the currently appointed AWOs in the Netherlands. An ideal situation appears be a team of 3R consulting experts consisting of a Replacement expert, a (bio)statistician and an AWO, in order to address each ‘R’ appropriately.

According to the participants the timing of addressing each of the 3Rs is crucial. Ideally, Replacement should be addressed at an early stage and at a high(er) responsibility level in the research chain. The responsibility for Replacement lies currently with the researcher. This, however, is not an ideal situation, since the development of Replacement possibilities usually does not lie within the domain of the individual researcher. In order to implement a full Replacement method in toxicology testing – which involves legally obligatory animal testing - the participation of multiple institutions and researchers over a longer time period is needed. Moreover, Replacement is generally not the first priority of the individual researcher. Therefore, this responsibility should be shifted to (teams at) higher hierarchical levels, such as project leaders, license holders, government and funding bodies. Only then development of Replacement methods can be prioritized and achieved by making the necessary resources available.

3. Bring license holders and funding bodies together for discussing topics on transparency and responsibility.

The debate on openness and transparency in the field of laboratory animal science in the Netherlands has been going on for a long time. The lack of transparency is counterproductive in science, as it creates difficulties for data exchange, and thus problems in coordination and cooperation of research. Additionally the lack of transparency also raises many critical questions in society and animal right groups. From the questionnaires it became clear that there is a need for more data exchange between research institutes [4, 5]. The responsibility for more openness lies primarily with the licensees and funding bodies (e.g. governmental funding and foundations such as the cancer fund and heart and kidney foundation). License holders and funding bodies have the possibilities and the power to enforce transparency. They can demand the mandatory publication of all results, including the so-called “negative results” (or neutral results). Depending on the circumstances, the degree of openness can be adapted to particular needs: the information from projects can be made

Chapter 4

70

anonymous for reasons of personal safety, and information can be temporarily withheld when it concerns a non-competition covenant.

4. Awareness in education for attitude development and best practice/experimental design.

In the FELASA category C compatible courses in the Netherlands the main focus lies on the 3R principles. This course is legally compulsory for all researchers and PhD students who will be involved in research with animal experiments. Often young scientist, prior to starting their scientific careers, attends this course. The workshop participants discussed the idea of a more continuous and possibly obligatory form of education for those involved in animal experimentation, to keep knowledge up-to-date. The survey results [4, 5] showed that ’researcher’s own motivation’ and ‘a good study preparation’ are two of the key factors for implementation of the 3Rs. It was therefore concluded that continuous professional education should focus on building the ‘proper’ attitude and on experimental design, in order to stimulate the motivation to implement the 3Rs in practice.

5. Accessible knowledge exchange. The current lack of transparency limits other needs within (laboratory animal)

science, such as the exchange of knowledge. When planning a new study, it is essential to take all other planned or ongoing studies into account in the same field of research by other groups. In order to stimulate collaborations and prevent unnecessary duplications, it is evident that this information needs to be shared.

During the workshop the idea was suggested to organize (inter)national meetings focussed on animal models to share knowledge and experiences. Similar types of meetings could also be organised per research field (e.g. disease types) and these meetings can easily be organised in combination with already existing (inter)national meetings and conferences. This also allows for more in-depth discussions on the pros and cons of the various animal models used in one specific field of research. A second issue that was addressed during the workshop was the necessity of publication of ‘negative’ (neutral) results. Knowledge gained from studies that gave ‘negative’ or neutral results is only rarely published, i.e. due to journal editorial policies. This information however, is very valuable for scientific reasons and necessary in order to prevent unnecessary duplications. Editorial boards of scientific journals can play an important role to make publication of these results possible and more attractive.

6. Facilitate and stimulate collaborations (communication) According to the workshop participants there seems to be a need for management

and coordination within the field of laboratory animal science in the Netherlands.


71

Issues such as lack of transparency and competition are currently important factors that limit research cooperation and knowledge exchange. The lively and passionate debates during the workshop demonstrate that these issues are not easily resolved, but that there is certainly much room for improvement of current practice. According to the workshop participants, the national coordination and communication should e.g. be organised by the NKCA (Netherlands Knowledge Centre for Alternatives), in order to define an action plan that identifies, prioritises and addresses the topics needed, on a national level.

Potential value of Systematic ReviewsFor both scientific and ethical reasons the design of a new animal experiment should encompass all relevant knowledge and information [21]. One way of achieving this so-called synthesis of evidence is by conducting Systematic Reviews (SRs) of animal studies. SRs have the potential to solve several of the issues raised in the consensus statements and advance the 3Rs principles in science. A SR can be defined as literature review to identify, appraise, select and synthesise all available relevant evidence to a specific research question. The conduct of a SR usually includes the following steps: 1. Formulate a specific question, 2. Define inclusion and exclusion criteria, 3. Locate studies, 4. Select studies, 5. Quality assessment, 6. Data extraction, 7. Analyse (if appropriate a meta-analysis) and present results, and 8. Interpret results [22]. The approach is systematic, thorough and leads to transparency on currently available data and its value.Since decades, SRs have been a well adopted methodology in the field of evidence-based medicine. This approach has led to major improvements in the methodology and reporting quality of clinical trials [22-24]. Major scientific and ethical improvements can be expected in by the introduction of SRs methodology in animal-based research too [25-29]. As CAMARADES has already shown the scientific value of SRs of animal studies in the field of neurology, it is very likely that the broad use of the SR methodology is of similar value in other areas of research involving animal studies [27]. From a scientific point of view one could make various arguments to support the conduct of SRs of animal studies [30]. Sir Iain Chalmers has explained some of these scientific arguments very well during the First International Symposium on SR in laboratory animal science [31]. A literature overview of all available relevant publications can reveal scientific gaps, e.g. were more research is needed or the presence of publication bias (unpublished studies with often negative or neutral results as mentioned in statement 5). The value of performing SRs lies in the fact that publication bias is made transparent [32]. This type of unpublished data can lead to an overestimation of the effects of a treatment, as demonstrated by SRs in the field of Stroke [25, 29]. A SR can include a meta-analysis in which data from multiple animal studies are statistically combined. This retrospective analysis evaluates both the coherence of the different animal studies and the contribution of the individual studies within a bigger research objective.

Chapter 4

72

There are also prospective reasons to conduct a SR as this literature overview can provide scientists with information to make more founded decisions for new experiments [23]. Useful information on 3Rs methods for a specific research question are hard to find as it is often scattered over various disciplines and different information sources or not even published at all [5, 6]. In this light it is not surprising that some scientist see the Thee Rs more as a burden than as a valuable addition to their research (also mentioned in consensus statement 1). As stated above SR seem an useful and scientifically consistent methodology to support the incorporation of the 3Rs principles in the planning of an animal experiment. Additionally SRs can be published in high impact factor journals and thus also contribute satisfactorily to a researcher’s quantitative and qualitative output. Which may make the conduct of a SR more appealing. From an ethical point of view an animal experiment should only be conducted when there is a clear need and necessity for the experiment and the objectives serve a greater good for the wellbeing of humans or animals. The experiments should be designed in such a way that they yield the most relevant and the most reliable results [30]. SRs seem a useful methodology for this purpose. Most relevant and reliable results are obtained when a model is selected for its properties to answer the research question, SR can help in this process [33]. How and why choices are made (such as the selection of an animal model) is not only valuable information for other researchers working in the same line of research. Also other stakeholders can advantage from this information. For example scientific committees can make more transparent why and how a line of research is selected (statements 3 and 5), funding bodies can use a SR as documentation on the current scientific quality to evaluate new research proposal and also AECs can use a SR in preparing their advice on approval of animal studies. A survey of published animal studies showed that over 80% did not use randomisation or blinding which are essential measures to reduce bias [28]. A quality assessment is one of the steps of a SR is to assess the methodological and reporting quality of the individual studies. In this step individual papers are scored for their internal (the extent to which bias is minimised) and external (the applicability of the results in other circumstances, translation) validity [22]. The conduct of SR of animal studies have there for the potential to improve transparency and thus raise the awareness for the need of correct experimental design as mentioned in statement 4. In the field of clinical research there is the CONSORT statement, to support the publication of various items in order to reduce the amount of bias. The endorsement of these guidelines by journals has lead to improvement in the completeness of reporting of clinical trials [24]. Similar developments are seen in animal research where recently, the GSPC and ARRIVE guidelines for planning and reporting of animal studies have been published [34, 35]. In line with the process in clinical research it is recommendable to make the animal-research guidelines also evolving documents and make them subject to changes as new evidence emerges.


73

SRs can also play an important role in the decision making from pre-clinical animal studies to clinical trials as described by Pound [36]. With two examples she shows that the safety of patients was jeopardized in clinical trials while this could have been prevented by conducting a SR of animal studies prior to starting the clinical trials [37, 38]. Individual animal studies showed some positive effects, however this effect was no longer found in a retrospective overall analysis of the animal studies. One could say that scientist have a moral duty to encompass all available data into the planning of a clinical trial. Not only regarding to patient safety but also with respect to the utility of results of animal studies. Conducting a SR is a multi-disciplinary process and requires involvement and collaboration of experts from the various disciplines needed, such as: a librarian, a statistician, a laboratory animal scientists and a field (medical) expert (e.g. when writing in oncology, a clinician working in the field of cancer should be part of the team). Conducting a SR stimulates multi-disciplinary involvement and collaboration as mentioned in statement 6 which will very likely also have a positive effect on the scientific quality. Education is needed to increase awareness of potential value of SRs of animal studies in science and to improve the skills needed to perform such a SR. Educational programs have been started at our university, both by successfully incorporating SR in the FELASA category C courses as well as an one week module on SRs of animal studies in the curriculum of biomedical science master students. These courses have branched out to other (inter)national universities such as Amsterdam, Lausanne and Bergen. It is a positive development that the Netherlands Organisation for Health Research and Development (ZonMw) has now issued a proposal to stimulate this type of education for animal researchers as continuing professional development [39].

Concluding remarks.

As the results of the workshop and other previous studies have shown, there is not one single solution to enhance the implementation of the 3Rs principles in science. We have argued that many of the items mentioned in the consensus statements may be addressed by conducting SRs of animal studies. However incorporating the 3Rs principles in sciences is not the only advantage of SRs and implementing these reviews will, therefore, enhance science beyond the Three Rs.

Acknowledgments

ZonMW (the Netherlands Organisation for Health Research and Development, grant nr.114 000 092) is greatly acknowledged for funding this research. The workshop participants are thanked for their active contributions and valuable input.

Chapter 4

74




2. ZonMw, Opbrengst ZonMw programma’s

op Vervanging, Vermindering en Verfijning

van dierproeven; analyse periode 2001-

2011. 2011, ZonMw: Den Haag.

3. Ormandy, E.H., C.A. Schuppli, and

D.M. Weary, Worldwide trends in the use

of animals in research: the contribution

of genetically-modified animal models.

Altern Lab Anim, 2009. 37(1): p. 63-8.

4. van Luijk, J., et al., Assessing the

application of the Three Rs: A survey

among Animal Welfare Officers in the

Netherlands. Lab Anim, 2012. Submitted.






39(5): p. 429-47.





37(3): p. 297-303.

7. Boyd Group, U., Advancing refinement

of laboratory animal use. Lab Anim, 1998.

32(2): p. 137-42.

8. Brønstad, A. and A.G.T. Berg, The role

of organizational culture in compliance

with the principles of the 3Rs. Lab Animal,

2011. 40(1): p. 22-26.

9. de Boo, J. and C. Hendriksen, Reduction

strategies in animal research: a review

of scientific approaches at the intra-

experimental, supra-experimental and

extra-experimental levels. Altern Lab

Anim, 2005. 33(4): p. 369-77.

10. Engeman, R.M. and S.A. Shumake,

Animal Welfare and the Statistical

Consultant. The American Statistician,

1993. 47(3): p. 229-233.

11. Howard, B., M. Hudson, and R. Preziosi,

More is less: reducing animal use by

raising awareness of the principles of

efficient study design and analysis. Altern

Lab Anim, 2009. 37(1): p. 33-42.

12. Lloyd, M.H., B.W. Foden, and S.E.

Wolfensohn, Refinement: Promoting the

three Rs in practice. Laboratory Animals,

2008. 42(3): p. 284-293.

13. Reed, B.T. and M. Jennings, Promoting

consideration of the ethical aspects of

animal use and implementation of the

3Rs. AATEX Special Issue, 2008. 14(Proc.

6th World Congress on Alternatives &

Animal Use in the Life Sciences, August

21-25, 2007, Tokyo, Japan): p. 131-135.

14. Wendel, A., Do we need a “Chair of

alternative methods”, and where? Altex,

2002. 19(2): p. 64-8.

15. Macleod, M.R., S. Ebrahim, and I.

Roberts, Surveying the literature from

animal experiments: systematic review and

meta-analysis are important contributions.

BMJ, 2005. 331(7508): p. 110.

16. Macleod, M.R., et al., Good laboratory

practice: preventing introduction of bias at

the bench. Stroke, 2009. 40(3): p. e50-2.

17. Hackam, D.G. and D.A. Redelmeier,

Translation of research evidence from

animals to humans. JAMA, 2006. 296(14):

p. 1731-2.

References


75

18. Macleod, M.R., et al., Pooling of animal

experimental data reveals influence of

study design and publication bias. Stroke,

2004. 35(5): p. 1203-8.

19. Perel, P., et al., Comparison of treatment

effects between animal experiments and

clinical trials: systematic review. BMJ,

2007. 334(7586): p. 197.

20. ZonMw, Landelijk zoeken naar en vinden

van de 3V’s. 2010, ZonMw: Den Haag.

21. Gaines Das, R., et al., Planning for

reduction. Altern Lab Anim, 2009. 37(1):

p. 27-32.




23. Clarke, M., S. Hopewell, and I. Chalmers,

Clinical trials should begin and end with

systematic reviews of relevant evidence:

12 years and waiting. Lancet, 2010.

376(9734): p. 20-1.

24. Moher, D., et al., Consolidated standards

of reporting trials (CONSORT) and

the quality of reporting of randomized

controlled trials. Cochrane Database of

Systematic Reviews 2010(3).

25. Sena, E.S., et al., Publication bias in

reports of animal stroke studies leads to

major overstatement of efficacy. PLoS

Biol, 2010. 8(3): p. e1000344.






Alzheimers Dis, 2012. 28(1): p. 191-209.




PLoS One, 2012. 7(2): p. e32296.

28. Kilkenny, C., et al., Survey of the quality

of experimental design, statistical analysis

and reporting of research using animals.

PLoS One, 2009. 4(11): p. e7824.

29. van der Worp, H.B., et al., Can animal

models of disease reliably inform human

studies? PLoS Med, 2010. 7(3): p.

e1000245.

30. de Vries, R.B., et al., Systematische

reviews van dierproeven - Over het etisch

belang voor dier én mens van het verhogen

van de wetenschappelijke kwaliteit van

dierexperimenteel onderzoek. Tijdschrift

voor Gezondheidzorg en Ethiek, 2010.

20(3).

31. Chalmers, I., Discussion and concluding

remarks 1st international symposium on

systematic reviews in laboratory animal

science. http://youtu.be/j5RDdkU2yxE,

2012.

32. Korevaar, D.A., L. Hooft, and G. ter Riet,

Systematic reviews and meta-analyses

of preclinical studies: publication bias in

laboratory animal experiments. Lab Anim,

2011. 45(4): p. 225-30.

33. de Vries, R., et al., Reducing the number

of laboratory animals used in tissue

engineering research by restricting the

variety of animal models. Articular cartilage

tissue engineering as a case study. Tissue

Eng Part B Rev, 2012.






Chapter 4

76



167-182.

35. Kilkenny, C., et al., Improving bioscience

research reporting: The ARRIVE guidelines

for reporting animal research. J Pharmacol

Pharmacother, 2010. 1(2): p. 94-9.




p. 514-517.

37. Horn, J., et al., Nimodipine in animal

model experiments of focal cerebral

ischemia: a systematic review. Stroke,

2001. 32(10): p. 2433-8.

38. Ciccone, A. and L. Candelise. Risk of

cerebral haemorrhage after thrombolysis:

systematic review of animal stroke models.

in the European Stroke Conference. 2003.

Valencia.

39. ZonMw, Programmatekst Meer Kennis

met Minder Dieren - Module Amendement

21, ZonMw, Editor. 2011: Den Haag.


77

Table 1 Affiliations of the workshop participants

Academia Industry Knowledge institute

Contract Research

Organisation

Other Total

Researchers 1 0 3 2 0 6

Animal welfare officers 4 2 2 1 0 9

Animal ethics commites members 0 0 2 0 1 3*

* The total number of AEC members was eight, but only three were solely AEC member, the other AEC members were an AEC member as a second profession. Their main profession was an AWO or a researcher and they participated in the workshop as such.

Table 2 The consensus statements formulated during the workhop and suggested key players

Consensus statement Suggested key players

1 Division of the Three Rs and cautious use of the term ‘alternatives’.

All involved in animal based research

2 Awareness for Replacement on a higher level and earlier stages of research, additionally appointment of a Replacement expert.

License holders, project leaders, Animal Ethics Committees, Scien-tific committees/boards within license holding organisations.

3 Bring license holders and funding bodies together for discussing the topics ‘transpa-rency’ and ‘responsibility’.

License holders, funding bodies (e.g. ZonMw and disease related funding agencies), government, public.

4 Awareness in education for building the ‘proper’ attitude and improve implementation of ‘best practices’/experimental design.

Dutch association for laboratory animal science (NVP), National Coordinators of the courses in laboratory animal science, FELASA.

5 Accessible knowledge exchange.- (inter)national meetings on animal models- Publication of ‘negative’ results

Animal care staff and technicians, researchers, AWOs, research departments, funding bodies, editorial boards of scientific journals.

6 Facilitate and stimulate collaborations (communication)

NKCA (Netherlands Knowledge Centre on Alternatives), govern-ment, politicians and society.

Jan RongenGerjon HanninkTony van TienenJudith van LuijkCarlijn Hooijmans

Osteoarthritis and Cartilage 23(8), 1242-1253, 2015

CHAPTER 5The protective effect of meniscus allograft transplantation on articular cartilage: a systematic review of animal studiesThe protective effect of meniscus allograft transplantation on articular cartilage: a systematic review of animal studies

Chapter 5

80

Abstract

Despite widespread reporting on clinical results, the effect of meniscus allograft transplantation on the development of osteoarthritis is still unclear. The aim of this study was to systematically review all studies on the effect of meniscus allograft transplantation on articular cartilage in animals.Systematic review and meta-analysis. Pubmed and Embase were searched for original articles concerning the effect of meniscus allograft transplantation on articular cartilage compared with both its positive (meniscectomy) and negative (either sham or non-operated) control in healthy animals. Outcome measures related to assessment of damage to articular cartilage were divided in five principal outcome categories. Standardized mean differences (SMD) were calculated and pooled to obtain an overall SMD and 95% confidence interval. 17 articles were identified, representing 14 original animal cohorts with an average timing of data collection of 24 weeks [range 4 weeks; 30 months]. Compared to a negative control, meniscus allograft transplantation caused gross macroscopic (1.45 [0.95; 1.95]), histological (3.43 [2.25; 4.61]) damage to articular cartilage, and osteoarthritic changes on radiographs (3.12 [1.42; 4.82]). Moreover, results on histomorphometrics and cartilage biomechanics are supportive of this detrimental effect on cartilage. On the other hand, meniscus allograft transplantation caused significantly less gross macroscopic (-1.19 [-1.84; -0.54]) and histological (-1.70 [-2.67; -0.74]) damage to articular cartilage when compared to meniscectomy. However, there was no difference in osteoarthritic changes on plain radiographs (0.04 [-0.48; 0.57]), and results on histomorphometrics and biomechanics did neither show a difference in effect between meniscus allograft transplantation and meniscectomy. Although meniscus allograft transplantation does not protect articular cartilage from damage, it reduces the extent of it when compared with meniscectomy.

Introduction

The menisci fulfill key biomechanical functions in the knee joint.[1] Unfortunately, meniscal injuries are quite common, accompanied by acute clinical symptoms such as knee pain, locking, and joint effusions. First documented treatments embraced swift and total meniscectomy to ameliorate acute symptoms.[2] However, an increased understanding of the osteoarthritic changes that occur after meniscectomy made clear that it is beneficial to save as much meniscal tissue as possible.[3-5] Meniscus allograft transplantation has been proposed as a promising treatment strategy for total meniscectomy already in the mid 1980s.[6] The initial goal of this treatment was to

a systematic review of animal studies

81

prevent, and possibly even reverse, the development of osteoarthritic changes in the knee joint.[6-11] Encouraged by the results of few animal studies clinical implantation was experimented upon.[12] However, a limited availability of donor menisci, and inconclusive results from meniscus allograft transplantation on development of osteoarthritis in humans, shifted its indication to treat localized pain after meniscectomy in a pre-selected patient population.[9, 13] Rosso et al. recently published a systematic review on the clinical and radiographic outcomes of meniscus allograft transplantation and its possible role in preventing osteoarthritis in humans.14 Despite the inclusion of 55 articles a meta-analysis could not be performed, mainly because of the lack of standardized evaluation methods to evaluate joint changes. The authors concluded that any chondroprotective effect in humans is still unclear. This result is not in accordance with their expectations based on the single animal study they refer to.[15] Although it is not uncommon for animal studies not to correspond (well) to results from clinical studies it does raise the question what the actual effect is of meniscus allograft transplantation on the articular cartilage in animals.[16] Therefore, we performed a systematic review and meta-analysis of all available preclinical studies we identified for this review, on the effect of meniscus allograft transplantation on the articular cartilage in animals. In addition, we discussed to what extent results from animals in meniscus related research can be translated to humans.

Methods

This systematic review investigates the effect of meniscus allograft transplantation on articular cartilage in animals. The inclusion criteria and method of analysis were specified in advance and documented in a protocol.[17]

Search strategy and selection of studies Pubmed and Embase were searched (last search performed July 29th, 2014) for original articles concerning the effect of meniscus allograft transplantation on articular cartilage compared with both its positive (meniscectomy) and negative (either sham or non-operated) controls in healthy animals. The search strategy, composed of three elements (meniscus, allograft, and animals), was developed in collaboration with information specialists from the medical library of the Radboud university medical center Nijmegen, the Netherlands.[18] To detect all animal studies search filters for Pubmed and Embase were used.[19, 20] The detailed search strategy is provided in supplementary file 1.Reference lists of the selected relevant (review-) papers were screened for potentially missed papers, and no restrictions (e.g. language or publication date) were imposed. Search results were imported in EROS (Early Review Organizing Software, developed

Chapter 5

82

by Institute of Clinical Effectiveness and Health Policy, Buenos Aires, Argentina) to remove duplicates, and randomly allocate references to two independent reviewers responsible for screening and selection (JR, GH). Discrepancies were resolved by discussion and if necessary a third reviewer was consulted (CH). Initially, during the screening phase, primary studies evaluating meniscus allograft transplantation in healthy animals were selected based on their title and abstract only. In the event that there was insufficient information to make a valid judgment, the whole publication was evaluatedFull-text copies of all publications eligible for inclusion were subsequently assessed and included when they met our pre-specified inclusion criteria: 1) a controlled interventional design (meniscectomy as a positive control and/or either sham or non-operated as a negative control); 2) description of (semi-) quantitative outcome measures related to articular cartilage damage (radiographic assessment, gross macroscopic assessment, histological/histochemical based grading, immunohistochemistry based grading, histomorphometry, magnetic resonance imaging (MRI), and/or biomechanical characterization).

Data extractionNext to bibliographic details, information was extracted related to: study design; animal model; intervention; outcome measures; and information related to exclusion of animals from analysis. Outcome measures related to assessment of damage to articular cartilage were divided in five principal outcome categories 1) Gross macroscopic assessment of damage (Grading or determining the area of articular cartilage with gross morphological changes , ICRS scores, Outerbridge scores, either with or without staining methods); 2) Medical imaging of changes related to osteoarthritis (plain radiographical (Kellgren & Lawrence) and MRI based classifications of morphological changes); 3) Histological histochemical grading of changes in articular cartilage (Mankin Grading method); 4) Histomorphometrics (any kind of quantitative study on microscopic images of articular cartilage); and 5) Biomechanical characterization of articular cartilage (tensile and compressive measures of stiffness). Raw data or group averages (mean, median), standard deviation (SD), standard error (SE) or ranges and number of animals per group (n) were extracted for all (semi-) continuous and ordinal outcome measures respectively. Attempts were made to obtain original data by contacting authors if results were presented incomplete or graphically only. If not otherwise possible, graphically presented data was converted to numerical data using digital ruler software (Plot Digitizer, University of South Alabama, USA).[21, 22]

Risk of bias assessmentThe internal validity of the included studies was assessed, by two reviewers independently (JR, GH), using SYRCLE’s risk of bias tool.[23] This tool is based on the Cochrane risk of bias tool [24] and has been adjusted for particular aspects of bias that


83

play a role in animal intervention studies. It contains 10 entries related to 6 types of bias (selection, performance, detection, attrition, reporting and ‘other’ bias). The score ‘yes’ indicates a low risk of bias, ‘no’ indicates high risk of bias, and ‘?’ indicates an unclear risk of bias. We used the provided signaling questions wherever possible.Reporting of experimental details on animals, methods and materials was expected to be poor.[25] To overcome the problem of judging too many items as unclear risk of bias two entries were added: 1) reporting of any measure of randomization, and 2) reporting of any measure of blinding. [23] For these two items, a ‘yes’ indicates reported, and a ‘no’ indicates unreported.

Data analysisA meta-analysis was performed whenever three or more independent comparisons per outcome category could be included (provided that outcome measure assessments were sufficiently comparable in terms of entity), and standardized mean differences (SMD) were calculated (SMD = the mean of the experimental group minus the mean of the control group divided by the pooled SDs of the two groups). Despite anticipated heterogeneity, the individual effect sizes were subsequently pooled to obtain an overall SMD and 95% confidence interval. We used a random effects model26 which takes into account the precision of individual studies and the variation between studies and weights each study accordingly. Heterogeneity was addressed by I2 which is the proportion of total variance explained by heterogeneity. With respect to the different time points of data collection across several studies, only results obtained equal or later than twelve weeks following index intervention were included in the meta-analysis. The latter is based on the assumption that, at least in sheep, damage to the articular cartilage is readily observed 3 months after meniscal destabilization procedures.[27] If multiple independent experimental groups were compared with the same control group within the same meta-analysis the number of animals in the control group was corrected by dividing it by the number of comparisons. In the case that more than one outcome measure corresponding to the same principal outcome category was presented, only one was included in the meta-analysis (continuous outcome measures were favored above semi quantitative and ordinal ones). For those outcome measures, of which results were presented separately for different anatomic regions in the knee joint, the tibia or its central weight bearing zone was used because this area is expected to demonstrate the highest degree of damage.[28] If contralateral knees were presented as paired non-operated controls, we solely used the paired controls of the meniscus allograft transplantation as negative controls. Because several authors presented and/or analyzed their ordinal data as continuous, and it was not possible to obtain raw data for each data set, results of ordinal scales were meta-analyzed as continuous data. In case the SD of the control group was zero, we used the SD of the experimental group instead to be able to calculate a SMD.

Chapter 5

84

Subgroup analyses were performed only if the subgroups contained a minimum of 3 independent comparisons. Subgroups were pre-specified in our protocol and analyses were planned for animal species (large/small), compartment of intervention (medial/lateral), and time of implantation after meniscectomy (delayed / immediate). Although not pre-specified, a subgroup analysis was additionally performed for timing of data collection as from meniscectomy (< 6 months (26weeks) / ≥ 6 months (26weeks)). Publication bias was addressed by means of a funnel plot, but only if at least ten studies could be included. In order to assess the robustness of our findings and in an attempt to further explain observed study heterogeneity, we performed a sensitivity analysis and investigated the effect of study quality and changing the time point of data collection from 3 to at least 6 months follow-up.

RESULTS

Study descriptivesThe search strategy retrieved 377 unique records; subsequent selection procedure resulted in 17 eligible articles, representing 14 original animal cohorts (figure 1).[15, 29-44] Both Aagaard et al.[36, 37] and Rijk et al.[32-34] confirmed that results of one single animal cohort were presented in more than one article. Overall, characteristics varied considerably between studies: Sheep (n=7), rabbits (n=7), goat (n=1), dogs (n=1), and rats (n=1), either males or females were used as animal models; they were operated upon uni- and bilaterally on the medial and lateral tibiofemoral compartment; and the follow up times ranged from 4 weeks to 30 months (Table 1).

Risk of bias and quality of reportingThe risk of bias assessment, summarized in Figure 2, shows that many items were scored unclear risk of bias which can be regarded as an indicator for poor reporting of the included animal studies.[45] For example, 9 of the 17 studies mentioned randomization at any level. However, none of these studies mentioned neither their methods of randomization nor sufficient details to judge its adequacy, and were therefore judged as unclear risk of bias. Twelve of the 17 studies stated that their experiment was blinded at any level, in all cases this implied that the outcome assessor was blinded. The individual scores of each study are presented in supplementary file 2.

Effect of meniscus allograft on articular cartilageOut of the 14 original animal cohorts, 20 independent experimental groups could be identified which underwent meniscus allograft transplantation and had a follow up longer than or equal to 12 weeks. Only Yamasaki et al. provided results prior to 3 months (4


85

or 8 weeks) and was therefore not included in any analysis.[44] Aagaard et al.[36, 37], Jiang et al.[40], Rijk et al.[32-34] and McNickle et al.[39] provided additional data by request, although the latter two were not able to retrieve all the requested information.

Gross macroscopic assessmentResults of studies included in the meta-analysis. Nine and 10 comparisons could be included in the meta-analysis concerning gross macroscopic damage between meniscus allograft transplantation with either meniscectomy or a negative control, respectively. The average timing of data collection from meniscectomy was 21.3 weeks (range [12weeks; 6 months]). Meniscus allograft transplantation demonstrated significantly less gross macroscopic damage when compared with meniscectomy (effect size -1.19 [-1.84; -0.54], I2 = 63%, figure 3a), but demonstrated more damage compared with a negative control (effect size 1.45 [0.95; 1.95], I2 = 42%, figure 3b).Results of studies not included in the meta-analysis. Elliot et al.[35] analyzed their results by using principal component analysis, providing factor loadings instead of descriptive results on gross macroscopic damage, and was therefore not included in the meta-analysis. Elliot et al.[35] described significant more degenerative changes in the morphological appearance of the articular cartilage following meniscal allograft transplantation compared with the non-operated control (in line with the meta-analysis), but differences between meniscectomy and meniscal allograft transplantation were not observed (not in line with the meta-analysis).

Medical imagingResults of studies included in the meta-analysis. Four comparisons could be included in the meta-analysis concerning osteoarthritic changes on plain radiographs between meniscus allograft transplantation with either meniscectomy or a negative control. The average timing of data collection from meniscectomy was 55.9 weeks (range [6 months; 30 months]). There was no difference in osteoarthritic changes between meniscus allograft transplantation and meniscectomy (effect size 0.04 [-0.48; 0.57], I2 = 0%, figure 4a). Meniscus allograft transplantation demonstrated significantly more osteoarthritic changes compared to a negative control (effect size 3.12 [1.42; 4.82], I2 = 69%, figure 4b).Results of studies not included in the meta-analysis. Whereas the studies included in the meta-analysis assessed osteoarthritic changes on plain radiographs, Kelly et al.[15] assessed morphologic changes in cartilage, bone, and bone marrow by using MRI. Because MRI and plain radiographs measure different entities of osteoarthritic changes the study of Kelly et al. [15] was not included in a meta-analysis. Kelly et al.[15] demonstrated less MRI based morphologic degenerative changes for meniscus allograft transplantation when compared with meniscectomy (not in line with the meta-analysis). However, more degenerative changes were observed for meniscus allograft

Chapter 5

86

transplantation when compared with the non-operated controls (in line with the meta-analysis). Spin echo T2 quantitative relaxation maps demonstrated better results for the non-operated controls when compared with meniscus allograft transplantation (in line with the meta-analysis), but there was no difference between meniscus allograft transplantation and meniscectomy (in line with the meta-analysis).

Histological/histochemical gradingResults of studies included in the meta-analysis. 11 comparisons could be included in the meta-analysis concerning histological histochemical grading of articular cartilage damage between meniscus allograft transplantation with either meniscectomy or a negative control. The average timing of data collection from meniscectomy was 26.5 weeks (range (12 weeks; 12 months)). Notably, the Mankin score was used by all authors, but often modified and presented for various anatomical regions. Meniscus allograft transplantation demonstrated less histological damage to articular cartilage compared with meniscectomy (effect size -1.70 [-2.67; -0.74], I2 = 84%, figure 5a). However, meniscus allograft transplantation showed more histological damage to cartilage compared with a negative control (effect size 3.43 [2.25; 4.61], I2 = 82%, figure 5b). Results of studies not included in the meta-analysis. Elliot et al.[35] analyzed their results by using principal component analysis, providing factor loadings instead of descriptive results on gross macroscopic damage, and was therefore not included in the meta-analysis. Elliot et al.[35] described more degenerative changes in the histological appearance of the articular cartilage following meniscal allograft transplantation compared with the non-operated control (in line with the meta-analysis). Differences in degenerative changes between meniscectomy and meniscal allograft transplantation were not observed (not in line with the meta-analysis).

HistomorphometricsThere were five studies that used any kind of quantitative study on microscopic images of articular cartilage to assess the effect of meniscus allograft on articular cartilage. The nature of these measurements was so divers (e.g. measuring different entities) that pooling of the individual effect sizes was not suitable. Results of individual studies are summarized in figure 6. Three of the 9 comparisons, of which 2 were independent, demonstrated a significant difference in damage to articular cartilage between meniscus allograft transplantation and meniscectomy, but in favor of different treatments. The remaining 6 comparisons, of which 3 were independent, did not demonstrate any significant difference in damage (figure 6a). Nine of the 13 comparisons, of which 5 were independent, demonstrated significant less damage to articular cartilage in favor for the negative control compared with meniscus allograft transplantation. The remaining 4 comparisons, of which 2 were independent, did not demonstrate any significant difference (figure 6b).


87

BiomechanicsTwo studies used biomechanical characterization to address the effect of meniscus allograft transplantation on articular cartilage. Compared to meniscectomy, one comparison did, and one did not demonstrate a significant difference in stiffness in favor of meniscus allograft transplantation (figure 7a). Compared to a negative control, two comparisons demonstrated a significant lower stiffness of articular cartilage after meniscus allograft transplantation (figure 7b).

Subgroup analysisBecause of the relative low number of included studies it was not possible to perform statistical robust subgroup analyses for all planned subgroups. For those subgroups that included more than 3 studies, the effect estimates were described in table 2. None of those subgroup analyses showed significant differences between the groups.

Publication biasBecause of the low number of studies that were included in the meta-analyses, a funnel plot was not created.

Sensitivity analysis Changing the time point for data collection from 3 to at least 6 months follow-up did not change the outcome of results. Due to poor reporting of experimental details the results from the risk of bias could not be implemented as a screening part in the sensitivity analysis.

DISCUSSION

The concept of meniscus allograft transplantation was conceived to prevent degeneration of articular cartilage in the meniscectomized knee.[6-11] So far, despite widespread reporting on clinical results, the effect of meniscus allograft transplantation on the development of osteoarthritis is still unclear.[14] This is not in line with expectations from the animal study referred to by the recently published meta-analysis.[14] We therefore conducted this study to systematically review all studies on the effect of meniscus allograft transplantation on articular cartilage in animals.From this systematic review and meta-analysis, which included studies with an average timing of data collection of 28 weeks (range (12 weeks; 30 months)), it becomes clear that compared to a negative control, meniscus allograft transplantation causes gross macroscopic and histological damage to articular cartilage, and osteoarthritic changes on plain radiographs. Moreover, results on histomorphometry and cartilage biomechanics support that meniscus allograft transplantation has a detrimental

Chapter 5

88

influence on articular cartilage. Although showing damage to articular cartilage, meniscus allograft transplantation causes significantly less gross macroscopic and histological damage to articular cartilage when compared to meniscectomy. However, this difference in favor of meniscus allograft transplantation is not observed for osteoarthritic changes on plain radiographs. Results on histomorphometrics and biomechanics do neither appear to show a clear difference in damage to articular cartilage between meniscus allograft transplantation and meniscectomy. Osteoarthritis is characterized by a molecular phase, a pre radiographic phase, and a recalcitrant radiographic phase with evident structural joint changes.[46] In this continuum of osteoarthritic stages we interpret our results as such that, although both meniscus allograft transplantation and meniscectomy cause structural joint changes, damage to articular cartilage is less extensive after meniscus allograft transplantation.

Some methodological issues which might hamper the interpretation of the experimental animal data and the subsequent translation to the clinical setting have to be discussed.First, the heterogeneity among the various animal studies was substantial. Unfortunately, unraveling this heterogeneity by performing subgroup analyses was not feasible due to too low number of independent comparisons within subgroups. The soundness of pooling different animal species and models within a single meta-analysis could be questioned. However, consistent results across species and models do provide some reassurance that the observed effect is reliable. Moreover, subgroup analysis did not demonstrate differences in results between large (sheep/goats) and middle (dogs/rabbits) animals. In addition, from a biomechanical point of view, the meniscus has the same function across different species: load distribution over two incongruent moving articular surfaces. Removing it will inflict pathological (peak) stresses causing articular cartilage to be damaged. To what extent this damage occurs, at what rate, and with what clinical symptoms could well be species specific. For example, it has been demonstrated that sheep show changes in kinematics after meniscectomy comparable to humans.[27] Rabbits only show little change in knee kinematics[47], and rats show only minor changes in static weight-bearing after meniscectomy.[48, 49] Unfortunately, due to too few comparisons, assessing differences in extent of damage between animal species could not be performed in this review. The minimum of twelve weeks as a time point of data collection used in the present study was based on the assumption that, at least in sheep, damage to the articular cartilage is readily observed 3 months after meniscal destabilization procedures. It is possible that adequate duration would be different for different animals/species. With the combination of consistent results across species and models, the theoretical background, and the results of the subgroup analysis in mind, we have no arguments that show that it might be better to only focus on large animals.


89

We did not differentiate for time from meniscectomy to meniscus allograft transplantation (e.g. delayed transplantation). It could well be that delayed meniscus allograft transplantation performs worse in comparison to immediate transplantation, because there is a time period in which the cartilage is overloaded. This seems as an important question to address, looking to the current clinical situation in which humans receive a meniscus allograft a substantial time period after the removal of their meniscus (Rosso et al. reported an overall weighted average time between meniscectomy and meniscus allograft transplantation of 15.16 years (range, 1.1-35.8 years)[14]). However, a fallacy here is that the current the indication of meniscus allograft transplantation is not the prevention of osteoarthritis but to treat localized pain post meniscectomy in a selected patient population. Not differentiating for time between meniscectomy and allograft transplantation theoretically underestimates the protective effect on damage to articular cartilage when compared to meniscectomy.Second, poor reporting of crucial pieces of information in the included articles is of serious concern, and limited the application of the SYRCLE’s risk of bias tool. Regrettably, this is not uncommon practice in animal studies, limiting the ability to draw reliable conclusions. This concern is also shared and addressed by others.[25, 45, 50-52] It is crucial that the poor reporting in animal studies is addressed and future studies should improve on this, allowing others to be able to replicate and build on previously published work. Ultimately, with better reporting, systematic reviews of high quality will become feasible. In recent years checklists have been developed to improve the quality of animal studies.[45, 50] Implementing such initiatives will improve the quality of individual scientific papers on animal experimentation. Moreover, as a consequence, numbers of animals used would be expected to fall, and will set in motion better translation to the clinic and increases patient safety.[16, 45, 53, 54] Because of the poor reporting of experimental details it was not possible to assess how the results of this review might have been affected if studies with a high risk of bias were excluded from the analysis.Third, results on long term effect of meniscus allograft transplantation on articular cartilage are not provided in the current systematic review. Our sensitivity analysis showed that results did not change if 6 instead of 3 months was used as a minimum time point of data collection. Damage to articular cartilage would be expected to be progressively more severe if even longer endpoints would have been used. Since both meniscus allograft transplantation and meniscectomy cause damage to articular cartilage it is imaginable that both interventions ultimately lead to end stages of osteoarthritis. To what extent meniscus allograft transplantation is able to postpone disabling end stages compared to meniscectomy, remains unknown.

Regarding translation from pre clinical animal models to clinical practice, several aspects should be addressed. First, consistent results across species and models do

Chapter 5

90

provide some reassurance that the observed effect is reliable, and that humans might respond similarly. Moreover, the minimal invasive arthroscopic procedures used in humans may have less detrimental effect on cartilage compared to the extensive open procedures used in animal experiments, which could improve the outcome of meniscal allograft transplantation. Finally, whereas graft sizing was only scarcely mentioned in animal experiments it is common practice in humans. From a biomechanical point of view, proper graft sizing leads to better load distribution, which is an important factor for a successful outcome of meniscal transplantation.[55] However, some aspects interfere with translation to clinical practice. First, in pre clinical animal models the meniscus allograft is transplanted in a “pristine” knee. Here, damage of articular cartilage is the result of both the surgical procedure and the meniscus allograft only. In contrast, clinically, a meniscus allograft is transplanted in a knee that has undergone a trauma large enough to cause considerable damage to the meniscus. This trauma may even well have damaged the cartilage directly, limiting the theoretical treatment effect of a meniscus allograft. Second, we demonstrated that both meniscectomy as well as meniscus allograft transplantation will cause degeneration of articular cartilage. This raises the question as to what extent the theoretically lesser damage to articular cartilage after meniscus allograft transplantation gives rise to a clinical meaningful reduction or delay in clinical (disabling) osteoarthritis, and subsequent need for total knee arthroplasty. However it should be noted that, not all patients who develop damage to the articular cartilage, or even radiographical established osteoarthritis, will end up with clinical disabling osteoarthritis.[56-58] Subsequently, a next question would be what the cost per health unit gain would be of meniscus allograft transplantation. These questions can only be addressed with information obtained from a (randomized) controlled clinical trial. Rosso et al. explain their inability to analyze the chondroprotective effect of meniscus allograft transplantation because of the lack of standardized evaluation methods (e.g. radiographically different grading scores were used to evaluate joint changes). However, a more important limitation in human studies is the lack of control groups. To be able to demonstrate any positive effect of meniscus allograft transplantation in humans, a control group of patients undergoing meniscectomy is imperative. Only 1 out of the 55 articles on human meniscus allograft transplantation identified by Rosso et al. used a control cohort of meniscectomized patients. However, it concerned a historical cohort and comparisons were made on subjective assessment rather than evaluating the status of articular cartilage.[59] This lack of control groups limits us to directly assess to what extent results from animal experiments in meniscus related research can be translated to humans. In conclusion, this systematic review with an average follow up of included studies of 28 weeks (range (12 weeks; 30 months)) demonstrates that, in animals, although meniscus allograft transplantation does not protect articular cartilage from damage, it


91

reduces the extent of it when compared with meniscectomy. Consistent results across species and models do provide some reassurance that this observed effect is reliable, and that humans might respond in a similar manner.

Chapter 5

92

1. Rongen JJ, van Tienen TG, van Bochove

B, Grijpma DW, Buma P. Biomaterials

in search of a meniscus substitute.

Biomaterials 2014; 35: 3527-3540.

2. Annandale T. Excision of the Internal

Semilunar Cartilage, Resulting in Perfect

Restoration of the Joint-Movements. Br

Med J 1889; 1: 291-292.

3. Petty CA, Lubowitz JH. Does

arthroscopic partial meniscectomy result

in knee osteoarthritis? A systematic

review with a minimum of 8 years’ follow-

up. Arthroscopy 2011; 27: 419-424.

4. Macnicol MF, Thomas NP. The knee

after meniscectomy. J Bone Joint Surg Br

2000; 82: 157-159.

5. Salata MJ, Gibbs AE, Sekiya JK. A

systematic review of clinical outcomes in

patients undergoing meniscectomy. Am J

Sports Med 2010; 38: 1907-1916.

6. Canham W, Stanish W. A study of the

biological behavior of the meniscus as a

transplant in the medial compartment of a

dog’s knee. Am J Sports Med 1986; 14:

376-379.

7. Milachowski KA, Weismeier K, Wirth CJ.

Homologous meniscus transplantation.

Experimental and clinical results. Int

Orthop 1989; 13: 1-11.

8. Keene GC, Paterson RS, Teague DC.

Advances in arthroscopic surgery. Clin

Orthop Relat Res 1987: 64-70.

9. Rodeo SA. Meniscal allografts--where

do we stand? Am J Sports Med 2001; 29:

246-261.

10. Arnoczky S, Warren R, McDevitt

C. Meniscal replacement using a

cryopreserved allograft. An experimental

study in the dog. Clinical Orthopaedics

and Related Research 1990: 121-128.

11. Powers D, Davenport M, Wisnewski P.

Glutaraldehyde-cross-linked meniscal

allografts: clinical, gross, and histological

results. Journal of investigative surgery

: the official journal of the Academy of

Surgical Research 1988; 1: 249-257.

12. Wirth CJ, Milachowski KA, Weismeier

K. [Meniscus transplantation in animal

experiments and initial clinical results]. Z

Orthop Ihre Grenzgeb 1986; 124: 508-512.

13. Veltri DM, Warren RF, Wickiewicz TL,

O’Brien SJ. Current status of allograft

meniscal transplantation. Clin Orthop

Relat Res 1994: 44-55.

14. Rosso F, Bisicchia S, Bonasia DE,

Amendola A. Meniscal Allograft

Transplantation: A Systematic

Review. Am J Sports Med 2014. : doi:

10.1177/0363546514536021

15. Kelly BT, Potter HG, Deng XH, Pearle

AD, Turner AS, Warren RF, et al. Meniscal

allograft transplantation in the sheep knee:

evaluation of chondroprotective effects.

Am J Sports Med 2006; 34: 1464-1477.

16. Pound P, Ebrahim S, Sandercock P,

Bracken MB, Roberts I, Reviewing Animal

Trials Systematically G. Where is the

evidence that animal research benefits

humans? BMJ 2004; 328: 514-517.

17. Rongen JJ, Hannink G, van Tienen TG,

van Luijk J, Hooijmans C. The translational

gap in research about meniscus allograft.

Available at: https://www.radboudumc.

nl/Research/Organisationofresearch/

Departments/cdl/SYRCLE/Documents/

Rongen_protocol_translat ional%20

REFERENCES


93

gap%20in%20research%20about%20

meniscus%20allograft.pdf.

18. Leenaars M, Hooijmans CR, van Veggel

N, ter Riet G, Leeflang M, Hooft L, et al.

A step-by-step guide to systematically

identify all relevant animal studies. Lab

Anim 2012; 46: 24-31.

19. Hooijmans CR, Tillema A, Leenaars

M, Ritskes-Hoitinga M. Enhancing



experimentation in PubMed. Lab Anim

2010; 44: 170-175.

20. de Vries RB, Hooijmans CR, Tillema A,

Leenaars M, Ritskes-Hoitinga M. Updated

version of the Embase search filter for

animal studies. Lab Anim 2014; 48: 88.

21. Hooijmans CR, Pasker-de Jong PC,

de Vries RB, Ritskes-Hoitinga M. The

effects of long-term omega-3 fatty acid

supplementation on cognition and

Alzheimer’s pathology in animal models

of Alzheimer’s disease: a systematic

review and meta-analysis. J Alzheimers

Dis 2012; 28: 191-209.

22. Hooijmans CR, de Vries RB, Rovers

MM, Gooszen HG, Ritskes-Hoitinga M.

The effects of probiotic supplementation

on experimental acute pancreatitis: a


PLoS One 2012; 7: e48811.

23. Hooijmans CR, Rovers MM, de Vries

RB, Leenaars M, Ritskes-Hoitinga M,

Langendam MW. SYRCLE’s risk of bias

tool for animal studies. BMC Med Res

Methodol 2014; 14: 43.

24. Higgins JP, Altman DG, Gotzsche

PC, Juni P, Moher D, Oxman AD, et al.

The Cochrane Collaboration’s tool for

assessing risk of bias in randomised

trials. BMJ 2011; 343: d5928.

25. Kilkenny C, Parsons N, Kadyszewski E,

Festing MF, Cuthill IC, Fry D, et al. Survey

of the quality of experimental design,

statistical analysis and reporting of

research using animals. PLoS One 2009;

4: e7824.

26. DerSimonian R, Laird N. Meta-analysis

in clinical trials. Control Clin Trials 1986; 7:

177-188.

27. Cake MA, Read RA, Corfield G, Daniel A,

Burkhardt D, Smith MM, et al. Comparison

of gait and pathology outcomes of three

meniscal procedures for induction of knee

osteoarthritis in sheep. Osteoarthritis

Cartilage 2013; 21: 226-236.

28. McDermott ID, Amis AA. The

consequences of meniscectomy. J Bone

Joint Surg Br 2006; 88: 1549-1556.

29. Mora G, Alvarez E, Ripalda P, Forriol

F. Articular cartilage degeneration after

frozen meniscus and Achilles tendon

allograft transplantation: experimental

study in sheep. Arthroscopy 2003; 19:

833-841.

30. Szomor ZL, Martin TE, Bonar F, Murrell

GA. The protective effects of meniscal

transplantation on cartilage. An

experimental study in sheep. J Bone Joint

Surg Am 2000; 82: 80-88.

31. Jackson DW, Mcdevitt CA, Simon

TM, Arnoczky SP, Atwell EA, Silvino NJ.

Meniscal Transplantation Using Fresh and

Cryopreserved Allografts - an Experimental-

Study in Goats. American Journal of Sports

Medicine 1992; 20: 644-656.

Chapter 5

94

32. Rijk PC, de Rooy TPW, Coerkamp

EG, Bernoski FP, van Noorden CJF.

Radiographic evaluation of the knee joint

after meniscal allograft transplantation.

An experimental study in rabbits. Knee

Surgery Sports Traumatology Arthroscopy

2002; 10: 241-246.

33. Rijk PC, Tigchelaar-Gutter W, Bernoski FP,

Van Noorden CJ. Histologic changes in

articular cartilage after medial meniscus

replacement in rabbits. Arthroscopy

2004; 20: 911-917.

34. Rijk PC, Tigchelaar-Gutter W, Bernoski

FP, Van Noorden CJF. Functional changes

in articular cartilage after meniscal

allograft transplantation: A quantitative

histochemical evaluation in rabbits.

Arthroscopy-the Journal of Arthroscopic

and Related Surgery 2006; 22: 152-158.

35. Elliott DM, Jones R, 3rd, Setton LA, Scully

SP, Vail TP, Guilak F. Joint degeneration

following meniscal allograft transplantation

in a canine model: mechanical properties

and semiquantitative histology of articular

cartilage. Knee Surg Sports Traumatol

Arthrosc 2002; 10: 109-118.

36. Aagaard H, Jorgensen U, Bojsen-

Moller F. Reduced degenerative articular

cartilage changes after meniscal allograft

transplantation in sheep. Knee Surg Sports

Traumatol Arthrosc 1999; 7: 184-191.

37. Aagaard H, Jorgensen U, Bojsen-Moller

F. Immediate versus delayed meniscal

allograft transplantation in sheep. Clinical

Orthopaedics and Related Research

2003: 218-227.

38. Cummins JF, Mansour JN, Howe Z,

Allan DG. Meniscal transplantation

and degenerative articular change:

An experimental study in the rabbit.

Arthroscopy 1997; 13: 485-491.

39. McNickle AG, Wang VM, Shewman EF,

Cole BJ, Williams JM. Performance of

a Sterile Meniscal Allograft in an Ovine

Model. Clinical Orthopaedics and Related

Research 2009; 467: 1868-1876.

40. Jiang D, Zhao LH, Tian M, Zhang JY,

Yu JK. Meniscus Transplantation Using

Treated Xenogeneic Meniscal Tissue:

Viability and Chondroprotection Study

in Rabbits. Arthroscopy-the Journal of

Arthroscopic and Related Surgery 2012;

28: 1147-1159.

41. Zwierzchowski TJ, Stasikowska-Kanicka

O, Danilewicz M, Fabis J. Assessment of

apoptosis and MMP-1, MMP-3 and TIMP-2

expression in tibial hyaline cartilage after

viable medial meniscus transplantation in

the rabbit. Archives of Medical Science

2012; 8: 1108-1114.

42. Wada Y, Amiel M, Harwood F, Amiel D.

Architectural remodeling of fresh meniscal

allograft transplantation in a rabbit joint:

Its long-term protective effect on articular

cartilage. Journal of Orthopaedic Science

1997; 2: 171-179.

43. Edwards DJ, Whittle SL, Nissen MJ,

Cohen B, Oakeshott RD, Keene GCR.

Radiographic changes in the knee

after meniscal transplantation - An

experimental study in a sheep model.

American Journal of Sports Medicine

1996; 24: 222-226.

44. Yamasaki T, Deie M, Shinomiya

R, Yasunaga Y, Yanada S, Ochi M.

Transplantation of meniscus regenerated


95

by tissue engineering with a scaffold

derived from a rat meniscus and

mesenchymal stromal cells derived from

rat bone marrow. Artif Organs 2008; 32:

519-524.

45. Hooijmans CR, Leenaars M, Ritskes-

Hoitinga M. A gold standard publication

checklist to improve the quality of animal

studies, to fully integrate the Three Rs, and

to make systematic reviews more feasible.

Altern Lab Anim 2010; 38: 167-182.

46. Kraus VB, Burnett B, Coindreau J, Cottrell

S, Eyre D, Gendreau M, et al. Application

of biomarkers in the development of

drugs intended for the treatment of

osteoarthritis. Osteoarthritis Cartilage

2011; 19: 515-542.

47. Mansour JM, Wentorf FA, Degoede KM.

In vivo kinematics of the rabbit knee in

unstable models of osteoarthrosis. Ann

Biomed Eng 1998; 26: 353-360.

48. Fernihough J, Gentry C, Malcangio

M, Fox A, Rediske J, Pellas T, et al.

Pain related behaviour in two models of

osteoarthritis in the rat knee. Pain 2004;

112: 83-93.

49. Bendele AM. Animal models of

osteoarthritis. J Musculoskelet Neuronal

Interact 2001; 1: 363-376.

50. Kilkenny C, Browne W, Cuthill IC,

Emerson M, Altman DG, Group

NCRRGW. Animal research: reporting in

vivo experiments: the ARRIVE guidelines.

Br J Pharmacol 2010; 160: 1577-1579.

51. Macleod MR, Ebrahim S, Roberts I.

Surveying the literature from animal

experiments: systematic review and meta-

analysis are important contributions. BMJ

2005; 331: 110.

52. Macleod M. Why animal research needs

to improve. Nature 2011; 477: 511.

53. Macleod MR, O’Collins T, Horky LL,

Howells DW, Donnan GA. Systematic

review and metaanalysis of the efficacy

of FK506 in experimental stroke. J Cereb

Blood Flow Metab 2005; 25: 713-721.

54. Hackam DG, Redelmeier DA. Translation

of research evidence from animals to

humans. JAMA 2006; 296: 1731-1732.

55. Dienst M, Greis PE, Ellis BJ, Bachus

KN, Burks RT. Effect of lateral meniscal

allograft sizing on contact mechanics of

the lateral tibial plateau: an experimental

study in human cadaveric knee joints. Am

J Sports Med 2007; 35: 34-42.

56. Massardo L, Watt I, Cushnaghan J,

Dieppe P. Osteoarthritis of the knee joint:

an eight year prospective study. Ann

Rheum Dis 1989; 48: 893-897.

57. Pengas IP, Assiotis A, Nash W, Hatcher

J, Banks J, McNicholas MJ. Total

meniscectomy in adolescents: a 40-year

follow-up. J Bone Joint Surg Br 2012; 94:

1649-1654.

58. McNicholas MJ, Rowley DI, McGurty D,

Adalberth T, Abdon P, Lindstrand A, et al.

Total meniscectomy in adolescence. A

thirty-year follow-up. J Bone Joint Surg Br

2000; 82: 217-221.

59. Wirth CJ, Peters G, Milachowski KA,

Weismeier KG, Kohn D. Long-term results

of meniscal allograft transplantation. Am J

Sports Med 2002; 30: 174-181.

Chapter 5

96

Tab

le 1

Stu

dy

char

acte

rist

ics

of

the

incl

uded

stu

die

s

Stu

dy ID

Ani

mal

cha

ract

eris

tics

Stu

dy c

hara

cter

istic

sA

llogr

aft c

hara

cter

istic

s

Ref

eren

ceSp

ecie

s (s

train

)se

xAg

e ʊ

Wei

ght

(kg)

ʊ

Expe

rimen

tal

grou

ps

(n p

er ti

me

of d

ata

colle

ctio

n)

Con

trol G

roup

s (n

per

tim

e of

da

ta c

olle

ctio

n)

Paire

d co

n-tro

ls¤

Med

or

La

t§

Sur-

gery

un

i/bi

ǂ

Tim

ing

data

co

llect

ion

from

m

enis

cec-

tom

y

Early

Po

stop

erat

ive

reha

bilit

atio

nɸ

Pre -

serv

a-tio

n

Gra

ft

Ster

iliza-

tion

Gra

ft Si

zing

Gra

ft fix

atio

n

Mor

a

2003

shee

p (*

)*

*30

-32

All(8

)C

(8);M

(8)

nom

edun

i6

Mo

clos

ed

confi

nem

ent

deep

fro

zen

**

trans

osse

ous

+ d

irect

su

ture

s

Kelly

20

06sh

eep

(CXR

)F

*70

-80

All(8

+8+

1)S(

2+2+

0);

M(1

2+12

+0)

yes

lat

uni

2, 4

, 12

mo

clos

ed

confi

nem

ent

fresh

*

wei

ght

mat

chin

g +

m

orph

ome-

trics

trans

osse

ous

+ d

irect

su

ture

s

Szom

or

2000

shee

p (m

blc)

**

40-5

0Al

l(8)

S(4)

; M(8

); Au

(8)

nom

edun

i16

wk

clos

ed

confi

nem

ent

fresh

**

Mite

k G

II su

ture

an-

chor

s

Jack

son

1992

goat

s (S

pani

sh)

F*

35-4

0 Al

l(10)

; cAl

l(10)

S(6)

; Au(

10)

yes

med

uni

6 M

ono

rest

rictio

n

fresh

&

cr

yop-

rese

r-ve

d

**

anch

oure

d bo

ne p

lugs

Rijk

20

02/

2004

/ 20

06

rabb

its

(NZW

)F

*3-

3.5

All(7

+7)

; dAl

l(0+

7)S(

2+2)

; M(7

+7)

yes/

no

×m

edun

i6

wk,

12

mo

no re

stric

tion

fresh

**

dire

ct s

utur

e

Ellio

tt

2002

dogs

(m

ongr

el)

**

25–3

5Al

l(10)

M(1

0)ye

sm

edun

i12

wk

no re

stric

tion

fresh

*

wei

ght

mat

chin

gdi

rect

sut

ure

Aaga

ard

19

99/

2003

shee

p (ic

elan

dic)

F 12

-18

mo

40-

55S-

M(6

+5)

; S-A

ll(0+

5); S

-dAl

l(0+

5);

M-A

ll(0+

5); M

-dAl

l(0+

5); A

ll-dA

ll(0+

5)¥

nom

edbi

3, 6

mo

no re

stric

tion

fresh

*

not

trans

osse

ous

sutu

re

Cum

min

s 19

97ra

bbits

(N

ZW)

M*

4.0-

4.5

All(8

+0)

; dAl

l(0+

8)M

(8+

8);

yes

med

uni

3, 6

mo

no re

stric

tion

deep

fro

zen

**

dire

ct s

utur

e

McN

ickl

e 20

09sh

eep

(Dor

set)

*±

1 yr

50-6

0Al

l(3+

6); s

All(3

+6)

M(2

+2)

yes

med

uni

2, 4

mo

*de

ep

froze

n

Bio

Cle

-an

se1

prot

ocol

*tra

nsos

seou

s +

dire

ct

sutu

res


97

Jian

g 20

12ra

bbits

(N

ZW)

M*

2.0-

2.5

All-M

(8+

8+8)

; XEN

-S(8

+8+

8) ¥

nom

edbi

6, 1

2,

24 w

kno

rest

rictio

nde

ep

froze

nga

mm

a irr

adia

tion

*di

rect

sut

ure

Zwie

r-zc

how

ski

2012

rabb

its

(NZ)

M*

3.5-

4.0

All†

(8);

AU(8

)S(

6)no

med

uni

6 m

ono

rest

rictio

ncu

lture

m

edi-

um

**

trans

osse

ous

+ d

irect

su

ture

s

Wad

a 19

97ra

bbits

(N

Z)*

7-8

mo

3.5-

4.0

All-M

(0+

16+

16+

15)

¥C

(12+

0+0+

0)no

med

bi0,

9, 1

2,

26 w

kno

rest

rictio

nfre

sh

*W

eigh

t/ ag

e m

atch

ing

dire

ct s

utur

e

Edw

ards

19

96sh

eep

(Mer

ino)

F*

65-7

0M

-C(1

); Al

l-C(3

); Al

l-M(3

); Au

-M(5

); M

-M(1

); M

-All(

1) ¥

nom

edbi

mea

n 21

.4

mo

[rang

e 13

-30]

clos

ed

confi

nem

ent

deep

fro

zen

**

bone

plu

gs

+ d

irect

su

ture

s

Yam

asak

i 20

08ra

ts (S

D)

*14

w

ks

*Al

l-M‡ (

*+*)

; msc

All-M

(*+

*)‡

nom

edbi

4 or

8 w

k*

deep

fro

zen

**

*

Ab

bre

viat

ions

: Mo

= m

ont

hs;

yr

= y

ears

; wk

= w

eeks

; All

= m

enis

cus

allo

gra

ft; c

All

= c

ryo

pre

serv

ed a

llog

raft

; dA

ll =

del

ayed

allo

gra

ft t

rans

pla

n-ta

tion;

S =

sh

am p

roce

dur

e; C

= u

nop

erat

ed c

ont

rol;

M =

men

isce

cto

my;

Xen

= x

eno

gra

ft; A

u =

men

iscu

s au

tog

raft

; msc

All

= m

enis

cus

allo

gra

ft

seed

ed w

ith m

esen

chym

al s

tro

mal

cel

ls d

eriv

ed f

rom

bo

ne m

arro

w; *

= n

ot

men

tione

d /

unkn

ow

n; ?

= u

ncle

ar;

ʊ =

ag

e an

d w

eig

ht

at s

tart

of

stud

y; C

XR

= C

olu

mb

ian

X R

amb

oui

llet;

NW

= N

ew Z

eala

nd; N

WZ

= N

ew z

eala

nd w

hite

; SD

= S

pra

gue

–Daw

ley;

MB

LC =

mer

ino

-bo

rder

leis

ter

cro

ssb

red

; ¤ =

kne

e co

ntra

late

ral t

o in

terv

entio

n se

rved

as

a p

aire

d u

nop

erat

ed c

ont

rol;

§ =

pro

ced

ures

on

eith

er m

edia

l or

late

ral m

enis

cus;

ǂ

= s

urg

ery

pro

ced

ures

eith

er u

ni-

or

bila

tera

lly w

ithin

one

ani

mal

; ɸ =

clo

sed

co

nfine

men

t (e

.g. l

imite

d s

pac

e to

mo

ve a

roun

d)

for

first

tim

e p

erio

d

afte

r in

dex

sur

ger

y;†

= m

enis

cus

allo

gra

ft w

as im

pla

nted

tw

o w

eeks

aft

er r

emo

val o

f th

e m

enis

cus

and

sto

red

in c

ultu

re m

ediu

m c

ont

aini

ng b

oth

fe

tal c

alf

seru

m a

nd a

ntib

iotic

s;‡

= t

he

men

iscu

s al

log

raft

was

tre

ated

with

ED

TA f

or

the

pur

po

se o

f d

ecal

cific

atio

n an

d f

reez

e th

awed

th

ree

times

to

kill

all

the

men

isca

l cel

ls; ¥

= c

om

bin

atio

ns o

f p

roce

dur

es p

erfo

rmed

on

bo

th le

gs

are

giv

en a

nd s

epar

ated

by

– si

gn.

× =

In t

he

2002

stu

dy

all

nono

per

ated

co

ntra

late

ral k

nees

ser

ved

as

pai

red

co

ntro

ls, w

her

eas

for

the

2004

and

200

6 st

udie

s o

nly

the

nono

per

ated

left

kne

e jo

ints

of

6 ra

b-

bits

wer

e se

lect

ed a

t ra

ndo

m b

efo

re s

urg

ery

to s

erve

as

a co

ntro

l gro

up.

Chapter 5

98

Tab

le 2

Eff

ect

size

s o

f su

bg

roup

ana

lysi

s Th

e ef

fect

siz

es a

nd 9

5% c

onfi

den

ce in

terv

als

(95%

CI)

fo

r d

iffer

ent

sub

gro

ups

are

pre

sent

ed. T

he

n re

flect

s th

e nu

mb

er o

f in

dep

end

ent

com

par

iso

ns. N

.e. =

no

t es

timat

ed. T

imin

g <

or ≥

6mo

= t

imin

g o

f d

ata

colle

ctio

n fr

om

men

isce

cto

my

earl

ier

or

equa

l/lat

er t

han

6 m

ont

hs

(26w

eeks

). L

arg

e an

imal

s =

sh

eep

/go

ats;

Mid

dle

ani

mal

s =

do

gs/

rab

bits

.

Sub

gro

ups

(n)

Co

mp

aris

on

Out

com

e m

easu

reE

ffec

t si

ze [

95%

CI]

Ani

mal

spe

cies

M

iddl

e (2

) / L

arge

(2)

Allo

graf

t vs.

men

isce

ctom

yM

edic

al im

agin

gn.

e. /

n.e

.

M

iddl

e (3

) / L

arge

(6)

Allo

graf

t vs.

men

isce

ctom

yG

ross

mac

rosc

opic

ass

essm

ent

-1.4

3 [-

2.00

; -0.

86] /

-1.

05 [-

2.11

, 0.0

1]

M

iddl

e (6

) / L

arge

(5)

Allo

graf

t vs.

men

isce

ctom

yH

isto

logi

cal-h

isto

chem

ical

gra

ding

-2.0

6 [-

3.64

; -0.

48] /

-1.3

5 [-

2.62

; -0.

09]

M

iddl

e (2

) / L

arge

(2)

Allo

graf

t vs.

neg

. con

trol

Med

ical

imag

ing

n.e.

/ n

.e.

M

iddl

e (2

) / L

arge

(8)

Allo

graf

t vs.

neg

. con

trol

Gro

ss m

acro

scop

ic a

sses

smen

tn.

e. /

1.32

[0.7

8; 1

.85]

M

iddl

e (6

) / L

arge

(5)

Allo

graf

t vs.

neg

. con

trol

His

tolo

gica

l-his

toch

emic

al g

radi

ng3.

02 [1

.60;

4.4

4] /

4.11

[1.9

1; 6

.31]

Tim

ing

allo

graf

t tra

nspl

anta

tion

Im

med

iate

(3) /

del

ayed

(1)

Allo

graf

t vs.

men

isce

ctom

yM

edic

al im

agin

g0.

13 [-

0.47

; 0.7

2] /

n.e

.

Im

med

iate

(8) /

del

ayed

(1)

Allo

graf

t vs.

men

isce

ctom

yG

ross

mac

rosc

opic

ass

essm

ent

-1.5

1 [-

1.92

; -1.

09] /

n.e

.

Im

med

iate

(8) /

del

ayed

(3)

Allo

graf

t vs.

men

isce

ctom

yH

isto

logi

cal-h

isto

chem

ical

gra

ding

-2.0

2 [-

3.19

; -0.

85] /

-0.9

8 [-

3.17

; 1.2

1]

Im

med

iate

(3) /

del

ayed

(1)

Allo

graf

t vs.

neg

. con

trol

Med

ical

imag

ing

2.81

[0.8

2; 4

.80]

/ n

.e.

Im

med

iate

(9) /

del

ayed

(1)

Allo

graf

t vs.

neg

. con

trol

Gro

ss m

acro

scop

ic a

sses

smen

t1.

30 [0

.85;

1.7

5] /

n.e

.

Im

med

iate

(8) /

del

ayed

(3)

Allo

graf

t vs.

neg

. con

trol

His

tolo

gica

l-his

toch

emic

al g

radi

ng3.

58 [2

.07;

5.1

0] /

3.26

[0.8

3; 5

.68]

Com

partm

ent o

f int

erve

ntio

n

M

edia

l (4)

/ La

tera

l (0)

Allo

graf

t vs.

men

isce

ctom

yM

edic

al im

agin

g0.

04 [-

0.48

; 0.5

7] /

n.e

.

M

edia

l (9)

/ La

tera

l (1)

Allo

graf

t vs.

men

isce

ctom

yG

ross

mac

rosc

opic

ass

essm

ent

-1.0

2 [-

1.66

; -0.

38] /

n.e

.

M

edia

l (10

) / L

ater

al (1

)A

llogr

aft v

s. m

enis

cect

omy

His

tolo

gica

l-his

toch

emic

al g

radi

ng-1

.26

[-2.

08; -

0.43

] / n

.e.

M

edia

l (4)

/ La

tera

l (0)

Allo

graf

t vs.

neg

. con

trol

Med

ical

imag

ing

3.12

[1.4

2; 4

.82]

/ n

.e.

M

edia

l (9)

/ La

tera

l (1)

Allo

graf

t vs.

neg

. con

trol

Gro

ss m

acro

scop

ic a

sses

smen

t1.

37 [0

.84;

1.8

9] /

n.e

.

M

edia

l (10

) / L

ater

al (1

)A

llogr

aft v

s. n

eg. c

ontro

lH

isto

logi

cal-h

isto

chem

ical

gra

ding

2.94

[1.9

0; 3

.98]

/ n

.e.


99

Tim

ing

of d

ata

colle

ctio

n

Ti

min

g <

6m

o (0

) / T

imin

g ≥

6mo

(4)

Allo

graf

t vs.

men

isce

ctom

yM

edic

al im

agin

gn.

e. /

0.0

7 [-

0.50

; 0.6

4]

Ti

min

g <

6m

o (6

) / T

imin

g ≥

6mo

(4)

Allo

graf

t vs.

men

isce

ctom

yG

ross

mac

rosc

opic

ass

essm

ent

-1.3

7 [-

1.90

; -0.

84] /

-0.

95 [-

2.48

; 0.5

8]

Ti

min

g <

6m

o (5

) / T

imin

g ≥

6mo

(6)

Allo

graf

t vs.

men

isce

ctom

yH

isto

logi

cal-h

isto

chem

ical

gra

ding

-3.1

0 [-

5.22

; -0.

99] /

-0.

93 [-

1.84

; -0.

01]

Ti

min

g <

6m

o (0

) / T

imin

g ≥

6mo

(4)

Allo

graf

t vs.

neg

. con

trol

Med

ical

imag

ing

n.e.

/ 3

.12

[1.4

2; 4

.82]

Ti

min

g <

6m

o (6

) / T

imin

g ≥

6mo

(4)

Allo

graf

t vs.

neg

. con

trol

Gro

ss m

acro

scop

ic a

sses

smen

t1.

42 [0

.73;

2.1

2] /

1.5

2 [0

.74;

2.3

1]

Ti

min

g <

6m

o (5

) / T

imin

g ≥

6mo

(6)

Allo

graf

t vs.

neg

. con

trol

His

tolo

gica

l-his

toch

emic

al g

radi

ng3.

90 [1

.85;

5.9

6] /

3.0

5 [1

.52;

4.5

9]

Chapter 5

100

Fig

ure

1 Fl

ow

dia

gra

m o

f th

e sy

stem

atic

rev

iew

and

met

a an

alys

is li

tera

ture

sea

rch

res

ults

IDE

NTI

FIC

ATI

ON

SC

RE

EN

ING

ELI

GIB

ILIT

Y

INC

LUD

ED

Rec

ord

s id

entifi

ed t

hro

ugh

d

atab

ase

sear

chin

g

(n =

588

)

Rec

ord

s sc

reen

ed

(n =

337

)R

eco

rds

excl

uded

(n

= 3

10)

Full-

text

art

icle

s as

sess

ed

for

elig

ibili

ty

(n =

67)

Stu

die

s in

clud

ed in

q

ualit

ativ

e sy

nth

esis

(n

= 9

1)

Ad

diti

ona

l rec

ord

s id

entifi

ed

thro

ugh

oth

er s

our

ces

(n =

0)

Rec

ord

s af

ter

dup

licat

es r

emo

ved

(n

= 3

77)

Full-

text

art

icle

s ex

clud

ed (

n =

50)

Dup

licat

es (

n =

6)

No

t a

pri

mar

y in

terv

entio

n st

udy

(n =

5)

No

t a

men

iscu

s al

log

raft

tra

nsp

lant

atio

n (n

= 7

)N

o p

rop

er c

ont

rol g

roup

(n

= 1

1)

No

qua

ntifi

able

mea

sure

on

cart

ilag

e (n

= 2

1)


101

Fig

ure

2 R

isk

of

bia

s, a

vera

ged

per

item

ye

s =

low

ris

k o

f b

ias,

no

= h

igh

ris

k o

f b

ias,

? =

unc

lear

ris

k o

f b

ias

Chapter 5

102

Fig

ure

3 E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: g

ross

mac

rosc

op

ic a

sses

smen

t Fo

rest

plo

ts o

f th

e in

clud

ed s

tud

ies

wh

ich

use

d g

ross

mac

rosc

op

ic a

sses

smen

t to

det

erm

ine

the

dam

age

of

artic

ular

car

tilag

e o

bse

rved

af

ter

men

iscu

s al

log

raft

tra

nsp

lant

atio

n co

mp

ared

with

men

isce

cto

my

(a)

and

a n

egat

ive

cont

rol (

b).

Fo

rest

plo

ts d

isp

lay

the

SM

Ds,

95

% c

onfi

den

ce in

terv

al a

nd r

elat

ive

wei

gh

t o

f th

e in

div

idua

l stu

die

s. T

he

dia

mo

nd in

dic

ates

th

e g

lob

al e

stim

ate

and

its

95%

co

nfid

ence

in

terv

al. s

-All

= s

teri

le m

enis

cus

allo

gra

ft; a

-All

= a

sep

tic m

enis

cus

allo

gra

ft; d

-All

= d

elay

ed m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion;

f-A

ll =

fr

esh

men

iscu

s al

log

raft

; c-A

ll =

cry

op

rese

rved

men

iscu

s al

log

raf

Fig

ure

3a. E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: g

ross

mas

cro

sco

pic

ass

esse

men

t (m

enis

cus

allo

gra

ft v

s m

enis

cect

om

y)

Fig

ure

3b. E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: g

ross

mas

cro

sco

pic

ass

esse

men

t (m

enis

cus

allo

gra

ft v

s ne

gat

ive

cont

rol)


103

Fig

ure

4 E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: m

edic

al im

agin

g

Fore

st p

lots

of

the

incl

uded

stu

die

s w

hic

h u

sed

pla

in r

adio

gra

ph

s to

det

erm

ine

chan

ges

rel

ated

to

ost

eoar

thri

tis o

bse

rved

aft

er m

enis

-cu

s al

log

raft

tra

nsp

lant

atio

n co

mp

ared

with

men

isce

cto

my

(a)

and

a n

egat

ive

cont

rol (

b).

Fo

rest

plo

ts d

isp

lay

the

SM

Ds,

95%

co

nfid

ence

in

terv

al a

nd r

elat

ive

wei

gh

t o

f th

e in

div

idua

l stu

die

s. T

he

dia

mo

nd in

dic

ates

th

e g

lob

al e

stim

ate

and

its

95%

co

nfid

ence

inte

rval

. d-A

ll =

d

elay

ed m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

Fig

ure

4a. E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: m

edic

al im

agin

g (

men

iscu

s al

log

raft

vs

men

isce

cto

my)

Fig

ure

4b. E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: m

edic

al im

agin

g (

men

iscu

s al

log

raft

vs

neg

ativ

e co

ntro

l)

Chapter 5

104

Fig

ure

5 E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: h

isto

log

ical

-his

toch

emic

al g

rad

ing

Fo

rest

plo

ts o

f th

e in

clud

ed s

tud

ies

wh

ich

use

d h

isto

log

ical

-his

toch

emic

al g

rad

ing

to

det

erm

ine

the

dam

age

of

artic

ular

car

tilag

e o

b-

serv

ed a

fter

men

iscu

s al

log

raft

tra

nsp

lant

atio

n co

mp

ared

with

men

isce

cto

my

(a)

and

a n

egat

ive

cont

rol (

b).

Fo

rest

plo

ts d

isp

lay

the

SM

D,

95%

co

nfid

ence

inte

rval

and

rel

ativ

e w

eig

ht

of

the

ind

ivid

ual s

tud

ies.

Th

e d

iam

ond

ind

icat

es t

he

glo

bal

est

imat

e an

d it

s 95

% c

onfi

den

ce

inte

rval

. d-A

ll =

del

ayed

men

iscu

s al

log

raft

tra

nsp

lant

atio

n.

Fig

ure

5a. E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: H

isto

log

ical

-his

toch

emic

al g

rad

ing

(m

enis

cus

allo

gra

ft v

s m

enis

cect

om

y)


105

Fig

ure

5b. E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: H

isto

log

ical

-his

toch

emic

al g

rad

ing

(m

enis

cus

allo

gra

ft v

s ne

gat

ive

cont

rol)

Fig

ure

5 E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: h

isto

log

ical

-his

toch

emic

al g

rad

ing

Fo

rest

plo

ts o

f th

e in

clud

ed s

tud

ies

wh

ich

use

d h

isto

log

ical

-his

toch

emic

al g

rad

ing

to

det

erm

ine

the

dam

age

of

artic

ular

car

tilag

e o

b-

serv

ed a

fter

men

iscu

s al

log

raft

tra

nsp

lant

atio

n co

mp

ared

with

men

isce

cto

my

(a)

and

a n

egat

ive

cont

rol (

b).

Fo

rest

plo

ts d

isp

lay

the

SM

D,

95%

co

nfid

ence

inte

rval

and

rel

ativ

e w

eig

ht

of

the

ind

ivid

ual s

tud

ies.

Th

e d

iam

ond

ind

icat

es t

he

glo

bal

est

imat

e an

d it

s 95

% c

onfi

den

ce

inte

rval

. d-A

ll =

del

ayed

men

iscu

s al

log

raft

tra

nsp

lant

atio

n.

Chapter 5

106

Fig

ure

6 E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: h

isto

mo

rph

om

etri

cs

Fore

st p

lots

of

the

incl

uded

stu

die

s w

hic

h u

sed

any

kin

d o

f h

isto

mo

rph

om

etri

cs t

o d

eter

min

e th

e d

amag

e o

f ar

ticul

ar c

artil

age

ob

serv

ed

afte

r m

enis

cus

allo

gra

ft tr

ansp

lant

atio

n co

mp

ared

with

men

isce

cto

my

(a)

and

a n

egat

ive

cont

rol (

b).

Fo

rest

plo

ts d

isp

lay

the

SM

D a

nd 9

5%

confi

den

ce in

terv

al o

f th

e in

div

idua

l stu

die

s. T

he

Fore

st p

lots

wer

e m

anua

lly a

dap

ted

so

th

at t

he

dir

ectio

n o

f ef

fect

has

th

e sa

me

mea

ning

ac

ross

th

e d

iffer

ent

out

com

e m

easu

res.

Rea

der

s sh

oul

d n

ote

th

at d

iffer

ent

out

com

e m

easu

res

with

in t

he

sam

e st

udy

are

not

ind

epen

den

t,

and

th

us s

ho

uld

no

t b

e p

oo

led

. dA

l = d

elay

ed m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion.

Fig

ure

6a. E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: h

isto

mo

rph

om

etri

cs (

men

iscu

s al

log

raft

vs

men

isce

cto

my)

Fig

ure

6b. E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: h

isto

mo

rph

om

etri

cs (

men

iscu

s al

log

raft

vs

neg

ativ

e co

ntro

l)


107

Fig

ure

7a. E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: b

iom

ech

anic

s (m

enis

cus

allo

gra

ft v

s m

enis

cect

om

y)

Fig

ure

7b. E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: b

iom

ech

anic

s (m

enis

cus

allo

gra

ft v

s ne

gat

ive

cont

rol)

Fig

ure

7 E

ffec

t o

f m

enis

cus

allo

gra

ft t

rans

pla

ntat

ion

on

cart

ilag

e: b

iom

ech

anic

s Fo

rest

plo

ts o

f th

e in

clud

ed s

tud

ies

wh

ich

use

d b

iom

ech

anic

al t

ests

to

det

erm

ine

the

dam

age

of

artic

ular

car

tilag

e o

bse

rved

aft

er m

enis

-cu

s al

log

raft

tra

nsp

lant

atio

n co

mp

ared

with

men

isce

cto

my

(a)

and

a n

egat

ive

cont

rol (

b).

Fo

rest

plo

ts d

isp

lay

the

SM

D a

nd 9

5% c

onfi

-d

ence

inte

rval

of

the

ind

ivid

ual s

tud

ies.

Chapter 5

108

Supplementary Table 1 Search strategy

Pubmed

Component 1: Meniscus Menisci, Tibial [Mesh] OR (knee [mesh] AND fibrocartilage [mesh]) OR (knee joint [mesh] AND fibrocartilage [mesh]) OR meniscal [tiab] OR meniscected [tiab] OR meniscectomie [tiab] OR meniscectomies [tiab] OR meniscectomised [tiab] OR meniscectomisized [tiab] OR meniscectomized [tiab] OR meniscectomy [tiab] OR menisceotomy [tiab] OR meniscetomy [tiab] OR menisci [tiab] OR meniscial [tiab] OR meniscopathies [tiab] OR meniscopathy [tiab] OR meniscopexia [tiab] OR meniscopexies [tiab] OR meniscopexy [tiab] OR meniscoplasty [tiab] OR menisco-tomies [tiab] OR meniscotomy [tiab] OR meniscus [tiab] OR meniscusectomy [tiab] OR meniscusectomy [tiab] OR menisectomies [tiab] OR menisectomized [tiab] OR menisectomy [tiab] OR (knee [tiab] AND fibrocartilage [tiab]) OR (knee [tiab] AND fibro-cartilage [tiab]) OR (knee [tiab] AND fibrocartilagenous [tiab]) OR (knee [tiab] AND fibro-cartilagenous [tiab]) OR (knee [tiab] AND fibrocartilageous [tiab]) OR (knee [tiab] AND fibrocartilages [tiab]) OR (knee [tiab] AND fibro-cartilages [tiab]) OR (knee [tiab] AND Semilunar [tiab]) OR (knees [tiab] AND fibrocartilage [tiab]) OR (knees [tiab] AND fibro-cartilage [tiab]) OR (knees [tiab] AND fibrocartilagenous [tiab]) OR (knees [tiab] AND fibro-cartilagenous [tiab]) OR (knees [tiab] AND fibro-cartilageous [tiab]) OR (knees [tiab] AND fibrocartilages [tiab]) OR (knees [tiab] AND fibro-cartilages [tiab]) OR (knees [tiab] AND semilunar [tiab]) OR (tibiafemoral [tiab] AND fibrocartilage [tiab]) OR (tibia-femoral [tiab] AND fibrocartilage [tiab]) OR (tibi-afemoral [tiab] AND fibro-cartilage [tiab]) OR (tibia-femoral [tiab] AND fibro-cartilage [tiab]) OR (tibiafemoral [tiab] AND fibrocartilagenous [tiab]) OR (tibiafemoral [tiab] AND fibrocartilagenous [tiab]) OR (tibia-femoral [tiab] AND fibrocartilagenous [tiab]) OR (tibiafemoral [tiab] AND fibro-cartilagenous [tiab]) OR (tibiafemoral [tiab] AND fibro-cartilagenous [tiab]) OR (tibia-femoral [tiab] AND fibro-cartilagenous [tiab]) OR (tibiafemoral [tiab] AND fibrocartilageous [tiab]) OR (tibiafemoral [tiab] AND fibrocar-tilageous [tiab]) OR (tibia-femoral [tiab] AND fibrocartilageous [tiab]) OR (tibiafemoral [tiab] AND fibrocartilages [tiab]) OR (tibia-femoral [tiab] AND fibrocartilages [tiab]) OR (tibiafemoral [tiab] AND fibro-cartilages [tiab]) OR (tibiafemoral [tiab] AND fibro-carti-lages [tiab]) OR (tibia-femoral [tiab] AND fibro-cartilages [tiab]) OR (tibiafemoral [tiab] AND semilunar [tiab]) OR (tibia-femoral [tiab] AND semilunar [tiab]) OR (tibiofemoral [tiab] AND fibrocartilage [tiab]) OR (tibio-femoral [tiab] AND fibrocartilage [tiab]) OR (tibiofemoral [tiab] AND fibro-cartilage [tiab]) OR (tibio-femoral [tiab] AND fibro-car-tilage [tiab]) OR (tibiofemoral [tiab] AND fibrocartilagenous [tiab]) OR (tibio-femoral [tiab] AND fibrocartilagenous [tiab]) OR (tibiofemoral [tiab] AND fibro-cartilagenous [tiab]) OR (tibio-femoral [tiab] AND fibro-cartilagenous [tiab]) OR (tibiofemoral [tiab] AND fibrocartilageous [tiab]) OR (tibio-femoral [tiab] AND fibrocartilageous [tiab]) OR (tibiofemoral [tiab] AND fibrocartilages [tiab]) OR (tibio-femoral [tiab] AND fibro-cartilages [tiab]) OR (tibiofemoral [tiab] AND fibro-cartilages [tiab]) OR (tibio-femoral [tiab] AND fibro-cartilages [tiab]) OR (tibiofemoral [tiab] AND semilunar [tiab]) OR (tibio-femoral [tiab] AND semilunar [tiab]) OR Semilunar Cartilage [tiab] OR Semilunar Cartilages [tiab] OR Semilunar fibrocartilage [tiab]

Component 2: Allograft Allografts [Mesh] OR Transplantation, Homologous [Mesh] OR Tissue Transplanta-tion [mesh] OR Allogeneic implant [tiab] OR Allogeneic implants [tiab] OR Allogenic implants [tiab] OR Allograft [tiab] OR Allo-graft [tiab] OR Allografted [tiab] OR Al-lo-grafted [tiab] OR Allografting [tiab] OR Allo-grafting [tiab] OR Allograftings [tiab] OR Allografts [tiab] OR Allo-grafts [tiab] OR Alloplastic implant [tiab] OR Alloplastic implants [tiab] OR Allotransplant [tiab] OR Allo-transplant [tiab] OR Allotransplantation [tiab] OR Allo-transplantation [tiab] OR Allotransplantations [tiab] OR Allo-transplanta-tions [tiab] OR Allotransplanted [tiab] OR Allo-transplanted [tiab] OR Allotransplanting [tiab] OR Allotransplants [tiab] OR Allo-transplants [tiab] OR Graft [tiab] OR Grafted [tiab] OR Grafting [tiab] OR Graftings [tiab] OR Grafts [tiab] OR Homo-graft [tiab] OR Homograft [tiab] OR Homografted [tiab] OR Homografting [tiab] OR Homo-graft-ing [tiab] OR Homograftings [tiab] OR Homografts [tiab] OR Homo-grafts [tiab] OR Homologous implants [tiab] OR Homotransplant [tiab] OR Homotransplantation [tiab] OR Homo-transplantation [tiab] OR Homotransplantations [tiab] OR Homo-transplan-tations [tiab] OR Homotransplanted [tiab] OR Homo-transplanted [tiab] OR Homo-transplants [tiab] OR Homo-transplants [tiab] OR Transplant [tiab] OR Transplantation [tiab] OR Transplantations [tiab] OR Transplanted [tiab] OR Transplanting [tiab] OR Transplants [tiab]

Component 3: Animal Search filter for animal studies [Hooijmans 2010]

Appendix chapter 5:


109

Embase

Component 1: Meniscus (exp knee / AND exp fibrocartilage/) OR (exp knee joint / AND exp fibrocartilage/) OR exp knee meniscus/ OR exp knee meniscus rupture/ OR exp meniscectomy/ OR exp meniscal surgery / OR meniscal .ti,ab. OR meniscected .ti,ab. OR meniscectomie .ti,ab. OR meniscectomies .ti,ab. OR meniscectomised .ti,ab. OR meniscectomis-ized .ti,ab. OR meniscectomized .ti,ab. OR meniscectomy .ti,ab. OR menisceotomy .ti,ab. OR meniscetomy .ti,ab. OR menisci .ti,ab. OR meniscial .ti,ab. OR menis-copathies .ti,ab. OR meniscopathy .ti,ab. OR meniscopexia .ti,ab. OR meniscopex-ies .ti,ab. OR meniscopexy .ti,ab. OR meniscoplasty .ti,ab. OR meniscotomies .ti,ab. OR meniscotomy .ti,ab. OR meniscus .ti,ab. OR meniscusectomy .ti,ab. OR meniscusectomy .ti,ab. OR menisectomies .ti,ab. OR menisectomized .ti,ab. OR menisectomy .ti,ab. OR (knee .ti,ab. AND fibrocartilage .ti,ab.) OR (knee .ti,ab. AND fibro-cartilage .ti,ab.) OR (knee .ti,ab. AND fibrocartilagenous .ti,ab.) OR (knee .ti,ab. AND fibro-cartilagenous .ti,ab.) OR (knee .ti,ab. AND fibrocartilageous .ti,ab.) OR (knee .ti,ab. AND fibrocartilages .ti,ab.) OR (knee .ti,ab. AND fibro-cartilages .ti,ab.) OR (knee .ti,ab. AND Semilunar .ti,ab.) OR (knees .ti,ab. AND fibrocartilage .ti,ab.) OR (knees .ti,ab. AND fibro-cartilage .ti,ab.) OR (knees .ti,ab. AND fibrocartilagenous .ti,ab.) OR (knees .ti,ab. AND fibro-cartilagenous .ti,ab.) OR (knees .ti,ab. AND fibro-cartilageous .ti,ab.) OR (knees .ti,ab. AND fibrocartilages .ti,ab.) OR (knees .ti,ab. AND fibro-cartilages .ti,ab.) OR (knees .ti,ab. AND semilunar .ti,ab.) OR (tibiafemoral .ti,ab. AND fibrocartilage .ti,ab.) OR (tibia-femoral .ti,ab. AND fibrocartilage .ti,ab.) OR (tibi-afemoral .ti,ab. AND fibro-cartilage .ti,ab.) OR (tibia-femoral .ti,ab. AND fibro-cartilage .ti,ab.) OR (tibiafemoral .ti,ab. AND fibrocartilagenous .ti,ab.) OR (tibiafemoral .ti,ab. AND fibrocartilagenous .ti,ab.) OR (tibia-femoral .ti,ab. AND fibrocartilagenous .ti,ab.) OR (tibiafemoral .ti,ab. AND fibro-cartilagenous .ti,ab.) OR (tibiafemoral .ti,ab. AND fibro-cartilagenous .ti,ab.) OR (tibia-femoral .ti,ab. AND fibro-cartilagenous .ti,ab.) OR (tibiafemoral .ti,ab. AND fibrocartilageous .ti,ab.) OR (tibiafemoral .ti,ab. AND fibrocar-tilageous .ti,ab.) OR (tibia-femoral .ti,ab. AND fibrocartilageous .ti,ab.) OR (tibiafemoral .ti,ab. AND fibrocartilages .ti,ab.) OR (tibia-femoral .ti,ab. AND fibrocartilages .ti,ab.) OR (tibiafemoral .ti,ab. AND fibro-cartilages .ti,ab.) OR (tibiafemoral .ti,ab. AND fibro-cartilages .ti,ab.) OR (tibia-femoral .ti,ab. AND fibro-cartilages .ti,ab.) OR (tibi-afemoral .ti,ab. AND semilunar .ti,ab.) OR (tibia-femoral .ti,ab. AND semilunar .ti,ab.) OR (tibiofemoral .ti,ab. AND fibrocartilage .ti,ab.) OR (tibio-femoral .ti,ab. AND fibro-cartilage .ti,ab.) OR (tibiofemoral .ti,ab. AND fibro-cartilage .ti,ab.) OR (tibio-femoral .ti,ab. AND fibro-cartilage .ti,ab.) OR (tibiofemoral .ti,ab. AND fibrocartilagenous .ti,ab.) OR (tibio-femoral .ti,ab. AND fibrocartilagenous .ti,ab.) OR (tibiofemoral .ti,ab. AND fibro-cartilagenous .ti,ab.) OR (tibio-femoral .ti,ab. AND fibro-cartilagenous .ti,ab.) OR (tibiofemoral .ti,ab. AND fibrocartilageous .ti,ab.) OR (tibio-femoral .ti,ab. AND fibrocar-tilageous .ti,ab.) OR (tibiofemoral .ti,ab. AND fibrocartilages .ti,ab.) OR (tibio-femoral .ti,ab. AND fibrocartilages .ti,ab.) OR (tibiofemoral .ti,ab. AND fibro-cartilages .ti,ab.) OR (tibio-femoral .ti,ab. AND fibro-cartilages .ti,ab.) OR (tibiofemoral .ti,ab. AND semilunar .ti,ab.) OR (tibio-femoral .ti,ab. AND semilunar .ti,ab.) OR Semilunar Carti-lage .ti,ab. OR Semilunar Cartilages .ti,ab. OR Semilunar fibrocartilage .ti,ab. OR knee disk .ti,ab.

Component 2: Allograft exp meniscal transplantation/ OR exp allograft/ OR exp allotransplantation/ OR exp Tissue Transplantation / OR Allogeneic implant .ti,ab. OR Allogeneic implants .ti,ab. OR Allogenic implants .ti,ab. OR Allograft .ti,ab. OR Allo-graft .ti,ab. OR Allografted .ti,ab. OR Allo-grafted .ti,ab. OR Allografting .ti,ab. OR Allo-grafting .ti,ab. OR Allograft-ings .ti,ab. OR Allografts .ti,ab. OR Allo-grafts .ti,ab. OR Alloplastic implant .ti,ab. OR Alloplastic implants .ti,ab. OR Allotransplant .ti,ab. OR Allo-transplant .ti,ab. OR Allo-transplantation .ti,ab. OR Allo-transplantation .ti,ab. OR Allotransplantations .ti,ab. OR Allo-transplantations .ti,ab. OR Allotransplanted .ti,ab. OR Allo-transplanted .ti,ab. OR Allotransplanting .ti,ab. OR Allotransplants .ti,ab. OR Allo-transplants .ti,ab. OR Graft .ti,ab. OR Grafted .ti,ab. OR Grafting .ti,ab. OR Graftings .ti,ab. OR Grafts .ti,ab. OR Homo-graft .ti,ab. OR Homograft .ti,ab. OR Homografted .ti,ab. OR Homografting .ti,ab. OR Homo-grafting .ti,ab. OR Homograftings .ti,ab. OR Homografts .ti,ab. OR Homo-grafts .ti,ab. OR Homologous implants .ti,ab. OR Homotransplant .ti,ab. OR Homotransplantation .ti,ab. OR Homo-transplantation .ti,ab. OR Homotransplantations .ti,ab. OR Homo-transplantations .ti,ab. OR Homotransplanted .ti,ab. OR Homo-trans-planted .ti,ab. OR Homotransplants .ti,ab. OR Homo-transplants .ti,ab. OR Transplant .ti,ab. OR Transplantation .ti,ab. OR Transplantations .ti,ab. OR Transplanted .ti,ab. OR Transplanting .ti,ab. OR Transplants .ti,ab.

Component 3: Animal Search filter for animal studies [de Vries 2011]

Chapter 5

110

Sup

ple

men

tary

Tab

le 2

Ind

ivid

ual r

isk

of

bia

s sc

ore

s

Mor

a 20

03K

elly

20

06S

zom

or

2000

Jack

son

1992

Rijk

20

04E

lliot

20

02A

agaa

rd

2003

Cum

min

s 19

97M

cNic

kle

2009

Jian

g 20

12Zw

ierz

chow

ski

2012

Wad

a 19

97E

dwar

ds

1996

Rijk

20

02R

ijk20

06A

agaa

rd

1999

Yam

asak

i 20

08

1se

lect

ion

bias

Was

the

allo

catio

n se

quen

ce a

de-

quat

ely

gene

rate

d an

d ap

plie

d?

??

??

??

??

??

??

??

??

?

2S

elec

tion

bias

Wer

e th

e gr

oups

si

mila

r at b

ase-

line

or w

ere

they

ad

just

ed fo

r con

-fo

unde

rs in

the

anal

ysis

?

??

??

??

??

??

??

??

??

?

3S

elec

tion

bias

Was

the

allo

catio

n to

the

diffe

rent

gr

oups

ade

quat

ely

conc

eale

d?

??

??

??

??

??

??

??

??

?

4Pe

rfor-

man

ce

bias

Wer

e th

e an

imal

s ra

ndom

ly h

ouse

d du

ring

the

expe

r-im

ent?

??

??

??

??

??

??

??

??

?

5Pe

rfor-

man

ce

bias

Wer

e th

e ca

regi

v-er

s an

d/or

inve

s-tig

ator

s bl

inde

d fro

m k

now

ledg

e w

hich

inte

rven

tion

each

ani

mal

re-

ceiv

ed d

urin

g th

e ex

perim

ent?

??

??

??

??

??

??

??

??

?

6D

etec

ti-on

bia

sW

ere

anim

als

sele

cted

at r

an-

dom

for o

utco

me

asse

ssm

ent?

??

??

??

??

??

??

??

??

?

7D

etec

ti-on

bia

sW

as th

e ou

tcom

e as

sess

or b

linde

d??

yes

yes

?ye

sye

sye

sye

s?

yes

yes

?ye

sye

sye

sye

s?


111

8A

ttriti

on

bias

Wer

e in

com

plet

e ou

tcom

e da

ta

adeq

uate

ly a

d-dr

esse

d?

?no

yes

noye

sye

sno

?ye

sye

sye

sye

sye

sye

sye

sno

?

9R

epor

-tin

g bi

asA

re re

ports

of

the

stud

y fre

e of

se

lect

ive

outc

ome

repo

rting

?

yes

yes

yes

?ye

sye

sye

sye

sno

yes

??

yes

?ye

sye

sye

s

10O

ther

Was

the

stud

y ap

pare

ntly

free

of

othe

r pro

blem

s th

at c

ould

resu

lt in

hi

gh ri

sk o

f bia

s?

?no

no?

no?

??

?no

??

no?

??

?

11A

dditi

o-na

lW

as it

sta

ted

that

th

e ex

perim

ent

was

rand

omiz

ed

at a

ny le

vel?

yes

yes

yes

noye

sno

yes

noye

sye

sno

noye

sno

yes

nono

12A

dditi

o-na

lW

as it

sta

ted

that

th

e ex

perim

ent

was

blin

ded

at a

ny

leve

l?

noye

sye

sno

yes

yes

yes

yes

noye

sye

sno

yes

yes

yes

yes

no

CHAPTER 6Systematic reviews of animal studies; missing link in translational research?

Judith van LuijkBrenda BakkerMaroeska RoversMerel Ritskes-HoitingaRob de VriesMarlies Leenaars

PlosOne 9(3), e89981, 2014

Chapter 6

114

Abstract

The methodological quality of animal studies is an important factor hampering the translation of results from animal studies to a clinical setting. Systematic reviews of animal studies may provide a suitable method to assess and thereby improve their methodological quality. The aims of this study were: 1) to evaluate the risk of bias assessment in animal-based systematic reviews, and 2) to study the internal validity of the primary animal studies included in these systematic reviews We systematically searched Pubmed and Embase for SRs of preclinical animal studies published between 2005 and 2012. A total of 91 systematic reviews met our inclusion criteria. The risk of bias was assessed in 48 (52.7%) of these 91 systematic reviews. Thirty-three (36.3%) SRs provided sufficient information to evaluate the internal validity of the included studies. Of the evaluated primary studies, 24.6% was randomized, 14.6% reported blinding of the investigator/caretaker, 23.9% blinded the outcome assessment, and 23.1% reported drop-outs.To improve the translation of animal data to clinical practice, systematic reviews of animal studies are worthwhile, but the internal validity of primary animal studies needs to be improved. Furthermore, risk of bias should be assessed by systematic reviews of animal studies to provide insight into the reliability of the available evidence.

Introduction

The majority of animal experiments is being carried out in the context of preclinical research, e.g. to test safety and efficacy of new treatments to improve healthcare. However, translating animal data to the human situation has been proven to be very challenging. Various factors influence this translation, such as biological differences between species, internal validity, differences in experimental design between animal studies and clinical trials, insufficient reporting, and publication bias [1]. Systematic reviews (SRs) of animal studies have the potential to reduce some of the challenges in the translation of animal data to clinical trials, for example by explicitly assessing the internal validity. SRs attempt to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. SRs of animal studies are still quite rare, but their number appears to be slightly increasing [2-4].However, little is known about the extent to which the available SRs include a risk of bias assessment, in which the internal validity of the included primary animal studies is evaluated. We therefore performed a systematic review of the risk of bias assessment in SRs of animal studies. Subsequently, we studied the internal validity of the individual studies included in these SRs.

SyStematic reviewS of animal StudieS, riSk of BiaS aSSeSSment

115

Materials and methods

Search strategy To find all SRs of animal studies published between 2005 and 2012, the following search strategy was carried out on 28 January 2013. To identify animal studies, the MEDLINE (PubMed platform) and EMBASE (OvidSP platform) databases were searched using the ‘Animal’ filter for PubMed by Hooijmans et al., [5] and the filter for EMBASE by De Vries et al., [6, 7] respectively. Since we were interested in SRs, we used the clinical query for SRs from PubMed, which we have adapted for Embase (see Text S1).

Study selection For the purpose of this study, a review was classified as a SR when at least all of the following items were reported: 1) the term Systematic Review 2) database(s) searched and 3) search terms. Selection was performed by two independent observers and disagreements were resolved through discussion (JvL, BB, ML). Only SRs aiming to inform human healthcare by reviewing a medical drug intervention were included, such as vitamin-based supplementations or stem cells treatment. Medical devices, such as prosthetics and scaffolds, and other types of interventions such as oxygen or heat were excluded. We also excluded SRs that were not written in English or could not be retrieved in full text. When supplementary data were available online, these were obtained.

Scoring procedure of SRsData on both the characteristics and methods used to assess risk of bias in the SRs were extracted by at least two independent reviewers (JvL, BB, ML). In SRs where both animal and human studies were included in the SR, only the animal data were evaluated. Disagreements between reviewers were resolved through discussion and if necessary a third reviewer was consulted.

Assessment risk of bias itemsThe methodology of quality assessment differs between SRs of animal studies [8]. For the purpose of this study we focused on the internal validity of primary studies. Therefore, we defined quality assessment as a risk of bias assessment. To fit this definition, the assessment had to include at least one of the following internal validity items: 1) randomized study design (selection bias), 2) blinding of investigator/caretaker (performance bias), 3) blinding of outcome assessment (detection bias) and 4) mentioning of drop-outs (attrition bias).

SR characteristicsAdditional information on the characteristics of the SRs was extracted: 1) the way in which the risk of bias was taken into account in the SR (e.g. conduct of subgroup

Chapter 6

116

analyses based on quality, exclusion of studies based on quality or a general comment/statement related to the study quality), 2) level of reporting detail on internal validity (e.g. score per item or a summary for quality per study) and 3) research area of the SR.

Data extraction Primary studiesSRs that provided detailed information on the required internal validity items were used to evaluate the internal validity of the included individual studies. Per SR, data were extracted on: total number of included studies and number of studies per item (randomised study design, blinding investigator/caretaker, blinding outcome assessment and drop-outs).

Results

Literature search and SR selectionWe identified 592 potentially eligible articles, of which 91 SRs met our inclusion criteria. Figure 1 shows the number of studies identified at each stage of the selection process. A complete list of the 91 references can be found elsewhere (see Table S2A).

Characteristics of included Systematic ReviewsThe number of published systematic reviews of animal studies increased over the last years from 6 in 2005 and 2, 6 and 12 in 2006, 2008 and 2010 to 32 in 2012, respectively (Figure 2). The 91 SRs included in this review cover a range of research topics. Most reviews (n=38; 41.8%) cover a neurological topic, of which 20 reviews (22.0%) pertained to stroke. The second largest group was on endocrinology (n = 11; 12.1%). Other topics included cardiovascular diseases, orthopaedics, infectious diseases, oncology, pharmacotoxicology, dentistry and gastroenterology. The complete list of topics and number of SRs per topic can also be found elsewhere (see Table S2A).

Risk of bias assessmentRisk of bias assessment in systematic reviewsNearly half of the SRs (n=43; 47.3%) did not assess any of the risk of bias items (figure 3). In 48 reviews (52.7%), one or more of our predefined risk of bias items were assessed. Thirty-three (36.3%) reviews also provided detailed information on the outcome of this assessment per individual study. Figure 3 shows that of the 91 SRs only 3 (3.3%) assessed all 4 internal validity items in their quality assessment. Twenty-two SRs (24.2%) assessed 3 items, of which 17 SRs (18.7%) did not assess drop-outs; the other 5 (5.5%) did not score blinding of the caretaker. Fourteen SRs (15.4%) assessed two items namely randomisation and blinding (of these, 13 SRs assessed blinding of the outcome assessment, in one SR


117

the type of blinding was unclear). Nine SRs (9.9%) assessed only one item, which in all cases was randomisation.

Risk of bias use in SRsOf the 48 SRs that assessed risk of bias of included individual studies, 45 (93.8%) referred to the internal validity of the primary studies in the results, discussion or conclusion section. This means that three SRs did not discuss the outcome of the risk of bias assessment in any way. In most reviews, (n=42; 87.5%) a general comment was made on the quality of the primary studies. In 25 SRs (52.1%), the primary study quality was used as a factor in the meta-analysis (e.g. subgroup analyses) and in three SRs the study quality was used as an exclusion criterion (see Table S2A).

Internal validity of primary studiesThirty-three SRs that provided detailed information on the risk of bias assessment were used to evaluate the internal validity of the included primary studies. These 33 SRs included a total of 2280 primary studies (median 18, range: 2 – 1152 primary studies). Most of these studies were on the subject of stroke or other neurological topics (see Table S2A and S2B).Figure 4 provides an overview of risk of bias scores of the individual animal studies per item (randomisation, blinding of caretaker/investigator, blinding of outcome assessment and drop-outs). As not all reviews scored all four items we evaluated (see figure 3), the number of primary studies varies per item in figure 4. Of the 2280 included primary studies, 562 (24.6%) were randomised. Blinding of the investigator/caretaker was scored for 546 (23.9%) primary studies, of which 80 (14.6%) were actually blinded. Blinding of the outcome assessment was scored for 2220 (97.4%) primary studies, of which 530 (23.9%) were indeed blinded. Drop-outs were scored in only 78 (3.4%) primary studies, of which 18 (23.1%) really did reported drop-outs. One study assessed blinding, without specifying the type of blinding. Therefore, the data of this study were not included in our results (see Table S2A and S2B).

Discussion

Our results show that the assessment of the methodological quality by systematic reviews of animal studies is quite poor. Half of the 91 evaluated SRs did not critically appraise the risk of bias in the included studies. Furthermore, the thirty-three reviews that did assess and report the risk of bias showed that the internal validity of most individual animal studies is poor as well. Therefore, there is a real risk that the outcomes of both, the individual studies and the subsequent SRs of these studies are biased.

Chapter 6

118

Our findings that the methodological quality of SRs is poor are in line with findings by Peters et al., who identified a number of deficiencies in the conduct and reporting of SRs and meta-analyses of animal studies. Peters et al. suggest that initiatives to improve the conduct and reporting of primary animal studies and of SRs of animal studies should go hand-in-hand [2]. Poor internal validity of animal studies has previously been demonstrated by Kilkenny et al. Of the 271 publications of animal studies they surveyed, only 13% had been randomised and 14% had blinded the outcome assessment [9]. We found slightly higher percentages, namely 24.6% randomisation and 23.9% for blinding. These higher percentages may be explained by two factors. First, our study contains a relative high number of stroke studies. Over the last decades, researchers in the field of stroke have been actively working on recommendations and guidelines for preclinical research in order to improve effective translation [10]. Second, over the last years, general awareness of the need for better reporting of animal studies has been steadily increasing.Although both the methodological quality of animal SRs and the internal validity of primary animal studies have been investigated before, they were studied separately by different research groups and more recent SRs of animal studies have not yet been taken into account. A major strength of our study is, therefore, that by updating and combining these evaluations in one study, we were able to gain more in-depth insight into the current state and level of available preclinical evidence.Some potential limitations should also be discussed. First, we have restricted ourselves to one type of SR, namely SRs of animal-based drug-intervention studies, which might hamper the generalization of our results to other SRs of animal studies. Although we excluded SRs of animal studies that are not directly related to clinical research, we consider it likely that the latter type of SRs are of lower methodological quality, as the SR methodology and measures to safeguard internal validity may not be as well established as in fields closely related to clinical research. Therefore, our restriction might have caused an overestimation of the methodological quality of SRs and the internal validity of primary studies in general. Second, it cannot be ruled out that a small proportion of the SRs did not assess certain internal validity items, because the experimental design of the included individual studies did not allow a risk of bias assessment (e.g. due to a lack of (independent) control groups). Third, some individual studies may have been less subject to bias than the SRs estimated due to a lack of (adequate) reporting of the randomization and/or blinding methods they actually used. Fourth, we have not investigated whether the SRs assessed the adequacy of the method of randomisation or blinding. Inadequate randomization and blinding in animal studies can cause overestimation of the effect size [11, 12] and thus may falsely inform other preclinical research or clinical trials. In principle, this means that even randomised studies could be subject to bias, namely when the randomisation method was not adequate for the study design. Similarly, some SRs assessed blinding but did not specify the level or


119

type of blinding. As long as the reporting of animal studies remains poor, however, these limitations are hardly avoidable.Adequate internal validity of animal studies has been described as one of the key factors for improving the translation of results to human studies [1]. SRs can be a useful method to evaluate and analyse (the quality of) available evidence. As previously stated, SRs of animal studies could profit from the use of guidelines [2]. Currently, there is no standard procedure available for conducting SRs of animal studies [8, 13]. This could be one of the reasons why so many animal-based SRs did not assess any of the risk of bias items. Valuable lessons can be learned here from the guidelines used in clinical research, such as the CONSORT and PRISMA statements. Guidelines for planning, conducting and reporting primary animal studies are already available [14, 15]. Even though the ARRIVE guidelines are adopted by many journals, the effect on publication standards of animal studies is still very minimal. Therefore, effective implementation of endorsement of these guidelines requires more attention [16]. As does education on this matter. A good education strategy regarding both the internal validity of animal studies and the SR methodology can help raise awareness for the current state of potentially biased animal data. Authors, as well as reviewers and editors, need to be aware of the potential risk of this bias in animal studies and how it can adequately be reduced to eventually produce high-quality research with reliable results for human healthcare.

Conclusions

To improve the translation of animal data to clinical practice, systematic reviews of animal studies are worthwhile, but the internal validity of the individual animal studies needs to be improved. Furthermore, risk of bias should be assessed by SRs of animal studies to provide insight into the reliability of available evidence.

Acknowledgments

The authors would like to acknowledge Alice Tillema, Nieky van Veggel, Jan Willem Weenink and Joppe Tra for their work in optimizing and conducting the search strategy.

Chapter 6

120

1. Hooijmans, C.R. and M. Ritskes-

Hoitinga, Progress in using systematic

reviews of animal studies to improve

translational research. PLoS Med, 2013.

10(7): p. e1001482.

2. Peters, J.L., et al., A systematic review

of systematic reviews and meta-analyses

of animal experiments with guidelines for

reporting. J Environ Sci Health B, 2006.

41(7): p. 1245-58.

3. Mignini, L.E. and K.S. Khan,

Methodological quality of systematic

reviews of animal studies: a survey of

reviews of basic research. BMC Med Res

Methodol, 2006. 6: p. 10.

4. Korevaar, D.A., L. Hooft, and G. ter Riet,

Systematic reviews and meta-analyses

of preclinical studies: publication bias in

laboratory animal experiments. Lab Anim,

2011. 45(4): p. 225-30.

5. Hooijmans, C.R., et al., Enhancing



experimentation in PubMed. Lab Anim,

2010. 44(3): p. 170-5.

6. de Vries, R.B., et al., A search filter for

increasing the retrieval of animal studies in

Embase. Lab Anim, 2011. 45(4): p. 268-

70.

7. de Vries, R.B., et al., Letter to the Editor.

Lab Anim, 2013.

8. Krauth, D., T.J. Woodruff, and L. Bero,

Instruments for assessing risk of bias and

other methodological criteria of published

animal studies: a systematic review.

Environ Health Perspect, 2013. 121(9): p.

985-92.

9. Kilkenny, C., et al., Survey of the quality

of experimental design, statistical analysis

and reporting of research using animals.

PLoS ONE, 2009. 4(11): p. e7824.

10. Stroke Therapy Academic Industry,

R., Recommendations for standards

regarding preclinical neuroprotective and

restorative drug development. Stroke,

1999. 30(12): p. 2752-8.

11. Macleod, M.R., et al., Good laboratory

practice: preventing introduction of bias at

the bench. Stroke, 2009. 40(3): p. e50-2.

12. Bebarta, V., D. Luyten, and K. Heard,

Emergency medicine animal research:

does use of randomization and blinding

affect the results? Acad Emerg Med,

2003. 10(6): p. 684-7.

13. Henderson, V.C., et al., Threats to validity

in the design and conduct of preclinical

efficacy studies: a systematic review of

guidelines for in vivo animal experiments.

PLoS Med, 2013. 10(7): p. e1001489.


research reporting: the ARRIVE guidelines

for reporting animal research. PLoS Biol,

2010. 8(6): p. e1000412.





the Three Rs, and to make systematic

reviews more feasible. Altern Lab Anim,

2010. 38(2): p. 167-82.

16. Baker, D., et al., Two Years Later:

Journals Are Not Yet Enforcing the ARRIVE

Guidelines on Reporting Standards for

Pre-Clinical Animal Studies. PLoS Biol,

2014. 12(1): p. e1001756.

References


121

Chapter 6

122

Fig

ure

1 Fl

ow

dia

gra

m t

he

syst

emat

ic r

evie

w li

tera

ture

sea

rch

res

ults

. A t

ota

l of

91 s

yste

mat

ic r

evie

ws

of

inte

rven

tion

anim

al s

tud

ies

wer

e in

clud

ed.

IDE

NTI

FIC

ATI

ON

SC

RE

EN

ING

ELI

GIB

ILIT

Y

INC

LUD

ED

Rec

ord

s id

entifi

ed t

hro

ugh

Pub

Med

an

d E

MB

AS

E s

earc

hin

g

(n =

19,

169)

Rec

ord

s sc

reen

ed

title

/ab

stra

ct

(n =

10,

764)

Rec

ord

s ex

clud

ed

(n =

10,

172)

N

ot

a S

R /

No

t a

SR

on

anim

al s

tud

ies

Full-

text

art

icle

s as

sess

ed f

or

el

igib

ility

(n

= 5

92)

Stu

die

s in

clud

ed in

q

ualit

ativ

e sy

nth

esis

(n

= 9

1)

Rec

ord

s af

ter

dup

licat

es r

emo

ved

(n

=10

,764

)

Full-

text

art

icle

s ex

clud

ed f

or

o

ne o

r m

ore

of

the

reas

ons

bel

ow

(n

= 5

01)

- N

ot

full

text

ava

ilab

le

- N

ot

in E

nglis

h

- N

ot

a S

R

- N

ot

a S

R o

f an

imal

stu

die

s -

No

t a

SR

of

inte

rven

tion

stud

ies


123

Figure 2 Number (n) of published SRs of intervention animal studies per publication year (2005-2012).

Figure 3 Percentages of SRs per number of internal validity item scored. Zero items by 47.3%, one item all randomisation, two items randomisation and one level of blinding, three items randomisation, blinding of caretaker/investigator and blinding of outcome assessment or randomisation, one level of blinding and drop-outs) and all four items by 3.3%.

Number of items scored

SR

s (%

)

00 1 2 3 4

10

20

30

40

50

Num

ber o

f SR

s

0

5

10

15

20

25

30

2005 2006 2007 2008 2009 2010 2011 2012

Year

35

Chapter 6

124

Figure 4 Percentage of primary animal studies assessed per validity item; yes or no.

Randomisation

Blinding handler

Blinding outcome

Drop-outs

0% 20% 40% 60% 80% 100%

Yes No

n=2280

n=546

n=2220

n=78


125

Chapter 6

126

Appendix chapter 6:

Supplementary file 1 Systematic Review filters

Pubmed (systematic review [ti] OR meta-analysis [pt] OR meta-analysis [ti] OR systematic literature review [ti] OR (systematic review [tiab] AND review [pt]) OR consensus development conference [pt] OR practice guideline [pt] OR cochrane database syst rev [ta] OR acp journal club [ta] OR health technol assess [ta] OR evid rep technol assess summ [ta]) OR ((evidence based[ti] OR evidence-based medicine [mh] OR best practice* [ti] OR evidence synthesis [tiab])AND (review [pt] OR diseases category[mh] OR behavior and behavior mechanisms [mh] OR therapeutics [mh] OR evaluation studies[pt] OR validation studies[pt] OR guideline [pt])) OR ((systematic [tw] OR systematically [tw] OR critical [tiab] OR (study selection [tw]) OR (predetermined [tw] OR inclusion [tw] AND criteri* [tw]) OR exclusion criteri* [tw] OR main outcome measures [tw] OR standard of care [tw] OR standards of care [tw]) AND (survey [tiab] OR surveys [tiab] OR overview* [tw] OR review [tiab] OR reviews [tiab] OR search* [tw] OR handsearch [tw] OR analysis [tiab] OR critique [tiab] OR appraisal [tw] OR (reduction [tw]AND (risk [mh] OR risk [tw]) AND (death OR recurrence))) AND (literature [tiab] OR articles [tiab] OR publications [tiab] OR publication [tiab] OR bibliography [tiab] OR bibliographies [tiab] OR published [tiab] OR unpublished [tw] OR citation [tw] OR citations [tw] OR database [tiab] OR internet [tiab] OR textbooks [tiab] OR references [tw] OR scales [tw] OR papers [tw] OR datasets [tw] OR trials [tiab] OR meta-analy* [tw] OR (clinical [tiab] AND studies [tiab]) OR treatment outcome [mh] OR treatment outcome [tw])) NOT (letter [pt] OR newspaper article [pt] OR comment [pt])

Embase (systematic review.ti. OR meta-analysis.pt. OR meta-analysis.ti. OR systematic literature review.ti. OR (systematic review.ti,ab. AND review.pt.) OR consensus development conference.pt. OR practice guideline.pt. OR cochrane database syst rev.ja. OR acp journal club.ja. OR health technol assess.ja. OR evid rep technol assess summ.ja.) OR ((evidence based.ti. OR evidence-based medicine/ OR best practice*.ti. OR evidence synthesis.ti,ab.) AND (review.pt. OR diseases category/ OR exp behavior and behavior mechanisms/ OR exp therapeutics/ OR evaluation studies.pt. OR validation studies.pt. OR guideline.pt.)) OR ((systematic.tw. OR systematically.tw. OR critical.ti,ab. OR (study selection.tw.) OR (predetermined.tw. OR inclusion.tw. AND criteri*.tw.) OR exclusion criteri*.tw. OR main outcome measures.tw. OR standard of care.tw. OR standards of care.tw.) AND (survey.ti,ab. OR surveys.ti,ab. OR overview*.tw. OR review.ti,ab. OR reviews.ti,ab. OR search*.tw. OR handsearch.tw. OR analysis.ti,ab. OR critique.ti,ab. OR appraisal.tw. OR (reduction.tw.AND (exp risk/ OR risk.tw.) AND (death OR recurrence))) AND (literature.ti,ab. OR articles.ti,ab. OR publications.ti,ab. OR publication.ti,ab. OR bibliography.ti,ab. OR bibliographies.ti,ab. OR published.ti,ab. OR unpublished.tw. OR citation.tw. OR citations.tw. OR database.ti,ab. OR internet.ti,ab. OR textbooks.ti,ab. OR references.tw. OR scales.tw. OR papers.tw. OR datasets.tw. OR trials.ti,ab. OR meta-analy*.tw. OR (clinical.ti,ab. AND studies.ti,ab.) OR exp treatment outcome/ OR treatment outcome.tw.)) NOT (letter.pt. OR newspaper article.pt. OR comment.pt.)


127

Supplementary file 2 Table 1 Characteristics SRsAssessment of internal validity: 1) Randomisation assessed, 2) Blinding caretaker/investigator assessed, 3) Blinding outcome assessment assessed, 4) Dropouts assessed. Use of internal validity: 5) General comment, 6) Used in meta-analysis (e.g. subgroup), 7) Exclusion criterion. Additional information: 8) Included in analyses of primary studies, 9) Field of research.

Reference 1 2 3 4 5 6 7 8 9

[1] * * * * * Endocrinology

[2] * * * * * Neurology

[3] * * * * * Oncology

[4] Endocrinology

[5] * * * * * * Stroke

[6] * * * * * * * Stroke

[7] * * * * Neurology

[8] Cardiovascular Research

[9] * * * * * Orthopeadics

[10] * * * * * * Stroke

[11] Neurology

[12] * * Pharmacotoxicology

[13] Oncology

[14] Infectious Diseases

[15] * * * * * * Dentistry

[16] * * * * * Stroke


[18] * * Dentistry

[19] Neurology

[20] Pharmacotoxicology

[21] Oncology

[22] * Neurology

[23] * * * * * Stroke

[24] * * * * * Neurology

[25] * * * * * Neurology

[26] * * * * * Stroke

[27] * Stroke

[28] * * Orthopeadics

[29] * * Stroke

[30] Pharmacology

[31] Endocrinology

[32] Endocrinology


[34] * * Orthopeadics

[35] * * * * * * * Gastroenterology

[36] * * * * * Neurology

Chapter 6

128

[37] * * * Infectious Diseases

[38] * * * * * Stroke

[39] Dentistry

[40] * * * * * Neurology

[41] Orthopeadics

[42] * * * Oncology

[43] Oncology

[44] Psychiatry

[45] Neurology

[46] Neurology

[47] * * * * * * Stroke

[48] Endocrinology

[49] Endocrinology


[51] * * * * * * Cardiovascular Research

[52] * * * * Stroke

[53] Neurology

[54] * * * * * Stroke

[55] * * * * * * Stroke

[56] * * * * * * Stroke

[57] * * Cardiovascular Research

[58] Orthopeadics

[59] * * * * * Endocrinology

[60] Orthopeadics

[61] * * Gastroenterology

[62] * * * * Stroke

[63] Ophthalmology



[66] * Cardiovascular Research

[67] Surgery

[68] Endocrinology

[69] * * * * * Neurology

[70] Gastroenterology

[71] Endocrinology

[72] * * * * * * Stroke

[73] * * * * * * Stroke

[74] * * * Endocrinology

[75] Neurology

[76] Neurology


129

[77] Neurology

[78] Pulmonology

[79] * * * * * Cardiovascular Research


[81] * * * * * Neurology

[82] Otolaryngology

[83] Nephrology

[84] * Pulmonology

[85]1 * ? ? * Cardiovascular Research

[86] * * * * * * Stroke

[87] * * * Stroke

[88] * * * * Stroke

[89] Surgery

[90] * * * * * * Endocrinology

[91] * * * * * * Neurology

n= 48 38 20 8 442 25 3 33

Footnotes: 1 Level of blinding was unclear2 In two reviews a comment was made about internal validity without a risk of bias

assessment

Chapter 6

130

Supplementary file 2 Table 2 Primary animal studies1) Total number of animals included, 2) Number of animals randomized, 3) Number of animals blinded caretaker/investigator, 4) Number of animals blinded outcome assessment, 5) Number of animals drop-outs. ? = number unclear, - = not scored

Reference 1 2 3 4 5

[1] 11 5 0 - 1

[2] 62 45 33 - -

[5] 17 1 5 2 -

[6] 18 9 12 8 9

[7] 85 21 21 - -

[9] 6 3 2 - 0

[10] 14 6 7 5 -

[15] 2 2 2 - 2

[16] 19 16 16 - -

[24] 12 2 5 - -

[25] 18 12 7 - -

[26] 27 2 1 - -

[35] 13 10 7 2 4

[36] 17 13 1 - 1

[37] 2 0 - - -

[40] 14 7 4 0 -

[47] 117 54 57 22 -

[51] 20 15 2 1 -

[52] 9 4 4 - -

[54] 25 6 7 0 -

[55] 13 4 4 2 -

[56] 29 6 2 1 -

[59] 4 0 0 0 -

[69] 254 40 38 - -

[72] 105 46 28 16 -

[73] 18 12 13 1 -

[74] 13 12 - - -

[79] 52 38 22 5 -

[81] 1152 108 184 - -

[85]3 45 12 ? ? -

[86] 6 2 1 0 -

[90] 11 11 0 0 1

[91] 70 38 45 15 -


131

Footnotes:3 Level of blinding unclear (not specified). Of the 45 primary studies: 15 were blinded

(33.3%), 14 not blinded (31.1%) and in 16 studies blinding was not mentioned (35.6%).

Chapter 6

132

1. Ainge, H., et al., A systematic review on

animal models of maternal high fat feeding

and offspring glycaemic control. Int J

Obes (Lond), 2011. 35(3): p. 325-35.

2. Akhtar, A.Z., J.J. Pippin, and C.B.

Sandusky, Animal studies in spinal

cord injury: A systematic review of

methylprednisolone. ATLA Alternatives to

Laboratory Animals, 2009. 37(1): p. 43-

62.

3. Amarasingh, S., M.R. Macleod, and I.R.

Whittle, What is the translational efficacy

of chemotherapeutic drug research in

neuro-oncology? A systematic review and

meta-analysis of the efficacy of BCNU

and CCNU in animal models of glioma. J

Neurooncol, 2009. 91(2): p. 117-25.

4. Auvinen, H.E., et al., Effects of high fat diet

on the Basal activity of the hypothalamus-

pituitary-adrenal axis in mice: a systematic

review. Horm Metab Res, 2011. 43(13): p.

899-906.

5. Banwell, V., E.S. Sena, and M.R. Macleod,

Systematic review and stratified meta-

analysis of the efficacy of interleukin-1

receptor antagonist in animal models of

stroke. J Stroke Cerebrovasc Dis, 2009.

18(4): p. 269-76.

6. Baryan, H.K., et al., Systematic review

and meta-analysis of the efficacy of statins

in experimental stroke. Int J Stroke, 2012.

7(2): p. 150-6.

7. Benatar, M., Lost in translation: treatment

trials in the SOD1 mouse and in human

ALS. Neurobiol Dis, 2007. 26(1): p. 1-13.

8. Berthelsen, L.O., A.T. Kristensen, and M.

Tranholm, Animal models of DIC and their

relevance to human DIC: A systematic

review. Thrombosis research, 2011. 128

(2): p. 103-116.

9. Borhanuddin, B., N.F. Mohd Fozi, and

I. Naina Mohamed, Vitamin e and the

healing of bone fracture: the current state

of evidence. Evid Based Complement

Alternat Med, 2012. 2012: p. 684510.

10. Bustamante, A., et al., Citicoline in pre-

clinical animal models of stroke: a meta-

analysis shows the optimal neuroprotective

profile and the missing steps for jumping

into a stroke clinical trial. J Neurochem,

2012. 123(2): p. 217-25.

11. Cadotte, D.W. and M.G. Fehlings, Spinal

cord injury: a systematic review of current

treatment options. Clin Orthop Relat Res,

2011. 469(3): p. 732-41.

12. Cave, G. and M. Harvey, Intravenous

lipid emulsion as antidote beyond local

anesthetic toxicity: a systematic review.

Acad Emerg Med, 2009. 16(9): p. 815-24.

13. Corpet, D.E. and F. Pierre, How good

are rodent models of carcinogenesis

in predicting efficacy in humans? A

systematic review and meta-analysis of

colon chemoprevention in rats, mice

and men. Eur J Cancer, 2005. 41(13): p.

1911-22.

14. Dare, A.J., et al., A systematic review of

experimental treatments for mitochondrial

dysfunction in sepsis and multiple organ

dysfunction syndrome. Free Radic Biol

Med, 2009. 47(11): p. 1517-25.

15. Dhingra, K. and K.L. Vandana, Prophylactic

vaccination against periodontal disease: a

systematic review of preclinical studies. J

Periodontol, 2010. 81(11): p. 1529-46.

Systematic Review References


133

16. England, T.J., C.L. Gibson, and P.M.W.

Bath, Granulocyte-colony stimulating

factor in experimental stroke and its effects

on infarct size and functional outcome:

A systematic review. Brain Research

Reviews, 2009. 62(1): p. 71-82.

17. Eyers, S., et al., The effect on mortality of

antipyretics in the treatment of influenza

infection: systematic review and meta-

analysis. Journal of the Royal Society of

Medicine, 2010. 103(10): p. 403-411.

18. Faggion, C.M., Jr., et al., Comparison

of the effects of treatment of peri-implant

infection in animal and human studies:

systematic review and meta-analysis. Clin

Oral Implants Res, 2010. 21(2): p. 137-47.

19. Farooqi, N., B. Gran, and C.

Constantinescu, Are current disease-

modifying therapeutics in multiple

sclerosis justified on the basis of

studies in experimental autoimmune

encephalomyelitis? Journal of

Neurochemistry, 2010. 115(4): p. 829-

844.

20. Fond, G., et al., Antipsychotic drugs:

pro-cancer or anti-cancer? A systematic

review. Med Hypotheses, 2012. 79(1): p.

38-42.

21. Fritz, H., et al., Vitamin A and retinoid

derivatives for lung cancer: a systematic

review and meta analysis. PLoS ONE,

2011. 6(6): p. e21107.

22. Frogley, C., et al., A systematic review of

the evidence of clozapine’s anti-aggressive

effects. Int J Neuropsychopharmacol,

2012. 15(9): p. 1351-71.

23. Garcia-Bonilla, L., et al., Evidence for the

efficacy of statins in animal stroke models:

a meta-analysis. J Neurochem, 2012.

24. Gibson, C. and S. Murphy, Benefits of

histone deacetylase inhibitors for acute

brain injury: A systematic review of animal

studies. Journal of Neurochemistry, 2010.

115(4): p. 806-813.

25. Gibson, C.L., et al., Progesterone for the

treatment of experimental brain injury; a

systematic review. Brain, 2008. 131(2): p.

318-328.

26. Gibson, C.L., et al., Estrogens and

experimental ischemic stroke: a

systematic review. J Cereb Blood Flow

Metab, 2006. 26(9): p. 1103-13.

27. Gibson, C.L., A.N. Murphy, and S.P.

Murphy, Stroke outcome in the ketogenic

state--a systematic review of the animal

data. J Neurochem, 2012. 123 Suppl 2: p.

52-7.

28. Gielkens, P.F., et al., Is there evidence

that barrier membranes prevent bone

resorption in autologous bone grafts

during the healing period? A systematic

review. Int J Oral Maxillofac Implants,

2007. 22(3): p. 390-8.

29. Harston, G., et al., The contribution

of L-arginine to the neurotoxicity of

recombinant tissue plasminogen activator

following cerebral ischemia: A review of

rtPA neurotoxicity. Journal of Cerebral

Blood Flow and Metabolism, 2010.

30(11): p. 1804-1816.

30. Hasani-Ranjbar, S., B. Larijani, and M.

Abdollah, A systematic review of iranian

medicinal plants useful in diabetes

Chapter 6

134

mellitus. Archives of Medical Science,

2008. 4(3): p. 285-292.

31. Hasani-Ranjbar, S., B. Larijani, and

M. Abdollahi, A systematic review of the

potential herbal sources of future drugs

effective in oxidant-related diseases.

Inflamm Allergy Drug Targets, 2009. 8(1):

p. 2-10.

32. Hasani-Ranjbar, S., et al., A systematic

review of the efficacy and safety of herbal

medicines used in the treatment of obesity.

World J Gastroenterol, 2009. 15(25): p.

3073-85.

33. Heard, K., N.R. Cleveland, and S. Krier,

Benzodiazepines and antipsychotic

medications for treatment of acute cocaine

toxicity in animal models--a systematic

review and meta-analysis. Hum Exp

Toxicol, 2011. 30(11): p. 1849-54.

34. Hirsch, B.P., et al., The effect of therapies

for osteoporosis on spine fusion: a

systematic review. Spine J, 2012.

35. Hooijmans, C.R., et al., The effects of

probiotic supplementation on experimental

acute pancreatitis: a systematic review

and meta-analysis. PLoS One, 2012.

7(11): p. e48811.






Alzheimers Dis, 2012. 28(1): p. 191-209.

37. Jefferies, S., et al., Systematic review and

meta-analysis of the effects of antipyretic

medications on mortality in Streptococcus

pneumoniae infections. Postgrad Med J,

2012. 88(1035): p. 21-7.

38. Jerndal, M., et al., A systematic review

and meta-analysis of erythropoietin in

experimental stroke. Journal of Cerebral

Blood Flow and Metabolism, 2010. 30(5):

p. 961-968.

39. Jung, R.E., D.S. Thoma, and C.H.

Hammerle, Assessment of the potential of

growth factors for localized alveolar ridge

augmentation: a systematic review. J Clin

Periodontol, 2008. 35(8 Suppl): p. 255-81.

40. Ker, K., P. Perel, and K. Blackhall, Beta-

2 receptor antagonists for traumatic brain

injury: A systematic review of controlled

trials in animal models. CNS Neuroscience

and Therapeutics, 2009. 15(1): p. 52-64.

41. Khojasteh, A., et al., Effects of different

growth factors and carriers on bone

regeneration: a systematic review. Oral

Surg Oral Med Oral Pathol Oral Radiol,

2012.

42. Kienle, G.S., et al., Viscum album L.

extracts in breast and gynaecological

cancers: a systematic review of clinical

and preclinical research. J Exp Clin

Cancer Res, 2009. 28: p. 79.

43. Kienle, G.S., R. Grugel, and H. Kiene,

Safety of higher dosages of Viscum album

L. in animals and humans--systematic

review of immune changes and safety

parameters. BMC Complement Altern

Med, 2011. 11: p. 72.

44. Kontis, D. and E. Theochari, Dopamine

in anorexia nervosa: a systematic review.

Behav Pharmacol, 2012. 23(5-6): p. 496-

515.

45. Kwon, B.K., et al., A systematic

review of non-invasive pharmacologic

neuroprotective treatments for acute


135

spinal cord injury. J Neurotrauma, 2011.

28(8): p. 1545-88.

46. Kwon, B.K., et al., A systematic review

of directly applied biologic therapies for

acute spinal cord injury. J Neurotrauma,

2011. 28(8): p. 1589-610.

47. Lees, J.S., et al., Stem cell-based therapy

for experimental stroke: a systematic

review and meta-analysis. Int J Stroke,

2012. 7(7): p. 582-8.

48. Lerchbaum, E. and B. Obermayer-

Pietsch, Vitamin D and fertility: a systematic

review. Eur J Endocrinol, 2012. 166(5): p.

765-78.

49. Lerchbaum, E. and B. Obermayer-

Pietsch, Mechanisms in endocrinology -

Vitamin D and fertility: A systematic review.

European Journal of Endocrinology, 2012.

166 (5): p. 765-778.

50. Li, Y., et al., The effect of heparin

administration in animal models of sepsis:

a prospective study in Escherichia coli-

challenged mice and a systematic review

and metaregression analysis of published

studies. Crit Care Med, 2011. 39(5): p.

1104-12.

51. Lim, W.Y., C.M. Messow, and C. Berry,

Cyclosporin variably and inconsistently

reduces infarct size in experimental

models of reperfused myocardial

infarction: a systematic review and meta-

analysis. Br J Pharmacol, 2012. 165(7): p.

2034-43.

52. Liu, J., et al., Systematic Review and Meta-

Analysis of the Efficacy of Sphingosine-

1-Phosphate (S1P) Receptor Agonist

FTY720 (Fingolimod) in Animal Models of

Stroke. Int J Neurosci, 2012.

53. Loef, M., G.N. Schrauzer, and H. Walach,

Selenium and Alzheimer’s disease: a

systematic review. J Alzheimers Dis, 2011.

26(1): p. 81-104.

54. MacDougall, N.J. and K.W. Muir,

Hyperglycaemia and infarct size in

animal models of middle cerebral artery

occlusion: systematic review and meta-

analysis. J Cereb Blood Flow Metab,

2011. 31(3): p. 807-18.

55. Macleod, M.R., et al., Systematic review

and metaanalysis of the efficacy of FK506

in experimental stroke. Journal of Cerebral

Blood Flow and Metabolism, 2005. 25(6):

p. 713-721.

56. Macleod, M.R., et al., Systematic review

and meta-analysis of the efficacy of

melatonin in experimental stroke. Journal

of Pineal Research, 2005. 38(1): p. 35-41.

57. Matthan, N.R., et al., A systematic

review and meta-analysis of the impact

of omega-3 fatty acids on selected

arrhythmia outcomes in animal models.

Metabolism, 2005. 54(12): p. 1557-65.

58. Mehrkens, A., et al., Tissue engineering

approaches to degenerative disc disease-

-a meta-analysis of controlled animal trials.

Osteoarthritis Cartilage, 2012. 20(11): p.

1316-25.

59. Muhlhausler, B.S., R.A. Gibson, and M.

Makrides, The effect of maternal omega-3

long-chain polyunsaturated fatty acid

(n-3 LCPUFA) supplementation during

pregnancy and/or lactation on body fat

mass in the offspring: a systematic review

of animal studies. Prostaglandins Leukot

Essent Fatty Acids, 2011. 85(2): p. 83-8.

Chapter 6

136

60. Naina Mohamed, I., et al., Vitamin e and

bone structural changes: an evidence-

based review. Evid Based Complement

Alternat Med, 2012. 2012: p. 250584.

61. Nicholson, I., A.M. Dalzell, and W. El-

Matary, Vitamin D as a therapy for colitis: a

systematic review. J Crohns Colitis, 2012.

6(4): p. 405-11.

62. O’Collins, V.E., et al., Preclinical drug

evaluation for combination therapy in acute

stroke using systematic review, meta-

analysis, and subsequent experimental

testing. J Cereb Blood Flow Metab, 2011.

31(3): p. 962-75.

63. Papathanassiou, M., et al., Vascular

Endothelial Growth Factor Inhibitors for

Treatment of Corneal Neovascularization:

A Meta-Analysis. Cornea, 2012.

64. Percie du Sert, N., et al., Cisplatin-

induced emesis: systematic review and

meta-analysis of the ferret model and

the effects of 5-HT receptor antagonists.

Cancer Chemother Pharmacol, 2011.

67(3): p. 667-86.

65. Perlroth, J., et al., Adjunctive use

of rifampin for the treatment of

Staphylococcus aureus infections: A

systematic review of the literature. Archives

of Internal Medicine, 2008. 168(8): p. 805-

819.

66. Pilcher, J., et al., The effect of hyperoxia

following cardiac arrest - A systematic

review and meta-analysis of animal trials.

Resuscitation, 2012. 83(4): p. 417-22.

67. Pommergaard, H.C., M.P. Achiam, and

J. Rosenberg, External coating of colonic

anastomoses: a systematic review. Int J

Colorectal Dis, 2012. 27(10): p. 1247-58.

68. Ranasinghe, P., et al., Efficacy and

safety of ‘true’ cinnamon (Cinnamomum

zeylanicum) as a pharmaceutical agent in

diabetes: a systematic review and meta-

analysis. Diabet Med, 2012. 29(12): p.

1480-92.

69. Rooke, E.D., et al., Dopamine agonists

in animal models of Parkinson’s disease:

a systematic review and meta-analysis.

Parkinsonism Relat Disord, 2011. 17(5):

p. 313-20.

70. Rossignol, D.A., Hyperbaric oxygen

treatment for inflammatory bowel disease:

a systematic review and analysis. Med

Gas Res, 2012. 2(1): p. 6.

71. Rumore, M.M. and K.S. Kim, Potential

role of salicylates in type 2 diabetes. Ann

Pharmacother, 2010. 44(7-8): p. 1207-21.

72. Sena, E., et al., Systematic review and

meta-analysis of the efficacy of tirilazad in

experimental stroke. Stroke, 2007. 38(2):

p. 388-394.

73. Sena, E.S., et al., Factors affecting the

apparent efficacy and safety of tissue

plasminogen activator in thrombotic

occlusion models of stroke: systematic

review and meta-analysis. J Cereb Blood

Flow Metab, 2010. 30(12): p. 1905-13.

74. Sun, G., et al., Adoptive infusion of

tolerogenic dendritic cells prolongs the

survival of pancreatic islet allografts:

a systematic review of 13 mouse and

rat studies. PLoS One, 2012. 7(12): p.

e52096.

75. Tetzlaff, W., et al., A systematic review of

cellular transplantation therapies for spinal

cord injury. J Neurotrauma, 2011. 28(8): p.

1611-82.


137

76. Trevitt, C.R. and J. Collinge, A

systematic review of prion therapeutics in

experimental models. Brain, 2006. 129(9):

p. 2241-2265.

77. Tsang, H. and T. Ho, A systematic review

on the anxiolytic effects of aromatherapy

on rodents under experimentally

induced anxiety models. Reviews in the

neurosciences, 2010. 21(2): p. 141-152.

78. Tuinman, P.R., et al., Nebulized

anticoagulants for acute lung injury - a

systematic review of preclinical and

clinical investigations. Crit Care, 2012.

16(2): p. R70.

79. van der Spoel, T.I., et al., Human

relevance of pre-clinical studies in stem

cell therapy: systematic review and

meta-analysis of large animal models of

ischaemic heart disease. Cardiovasc Res,

2011. 91(4): p. 649-58.

80. Vang, O., et al., What is new for an

old molecule? Systematic review

and recommendations on the use of

resveratrol. PLoS ONE, 2011. 6 (6): p.

e19881.

81. Vesterinen, H.M., et al., Improving the

translational hit of experimental treatments

in multiple sclerosis. Mult Scler, 2010.

16(9): p. 1044-55.

82. Waissbluth, S., J. Pitaro, and S.J. Daniel,

Gene therapy for cisplatin-induced

ototoxicity: a systematic review of in vitro

and experimental animal studies. Otol

Neurotol, 2012. 33(3): p. 302-10.

83. Wang, Y., et al., Systematic Review

and Meta-analysis of MSC Therapy for

Impaired Renal Function in Small Animal

Models. Nephrology (Carlton), 2012.

84. Warner, D. and S.E. Brietzke, Mitomycin

C and airway surgery: How well does it

work? Otolaryngology Head and Neck

Surgery, 2008. 138(6): p. 700-709.

85. Weigl, M., et al., A systematic review

of currently available pharmacological

neuroprotective agents as a sole

intervention before anticipated or induced

cardiac arrest. Resuscitation, 2005. 65(1):

p. 21-39.

86. Wheble, P.C.R., E.S. Sena, and M.R.

Macleod, A systematic review and meta-

analysis of the efficacy of piracetam and

piracetam-like compounds in experimental

stroke. Cerebrovascular Diseases, 2008.

25(1-2): p. 5-11.

87. White, A. and A. Murphy, Administration

of thiazolidinediones for neuroprotection in

ischemic stroke: A pre-clinical systematic

review. Journal of Neurochemistry, 2010.

115(4): p. 845-853.

88. Willmot, M., et al., Nitric oxide synthase

inhibitors in experimental ischemic stroke

and their effects on infarct size and

cerebral blood flow: A systematic review.

Free Radical Biology and Medicine, 2005.

39(3): p. 412-425.

89. Yiannakopoulou, E., et al.,

Pharmacological modulation of oxidative

stress response in minimally invasive

surgery: systematic review. Surg Laparosc

Endosc Percutan Tech, 2012. 22(3): p.

200-4.

90. Zhang, J., et al., Systematic review of

the renal protective effect of Astragalus

membranaceus (root) on diabetic

nephropathy in animal models. Journal

Chapter 6

138

of Ethnopharmacology, 2009. 126(2): p.

189-196.

91. Zoerle, T., et al., Pharmacologic

reduction of angiographic vasospasm in

experimental subarachnoid hemorrhage:

systematic review and meta-analysis. J

Cereb Blood Flow Metab, 2012. 32(9): p.

1645-58.


139

CHAPTER 7Effects of database selection and language restriction in systematic reviews and meta-analyses of animal studies

CHAPTER 7Effects of database selection and language restriction in systematic reviews and meta-analyses of animal studies

Judith van LuijkEmily SenaGillian CurrieKimberley WeverAlice TillemaMarlies LeenaarsMerel Ritskes-HoitingaRob ScholtenBart van der Worp

(In preparation)

Chapter 7

142

Abstract

The number of published systematic reviews and meta-analyses of animal studies has increased rapidly over the last decade. Still the use of systematic review for animal studies is a developing field. For one, there is uncertainty surrounding the added value of using multiple bibliographic databases to identify studies and whether there is an impact of language restrictions in their selection.To investigate to what extent database selection and language restriction in the search strategy of systematic reviews of animal studies affect the outcome of the meta-analysis.Systematic reviews of which data for meta-analysis are available in the CAMARADES database.There was a large overlap between PubMed and Embase in retrievability of studies included in the reviews (>85% overlap in 2/3 of the evaluated reviews). Restricting a search to either PubMed or Embase resulted in missing up to 20% and 4% unique studies respectively. Restricting study retrieval to English studies only, could result in missing up to 13% relevant studies. An effect of these missed studies (both due to database selection or language restriction) on the meta-analysis could neither be demonstrated nor refuted.Restriction of a search to a single database or to studies published in English can lead to missing relevant studies for systematic reviews of animal studies and may affect the outcome of meta-analyses. How the meta-analysis of the review will be affected if these studies had been missed, differs per review. The authors discuss some potential strategies and recommendations how the issue of database selection and language restriction in systematic reviews of animal studies can be addressed.

IntroductionThe number of published systematic reviews of animal studies has increased rapidly over the last decade [1]. Their use has been driven largely by a desire to provide empirical evidence to inform improvements in the validity of preclinical studies [2]. These reviews provide comprehensive and less biased summaries of research findings, but as with any research method, are still susceptible to bias. The use of systematic review for animal studies is a developing field of research with a number of different approaches to its conduct and reporting [1, 3]. A fundamental step in the systematic review process is the identification and selection of relevant studies. There is uncertainty surrounding the added value of using multiple bibliographic databases to identify studies and whether there is an impact of language restrictions in their selection. The reliability of the findings of a systematic review is critically dependent on its completeness. For systematic reviews of human

SyStematic reviewS of animal StudieS, Search proceSS

143

clinical research, it has been shown that only searching Medline does not identify all controlled clinical trials [4]. Furthermore, only searching MEDLINE may bias summary estimates in a meta-analysis as primary studies indexed in MEDLINE are associated with larger treatment effects [5]. To date, studies that have assessed the effect of language restrictions in systematic review report contradictory findings. It has been suggested that investigators working in non-English speaking countries are more likely to publish positive results of clinical trials in international English-language journals whereas neutral or negative findings are published in local journals. There are also indications for a correlation between a lower quality score and studies published in another language than English [6].A more recent study found no evidence of systematic bias from the use of language restriction in systematic review-based meta-analyses in medicine [7]. However, other studies report that language bias can be more prominent in different fields of medicine [8], and can depend on the language that is restricted [9]. For systematic reviews of animal studies we were unable to identify empirical evidence to either support or oppose the added value of searching multiple databases instead of a single one, and on including studies published in any language rather than English alone.The most frequently searched biomedical databases are PubMed, Embase and Web of Science (WoS). PubMed is freely available and commonly used to access MEDLINE. Embase is less often available to researchers and is accessed via paid subscription through a range of platforms. Since early 2010, records that were previously only available in MEDLINE can also be retrieved via Embase. Both PubMed and Embase allow for a comprehensive search using a thesaurus structure for indexing: MeSH for MEDLINE via PubMed and EMTREE for Embase. PubMed also contains recent studies that are yet to be indexed. This feature is not available in Embase. Other databases, such as WoS and Scopus, do not use such all-encompassing thesaurus indexing, which results in less specificity and substantially more search hits [10]. We have assessed the effects of the use of multiple databases and language restrictions in systematic reviews of animal studies. We used data from published systematic reviews curated on the Collaborative Approach to Meta-Analysis and Review of Animal Data in Experimental Studies (CAMARADES) database to investigate the effects of methodological choices of primary study identification and selection. Specifically, we aimed to gain insight into the number of studies each database contributed to each systematic review and to evaluate the effect of limiting the search for primary studies to PubMed or Embase and to studies published in English on the results of meta-analyses.

Chapter 7

144

Methods

Study selectionWe selected all published systematic reviews up until January 2015 curated in the Collaborative Approach to Meta-Analysis and Review of Animal Data in Experimental Studies (CAMARADES) database that met the following inclusion criteria: 1) two or more online databases were searched, 2) no language restriction was applied in the search or selection of primary studies, 3) a minimum of 25 primary studies were included in the systematic review to allow for sub-group analyses and 4) the systematic review was published in a peer-reviewed journal.

Data collectionTo identify whether the included primary studies were available in PubMed, Embase and/or WoS, an information specialist (AT) of the Radboud University Medical Library, used the original bibliographic management files (Reference Manager or EndNote) of the included primary studies for each systematic review to run batch search scripts per database. We also classified each primary study as written in English if published in an English-language journal. In cases where a study was published in a non-English or bi-lingual journal we checked the language of the full text version of the study stored in the CAMARADES database.

Stratified meta-analysisFor each systematic review, we calculated the total number of studies that could be retrieved from each database and assessed the overlap in retrieved studies between databases and the number of studies that were uniquely found in a single database. For each systematic review we identified the predefined primary outcome measure. If the primary outcome was not clearly stated we contacted the corresponding author to ascertain which outcome was used as the primary outcome. We abstracted the following information from each study: publication year, primary outcome measure, animal cohort, effect size and standard error. For each independent comparison, we then calculated the effect size for the primary outcome as a standardized mean difference (SMD) and corresponding 95% confidence interval (CI). We pooled these SMDs using random effects meta-analysis, to account for anticipated heterogeneity [11]. We then partitioned heterogeneity (based on Cochran’s Q [12]) using stratified meta-analysis to investigate the impact of the database searched and of language restrictions. In stratified meta-analyses, the heterogeneity statistics for each stratum are summed and then subtracted from the total heterogeneity. This residual heterogeneity between the strata follows a Chi2 distribution. To test whether there was a significant proportion of residual heterogeneity we used the Chi2 test with n-1 degrees of freedom to test for a significant difference between the strata.


145

For each of the included systematic reviews we performed 4 analyses where we stratified by: (i) retrievable from PubMed (Y/N), (ii) retrievable from Embase (Y/N), (iii) retrievable from either PubMed or Embase (Y/N), and (iv) English language publication (Y/N). These analyses were performed in the CAMARADES database (Microsoft Office Access 2007). To allow for these multiple analyses, we adjusted our significance level to P<0.013 using Bonferroni-Holm correction. We did not assess for studies retrievable from WoS as this database was not used in all reviews.

Meta-analysis of meta-analysesTo assess the overall effect of database and language restriction on meta-analysis, we combined the results of the stratified meta-analyses in one meta-meta-analysis. For example, to study the overall effect of language restriction, we used the SMD of English and non-English studies to calculate the difference between SMDs (ΔSMDs) and corresponding 99% confidence interval (CI), for each review. These ΔSMDs were then pooled using random effects meta-analysis. The same approach was used to determine the effect of database restriction, resulting in a total of 4 meta-meta-analyses (all performed using Stata/SE 11.2 software).

Results

A total of 15 systematic reviews [13-27] met our selection criteria (Figure 1). The number of studies included in these systematic reviews ranged from 25 to 252, with a median of 73 (Table 1). The total number of primary studies was 1314. The topics varied between reviews, however they were all neuroscience related (stroke, glioma, spinal cord injury, bone cancer pain and Parkinson’s disease).

Effect of database and language restriction on search resultsTable 1 shows the characteristics of the reviews we evaluated. All reviews used MEDLINE (via PubMed) and Embase to search for primary studies. Ten systematic reviews (67%) searched WoS, including one review that used the Web of Knowledge, which includes WoS [16]. Five systematic reviews (33%) searched BioSIS and in eight reviews (53%) some type of hand searching (e.g. reference lists of included studies) was performed. For three reviews, all included studies could have been retrieved either from PubMed or Embase (100% overlap). Out of all the 1314 studies, 1131 (86%) could have been retrieved from PubMed and 1169 (89%) from Embase. A search in PubMed yielded unique studies for five reviews, if PubMed had not been searched this could have resulted in missing studies up to 4% per review. The Embase search yielded unique studies for nine reviews; without an Embase search up to 20% of the studies would have been missed per review. Five reviews did not search WoS, nevertheless we

Chapter 7

146

assessed retrievability of the included studies in these reviews for this database. Five reviews used a database for which retrievability could not be checked namely Biosis. Unique studies from other sources therefore represent studies that could not have been retrieved from PubMed, Embase or Web of Sciences and where thus retrieved via other databases or via hand searching (See Table 1).In six reviews, only studies published in English met the authors’ inclusion criteria despite there being no language restriction. If a language restriction had been applied in the other reviews, up to 13% of the studies would have been missed. (See Table 1).

Effect of database and language restriction on individual meta-analysesIn ten reviews, infarct volume (IV) was the primary outcome measure for the meta-analysis [15, 17-20, 23-27]. In the other five reviews the primary outcome measure was neurobehavioral score (NBS) or extent of pain-related behavior [13, 14, 16, 21, 22]. Table 2 shows details on the number of studies and experimental comparisons included in the meta-analysis of each review. For four reviews, no analysis could be performed regarding database and language restrictions, since all studies included in their meta-analysis were retrievable from both PubMed and Embase and were published in English [17, 18, 21, 26]. Table 2 lists the reviews and the respective amounts of heterogeneity in each stratum for all four analyses. Ten out of the 11 reviews included studies retrievable with PubMed, as well as studies that were not [13-16, 19, 20, 22-25]. For five of these reviews the extent of heterogeneity differed significantly between the two groups. The same ten reviews included studies retrievable from Embase, as well as studies that were not. In five of these reviews heterogeneity between the two groups differed significantly. Six reviews included studies from PubMed or Embase, as well as additional studies from other databases [13, 14, 19, 22-24]. In one of these reviews, the extent of heterogeneity differed significantly between the two groups(PubMed or Embase; yes/no). Eight reviews had both English and non-English studies included in the meta-analysis [13, 14, 16, 19, 20, 22, 24, 27]. In four of these reviews the extent of heterogeneity between the two groups differed significantly.

Meta-meta-analyses on the overall effect of database and language restriction Figures 2-5 show the meta-meta-analysis forest plots of our four analyses. Per review the effect size is plotted as ΔSMD with a CI of 99%. Figure 2 shows the first subgroup analysis (PubMed; Y/N). Overall and the majority of reviews (seven) show no effect of database selection for PubMed. For three reviews, the overall effect size of studies retrievable from PubMed was significantly lower than that of other studies. The summary ΔSMD of the ten reviews was -0.59 (99%, CI -1.24


147

to 0.07), and heterogeneity in this analysis was high (I2 = 82%), indicating that the effect of restriction to PubMed is variable.Figure 3 shows the second subgroup analysis (Embase; Y/N). Overall and the majority of reviews (seven) show no effect of database selection for Embase. For three reviews, the overall effect size of studies retrievable from Embase was significantly lower than that of other studies. The summary ΔSMD of the ten reviews was -0.78 (99%, CI -1.81 to 0.25), and heterogeneity in this analysis was high (I2 = 88%), indicating that the effect of restriction to Embase is variable.Figure 4 shows the third subgroup analysis (PubMed or Embase Y/N). Overall and the majority of reviews (four) show no effect of search restriction to PubMed or Embase. For two reviews, the overall effect size of studies retrieved from PubMed or Embase was significantly lower than that of studies from other sources. The summary ΔSMD of the six reviews was -0.45 (99%, CI -0.99 to 0.10), and heterogeneity in this analysis was moderate (I2 = 50%), indicating that the effect of restriction to PubMed and Embase is less variable between reviews.Figure 5 shows the fourth subgroup analysis (English Y/N). The majority of reviews (six) show no effect for language restriction. For two reviews, the overall effect size of English studies was higher than of non-English studies. The summary ΔSMD of the eight reviews was -0.68 (99%, CI -2.29 to 0.94), and heterogeneity in this analysis was high (I2 = 96%), indicating that the effect of language restriction is variable.

Discussion

The aim of this study was to investigate to what extent database selection and language restriction in the search strategy of systematic reviews of animal studies may affect the outcome of the meta-analysis. We assessed the effect of these restrictions on between study variation (heterogeneity) and overall effect size per review. Our results show that on average an effect of database choice or language restriction on the outcome of the meta-analysis could neither be demonstrated nor refuted. However, heterogeneity was present, indicating that the results are dependent on characteristics of a specific review. This indicates that the research area of the review might be an important factor. In two reviews on stem-cell research, restricting the search to either pubmed or Embase resulted in considerably smaller effect sizes. A second potential factor might be study quality (impact of bias), for example it is possible that the quality of studies found in PubMed differ from studies found in Embase. However, study quality was beyond the scope of this study, but could be interesting to take into account in future methodology research .Some potential limitations should be discussed. Firstly, we used reviews available in the CAMARADES database, of which most are focused on a neurology related topic.

Chapter 7

148

The extent to which database selection or language restriction affects the outcome of a systematic review may be different in other research areas.Secondly, we focused on the effect on meta-analysis, but a systematic review is more than a meta-analysis alone. By applying any search restrictions, potentially relevant evidence can be missed. Even though search restrictions might not always influence the results of a meta-analysis, they might jeopardize the purpose of a systematic review, namely to provide a complete and objective overview of all available evidence. Additionally, we have only included reviews that had a minimum of 25 primary studies in the primary meta-analysis, thus relatively large reviews in terms of animal studies. A previous study showed that a minority (36%) of the reviews include >25 primary animal studies [1]. A third limitation is the potential discrepancy between our retrievability analysis and the search strategy used in the original reviews. For our analyses we checked the availability of included studies in PubMed, Embase and Web of Science (WoS), irrespective of the search strategy and databases that were searched in the original reviews. In other words, we did not take the effectiveness of the search strategy that was used in the original reviews into account. Therefore there might be a discrepancy between the retrievability of a study in a database in our analysis and the original review. Use of multiple databases in a search strategy of a systematic review is a way of increasing chances to retrieve relevant studies [10]. Additionally five reviews originally did not search WoS, but we did check the retrievability of the included studies in WoS. In theory it is possible that if WoS had been searched with an appropriate strategy, additional relevant studies could have been found in this database. For this reason we did not perform specific WoS analysis and instead only provided descriptive data in Table 1. Fourthly, retrievability of studies per database can be differ depending on the interface used or the subscription type the institution has. To update or re-use a search strategy of a systematic review, it is important that this information is reported [10]. An effect of database choice and language restriction in the search strategy of systematic reviews of animal studies on the primary meta-analysis could neither be demonstrated nor refuted. Even though the effect on the meta-analysis is inconclusive, it has been shown that database choice and language restriction can result in missing relevant studies which can compromise the completeness of the systematic review itself. We therefore suggest a similar strategy for systematic reviews of animal studies as for clinical systematic reviews; to increase search accuracy by searching a range of databases even though the content of various databases may overlap to a great extent [4, 5]. However due to the large overlap between PubMed and Embase, searching both of these databases might be considered a waste of time and resources. A more fruitful strategy could be to search Embase or PubMed and one or two additional databases (e.g. Web of Science or Scopus) whether or not in combination with a hand search (e.g. reference lists or relevant conference proceedings). For further fine-tuning


149

of guidelines for the search strategy of systematic reviews of animal studies, it could be useful to 1) assess the effect of database choice and language restriction in a wider range of high quality systematic reviews of animal studies (in multiple research areas), and 2) assess if there is a relationship between study quality and database or language they are in. Unfortunately, data and sufficient details to perform such analysis on a larger scale are not yet available. Nonetheless, the conduct and publication of systematic reviews of animal studies is gradually increasing. Adequate reporting of these reviews will enable better methodological research for this developing field. Initiatives such as reporting guidelines for primary studies (e.g. the ARRIVE guidelines), reporting guidelines for systematic reviews of animal studies (akin to the PRISMA guidelines for clinical reviews), publication of review protocols (in the near future in Prospero) and adequate education and training on these matters are therefore of great value.

Acknowledgements

The authors would like to thank the authors and collaborators of the CAMARADES reviews for providing the required information and answers during the process of this study. H.B. van der Worp is supported by a grant from the Dutch Heart Foundation (2010T075).

Chapter 7

150

1. van Luijk, J., et al., Systematic reviews of

animal studies; missing link in translational

research? PLoS One, 2014. 9(3): p.

e89981.

2. Sena, E.S., et al., Systematic reviews

and meta-analysis of preclinical studies:

why perform them and how to appraise

them critically. J Cereb Blood Flow Metab,

2014. 34(5): p. 737-42.

3. Hooijmans, C.R., et al., SYRCLE’s risk of

bias tool for animal studies. BMC Med Res

Methodol, 2014. 14: p. 43.

4. Suarez-Almazor, M.E., et al., Identifying

clinical trials in the medical literature with

electronic databases: MEDLINE alone

is not enough. Control Clin Trials, 2000.

21(5): p. 476-87.

5. Sampson, M., et al., Should meta-

analysts search Embase in addition to

Medline? J Clin Epidemiol, 2003. 56(10):

p. 943-55.

6. Egger, M., G. Smith, and D. Altman,

Systematic Rieviews in Health Care Meta-

analysis in context. 2007. p. 60-62.

7. Morrison, A., et al., The effect of English-

language restriction on systematic review-

based meta-analyses: a systematic review

of empirical studies. Int J Technol Assess

Health Care, 2012. 28(2): p. 138-44.

8. Pan, Z., et al., Local literature bias in

genetic epidemiology: an empirical

evaluation of the Chinese literature. PLoS

Med, 2005. 2(12): p. e334.

9. Egger, M., et al., How important are

comprehensive literature searches and

the assessment of trial quality in systematic

reviews? Empirical study. Health Technol

Assess, 2003. 7(1): p. 1-76.

10. Leenaars, M., et al., A step-by-step guide

to systematically identify all relevant animal

studies. Lab Anim, 2012. 46(1): p. 24-31.

11. Vesterinen, H.M., et al., Meta-analysis

of data from animal studies: a practical

guide. J Neurosci Methods, 2014. 221: p.

92-102.

12. Cochran, W.G., The Combination of

Estimates from Different Experiments.

Biometrics, 1954. 10(1): p. 101-129.

13. Antonic, A., et al., Stem cell

transplantation in traumatic spinal cord

injury: a systematic review and meta-

analysis of animal studies. PLoS Biol,

2013. 11(12): p. e1001738.

14. Currie, G.L., et al., Animal models of bone

cancer pain: systematic review and meta-

analyses. Pain, 2013. 154(6): p. 917-26.

15. Egan, K.J., et al., Exercise reduces infarct

volume and facilitates neurobehavioral

recovery: results from a systematic

review and meta-analysis of exercise in

experimental models of focal ischemia.

Neurorehabil Neural Repair, 2014. 28(8):

p. 800-12.

16. Frantzias, J., et al., Treatment of

intracerebral hemorrhage in animal

models: meta-analysis. Ann Neurol, 2011.

69(2): p. 389-99.

17. Gibson, C.L., et al., Estrogens and

experimental ischemic stroke: a

systematic review. J Cereb Blood Flow

Metab, 2006. 26(9): p. 1103-13.

18. Hirst, T.C., et al., Systematic review and

meta-analysis of temozolomide in animal

models of glioma: was clinical efficacy

predicted? Br J Cancer, 2013. 108(1): p.

64-71.

References


151

19. Lees, J.S., et al., Stem cell-based therapy

for experimental stroke: a systematic

review and meta-analysis. Int J Stroke,

2012. 7(7): p. 582-8.

20. McCann, S.K., et al., Efficacy of

antidepressants in animal models of

ischemic stroke: a systematic review and

meta-analysis. Stroke, 2014. 45(10): p.

3055-63.

21. Pedder, H., et al., Systematic review and

meta-analysis of interventions tested in

animal models of lacunar stroke. Stroke,

2014. 45(2): p. 563-70.

22. Rooke, E.D., et al., Dopamine agonists

in animal models of Parkinson’s disease:


Parkinsonism Relat Disord, 2011. 17(5):

p. 313-20.






Flow Metab, 2010. 30(12): p. 1905-13.

24. van der Worp, H.B., et al., Hypothermia in

animal models of acute ischaemic stroke:


Brain, 2007. 130(Pt 12): p. 3063-74.

25. Vesterinen, H.M., et al., Systematic

review and stratified meta-analysis of the

efficacy of RhoA and Rho kinase inhibitors

in animal models of ischaemic stroke.

Syst Rev, 2013. 2: p. 33.

26. Willmot, M., et al., A systematic review

of nitric oxide donors and L-arginine in

experimental stroke; effects on infarct

size and cerebral blood flow. Nitric Oxide,

2005. 12(3): p. 141-9.

27. Willmot, M., et al., Nitric oxide synthase

inhibitors in experimental ischemic stroke

and their effects on infarct size and

cerebral blood flow: a systematic review.

Free Radic Biol Med, 2005. 39(3): p. 412-

25.

Chapter 7

152

Tab

le 1

Ch

arac

teri

stic

s o

f th

e ev

alua

ted

sys

tem

atic

rev

iew

s.

Per

Rev

iew

: Rev

iew

ID (

nam

e 1st

aut

ho

r an

d y

ear)

, To

tal s

tud

ies

(n)

incl

uded

, ret

riev

able

stu

die

s (n

,%)

fro

m P

ubM

ed (

PM

) an

d E

mb

ase

(Em

b)

and

th

eir

ove

rlap

and

ret

riev

able

stu

die

s (n

,%)

fro

m W

eb o

f S

cien

ce (

Wo

S)

and

oth

er d

atab

ases

(e.

g. B

iosi

s o

r h

and

sea

rch

).

Num

ber

of

uniq

ue s

tud

ies

(n,%

) p

er d

atab

ase.

No

n-E

nglis

h s

tud

ies

(n,%

). S

earc

hed

dat

abas

e (P

ubM

ed, E

mb

ase,

Web

of

Sci

ence

/Kno

w-

led

ge,

Bio

SIS

and

/or

han

d s

earc

h).

Rev

iew

IDA

nton

ic

2013

Cur

rie

2013

Ega

n 20

14Fr

antz

ias

2011

Gib

son

2006

Hirs

t 20

13Le

es

2012

McC

ann

2014

Ped

der

2014

Roo

ke

2011

Sen

a 20

10vd

Wor

p 20

07Ve

ster

inen

20

13W

illm

ot

2005

(d)

Will

mot

20

05 (

i)

Stu

dies

incl

uded

in

revi

ewn

156

150

4088

2759

117

4455

252

102

101

2525

73

Ret

rieva

ble

from

P

ubM

ed

n13

810

732

8127

5711

039

5519

687

8123

2573

%88

7180

9210

097

9489

100

7885

8092

100

100

Ret

rieva

ble

from

E

mba

se

n15

111

039

8327

5310

642

5520

692

8325

2572

%97

7398

9410

090

9195

100

8290

8210

010

099

Ove

rlap

Pub

Med

/E

mba

se

n13

699

3276

2753

101

3855

196

8680

2325

72

%87

6680

8610

090

8686

100

7884

7992

100

99

Ret

rieva

ble

from

WoS

n13

010

833

6727

5310

139

5520

194

8124

2572

%83

7283

7610

090

8689

100

8092

8096

100

99

Ret

rieve

d ot

her

so

urce

s*

n3

180

00

21

00

469

170

00

%2

120

00

31

00

189

170

00

Uni

que

in P

ubM

edn

06

02

02

30

00

10

00

1

%0

40

20

33

00

01

00

01

Uni

que

in E

mba

sen

87

84

00

44

06

22

10

0

%5

520

50

03

90

22

24

00

Uni

que

in W

oSn

414

10

01

11

012

55

00

0

%3

93

00

21

20

55

50

00

Uni

que

in o

ther

so

urce

s

n3

183

00

21

00

344

120

00

%2

128

00

31

00

134

120

00


153

Non

-Eng

lish

st

udie

s

n9

125

70

110

10

30

50

00

%6

813

80

29

20

10

50

00

Dat

abas

es s

earc

hed

in

rev

iew

PM

, E

mb,

W

oS

PM

, E

mb,

W

oS,

Han

d

PM

, E

mb,

W

oS

PM

, E

mb,

W

oK,

Han

d

PM

, E

mb,

W

oS,

Han

d

PM

, E

mb

PM

, E

mb,

W

oS,

bio-

SIS

PM

, E

mb,

W

oS,

bioS

IS

PM

, E

mb,

W

oS,

Han

d

PM

, E

mb,

B

io-

SIS

PM

, E

mb,

B

io-

SIS

, H

and

PM

, E

mb,

B

ioS

IS,

Han

d

PM

, E

mb,

W

oS

PM

, E

mb,

W

oS,

Han

d

PM

, E

mb,

H

and

Chapter 7

154

Fig

ure

1 Fl

ow

char

t o

f th

e se

lect

ion

pro

cess

fo

r an

alys

is.

IDE

NTI

FIC

ATI

ON

&

SC

RE

EN

ING

ELI

GIB

ILIT

Y

INC

LUD

ED

Num

ber

of

pub

lish

ed

syst

emat

ic r

evie

w a

nim

al

stud

ies

of

wh

ich

dat

a is

ava

ilab

le

in t

he

CA

MA

RA

DE

S

dat

abas

e (n

= 2

5)

Sys

tem

atic

rev

iew

s as

sess

ed

for

elig

ibili

ty

(n =

25)

Stu

die

s in

clud

ed in

q

ualit

ativ

e sy

nth

esis

(n

= 1

5)

Stu

die

s in

clud

ed in

q

uant

itativ

e sy

nth

esis

(n

= 1

1)

Sys

tem

atic

rev

iew

s ex

clud

ed

(n =

10)

1 =

onl

y o

ne d

atab

ase

sear

ched

9

= <

25 p

rim

ary

stud

ies

incl

uded


155

Tab

le 2

Ove

rvie

w o

f m

eta-

anal

yses

(re

view

s) in

clud

ed in

th

e an

alys

is.

Per

rev

iew

: Rev

iew

ID (

Nam

e 1st

aut

ho

r ye

ar).

In g

ray:

Det

ails

of

the

pri

mar

y m

eta-

anal

ysis

(M

A)

per

rev

iew

; O

utco

me

mea

sure

: Neu

rob

e-h

avio

ral S

core

(N

BS

) o

r In

farc

t Vo

lum

e (I

V),

Num

ber

(n)

of

stud

ies

incl

uded

in t

he

MA

, to

tal n

umb

er o

f co

mp

aris

ons

(n)

in t

hes

e st

udie

s in

clud

ed in

th

e M

A, t

ota

l het

ero

gen

eity

(Q

) in

th

e M

A. I

n w

hite

: Fo

ur a

naly

ses

per

rev

iew

: Pub

Med

, Em

bas

e, P

ubM

ed o

r E

mb

ase,

Eng

lish

la

ngua

ge.

Per

ana

lysi

s: F

or

two

str

ata

(yes

/no

), n

umb

er o

f st

udie

s (n

) p

er s

trat

um a

nd h

eter

og

enei

ty (

Q),

res

idua

l het

ero

gen

eity

(Q

) an

d

sig

nific

ance

tes

t (p

-val

ue)

on

Ch

i2 d

istr

ibut

ion.

Rev

iew

IDA

nton

ic

2013

Cur

rie

2013

Ega

n 20

14Fr

antz

ias

2011

Lees

20

12M

cCan

n 20

14R

ooke

20

11S

ena

2010

vd W

orp

2007

Vest

erin

en

2013

Will

mot

20

05(i)

Primary

MA

Out

com

e m

easu

reN

BS

NB

SIV

NB

SIV

IVN

BS

IVIV

IVIV

Tota

l stu

dies

(n)

151

103

2776

7827

190

8497

2552

Tota

l com

paris

ons

(n)

306

113

6514

918

754

601

202

222

4114

8

Tota

l het

erog

enei

ty (

Q)

1513

1109

251

632

685

120

1761

385

676

121

428

PubMed

Yes

n27

599

6313

818

351

554

170

189

3914

8

Q13

2992

124

658

563

910

717

1029

758

112

042

8

No

n31

142

114

347

3233

20

Q12

616

84

3344

644

8892

0.3

-

Res

idua

l Q58

200.

715

37

60.

23

0.03

NA

p-va

lue

<0.

000*

<0.

000*

0.4

<0.

000*

0.09

0.00

8*0.

01*

0.7

0.08

0.9

NA

Embase

Yes

n29

910

363

138

185

5157

819

421

739

148

Q14

6193

924

658

564

110

717

3137

366

912

042

8

No

n7

102

112

323

85

20

Q33

129

433

0.27

628

114

0.3

-

Res

idua

l Q39

400.

715

447

31

20.

03N

A

p-va

lue

<0.

000*

<0.

000*

0.4

<0.

000*

<0.

000*

0.00

8*0.

10.

30.

10.

9N

A

Chapter 7

156

PubMed or EmbaseYe

sn

282

109

6514

918

554

577

178

194

4114

8

Q13

8210

8925

163

268

312

017

4130

958

812

142

8

No

n24

40

02

024

2428

00

Q92

17-

-1

-17

7487

--

Res

idua

l Q18

2N

AN

A0.

9N

A4

21

NA

NA

p-va

lue

<0.

000*

0.1

NA

NA

0.3

NA

0.06

0.1

0.2

NA

NA

English

Yes

n26

699

6513

918

451

556

202

212

4114

4

Q97

789

625

157

864

310

716

8638

564

812

142

1

No

n40

140

103

345

010

04

Q41

018

9-

3742

675

-26

-0.

8

Res

idua

l Q12

524

NA

160.

037

0.5

NA

2N

A6

p-va

lue

<0.

000*

<0.

000*

NA

<0.

000*

0.9

0.00

8*0.

5N

A0.

1N

A0.

02


157

Fig

ure

2 Fo

rest

plo

t o

f th

e m

eta-

met

a-an

alys

is o

f ΔS

MD

of

Pub

Med

vs.

no

n-P

ubM

ed s

tud

ies

with

a C

I of

99%

. Th

e le

ft s

ide

of

the

zero

line

ind

icat

es

a lo

wer

eff

ect

size

(E

S)

of

the

met

a-an

alys

is (

MA

) o

f a

revi

ew w

ith s

tud

ies

retr

ieve

d f

rom

onl

y P

ubM

ed. T

hre

e re

view

s sh

ow

sig

nific

antly

lo

wer

eff

ect

size

s, o

vera

ll an

eff

ect

coul

d b

e ne

ither

dem

ons

trat

ed n

or

refu

ted

.

Chapter 7

158

Fig

ure

3 Fo

rest

plo

t o

f th

e m

eta-

met

a-an

alys

is o

f ΔS

MD

of

Em

bas

e vs

. no

n-E

mb

ase

stud

ies

with

a C

I of

99%

. Th

e le

ft s

ide

of

the

zero

-lin

e in

dic

ates

a

low

er e

ffec

t si

ze (

ES

) in

th

e m

eta-

anal

ysis

(M

A)

of

a re

view

with

stu

die

s re

trie

ved

fro

m o

nly

Em

bas

e. T

hre

e re

view

s sh

ow

sig

nific

antly

lo

wer

eff

ect

size

s, o

vera

ll an

eff

ect

coul

d n

eith

er b

e d

emo

nstr

ated

no

r re

fute

d.


159

Fig

ure

4 Fo

rest

plo

t o

f th

e m

eta-

met

a-an

alys

is o

f ΔS

MD

of

Pub

Med

or

Em

bas

e vs

. no

t P

ubm

ed/E

mb

ase

stud

ies

with

a C

I of

99%

. Th

e le

ft s

ide

of

the

zero

-lin

e in

dic

ates

a lo

wer

eff

ect

size

(E

S)

in t

he

met

a-an

alys

is (

MA

) o

f a

revi

ew w

ith s

tud

ies

retr

ieve

d f

rom

onl

y P

ubm

ed o

r E

mb

ase.

Tw

o r

evie

ws

sho

w s

igni

fican

tly lo

wer

eff

ect

size

s, o

vera

ll an

eff

ect

coul

d n

eith

er b

e d

emo

nstr

ated

no

r re

fute

d.

Chapter 7

160

Fig

ure

5 Fo

rest

plo

t o

f th

e m

eta-

met

a-an

alys

is o

f ΔS

MD

of

Eng

lish

vs.

no

n-E

nglis

h s

tud

ies

with

a C

I of

99%

. Th

e le

ft s

ide

of

the

zero

-lin

e in

dic

ates

a

low

er e

ffec

t si

ze (

ES

) in

th

e m

eta-

anal

ysis

(M

A)

of

a re

view

with

stu

die

s in

th

e E

nglis

h la

ngua

ge

onl

y. T

wo

rev

iew

s sh

ow

sig

nific

antly

lo

wer

eff

ect

size

s, o

vera

ll an

eff

ect

coul

d n

eith

er b

e d

emo

nstr

ated

no

r re

fute

d.


161

CHAPTER 8General discussionHow to improve responsible use of animals in research; about 3Rs and SRsHow to improve responsible use of animals in research;

Judith van LuijkMarlies LeenaarsRob de VriesGert-Jan van der WiltMerel Ritskes-Hoitinga

(In preparation)

Chapter 8

164

Background

Quality, reproducibility, reporting and translatability of animal studies have increasingly become topics of debate over the last 5-10 years [1-3]. The aim of this thesis was to contribute to responsible use of animals in research by studying strategies to improve scientific quality and animal welfare in animal-based research. First the use and implementation of the so-called 3Rs (of Replacement, Reduction and Refinement [4]) in the Netherlands were evaluated. Effective strategies to optimally implement the 3Rs appeared to be lacking, which hampers responsible animal use in research. A new strategy was introduced, namely systematic reviews of animal studies. This methodology may provide new ways of further improving scientific quality, reducing wasteful research, and evaluating the translational value of animal research [5, 6]. This final chapter puts the main findings of this thesis in the context of the major developments in the field of laboratory animal sciences related to the 3Rs and systematic reviews in the recent years. The final part of the discussion contains recommendations on how the systematic review methodology can contribute to responsible use of animals in research.

Background of the 3R principlesThe 3R principles were introduced by Russell and Burch in 1959 and became well known in the laboratory animal sciences in the 70’s. The 3Rs stand for Replacement, Reduction and Refinement and are defined as follows [7]:- Replace: avoid or replace the use of (living) animals wherever possible, - Reduce: employ strategies that will result in fewer animals being used and which

are consistent with sound experimental design- Refine: modify husbandry or experimental procedures to minimize pain and

distress. Later this definition was expanded to also encompass the improvement of welfare (e.g. by providing enrichment).

Over the years, the 3Rs have been incorporated in the legislation on animal experimentation of many countries, including the Netherlands. The first Dutch legislation on animal experimentation dates back from 1977, its most recent update was in 2014. In the EU, the Directive 86/609 on animal experimentation was first accepted in 1986 and updated in 2010 (2010/63/EU). In Directive 2010/63/EU the 3Rs are an important basic principle to obtain scientifically and ethically justifiable animal experiments.

3R practice in the NetherlandsA survey held in 2009 among animal researchers in Nijmegen (the Netherlands) identified a gap between legal requirements regarding the 3Rs and the lack of practical guidelines on how to implement them [8]. One of the main outcomes of this survey was that researchers expressed to have difficulties in searching for -and thus optimal

General discussion

165

implementation of- available and relevant 3R possibilities. This led to the establishment of the 3R Research Centre (3RRC) at the animal facility of the Radboud university medical center (Radboudumc), funded by the license holder. The 3RRC was a service centre conducting specific 3R literature searches for researchers . The assistance by the centre was highly appreciated, as 3R information is scattered over more than 100 websites and databases, which often require different approaches for an efficient search. After 2 years, funding by the license holder was discontinued and researchers had to cover the service costs themselves. Even though researchers were satisfied with this service, funding seemed to be an issue. As a result, requests for help stagnated quickly and eventually came to a halt.It was hypothesised that the situation in Nijmegen was not unique and that potentially more animal-based researchers struggled with 3R search and implementation. To gain better insights into how the 3Rs were implemented on a national level, the 3RRC conducted a national survey among researchers, animal welfare officers and animal ethics committee members in the Netherlands to assess their opinion and practice regarding 3R implementation. Questionnaires were developed and sent out to recipients affiliated with academia, industry and non-governmental organisations (NGO’s). The results of these surveys were in line with the previous findings of the local survey [8] and of surveys from other countries including Denmark and Canada [9, 10]. The findings suggest that there is much room for improvement. For example, it is indicated that many potential 3R possibilities are missed as knowledge and skills to effectively search them are lacking. Instead, information on 3R methods is generally obtained via personal communication, mostly with local professionals (e.g. animal care staff, animal welfare officer and other researchers within their own organisation). This impairs optimal implementation, as latest developments can easily be missed. Another interesting finding was that some respondents find it difficult to address the 3Rs as a single concept, as the implementation of a Replacement alternative requires a different strategy (e.g. in development and in validation and regulation requirements) compared to applying Reduction and Refinement possibilities in a planned animal experiment [11, 12]. Apparently, available 3R possibilities are not always optimally searched for nor implemented. This is jeopardizing the realisation of their aim, namely increasing scientific quality and animal welfare in animal-based research. So the question remains, how can responsible use of animals be achieved?In 2010, a follow-up meeting was organised with the survey respondents to explore possibilities to improve animal-based research, with a primary focus on 3R implementation. It was the first time in the Netherlands that researchers, animal welfare officers and animal ethic committee members from academia, industry and NGO’s came together to discuss issues regarding 3R implementation. The main outcomes of discussions indicated that there is not one single solution and that other strategies are needed to improve animal-based research besides the 3Rs. Main issues that occurred

Chapter 8

166

repeatedly during the plenary discussions were: the need for more transparency within animal-based research, including sharing of (negative) data and knowledge, as well as awareness of good experimental practice and the need for more education in this respect [13]. These issues indicates that –besides 3R implementation- additional strategies are needed to further improve animal-based research and thereby ensure responsible animal use. Systematic reviews of animal studies can be a valuable addition, as will be further discussed in the following section.

Systematic reviews of animal studiesA systematic review is a comprehensive and reproducible method to summarize and synthesize available evidence regarding a specific question. The methodology plays a key role in clinical research in creating transparency regarding the available body of evidence, including providing indications of potentially unpublished (neutral/negative) data (publication bias). The methodology also provides transparency regarding methodological quality, which has led to awareness of these issues and has provided leads for improvement [14]. In clinical research, systematic reviews have been performed for over 20 years and are considered the highest level of evidence in the chain of evidence-based medicine [14]. Besides medical research, systematic reviews are conducted in a number of other research areas including social sciences, economics and environmental sciences. Even though the conduct and publication of systematic reviews of animal studies have gradually increased in the last decade, the numbers are still extremely low compared to clinical systematic reviews. This is surprising as preclinical animal studies form the basis for clinical trials. Systematic reviews and meta-analyses of animal studies that have been published in the past decade have provided structured overviews and new insights in a variety of research areas. They provide valuable insights in the scientific quality of animal studies and useful information for the experimental design of future research, including 3R information. De Vries et al. [15] used the systematic review methodology to summarize tissue-engineered constructs that have the potential to be a Replacement for animal use in toxicological testing, drug screening and basic physiology. A systematic review by Sena et al. presents a cumulative meta-analysis on the effect of tPA on infarct volume in stroke, containing data from 3388 animals from studies published between 1989 and 2008 [16]. A stable effect could already have been found around 2001 based on data of ‘only’ ~1500 animals. Cumulative meta-analysis could therefore be a valuable tool summarizing and evaluating available evidence and thus avoiding unnecessary duplication of animal experiments (Reduction). A more evidence-based choice of animal models can also help in reducing animal studies with less suitable animal models. A systematic review by Sloff et al. provided an overview of tissue engineering animal models for urinary diversion and showed that data of larger animal models (e.g., pigs/dogs instead of rats/rabbits) appeared to translate

General discussion

167

better to humans [17]. De Vries et al. [18] summarized animal models used in articular cartilage tissue engineering in a systematic review and provided evidence-based recommendations on how to reduce the number of animals used based on limitations and predictive value for each model. A systematic review by Kleikers et al. revealed that most NOX2 studies in stroke were insufficiently powered [19]. Focussing purely on Reduction of use of animals -without taking statistical principles into account- can lead to underpowered studies. Studies with insufficient power are prone to incorrect interpretation, which can contribute to a waste of animals and resources. Many systematic reviews of animal studies have revealed shortcomings in the experimental design and/or reporting of animal studies and valuable recommendations for future (pre-)clinical research have been based on these findings [20-22]. The authors of a systematic review on animal studies in intestinal anastomosis research concluded that the quality and reporting of animal studies in this field is currently so poor that it is impossible to infer any valid conclusions from them [23].The issue of poor experimental design and reporting of preclinical animal studies is observed repeatedly in systematic reviews of animal studies [16, 17, 20-25]. In recent years, various reporting guidelines have been developed with the aim to improve reporting of animal studies [26-28]. These developments are similar to the clinical situation, where systematic reviews contributed to the exposure of inadequate reporting of randomized controlled trials (RCTs), which in turn fuelled the development of reporting guidelines such as the CONSORT statement and improved adequate reporting of RCTs [29]. The most well-known reporting guideline for animal studies is the ARRIVE guideline, which by 2014 was already incorporated in the instructions to authors by more than 400 journals. Unfortunately, at that time this guideline had not yet led to major improvements in reporting quality of animal studies [30]. By conducting more systematic reviews of animal studies in a variety of research areas, we gain better insights in the magnitude of the issue of poor reporting/methodological quality, which is expected to support better use and endorsement of these reporting guidelines.

Milestones in systematic reviews of animal studiesTo our knowledge, the first systematic review of animal studies was conducted in 2001 by Horn et al. [31]. Since then the number of published systematic reviews and meta-analyses has gradually increased [24, 32]. In 2006, CAMARADES began as the “Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Stroke” for investigators interested in the translational failures in the field of stroke. Over time the ‘S’ of stroke was replaced by the ‘S’ of studies, indicating a broader implementation and development of the systematic review and meta-analysis methodology [33]. During the FELASA/Scand-Las meeting in 2010 in Helsinki, Prof. Dr. Scholten, director of the Dutch Cochrane Centre, addressed the Laboratory animal science community by saying: “Do like us. Just start performing systematic reviews

Chapter 8

168

and learn as you go”. During the 1st international symposium on systematic reviews of animal studies in Nijmegen in 2012, the 3R Research Centre officially changed its focus from the 3Rs to systematic reviews and launched their new name: SYRCLE. Since then SYRCLE has mainly focused on the development of tools and training programmes for the conduct of systematic reviews of animal studies [34]. During the 8th World Congress on Alternatives and Animal use in the Life Sciences (WC8) in Montréal, Canada, the “Montréal Declaration on the Synthesis of Evidence to Advance the 3Rs principles in Science” was adopted [35]. This declaration called for a change in the culture of planning, conducting, reporting and reviewing animal studies by improving synthesis of available evidence, for example by conducting systematic reviews. Internationally, many initiatives have embraced the systematic review methodology for animal-based research, for example the NC3Rs, SABRE (a patient driven initiative for improved translation), Evidence-based Research Network, Evidence-Based Toxicology Collaboration, Evidence Synthesis International, the European Food Safety Authority, the National Toxicology Program/OHAT, the Environmental Protection Agency (EPA), AUGUST (Aarhus University Group for Understanding Systematic reviews and meta-analyses in Translational Preclinical Science), evidence-based veterinary sciences and many more. Also the Cochrane Collaboration gradually starts to evaluate and incorporate evidence of pre-clinical animal studies. For the first time in history, a special session on the synergy of systematic reviews of animal and clinical studies was organised by SYRCLE during the Cochrane colloquium in 2013 in Quebec Canada, which resulted in an editorial in the Cochrane Database[36].In the Netherlands, the parliament accepted two motions on the topic. The first motion, “Make systematic reviews the norm in animal-based research, similar to clinical research”, was accepted in 2012. This motion has led to funding of a research programme by the government on the development of guidelines [34], open access publication of negative/neutral results [37], SR training programs for professionals [37] and implementation research among stakeholders [38]. ZonMw (The Netherlands Organisation for Health Research and Development) was the first funding body to provide funding specifically for the conduct (methodology development, training and coaching) of systematic reviews of animal studies. This funding programme was part of their “More knowledge with fewer animals”-programme. A programme dedicated to stimulate the increase of scientific knowledge while simultaneously stimulating a reduction in the number of animals used [39]. The second motion that was passed urged to “make education on systematic reviews an obligatory part of the Dutch laboratory animal science courses (EU/2010/63 art.23.2.b)”. This motion was accepted in 2013 and has led to the development of an e-learning module (funded by the Dutch Ministry of Economic affairs), which was officially launched in October 2015 by EU commissioner Susanna Louhimies [40]. A recent report published by a Dutch think tank on animal-free innovations acknowledges the systematic review

General discussion

169

methodology as an innovative method to generate new knowledge without using new animal experiments [41].

Challenges and recommendations

The aim of this thesis was to investigate ways to improve scientific quality and responsible use of animals in research, with an initial focus on optimal 3R-implementation. It was demonstrated that optimal practices on 3R-search and implementation are currently lacking and it was suggested that additional strategies are needed to ensure high quality research and responsible use of animals. Systematic reviews have shown to be a powerful strategy to promote responsible use of animals by providing new evidence-based input for future research, including relevant 3R information. The final part of this chapter will provide recommendations on how the systematic review methodology can contribute to improved animal-based research and more translational transparency.

1. Increase the numbers of systematic reviews in a variety of animal-based research areas

Why First of all, systematic reviews have a great potential to be of high scientific value for animal-based research fields (e.g. medicine, toxicology, environmental science and veterinary science) as they provide transparent overviews of the available evidence. By reanalysing existing data, new insights can be gained without performing new animal studies. Already published reviews have showed that the following insights can be gained: (1) a more evidence-based choice of animal models, (2) insights in efficacy and safety of new therapies from animal studies before planning clinical trials, and (3) identification of relevant factors for possible mechanisms of disease and therapies. Additionally, gained insights can be used to determine ‘if’ and ‘how’ future research should be setup. Secondly, by increasing the number of published systematic reviews of animal studies, more evidence will be generated to inform best-practices for conducting these reviews. This can promote both methodology development and efficiency in the conduct of systematic reviews.

How Simply by just keep doing it. The research groups SYRCLE and CAMARADES started performing systematic reviews of preclinical animal studies, which have led to various high-impact papers. Additionally, both groups have been very successful in the development of tools and guidelines for the systematic review methodology for animal studies. Other research areas could follow-up on this by developing systematic review tools tailored to their line of research. It is highly recommended to establish research

Chapter 8

170

groups specifically devoted to the conduct of systematic reviews including methodology development and training programmes (similar to SYRCLE, CAMARADES, EBTOX and AUGUST) to speed-up progress worldwide. SYRCLE has initiated a worldwide ambassador network, aiming at appointing individuals that stimulate teaching and research on systematic review in their home countries and institutes.

2. Stimulate better reporting of animal studies. Why Use of animals in research is often justified by their proclaimed scientific

value. The aim of most preclinical animal studies is to provide useful insights for human medicine (e.g. disease development or intervention efficacy). It has been argued, however, that the majority of animal studies do not translate well to the human situation. In order to identify factors crucial for maximising the translational value of animal data, the circumstances under which these data were obtained need to be fully transparent. This requires the reporting of various details of animal studies regarding study design, characteristics and quality measures. Reporting guidelines for animal studies such as the ARRIVE guideline can therefore be of great value. Even though, the ARRIVE is currently endorsed by more than 600 journals, it seems that many authors, reviewers and editors still don’t use them.

How Firstly, more journals should endorse reporting guidelines for animal studies, and additionally provide guidance for their peer reviewers. The NC3Rs recently published a checklist of the ARRIVE guideline, which can help to ease the process for the authors, reviewers and editors. Secondly, adequate reporting of animal studies should be part of the training for all researchers directly or indirectly involved in animal based-research. Ideally, they should be discussed as early as in the bachelor training of future scientists and not only in the obligatory laboratory animal sciences course. And thirdly, as mentioned under recommendation 1, systematic reviews of animal studies can help to gain insight into the magnitude of the issue and create awareness for the value of these guidelines and the need for using them.

3. Perform a comprehensive search for relevant evidence before designing a new animal study.

Why A full systematic review is not always feasible and certainly not for each new animal experiment. Nevertheless, the first steps of the systematic review methodology can always be used to accumulate available relevant literature in a structured and comprehensive way. A comprehensive search can increase transparency and help reduce the introduction of bias in the search process. Objective literature overviews can be used to informing

General discussion

171

new animal studies, including 3R information. Unfortunately, knowledge on how to build a comprehensive search is often lacking.

How A comprehensive search can be performed in one or more online databases (e.g. PubMed). Strategies to build a comprehensive search often vary between databases, therefore it is highly recommended to make use of online tutorials, follow training in comprehensive search techniques, use available search filters and/or consult an information specialist. Research authorities such as animal ethics committees, animal welfare bodies, senior research staff and funding bodies can stimulate this evidence-based approach by asking for more details on how evidence was searched and selected before approving new animal-based studies. For example, in case in the application for a new animal study it is stated that no 3R methods are available, the researcher should provide more details on the search process (search terms, databases used etc.). And, if necessary, the search strategy can be assessed by an information specialist regarding sensitivity and specificity (meaning if adequate measures were taken to increase retrieval of relevant papers and decrease retrieval of irrelevant papers, respectively).

4. Provide and endorse training in comprehensive searching and systematic reviews of animal studies.

Why The previous recommendations and paragraphs describe the added value of (aspects of) systematic reviews of animal studies. However, skills and knowledge to use these methodologies often appears to be insufficient or lacking. This expresses the need for suitable training programs in both comprehensive search techniques and the systematic review methodology.

How To maximize output of systematic reviews and their potential contribution to improving animal-based research, it is highly recommended to provide training to all directly or indirectly involved in animal-based research. However, some only need to become familiar with the principles of the methodology, whereas others need to be able to conduct the full process. Institutes involved in animal-based research could provide training to their staff on comprehensive searching techniques to promote evidence-based experimental design. These programmes could then be tailored to the databases and software used at that institute to increase efficiency. Also students (bachelor, master and PhD) should be taught and examined on their search skills throughout the course of their training. To further promote evidence-based design, training in systematic reviews could be provided. This is expected to contribute to a major extent to a more responsible use of animals in research as described under recommendations 1 and 2.

Chapter 8

172

In conclusion

Various examples of systematic reviews and meta-analyses of animal studies have shown that the basic principles of clinical systematic reviews methodology can be translated to preclinical animal research. Systematic reviews and meta-analyses of animal studies can be used to provide an overview of available evidence, reduce unnecessary duplications of animal studies, provide new insights, identify knowledge gaps, provide useful information for new (pre)clinical research (including 3R information), and last but not least provide transparency in quality, validity and predictive value of animal studies for humans.

General discussion

173

1. Ioannidis, J.P., et al., Increasing value

and reducing waste in research design,

conduct, and analysis. Lancet, 2014.

383(9912): p. 166-75.

2. Pound, P. and M.B. Bracken, Is animal

research sufficiently evidence based to

be a cornerstone of biomedical research?

BMJ, 2014. 348: p. g3387.

3. van der Worp, H.B., et al., Can animal

models of disease reliably inform human

studies? PLoS Med, 2010. 7(3): p.

e1000245.




5. de Vries, R.B., et al., The usefulness of

systematic reviews of animal experiments

for the design of preclinical and clinical

studies. ILAR J, 2014. 55(3): p. 427-37.

6. Hooijmans, C.R. and M. Ritskes-

Hoitinga, Progress in using systematic

reviews of animal studies to improve

translational research. PLoS Med, 2013.

10(7): p. e1001482.

7. CCAC. June 2016; Available from: http://

www.ccac.ca/en_/standards/threer.





37(3): p. 297-303.

9. Nøhr, R., T.B. Lund, and J. Lassen, The

Danish 3R survey. 2016.

10. Fenwick, N., P. Danielson, and G.

Griffin, Survey of Canadian animal-based

researchers’ views on the Three Rs:

replacement, reduction and refinement.

PLoS One, 2011. 6(8): p. e22478.

11. van Luijk, J., et al., Assessing the

application of the 3Rs: a survey among

animal welfare officers in The Netherlands.

Lab Anim, 2013. 47(3): p. 210-9.






39(5): p. 429-47.

13. van Luijk, J., et al., Outcomes of a Dutch

workshop on improvements for the 3Rs in

daily practice. ALTEX, 2012. 29(4): p. 440-

3.

14. Cochrane_Collaboration. Available

from: http://www.cochrane.org/.

15. de Vries, R.B., et al., The potential of tissue

engineering for developing alternatives to

animal experiments: a systematic review.

J Tissue Eng Regen Med, 2015. 9(7): p.

771-8.






Flow Metab, 2010. 30(12): p. 1905-13.

17. Sloff, M., et al., Tissue engineering in

animal models for urinary diversion: a

systematic review. PLoS One, 2014. 9(6):

p. e98734.

18. de Vries, R.B., et al., Reducing the

number of laboratory animals used in

tissue engineering research by restricting

the variety of animal models. Articular

cartilage tissue engineering as a case

study. Tissue Eng Part B Rev, 2012. 18(6):

p. 427-35.

References

Chapter 8

174

19. Kleikers, P.W., et al., A combined pre-

clinical meta-analysis and randomized

confirmatory trial approach to improve data

validity for therapeutic target validation. Sci

Rep, 2015. 5: p. 13428.




PLoS One, 2012. 7(2): p. e32296.

21. Rongen, J.J., et al., The protective effect

of meniscus allograft transplantation on

articular cartilage: a systematic review of

animal studies. Osteoarthritis Cartilage,

2015. 23(8): p. 1242-53.

22. Groenink, L., et al., Pharmacological and

methodological aspects of the separation-

induced vocalization test in guinea pig

pups; a systematic review and meta-

analysis. Eur J Pharmacol, 2015. 753: p.

191-208.

23. Yauw, S.T., et al., Systematic review

of experimental studies on intestinal

anastomosis. Br J Surg, 2015. 102(7): p.

726-34.

24. van Luijk, J., et al., Systematic reviews of

animal studies; missing link in translational

research? PLoS One, 2014. 9(3): p.

e89981.

25. Sloff, M., et al., Tissue engineering of

the bladder--reality or myth? A systematic

review. J Urol, 2014. 192(4): p. 1035-42.





the Three Rs, and to make systematic

reviews more feasible. Altern Lab Anim,

2010. 38(2): p. 167-82.


research reporting: the ARRIVE guidelines

for reporting animal research. PLoS Biol,

2010. 8(6): p. e1000412.

28. Festing, M.F., The design and statistical

analysis of animal experiments. ILAR J,

2002. 43(4): p. 191-3.

29. Moher, D., et al., CONSORT 2010

explanation and elaboration: updated

guidelines for reporting parallel group

randomised trials. BMJ, 2010. 340: p.

c869.

30. Baker, D., et al., Two years later:

journals are not yet enforcing the ARRIVE

guidelines on reporting standards for pre-

clinical animal studies. PLoS Biol, 2014.

12(1): p. e1001756.

31. Horn, J., et al., Nimodipine in animal

model experiments of focal cerebral

ischemia: a systematic review. Stroke,

2001. 32(10): p. 2433-8.

32. Peters, J.L., et al., A systematic review

of systematic reviews and meta-analyses

of animal experiments with guidelines for

reporting. J Environ Sci Health B, 2006.

41(7): p. 1245-58.

33. CAMARADES. December 2015;

Available from: http://www.dcn.ed.ac.uk/

camarades/.

34. SYRCLE. December 2015; Available

from: http://www.syrcle.nl/.

35. Leenaars, M.R.-H., M.; Griffin, G.;

Ormandy, E., Background to the Montréal

Declaration on the Synthesis of Evidence

to Advance the 3Rs Principles in Science,

as Adopted by the 8th World Congress

on Alternatives and Animal Use in the Life

General discussion

175

Sciences, Montréal, Canada, on August

25, 2011. ALTEX, 2011.

36. Ritskes-Hoitinga, M., et al., Systematic

reviews of preclinical animal studies can

make significant contributions to health

care and more transparent translational

medicine. Cochrane Database Syst Rev,

2014(3): p. ED000078.

37. ZonMW. Available from: http://www.

zonmw.nl/en/.

38. Swankhuisen, C. and I. Smit, Systematic

reviews in the laboratory animal domain.

January 2014, Tabula rasa.

39. ZonMW_MDMD. Available from: http://

www.zonmw.nl/en/programmes/more-

knowledge-with-fewer-animals/general-

information/.

40. SYRCLE. E-learning module systematic

reviews of animal studies. December

2015; Available from: https://syrcle.

ekphost.nl/.

41. In transitie! 2015, Danktank Aanvullende

Financiering alternatieven voor

dierproeven.

176

ENGLISH SUMMARY

English Summary

Everyone will agree that the use of animals in research is controversial. Balancing “if”, “when” and “how” animals should be used is a continued topic of debate. The aim of this thesis is to inform this debate by evaluating strategies to improve scientific quality and responsible use of animals in research. Chapter one introduces the two evaluated strategies namely; 1) the in laboratory animal sciences well established 3R-principles of Replacement, Reduction and Refinement of animal research and 2) an upcoming methodology in animal-based research, namely systematic reviews of animal studies, as a new strategy to improve animal-based research.

To gain insights into the current state of affairs regarding the 3R-implementation in the Netherlands, surveys were sent out to three main profession groups namely: scientists involved in animal-based research, animal welfare officers and animal ethic committee members. The surveys covered four main topics: 1) view on the 3R-principles, 2) information sources and search strategies, 3) implementation strategies in daily practice and 4) needs for future improvement. Each profession group received an adapted survey with questions relevant to their professional tasks and responsibilities. Chapter two describes the outcomes of the survey among scientists and Chapter three the outcomes of the surveys among animal welfare officers and animal ethics committee members. Respondents represented a variety of research disciplines affiliated with different institutional backgrounds. The 3Rs are very well known and perceived as an important concept by respondents of all the three profession groups. Nevertheless, skills and resources to adequately search for relevant 3R information are often lacking, resulting in missing and thus not implementing 3R methods in research. Additionally, various respondents expressed to have difficulties with addressing the 3R-principles as one concept as implementation of a replacement alternative requires a different strategy than Reduction and Refinement.

The answers to part 4 of the survey ‘needs for future improvement’ were discussed in more detail during a one day workshop with respondents representing all three profession groups. Chapter four describes the six consensus statements that were formulated during the workshop as suggestions for improved animal-based research. Regarding the 3R-principles it was suggested to split the 3Rs in two concepts namely as Replacement and best practices (Reduction & Refinement). For optimal implementation of replacement alternatives it was suggested that all stakeholders (including legislators and funding bodies) should be involved in the development process. Other consensus statements covered issues such as experimental design, data sharing, transparency and collaboration. The diversity and complexity of issues raised during the workshop suggest that besides the well established 3R-principles, other strategies are needed to further improve animal-based research and ensure responsible use of animals.

177

ENGLISH SUMMARY

As introduced in Chapter one, systematic reviews of animal studies might be a powerful method to promote responsible use of animals in research, especially because they can be used to address some of the issues raised in Chapter four such as: sharing of (neutral/negative) data and awareness for optimal experimental study design. Nevertheless, systematic reviews of animal studies are a relatively new concept. Numbers of published systematic reviews are rapidly increasing, but still low compared to clinical systematic reviews of human trials. Despite the fact that many safety and efficacy animal studies need to be done for protecting humans.

Chapter five shows an example of such a systematic review of animal studies. The aim of this review was to systematically appraise and summarize all available evidence on the effect of meniscus allograft transplantation on articular cartilage in animals. By providing an overview of the available animal studies, the authors concluded that meniscus allograft transplantation does not protect articular cartilage from damage, but it reduces the extent of it when compared with meniscectomy. The consistent results across species and models might be an indication that a comparable treatment of the human knee joint will show similar results. The systematic review also exposed the reporting and methodological quality of the included animal studies. Crucial information was often not reported, which hampered the evaluation of study quality and limiting the ability to draw reliable conclusions. Unfortunately, other systematic reviews of animal studies in a variety of research areas show similar concerns when it comes to reporting and methodological quality of primary animal studies.

The step of quality assessment of primary studies in the systematic review methodology is very valuable to expose and monitor quality and thus validity of animal studies. However as there are no clear guidelines on how systematic reviews of animal studies should be conducted, the quality assessment can differ per review. Chapter six describes a study in which 91 systematic reviews of animal studies were assessed on the used quality assessment. Additionally, the overall quality of the primary studies included in these reviews was assessed. Just a little over half (52.7%) of the reviews assessed the primary studies on selection bias, performance bias, detection bias or attrition bias. Thirty-three systematic reviews provided sufficient information to evaluate the internal validity of the included studies. Of the evaluated primary studies, 24.6% was randomized, 14.6% reported blinding of the investigator/caretaker, 23.9% blinded the outcome assessment, and 23.1% reported drop-outs. This indicates that there is much room for improved reporting and/or methodological quality of primary studies and need for training and guidelines on the conducts of systematic reviews of animal studies.

178

ENGLISH SUMMARY

Perhaps the most vital but also the most time consuming step of the systematic review methodology is the search for and screening of papers. There is uncertainty surrounding the added value of using multiple bibliographic databases to identify studies and whether there is an impact of language restrictions in their selection. Chapter seven investigates to what extent database selection and language restriction in the search strategy of systematic reviews of animal studies affect the outcome of the meta-analysis. There appears to be a large overlap between PubMed and Embase in retrievability of studies included in the reviews (>85% overlap in 2/3 of the evaluated reviews). Restricting a search to either PubMed or Embase resulted in missing up to 20% and 4% unique studies respectively. Restricting study retrieval to English studies only, could result in missing up to 13% relevant studies. An effect of these missed studies (both due to database selection or language restriction) on the meta-analysis could be demonstrated nor refuted. Restriction of a search to a single database or to studies published in English can lead to missing relevant studies for systematic reviews of animal studies and may affect the outcome of meta-analyses. Although how the meta-analysis of the review would have been affected if these studies had been missed, differs per review. Potential strategies and recommendations on how the issue of database selection and language restriction in systematic reviews of animal studies can be addressed, are discussed in more detail in this chapter.

The concluding Chapter eight summarises the main outcomes of this thesis and describes some key developments in the field of systematic reviews of animal studies. The chapter concludes with four recommendations related to the question: How can systematic reviews of animal studies contribute to improved scientific quality of animal studies; 1) Increase the numbers of systematic reviews in a variety of animal based-research areas. 2) Stimulate better reporting of animal studies. 3) Perform a comprehensive search for relevant evidence before designing a new animal study. And 4) Provide and endorse training in comprehensive searching and systematic reviews of animal studies.

Use of animals in research is justified by their translational value, however true translational value of animal models can only be assessed when animals are used responsibly and animal studies comply to the highest standards for animal welfare and scientific quality. Insights from systematic reviews of animal studies have shown that this type of assessment is currently not optimally feasible. By providing education in better reporting, literature search and systematic reviews, this will lead to quality improvements in preclinical science, thereby contributing to the implementation of the 3Rs and improved patient health care.

179

ENGLISH SUMMARY

180

NEDERLANDSE SAMENVATTING

Nederlandse Samenvatting

Iedereen zal het erover eens dat het gebruik van dieren in (bio)medisch onderzoek controversieel is. Het balanceren tussen “of”, “wanneer” en “hoe” dieren gebruikt mogen worden, blijft onderwerp van discussie. Het doel van dit proefschrift is om een bijdrage te leveren aan dit debat. In dit proefschrift zijn twee strategieën geëvalueerd die ingezet kunnen worden om de wetenschappelijke kwaliteit van dieronderzoek te verbeteren en daarmee verantwoord proefdiergebruik te bewerkstelligen. Hoofdstuk een introduceert de twee geëvalueerde strategieën; als eerste het 3V-principe van Vervanging, Vermindering en Verfijning. Dit 3V-principe vormt sinds de jaren-70 de hoeksteen van de proefdierkunde. De tweede strategie is een opkomende methode binnen de proefdierkunde, namelijk systematic reviews van dierstudies.

Om het effect van het 3V-principe te evalueren is begonnen met het verkrijgen van inzicht in 3V-implementatie in Nederland. Hiervoor zijn enquêtes uitgestuurd naar drie beroepsgroep: onderzoekers die betrokken zijn bij diergebonden onderzoek, proefdierdeskundigen en leden van dierexperimentencommissies (DEC). De enquêtes omvatte vier hoofdthema’s: 1) perceptie van de 3V-principes, 2) 3V informatiebronnen en zoekstrategieën, 3) implementatie strategieën in de dagelijkse praktijk en 4) wensen voor toekomstige verbeteringen. Iedere beroepsgroep ontving een aangepaste enquête met specifieke vragen gerelateerd aan hun professionele taken en verantwoordelijkheden. Hoofdstuk twee beschrijft de resultaten van de enquête onder onderzoekers en Hoofdstuk drie de resultaten van de enquêtes onder de proefdierdeskundigen en DEC-leden. De respondenten vertegenwoordigen een verscheidenheid aan onderzoeksdisciplines binnen zowel universiteiten als ook de (biomedische) industrie. Uit de enquêtes onder alle drie de groepen bleek dat de 3V’s zeer goed bekend zijn en ook gezien worden als een relevant concept binnen proefdiergebonden onderzoek. Vaardigheden en middelen om op adequate wijze te zoeken naar relevante 3V-informatie ontbreken vaak, wat resulteert in het missen en dus niet toepassen van relevante 3V- methoden in onderzoek.

De antwoorden op deel 4 van het enquêtes (wensen voor toekomstige verbetering) zijn gebruikt als startpunt tijdens een verdiepende eendaagse workshop met respondenten van alle drie de enquêtes. Hoofdstuk vier beschrijft de zes consensus statements die tijdens de workshop als suggesties voor verbetering van diergebonden onderzoek zijn geformuleerd. Ten aanzien van de 3V-principes werd voorgesteld om de 3V’s te splitsen in twee begrippen namelijk in “Vervanging” en “best practices” (Vermindering & Verfijning). Voor een optimale ontwikkeling van vervangingsalternatieven werd gesuggereerd dat alle belanghebbenden (inclusief wetgevers en financierders) betrokken dienen te worden bij het ontwikkelingsproces. De overige consensusstatements

181


omvatte onder andere belang van goed experimenteel ontwerp, het delen van data (inclusief negatieve en neutrale data), en meer transparantie en samenwerking. De diversiteit en complexiteit van de geopperde wensen ten aanzien van verbetering van proefdiergebonden onderzoek laat zien dat naast de 3V-principes andere strategieën nodig zijn om verantwoord diergebruik binnen onderzoek te kunnen waarborgen.

Zoals reeds geïntroduceerd in hoofdstuk een, zouden systematic reviews van dierstudies een krachtige methode kunnen zijn om verantwoord diergebruik in onderzoek te bevorderen. Vooral ook omdat deze methode ingezet kan worden om enkele aandachtspunten zoals beschreven in hoofdstuk vier (bv. het delen van data en verantwoord opzetten van nieuwe proefdieronderzoek) aan te pakken. Momenteel worden systematic reviews van dierstudies slechts nog incidenteel en niet gestandaardiseerd uitgevoerd. Het aantallen gepubliceerde systematic reviews van dierstudies groeit snel, maar is nog steeds relatief laag in vergelijking met systematic reviews van humane studies. In de volgende hoofdstukken zal verder op de systematic review methodologie voor dierstudies.

Hoofdstuk vijf is een voorbeeld van een systematic review van dierstudies. Het doel van dit onderzoek was om op systematische wijze alle beschikbare gegevens te zoeken, selecteren en beoordelen op kwaliteit en samen te vatten voor de specifieke onderzoeksvraag: “What is the effect of meniscus allograft transplantation on articular cartilage in animals?” Door het maken van een dergelijk overzicht kon geconcludeerd worden dat meniscus transplantatie kraakbeen schade niet voorkomt, maar dat de schade wel kan worden verminderd ten opzichte van meniscectomie. De vergelijkbare resultaten die gevonden zijn in verschillende diermodellen en diersoorten kunnen een indicatie zijn dat een vergelijkbare behandeling van het menselijke kniegewricht ook vergelijkbare resultaten zal laten zien. Daarnaast laat de systematic review ook de kwaliteit van de methodologische opzet en rapportage van de geïncludeerde studies zien. Zo bleek dat cruciale informatie vaak niet was beschreven. Hoordoor kon de kwaliteit van de studie niet altijd kon worden geëvalueerd, wat vervolgens de mogelijkheid om betrouwbare conclusies te trekken belemmert. Helaas is dit review hierin niet uniek, diverse systematic reviews van dierstudies uit verschillende onderzoeksgebieden laten een zorgwekkende situatie zien op gebied van methodologische kwaliteit en/of rapportage van primaire dierstudies. Kwaliteitsbeoordeling van primaire studies een zeer waardevolle stap in het systematic review proces. Niet alleen voor het transparant maken van de huidige stand van zaken, maar ook voor het bewaken van veranderingen in kwaliteit over de tijd.

Systematic reviews van dierstudies worden nog niet gestandaardiseerd uitgevoerd, dit komt o.a. doordat er nog geen eenduidige richtlijnen bestaan over hoe systematic

182


reviews van dierstudies optimaal kunnen worden uitgevoerd. Hoofdstuk zes beschrijft een review van systematic reviews van dierstudies. Hierin zijn 91 systematic reviews beoordeeld op het gebruik van een kwaliteitsbeoordeling voor de primaire studies. Daarnaast is er ook gekeken naar de kwaliteit van de primaire studies in deze 91 systematic reviews. Iets meer dan de helft (52,7%) van de reviews beoordeelde de primaire studies op genomen maatregelen om de volgende vormen van bias te voorkomen: selectie bias, performance bias, detectie bias of attrition bias. Drieëndertig systematic reviews rapporteerde gedetailleerde informatie over de kwaliteitsbeoordeling per primaire studie, dit leverde het volgende beeld op: 24,6% van de primaire studies was gerandomiseerd (selection bias), 14,6% rapporteerde blindering van de onderzoeker / verzorger (performance bias), 23,9% blindering bij het meten van de uitkomstmaat (detection bias), en 23,1% rapporteerde drop-outs (attrition bias). Dit geeft aan dat er nog veel ruimte voor verbetering is op gebied van methodologische kwaliteit en/of de rapportage daarvan. Dat slechts de helft van de reviews een kwaliteitsbeoordeling uitgevoerd had, laat zien dat er behoefte is aan richtlijnen en training op het gebied van systematic reviews dierstudies.

De misschien wel de meest vitale, maar ook meest tijdrovende stap in het systematic review proces is het zoeken naar en screening van artikelen. Door het ontbreken van richtlijnen op het gebied van systematic reviews van dierstudies is er onzekerheid over de toegevoegde waarde van het doorzoeken van meerdere bibliografische databases is. Daarnaast is ook niet bekend wat het effect van taalbeperkingen (bijvoorbeeld door alleen studies in de Engelse taal te includeren) is op de uitkomsten van systematic reviews van dierstudies. Hoofdstuk zeven beschrijft een studie waarin is gekeken in hoeverre database selectie en taal beperking in de zoekstrategie van systematic reviews van dierstudies invloed kan hebben op de uitkomst van de meta-analyse. Allereerst blijkt er een grote overlap te zijn tussen PubMed en Embase in vindbaarheid van de artikelen (> 85% overlap in 2/3 van de reviews). Het beperken van een zoekopdracht tot PubMed ofwel Embase resulteerde in het missen van tot 20% en 4% relevante artikelen. Het beperken van studie selectie naar alleen Engelstalige artikelen zou kunnen leiden tot het missen van 13% relevante studies. Een significant effect van deze gemiste studies (zowel van database selectie als van taal restrictie) op de meta-analyse kon niet worden aangetoond of weerlegd. Beperken van de zoekstrategie naar een enkele database of selectie van enkel Engelstalige artikelen kan leiden tot het missen van relevante studies wat wel tot een vertekening van de uitkomsten van de systematic review kan leiden. De mate van vertekening kan variëren per vakgebied. In de discussie van dit hoofdstuk worden enkele strategieën en aanbevelingen beschreven over hoe om gegaan kan worden met database selectie en taal beperking in systematic reviews van dierproeven.

183


Het afsluitende Hoofdstuk acht geeft een overzicht van de belangrijkste resultaten van dit proefschrift en beschrijft een aantal belangrijke ontwikkelingen op het gebied van systematic reviews van dierproeven binnen de proefdierkunde en daarbuiten. Het hoofdstuk wordt afgesloten met vier aanbevelingen met betrekking tot de vraag: Hoe kunnen systematic reviews van dierstudies bijdragen aan een betere wetenschappelijke kwaliteit van dierproeven; 1) Uitvoeren van meer systematic reviews in verschillende dierproef gebonden onderzoeksgebieden . 2) Het stimuleren van een betere rapportage van dierproeven. 3) Het uitvoeren van een zogenaamde comprehensive search naar relevante literatuur voor het ontwerpen van nieuwe dierstudies. En 4) Aanbieden van onderwijsprogramma’s op gebied van systematic reviews dierstudies en comprehensive search strategies.

Het gebruik van dieren in wetenschappelijk onderzoek wordt gerechtvaardigd door hun translationele waarde. De translationele waarde van diermodellen kan echter alleen worden beoordeeld wanneer de dieren op verantwoorde wijze worden ingezet in onderzoek en de dierstudies zelf voldoen aan de hoogste normen voor dierenwelzijn en wetenschappelijke kwaliteit. Uit inzichten verkregen uit systematic reviews van dierstudies blijkt dit in de praktijk binnen diverse onderzoeksgebieden vaak nog lang niet het geval te zijn. Het aanbieden van onderwijs in hoe dierstudies op een adequate wijze gerapporteerd kunnen worden, het uitvoeren van gedegen literatuuronderzoek en de systematic review methodologie, kan bijdrage aan de nodige kwaliteitsverbetering van preklinische dieronderzoek. Op deze wijze kan een hoogwaardig en transparante basis gelegd voor de ontwikkelen van nieuwe en veilige therapieën voor de patiënt.

184

DANKWOORD

Dankwoord

Merel, jij bood mij de kans om binnen jouw onderzoeksgroep (toen nog het 3R Research Center) mijn afstudeerstage door te zetten in een promotie project. Ik ben je enorm dankbaar voor je vertrouwen in mij en de ruimte die je me hebt gegeven om te ontwikkelen, niet alleen als onderzoeker, maar ook als ‘ICT-nerd’ en PR medewerker, en vooral ook als docent. Jouw visie op de proefdierkunde en je lef om soms wat onconventioneel te handelen om de boel eens flink wakker te schudden, zijn enorm inspirerend. Dank dat je promotor wilde zijn.

Gert-Jan, na de project- en voortgangsgesprekken met jou liep ik altijd de deur uit met nieuwe inzichten en diep onder de indruk van jouw scherpe analytische vermogen. Wanneer ik het overzicht kwijt was, wist jij het grote kader weer te schetsen. Sinds 1 januari 2017 zijn we met onze onderzoeksgroep SYRCLE officieel onderdeel geworden van ‘jouw’ sectie HTA binnen de Department for Health Evidence. Erg leuk om te zien hoe er in zo’n korte tijd al mooie ideeën ontkiemen op gebied van gezamenlijk onderzoek en onderwijs. Dank dat je promotor wilde zijn.

Rob, je bent bij dit project betrokken als directeur van het Dutch Cochrane Centre. Naast de inhoudelijke uitdagingen, doorzag jij ook de uitdaging in het proces in dit project. Waar normaal een promovendus zo’n 2 á 3 begeleiders heeft, had ik er –omdat het nog zo’n nieuw onderzoeksterrein is- vijf. Zo nu en dan stuurde je me een kort opbeurend mailtje, vaak maar met een paar regels. Een hart onder de riem om mij en mijn onderzoek weer op de rit te krijgen. Dank dat je promotor wilde zijn.

Marlies, door jou ben ik bij het 3R Research Center terecht gekomen nadat ik een lezing van jou hoorde op de Hogeschool van Utrecht. En jij bent ook degene die me het laatste zetje gaf om ja te zeggen tegen dit promotie-onderzoek. Op een boot tijdens het FELASA symposium in Helsinki deelde je jouw visie dat dit onderzoek wel eens van grote waarde voor de proefdierkunde zou kunnen zijn. In de afgelopen 7 jaar is er ontzettend veel gebeurd; samen met vele andere inzichten, heeft ons onderzoek ook inzicht gegeven in de vaak lage kwaliteit en vertaalbaarheid van dierstudies, het kwam zelfs tot Kamervragen en moties in de Tweede Kamer. Gezien de vele (inter)nationale verzoeken voor trainingen, lezingen en symposia die we nu als SYRCLE ontvangen durf ik wel te concluderen dat je visie van toen een underestimation was van wat het uiteindelijk allemaal teweeg heeft gebracht. Ik ben ontzettend trots dat ik daar met dit proefschrift aan heb mogen bijdragen. Dank dat je co-promotor wilde zijn.

Bart, via jou heb ik de onderzoeksgroep CAMARADES leren kennen, hetgeen het methodologische onderzoek naar de systematic review methodologie van dierstudies

185

DANKWOORD

in dit proefschrift heeft mogelijk gemaakt. Je ervaringen als pioneer systematic reviewer en clinicus waren van grote waarde. Dank dat je co-promotor wilde zijn.

Dear CAMARADES-crew, after the first systematic review of animal studies by Janneke Horn in 2001, you were the pioneers who introduced the systematic review and meta-analysis methodology to the pre-clinical stroke research field. The CAMARADES database is a goldmine for systematic reviews of systematic reviews. Chapter 7 of this thesis would not have been possible without it. Thank you all for all the educational, inspiring and fun visits to CAMARADES at the University of Edinburgh in Scotland. I see a bright future for CAMARADES, SYRCLE and many more other international collaborations to come.

Dank aan ZonMw en Stichting Reinier Post voor het gestelde vertrouwen in het werk van SYRCLE. Dankzij deze financiële ondersteuning hebben we de geleerde lessen uit dit proefschrift kunnen vertalen naar vele nationale trainingen begeleidingsprogramma’s, waarmee ook een grote impact op onderwijs en onderzoek op nationaal en internationaal niveau is bereikt.

Alice, ‘onze’ informatie specialist bij de medische bibliotheek. Ik heb ontzettend veel van je geleerd op het gebied van grondig literatuur zoeken. Samen hebben we ook onderwijs voor literatuur zoeken voor systematic reviews doorontwikkeld en ontelbare keren gegeven. Heel veel dank voor al je hulp bij dit proefschrift.

Het grootste deel van mijn promotie heb ik op het CDL (Centraal Dierenlaboratorium) doorgebracht. Dank aan alle collega’s daar voor de leuke dagjes-uit, de gezellige (on)zinnige gesprekken tijdens de koffiepauzes. Ook dank aan de voormalig CDL bedrijfsleider Kees Hagenaar die slechts met een blik kon zeggen: “Judith leg dat ei en maak dat boekje nou eens af”. Kees, het is gelukt!

Carlijn en Kim (CH en KW), lieve collega’s, kamergenoten en methodologische toppers van het eerste uur! Ik heb met jullie gelachen (lachen, lachen, lachen), gehuild, gegeten (o.a. beschuit met muisjes en veel chocola), gedronken, gevlogen, auto gereden, gefietst, gezongen, gepubquized en trampoline gesprongen. Later kwam ook Cathalijn (CL) erbij, gelukkig ook al zo’n choc-a-holic. Dank meiden voor al jullie input, steun en gezelligheid.

Rob (Ropperdepop of RdV) de collega die ik het meest en het langst achter elkaar gezien heb door de vele reizen die we samen hebben mogen maken om systematic review trainingen te geven. Ik ben super blij dat jij tijdens de verdediging van mijn proefschrift als paranimf naast mij staat. Met jouw rust, wijsheid en (Italic Friday) humor

186

DANKWOORD

heb je me al door vele moeilijke momenten heen gesleept. Als het allemaal achter de rug is drink ik er graag een goede Whisky uit jouw collectie op!

Lieve Eef (Eva) het begon eigenlijk na onze studie pas, samen stappen in Utrecht en een prachtige reis naar Japan, een lang gekoesterde droom van mij. Waar ik soms sta te twijfelen, pak jij door en daardoor ben je echt een inspiratie voor mij. Ik ben daarom ook super trots en blij dat je tijdens de verdediging naast me staat als paranimf, ik weet dat jij me er met je enthousiasme doorheen sleept. Robert, ik mocht naast Eva staan als getuige toen jij haar volmondig het ja-woord gaf. Super om te zien hoe jouw vriendschap en die van je (de leuke) vrienden werkt als een soort spons, wie in de buurt komt wordt vrijwel direct opgenomen. Dank voor jullie grenzeloze gezelligheid in Athene, Utrecht, Maartensdijk, Hilversum, Nijmegen, Tilburg en straks ook Soesterberg.

Lieve Jessica en Anne, mijn dierbare vriendinnetjes van heel vroeg. We zijn samen opgegroeid, van jonge naïeve meiden naar stoere, zelfstandige ‘jonge’ vrouwen die allemaal grote mensen dingen doen zoals huizen kopen, kinderen krijgen en verre reizen maken. En nu dus ook een proefschrift schrijven, ik ben blij dat ik ook dit met jullie mag delen, op naar de volgende mijlpalen!

Marije, ouwe CDL-er! Het begon allemaal met een BBQ-tje op mijn dakterras na het werk, gewoon lekker in chill-broek alsof we elkaar al jaren kenden. Na jaren samen eten, sporten, stappen en delen van lief en leed, heb je het gemoedelijke Nijmegen verruild voor de stadse-fratsen van Amsterdam. Dank voor je altijd luisterende oor, kritische reflecties, heerlijke sarcasme en verhelderende inzichten.

Mieke en Nathalie, mijn twee ‘culturele’ vriendinnetjes uit Nijmegen die ook naar Amsterdam vertrokken. We zien elkaar niet zo vaak, maar ik weet dat jullie er (voor mij) zijn en dat is een fijne gedachte.

Miranda, nog zo’n oud CDL-er, eentje van het eerste uur. Samen hebben we veel lol gehad op het CDL in de Activiteiten Commissie en daarbuiten. Onze vrijdagmiddag borrels/diners bij van Buren met Nathalie en later ook Claudia zijn le-gen-da-risch. Wat daar allemaal boven de van Buren-Burgers besproken is… Ik zou er nog een boekje mee kunnen vullen. Of wie weet misschien toch ooit nog die eigen smartlap op de Dag van het Levenslied?

Dirk, Barbara, Thomas en Janneke (en Jip), jullie zijn bijna als een tweede familie voor mij. Dank voor -zoals jullie het zelf noemen- jullie bijdrage aan de wetenschap, in de vorm van een wekelijkse warme maaltijd, sporten, wintersport, zeilen, goede gesprekken, nerd-talk en onzin. Opa Leo, het boekje is eindelijk af! Familie Beekwilder, dank dat ik

187

DANKWOORD

regelmatig op jullie heerlijke huis mocht passen. Verschillende delen van dit proefschrift zijn aan jullie eettafel geschreven, onder streng toeziend oog van poes Lotje.

Familie van Bavel, ineens was ik daar in het leven van jullie zoon, broer en vader, dank voor jullie oprechte interesse en Brabantse gezelligheid. Jack, bedankt dat je op de valreep nog even als kritische ‘leek’ hebt meegelezen.

Lieve Erik, ook jij hebt een lange studieweg doorlopen: van VBO heb jij helemaal geploeterd naar WO, dat getuigt van ongelofelijk doorzettingsvermogen (net als het uitlopen van de 4-daagse trouwens). Jij legde de lat hoog, waar ik als kleine zus natuurlijk niet voor onder wilde doen. En dus ploeterde ik ook door met dit boekje als resultaat (maar die 4-daagse ga ik echt nooit lopen en al zeker de 50km niet). Dank aan jou en Dorine voor jullie steun, goede gesprekken, scherpe vragen, lieve kaartjes en natuurlijk het allergrootste plezier: trotse tante zijn van Sylt!

Lieve pap en mam, het dankwoord aan jullie is het moeilijkste te verwoorden, want die dank is het grootst. We praten altijd heel veel, maar dit zeg ik niet zo vaak: dank jullie wel voor jullie liefde, vertrouwen, steun en vrijheid om te studeren en te reizen. Welke keus ik ook maak, ik voel me door jullie altijd gesteund. Fijn om in drukke tijden bij jullie aan te mogen schuiven, lekker mee-eten, samen een spelletje te spelen en bij te praten over werk (hoe gaat het met je boekje?) en wat er in Oeffelt is gebeurd. Het voelt allemaal zo vanzelfsprekend dat ik soms vergeet hoe bijzonder het is.

Lieve Matthijs, geheel onverwacht kruisten onze wegen weer en sloeg de vonk over. Ik ben heel benieuwd waar de toekomst ons zal brengen, maar laten we beginnen met een biertje want mijn boekje is klaar. Xx

Dank aan iedereen die ik onderweg naar dit proefschrift ben tegen gekomen. Je hebt je sporen achtergelaten, en mij -op welke manier dan ook- een stap verder geholpen!

188

CURRICULUM VITAE

About the author

Name Antonia Katharina Romelia van Luijk (Judith)Date of Birth 16 April 1982Place of Birth Oeffelt, the NetherlandsEmail [email protected]

Antonia Katharina Romelia (Judith) van Luijk was born in Oeffelt, the Netherlands on April 16th 1982. She successfully completed a vocational education (MBO) in clinical chemistry at the Gilde Opleidingen in Blerick in 2002. After nearly a year abroad (Curaçao) she started pursuing an academic career by first taking an applied Bachelor in Zoology at the Applied University of Utrecht (HBO), followed by a Master in Animal Behaviour at the Wageningen University (WO). Her master thesis resulted in a PhD project. She is passionate about animal welfare and responsible animal use in research, which has been the driving force in her academic career.

Judith started her PhD project at the 3R research Centre (now SYRCLE: SYstematic Review Centre for Laboratory animal Experimentation) of the Radboud University, Nijmegen, the Netherlands. She has investigated strategies to stimulate more responsible animal use in research, including optimal 3R implementation. In her thesis she has introduced the systematic review methodology as a science driven and animal-based approach to promote better 3R implementation and improve experimental design of future animal studies. This will contribute to increased validity of animal studies, being an essential step towards improving translatability of animal data to the human situation.

Over the years Judith has become involved in the development of various training programs in systematic reviews of animal studies, nationally and internationally. Currently Judith is successfully coordinating post-graduate training programs in systematic reviews of animal studies at the Radboud university medical center and Radboud University. In addition she coaches researchers conducting their own systematic reviews and travels the world as one of the senior SYRCLE-trainers. Her teaching evaluations by course participants range from good to outstanding.

189

CURRICULUM VITAE

Professional Experience

2015–current Coordinator, trainer and coach in systematic reviews of animal studies. Organising one day hands-on training in Systematic reviews of animal studies at the Radboudumc/Radboud University (Funded by Stichting Reinier Post)

2014–current Coordinator of and tutor in one week course during the Radboud Summer School on quality and translatability of animal studies

2014–current Trainer and coach in systematic reviews of animal studies for various (inter)national training programs and workshops

2010–2016 PhD student at 3R Research Centre/SYRCLE, Radboud University Medical Center, Nijmegen, The Netherlands www.SYRCLE.nl

2009–2010 Master thesis at 3R Research Centre (now SYRCLE), Radboud University Medical Center, Nijmegen, the Netherlands

2006–2007 Student member of the National animal ethics committee for animal use in education

Publications

Scientific publications- Camargo WA, de Vries R, van Luijk J, Bronkhorst EM, Hoekstra JW, van den

Beucken J, et al. Diabetes mellitus and bone regeneration: a systematic review and meta-analysis of animal studies. Tissue engineering Part B, Reviews. 2016. [Epub ahead of print]

- Rongen JJ, Hannink G, van Tienen TG, van Luijk J, Hooijmans CR. The protective effect of meniscus allograft transplantation on articular cartilage: a systematic review of animal studies. Osteoarthr Cartilage. 2015;23(8):1242-53.

- Li J, Hernanda PY, Bramer WM, Peppelenbosch MP, van Luijk J, Pan QW. Anti-Tumor Effects of Metformin in Animal Models of Hepatocellular Carcinoma: A Systematic Review and Meta-Analysis. Plos One. 2015;10(6).

- de Vries RBM, Hooijmans CR, Langendam MW, van Luijk J, Leenaars M, Ritskes-Hoitinga M, et al. A protocol format for the preparation, registration and publication of systematic reviews of animal intervention studies. Evidence-based Preclinical Medicine. 2015(2):1–9.

- van Luijk J, Bakker B, Rovers MM, Ritskes-Hoitinga M, de Vries RBM, Leenaars M. Systematic Reviews of Animal Studies; Missing Link in Translational Research? Plos One. 2014;9(3).

190

- van Luijk J, Leenaars M, Hooijmans C, Wever K, de Vries R, Ritskes-Hoitinga M. Towards Evidence-Based Translational Research: The pros and cons of conducting systematic reviews of animal studies. Altex-Altern Anim Ex. 2013;30(2):256-7.

- van Luijk J, Cuijpers Y, van der Vaart L, de Roo TC, Leenaars M, Ritskes-Hoitinga M. Assessing the application of the 3Rs: a survey among animal welfare officers in The Netherlands. Laboratory animals. 2013;47(3):210-9.

- van Luijk J, Leenaars M, van Dongen AM, van der Vaart L, Ritskes-Hoitinga M. Outcomes of a Dutch workshop on improvements for the 3Rs in daily practice. Altex-Altern Anim Ex. 2012;29(4):440-3.

- van Luijk J, Cuijpers Y, van der Vaart L, Leenaars M, Ritskes-Hoitinga M. Assessing the Search for Information on Three Rs Methods, and their Subsequent Implementation: A National Survey among Scientists in The Netherlands. Atla-Altern Lab Anim. 2011;39(5):429-47.

- Salomons AR, van Luijk JAKR, Reinders NR, Kirchhoff S, Arndt SS, Ohl F. Identifying emotional adaptation: behavioural habituation to novelty and immediate early gene expression in two inbred mouse strains. Genes Brain Behav. 2010;9(1):1-10.

Other- Interview (in Dutch): Oude stoffen – oude protocollen. Toxiciteit testen zonder

labdieren? C2W Life sciences 2011 107(10) p15.

Conference contributions

Presentations - Van Luijk, J. ‘Improving transparency on quality and translatability of animal studies

using a new science driven approach’ EUSAAT annual congress, 24-27 August 2016. Linz, Austria.

- Van Luijk, J. ‘Laboratory Animal Science 2.0, from the 3Rs to SRs.’ Symposium 60 year Central animal facility; Changing perspective on the impact of animal studies on health care, October 2015. Nijmegen The Netherlands.

- Van Luijk, J. ‘Systematic Reviews of animal studies- Definitions, Challenges and Implementation.’ 9th World congress on alternatives and animal use in the life sciences, August 2014. Prague, Czech Republic. Keynote speaker session III-4

- Van Luijk, J. ‘Comprehensive search strategies in systematic reviews of animal studies’ 3rd International symposium on systematic review and meta-analysis of laboratory animal studies, 13-14 November 2014, The Embassy of Australia, Washington DC, USA. Invited speaker

- Van Luijk, J. ‘Systematic reviews of animal studies; Pitfalls and opportunities’ LASA winter Meeting Birmingham UK 25-27 November 2014 Invited speaker

CURRICULUM VITAE

191

- van Luijk, J. ‘Systematic Reviews of animal studies; the Benefits for Refinement, Animal welfare and Quality of Science’ SGV meeting, November 2013. Lausanne, Switzerland Invited speaker

- van Luijk, J. ‘Internal validity of animal studies; a potential threat to translational research’ NCEBP PhD Retreat, September 2013. Wageningen, the Netherlands. Awarded Best Presentation

- van Luijk, J. ‘Betere dierproeven (improved animal experimentation)’ Brilliant Failure Award (Health) by ZonMw, September 2013. Lunteren, the Netherlands. Nominee

- van Luijk, J. ‘De dierverzorger/biotechnicus als waardevolle schakel voor het toepassen van de 3V’s’ Biotechnische dagen Veldhoven the Netherlands, 9-11 November 2010 Invited speaker

- van Luijk J ’Improving implementation of 3R knowledge - which way to go?’ FELASA-congress, Helsinki Finland, 14-16 June 2010. Nominee Yong scientist award

- van Luijk J ‘Improving implementation of 3R knowledge – outcome of a national workshop’ FELASA-congress, Helsinki, Finland, 14-16 June 2010.

Posters

- Van Luijk J, M. Leenaars, M. Ritskes-Hoitinga. ‘Improving transparency on quality and translatability of animal studies using a new science driven approach’ FELASA congress, Brussels (Belgium), 13-16 June 2016.

- Leenaars M, J. Van Luijk, C. Hooijmans, M. Ritskes-Hoitinga, R. de Vries, K. Wever. ‘Unique e-learning module on systematic reviews and meta-analysis of animal studies; an interactive introduction to the basic principles and power of this methodology’ FELASA congress, Brussels (Belgium), 13-16 June 2016

- van Luijk J, M. Leenaars, R. de Vries, C. Hooijmans, M. Ritskes-Hoitinga. ‘Systematic Reviews of animal studies: assessing translational value’ NCEBP PhD retreat, Wageningen (The Netherlands), 29-30 September 2011.

- van Luijk J, M. Leenaars, R. de Vries, C. Hooijmans, M. Ritskes-Hoitinga. ‘How the 3Rs can benefit from Systematic Reviews’ 8th World congress on alternatives and animal use in the life sciences, Montréal (Canada), 21-25 Augustus 2011.

- van Luijk, J, Y. Cuijpers, L. van der Vaart, M. Leenaars, M. Ritskes-Hoitinga. ‘Enhancing implementation of the 3Rs in daily practice – which way to go?’ 8th World congress on alternatives and animal use in the life sciences, Montréal (Canada), 21-25 Augustus 2011.

Other- Chair ‘laptop session’ Implementing systematic reviews of animal studies – the

next step, Radboudumc Nijmegen the Netherlands, 16 October 2015.

CURRICULUM VITAE

192

- Organizing committee Symposium 60 year Central animal facility; Changing perspective on the impact of animal studies on health care, Nijmegen The Netherlands, October 2015.

- Chair session 3: ‘Exciting new techniques for systematic review and meta-analysis’ 3rd international symposium on systematic review and meta-analysis of laboratory animal studies, The Embassy of Australia, Washington DC, USA, 13-14 November 2014.

- Chair ‘laptop session’ Inspiration day on synthesis of evidence, NWO, Den Haag, the Netherlands, 2 July 2014.

Trainer

- One day workshop: Hands-on training in systematic reviews, Nijmegen, Amsterdam, Utrecht (the Netherlands), Geneva, Lausanne, Zurich, Basel (Switzerland), Italy (Varese), Aarhus (Denmark), Voss (Norway), Stockholm (Sweden), Sao Paulo (Brazil). – Topics: “Comprehensive search strategies”, “Risk of bias assessment” and “data extraction and meta-analysis”

- Laboratory Animal Science course, Radboudumc Nijmegen (the Netherlands), AMC and KNAW Amsterdam (the Netherlands), Aarhus University (Denmark) – Topics: “Introduction to the systematic review methodology”, “Comprehensive search strategies”, “Reporting and publication bias”

- Radboud Summer School, Nijmegen (the Netherlands) – Topics: “Translational challenges”, “Risk of bias assessment” and “Randomisation and blinding of experimental design”

- Systematic reviews at EFSA, European Food and Safety Authority, Parma – Topics: “Introduction to the systematic review methodology” and “Comprehensive search strategies”

- Master of Science in Laboratory Animal Science, RWTH Aachen University (Germany) – Topics: “Experimental design and reporting, 3Rs”, “Systematic reviews and searching”, ”Systematic reviews advantages and challenges”

- Leuk om te leren, Weekendschool Arnhem – Topics: “Dierproeven – waarom en hoe doen ze dat?”

CURRICULUM VITAE

193

CURRICULUM VITAE

The next steps towards responsible animal-based research

Documents

Transcript of The next steps towards responsible animal-based research