|Year : 2019 | Volume
| Issue : 3 | Page : 77-84
Comparison of adherence to ARRIVE guidelines in animal research articles published in the years 2009 and 2016 in pharmacology journals: An observational study
Sharmila Jalgaonkar, Tausif Mapara, Apoorv Verma, Mohsin Sayyed
Department of Pharmacology and Therapeutics and Seth GS Medical College and KEM Hospital, Mumbai, Maharashtra, India
|Date of Submission||04-Jun-2019|
|Date of Decision||24-Sep-2019|
|Date of Acceptance||12-Sep-2019|
|Date of Web Publication||19-Nov-2019|
Department of Pharmacology and Therapeutics and Seth GS Medical College and KEM Hospital, Parel, Mumbai, Maharashtra
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Objective: To evaluate and compare the adherence of the ARRIVE guidelines before and after its introduction by the articles published by two leading journals publishing animal research articles. Materials and Methods: Literature of original research articles of animal studies was obtained from two leading open-access journals. A combined total of 100 articles were selected from these journals from the years 2009 and 2016 (50 articles from each year). Each article was assessed by two investigators independently. The percentage of articles adhering to each item in the ARRIVE guidelines was calculated for both 2009 and 2016. The percentage of overall adherence of the articles to all recommendations was calculated. Completeness of reporting of an article was also assessed. The percentage of overall adherence of the articles to all recommendations was calculated. The adherence to the ARRIVE guidelines in the two journals was statistically compared using Chi-square test. P = 0.05 was considered as a measure of statistical significance. Results: A combined total of 100 articles from the two journals, Indian Journal of Pharmacology and Journal of Pharmacology and Experimental Therapeutics, during the years 2009 and 2016 (50 articles from each year) were studied for adherence to ARRIVE and completeness of studies. Despite introduction of the ARRIVE guidelines, 8 of 38 unreported items in articles of 2009 are still unreported in 2016. A statistically insignificant improvement was observed in total adherence to 22 of 38 recommendations of the ARRIVE checklist by >50% of the articles in 2016. A statistically significant improvement was observed in completeness of reporting on 14/38 recommendations, 6 years after the introduction of the ARRIVE guidelines. The study revealed that ~58% of recommendations of the ARRIVE guidelines are being followed by the articles selected from 2016. Conclusion: No significant improvement in quality of reporting of animal research published by two reputed journals was observed even 6 years after the introduction of the ARRIVE guidelines. However, the study revealed improvement in the completeness of reporting of an animal research article. Active endorsement of the ARRIVE guidelines by editors of journals publishing animal research is recommended.
Keywords: Animals in Research: Reporting In vivo Experiments, animal studies, publication
|How to cite this article:|
Jalgaonkar S, Mapara T, Verma A, Sayyed M. Comparison of adherence to ARRIVE guidelines in animal research articles published in the years 2009 and 2016 in pharmacology journals: An observational study. J Pharmacol Pharmacother 2019;10:77-84
|How to cite this URL:|
Jalgaonkar S, Mapara T, Verma A, Sayyed M. Comparison of adherence to ARRIVE guidelines in animal research articles published in the years 2009 and 2016 in pharmacology journals: An observational study. J Pharmacol Pharmacother [serial online] 2019 [cited 2020 May 25];10:77-84. Available from: http://www.jpharmacol.com/text.asp?2019/10/3/77/271232
| Introduction|| |
Scientific publications enable the transmission of results and ideas throughout the scientific community. Over the years, the number of scientific articles published across various fields has increased. These articles aim to promote the scientific understanding and health care. Quality of reporting is very important in medical research. Poor reporting of the studies may lead to poor assessment of the scientific research and misinterpretation of the results of the study. Accurate and transparent reporting is vital to allow the reader to assess the methods of the study and the reliability and importance of the scientific findings. This is particularly necessary for scientific research using animals, as poorly designed experiments and reporting omissions can raise both ethical and scientific concerns.
The animal studies are a vital part of biomedical research and have made a significant contribution to human health and welfare. Biomedical animal research is used to pervade human research and practice on the assumption that animal models closely relate to and are comparable to humans with respect to physiology and response to drugs and disease.
Reproducibility of research findings is the hallmark of scientific evidence. However, a big concern today is poor design, inadequate analysis, and reporting of the animal experiments. This makes these experiments scientifically invalid and irreproducible leading to a waste of animal and scientific resources. Furthermore, poorly written articles reduce the impact of the studies and hamper their inclusion in systematic reviews and meta-analysis. Thus, a well-designed and well-conducted research must have accurate and adequate reporting of experimental details.
Translation of animal study data to humans with sufficient reliability remains a challenge owing to biological differences between species, internal validity, differences in experimental designs between animal and clinical studies, insufficient reporting, and publication bias. Low translational studies can be of three types – poorly reported well-conducted studies, poorly reported poorly conducted studies, and poorly conducted but falsely reported studies. The reason for poor reporting can be attributed to sheer negligence in reporting, absence of reporting guidelines altogether, or lack of awareness of existing reporting guidelines. Improving reporting standards by the following existing guidelines and their endorsement by editors of journals can help curb poor reporting. Mere knowledge of reporting guidelines, area of research, and ethical issues does not rule out the last type of low translational valued studies. However, creating herd knowledge is an optimistic beginning toward eventual change in attitude and practice of conduction of studies and hence their reporting.
To improve the standard of preclinical animal experiments and impart more transparency and accuracy in their reporting through publications, the ARRIVE guidelines were first published in 2010 and further revised in 2014. ARRIVE stands for the Animal Research: Reporting In vivo Experiments. The ARRIVE statement is a 20-item checklist which guides the author on reporting various parts of the article right from the abstract to the discussion. It details the minimum information that should be reported when using animals in a research study. The goal is to ensure that the reader can assess the methodological rigor of the experiment, and other scientists can evaluate and reproduce the methods.
A summary of the areas encompassed by the ARRIVE guidelines is covered in [Figure 1].
|Figure 1: Summary of the areas encompassed by the Animals in Research: Reporting In vivo Experiments guidelines|
Click here to view
Over 300 journals' editors have started following ARRIVE guidelines for publishing animal research. However, certainty of the impact of introduction of the ARRIVE guidelines remains debatable because of little to no compliance by authors, reviewers, and journal editors. Various studies show mixed results. A study conducted by Leung et al. found that none of the journals reviewed by them actually had 100% adherence to the ARRIVE guidelines.
Indian Journal of Pharmacology (IJP) and Journal of Pharmacology and Experimental Therapeutics (JPET) are leading, reputed, and internationally recognized journals in the field of pharmacology published from India and the United States of America, respectively. A major part of the studies published in IJP and JPET is constituted by animal research. It was, thus, considered worthwhile to assess the adherence to the ARRIVE guidelines in the reporting of published animal studies in these two reputed journals. Thus, to compare the adherence of the guidelines, we decided to review articles published by IJP and JPET in 2009 and 2016, that is, before and after the formulation of the ARRIVE guidelines.
| Materials and Methods|| |
The study was submitted to the Institutional Ethics Committee for approval and was granted exemption from review. To determine the adherence of the ARRIVE guidelines, we conducted a comprehensive and systematic literature search for original research articles of animal studies published by two leading journals, IJP and JPET, which regularly publish animal studies. We downloaded the original research articles published between January and December 2009 and between January and December 2016. This corresponds to the time period of 1 year before the publication of the ARRIVE guidelines and 2 years after the revision of the ARRIVE guidelines. After screening and excluding articles (short communications, letter to editors, case reports, review articles, any research articles involving human participants, and in vitro experimental studies were not taken into account) not meeting our inclusion criteria of original articles on animal studies only, a total of 25 articles were randomly chosen from each journal by means of a computer-generated randomization table for year 2009 and then for year 2016 using SAS (Statistical Analysis System) programming for randomization. The study outline has been depicted in the [Figure 2].
A total of 100 articles (50 from each journal) were analyzed using the ARRIVE guidelines checklist and checked for adherence. Each article was assessed by two investigators independently. Each recommendation in the ARRIVE guideline was categorized as yes, no, or partial. “Yes” indicated that all information pertaining to particular recommendation are provided. “No” indicated that information related to a particular recommendation is not provided at all. If some information of particular recommendation was provided, it was categorized as “partial.” Any discrepancies were resolved by reanalysis of studies, and a final decision was reached by discussion and deliberation among all investigators. The data were summarized as numbers and percentages using Microsoft Excel (Version 2018). The percentage of articles having adherence for each item was calculated for both 2009 and 2016. The percentage of overall adherence of articles to all recommendations was calculated. The adherence to the ARRIVE guidelines in the two journals was statistically compared using Chi-square test. Adherence of the recommendation by >50% of the articles was considered optimum. We used the traditional P = 0.05 as a measure of statistical significance. GraphPad Instat version 3(GraphPad Software company by Dr. Harvey Motulsky located at California San Diego, USA) was the software used for statistical analysis.
| Results|| |
We selected a total of 100 articles from the two journals, IJP AND JPET, during the years 2009 and 2016 (50 articles from each year) and studied the adherence to the ARRIVE guidelines of all these articles [Table 1]. [Table 2] denotes the ARRIVE recommendation reported by all the articles during 2009 and 2016, whereas [Table 3] gives the ARRIVE recommendations reported by none of the articles during 2009 and 2016. Recommendations showing a significant increase in reporting in 2016 after the implementation of the ARRIVE guidelines are mentioned in [Table 4].
|Table 1: Effect of ARRIVE guidelines on quality of animal studies reporting using ARRIVE adherence checklist (n=50)|
Click here to view
|Table 2: ARRIVE recommendation reported by all the articles during 2009 and 2016|
Click here to view
|Table 3: ARRIVE recommendations reported by none of the articles during 2009 and 2016|
Click here to view
|Table 4: Recommendations showing a significant increase in reporting in 2016 after the implementation of ARRIVE guidelines|
Click here to view
| Discussion|| |
At the inception of this study, we could hardly find a study comparing the adherence to the ARRIVE guidelines by published animal research articles before and after the introduction of the ARRIVE guidelines. The ARRIVE guidelines lays down standards for reporting in all sections of published articles as follows: the introduction (the background and objectives of the study), the methods (an ethical statement, description of the study design, experimental procedures and animals, housing and husbandry, sample size, and statistical methods), the results (numbers analyzed and adverse events), the discussion (interpretation of the data, their implications, and potential for translation), and the acknowledgments. In our study, we analyzed the completeness of studies and adherence to the ARRIVE guidelines by conducting literature search of a total of 100 original animal research articles published in IJP and JPET during the years 2009 and 2016 (50 articles from each year). We also analyzed the effect of the ARRIVE guidelines on quality of reporting of animal studies 1 year before and 6 years after the endorsement of the ARRIVE guidelines by these journals using the ARRIVE adherence checklist [Figure 3].
|Figure 3: Effect of ARRIVE guidelines on quality of animal studies reporting using Animals in Research: Reporting In vivo Experiments adherence checklist|
Click here to view
The completeness of study with respect to the ARRIVE guidelines assessed using the ARRIVE adherence checklist was not optimum in the year 2009 as only 15 of 38 recommendations were properly reported by >50% of the articles published before the introduction of ARRIVE guidelines. However, in the year 2016, 22 of 38 recommendations were followed by > 50% of the articles published after the introduction of the ARRIVE guidelines. This difference was not found to be statistically significant using the Chi-square test (P = 0.108). Hence, our study shows no significant improvement in the quality of reporting by reputed journals publishing animal research.
In our study, we found that 5/38 recommendations, as mentioned in [Table 2], were reported by all the articles (100%) published during both 2009 and 2016. This could be because of them being the essential elements for any research study to be published.
However, the adherence to recommendation listed in [Table 3] was not done by any article (0%) of both the journals published in 2009 as well as 2016. This constituted 8 of total 38 recommendations given by the ARRIVE guidelines. The ARRIVE guidelines are based on the principles of the three Rs (replacement, refinement, or reduction). Yet, implications of the experimental methods on these principles were not described by any article.
There was also statistically significant improvement in completeness of reporting on 14/38 recommendations 6 years after the ARRIVE guidelines after published, as shown in [Table 4]. The reason behind this can be attributed to increased awareness of these guidelines among the researchers. Furthermore, such guidelines increase the understanding of researchers on how to correctly publish their studies and what information should be provided in the research article. Six years after the introduction of the ARRIVE guidelines, a significant improvement in understanding of the researchers to write the abstract of the research article properly was seen in our study. A significant improvement was also seen in reporting of sufficient scientific background of the research conducted. However, even after the formulation of guidelines, researchers are not able to explain the reason for choosing the animal species and the model to achieve the objectives of the study. A similar finding was reported in a study by Hotwani et al., 2017, assessing adherence to the ARRIVE guidelines. Introduction of the ARRIVE guidelines increased the reporting of randomization procedure during allocation of animals as well as blinding during assessment of results. This is a positive change compared to 2009 where ~ 85% of the studies were not reporting them. Randomization eliminates selection bias and ensures baseline comparability of different groups. Blinding also helps to eliminate the bias in results. Despite this, randomization and blinding were reported in <50% of the articles of both the journals. The ARRIVE guidelines helped in proper detailed reporting of experimental procedure with respect to its methodology timing and place. However, site of the procedure was reported in mere 22% of the articles published in both the journals, even 6 years after the ARRIVE guidelines were published. There is a significant improvement in reporting the details of animals used and their housing and husbandry conditions. Proper detailing of animals used in the research helps in ensuring the validity and reproducibility of results. Introduction of the ARRIVE guidelines caused a statistically significant improvement in reporting of primary and secondary experimental outcomes and their assessment. They were reported by 100% of the articles on animal studies published in 2016 as compared to 78% in the year 2009. A significant improvement was also seen in reporting the limitations of the studies as well as the translation of research findings into any relevance to humans. The ARRIVE guidelines also soared the listing of funding sources and their role in the study significantly.
Our study showed that in 96% of the articles in the year 2016 from both the journals, details of sample size calculation were reported. This is in contrast to the findings from a study of Hotwani et al. in 2017, where they claimed poor reporting of sample size calculation. Proper and correct sample size calculation powers the study to find a significant difference between treatment groups. Thus, the ARRIVE guidelines are promoting the conduct of studies with sufficient power.
Widespread lack of transparent quality reporting of study design and its implementation is one important issue with animal studies. Inadequate reporting of key aspects of experimental design may reduce the impact of studies and act as a barrier to translation by preventing reliability or inclusion in meta-analysis.
The ARRIVE guidelines were developed to address the lack of reproducibility in biomedical animal studies and improve the communication of research findings. This study shows improvement in reporting of animal studies after the publication of the ARRIVE guidelines in the selected journals. Our study shows that almost 58% of recommendations of the ARRIVE guidelines are now being followed by animal research articles in these journals. Despite publishing author guidelines by both the journals to report the animal research as per the ARRIVE guidelines only, almost half of the recommendations are not being followed by the authors, including five recommendations which are not being reported at all. Thus, this study will sensitize the authors about filling up these lacunae.
In our study, we analyzed the articles on animal studies published in only two journals. This could be the only limitation of our study. Nevertheless, this is the first study of its kind to compare the quality and completeness of reporting of animal studies before and after the introduction of the ARRIVE guidelines. This study gives an insight into the importance of adhering to these guidelines. It also apprises us of the need to routinely evaluate the quality of reporting of research articles in the reputed journals.
Full implementation of every recommendation of the ARRIVE guidelines seems highly impractical and very unlikely in the future. However, journals must make certain recommendations of the ARRIVE guidelines a compulsion before any research article will be published. This will compel all the authors to adhere to the recommendations as per the ARRIVE checklist. The authors should also be made to fill the ARRIVE checklist while submitting their manuscript. Journals should also not restrict the methodology section by words as this will encourage comprehensive reporting of the material and methods used. According to recommendations of Baker et al. 2014, the recommendations of the ARRIVE guidelines that involve reporting the justification of selection of species and strain of animal used and the route and timing of delivery of agents, often, form a part of the ethical review process  of the study which is currently being reported. Hence, there is no need to repeat this information in an article.
| Conclusion|| |
The present study reveals a modest improvement in the completeness of study and quality of reporting of animal research published by two reputed journals 6 years after the introduction of the ARRIVE guidelines. Frequent studies including more number of journals need to be conducted to further gauge reporting trends. Findings of this study will help us to focus more on the areas of deficient animal studies' reporting. It also signifies the need to enhance the awareness of the ARRIVE guidelines among researchers as well as editors of the journals as these guidelines can certainly improve the quality of reporting of animal-based experiments. We strongly recommend the strict adherence of the ARRIVE guidelines checklist by the animal researchers while reporting their study. We recommend editors of journals publishing animal research to actively endorse these ARRIVE guidelines in their guidelines for authors.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Kilkenny C, Parsons N, Kadyszewski E, Festing MF, Cuthill IC, Fry D, et al.
Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS One 2009;4:e7824.
Greek R, Rice MJ. Animal models and conserved processes. Theor Biol Med Model 2012;9:40.
Jilka RL. The road to reproducibility in animal research. J Bone Miner Res 2016;31:1317-9.
Festing MF, Nevalainen T. The design and statistical analysis of animal experiments: Introduction to this issue. ILAR J 2014;55:379-82.
van Luijk J, Bakker B, Rovers MM, Ritskes-Hoitinga M, de Vries RB, Leenaars M, et al.
Systematic reviews of animal studies; missing link in translational research? PLoS One 2014;9:e89981.
Kilkenny C, Browne W, Cuthill IC, Emerson M, Altman DG, NC3Rs Reporting Guidelines Working Group, et al.
Animal research: Reporting in vivo
experiments: The ARRIVE guidelines. Br J Pharmacol 2010;160:1577-9.
Percie du Sert N, Hurst V, Ahluwalia A, Alam S, Altman D, Avey M et al
. Revision of the ARRIVE guidelines: rationale and scope. BMJ Open Science. 2018;2 (1):e000002.
Baker D, Lidster K, Sottomayor A, Amor S. Two years later: Journals are not yet enforcing the ARRIVE guidelines on reporting standards for pre-clinical animal studies. PLoS Biol 2014;12:e1001756.
Leung V, Rousseau-Blass F, Beauchamp G, Pang DS. ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal research: Reporting of in vivo
experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia. PLoS One 2018;13:e0197882.
Hotwani JH, Deshmukh KR, Mehta MN. Adherence to ARRIVE guidelines in animal research articles published in a National and International Pharmacology Journal. Int J Basic Clin Pharmacol 2017;6:1696-700.
Vesterinen HM, Egan K, Deister A, Schlattmann P, Macleod MR, Dirnagl U, et al.
Systematic survey of the design, statistical analysis, and reporting of studies published in the 2008 volume of the journal of cerebral blood flow and metabolism. J Cereb Blood Flow Metab 2011;31:1064-72.
McCance I. Assessment of statistical procedures used in papers in the Australian veterinary journal. Aust Vet J 1995;72:322-8.
Karp NA, Meehan TF, Morgan H, Mason JC, Blake A, Kurbatova N, et al.
Applying the ARRIVE guidelines to an in vivo
database. PLoS Biol 2015;13:e1002151.
[Figure 1], [Figure 2], [Figure 3]
[Table 1], [Table 2], [Table 3], [Table 4]