Table of Contents  
RESEARCH PAPER
Year : 2011  |  Volume : 2  |  Issue : 2  |  Page : 85-89  

Quality of reporting statistics in two Indian pharmacology journals


Department of Pharmacology, Govt. Medical College, Surat, Gujarat - 395 001, India

Date of Web Publication6-Jun-2011

Correspondence Address:
Jaykaran
Department of Pharmacology, Govt. Medical College, Surat, Gujarat - 395 001
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/0976-500X.81897

Rights and Permissions
   Abstract 

Objective: To evaluate the reporting of the statistical methods in articles published in two Indian pharmacology journals. Materials and Methods: All original articles published since 2002 were downloaded from the journals' (Indian Journal of Pharmacology (IJP) and Indian Journal of Physiology and Pharmacology (IJPP)) website. These articles were evaluated on the basis of appropriateness of descriptive statistics and inferential statistics. Descriptive statistics was evaluated on the basis of reporting of method of description and central tendencies. Inferential statistics was evaluated on the basis of fulfilling of assumption of statistical methods and appropriateness of statistical tests. Values are described as frequencies, percentage, and 95% confidence interval (CI) around the percentages. Results: Inappropriate descriptive statistics was observed in 150 (78.1%, 95% CI 71.7-83.3%) articles. Most common reason for this inappropriate descriptive statistics was use of mean ± SEM at the place of "mean (SD)" or "mean ± SD." Most common statistical method used was one-way ANOVA (58.4%). Information regarding checking of assumption of statistical test was mentioned in only two articles. Inappropriate statistical test was observed in 61 (31.7%, 95% CI 25.6-38.6%) articles. Most common reason for inappropriate statistical test was the use of two group test for three or more groups. Conclusion: Articles published in two Indian pharmacology journals are not devoid of statistical errors.

Keywords: Inappropriate statistics, Indian Journals, Pharmacology


How to cite this article:
Jaykaran, Yadav P. Quality of reporting statistics in two Indian pharmacology journals. J Pharmacol Pharmacother 2011;2:85-9

How to cite this URL:
Jaykaran, Yadav P. Quality of reporting statistics in two Indian pharmacology journals. J Pharmacol Pharmacother [serial online] 2011 [cited 2019 Jul 15];2:85-9. Available from: http://www.jpharmacol.com/text.asp?2011/2/2/85/81897


   Introduction Top


Statistics is a tool in the hand of a researcher by which he can analyze his study findings. If statistics methods used in the study are inappropriate, the conclusions drawn from the study become questionable. Studies with poor methodological quality and poor statistics cannot prove or disprove study hypothesis with certainty. So conduction of these kind of studies raises many ethical issues like exposure of participants to risk of new intervention, deprivation of participants to established treatment, unnecessary use of animals in experimental studies, misuse of resources, and wrong clinical judgments on the basis of these studies once they get published. [1],[2],[3],[4],[5] Despite publication of various guidelines related to the reporting of various methodological and statistical parameters of a study, it has been observed that quality of statistical reporting is poor in various biomedical journals. [6],[7] Various surveys done for the articles published in western medical journals indicate that statistical error in the published article is a common phenomenon and error rate may vary from 30% to 90%. [8],[9],[10],[11] Although many surveys are done for statistical reporting in western journals, data are lacking for studies published in Indian medical journals. Some small studies done for articles published in Indian medical journals observed the same phenomenon of poor reporting of various statistical parameters. [12],[13] It is observed that data related to the statistical reporting of articles published in pharmacology journals of India are lacking. So this study was designed with the aim of evaluating articles published in Indian pharmacology journals (Indian Journal of Pharmacology (IJP) and Indian Journal of Physiology and Pharmacology (IJPP)) for statistical reporting. These two pharmacology journals are widely circulated and Pubmed-indexed Indian pharmacology journals; hence they were selected for evaluation.


   Materials and Methods Top


All articles published in IJP and IJPP between 2002 to the latest issue of 2010 were downloaded from journals website ( www.ijp-online.com and www.ijpp.com ). In case of IJPP, articles published since 2002 were available on website. So to maintain uniformity for both journals, all articles which were published in or after 2002 were downloaded. Only original studies were considered for analysis. Short communications, research letters, and letter to editors were not taken into account. In case of IJPP, only articles related to pharmacology were downloaded. All articles were evaluated independently by first (J.K.) and second author (P.Y.). These articles were appraised for quality of reporting of descriptive statistics and quality of reporting of inferential statistics. Descriptive statistics is evaluated on the basis of appropriate reporting of data as mean, median, or frequency with the central tendencies. Inferential statistics was evaluated on the basis of reporting of assumptions of statistical tests and inappropriateness of statistical tests. Common methods of statistical analysis were also noted. Common reasons for inappropriate descriptive statistics and common reasons for inappropriate statistical tests were also noted. Any disagreements between two authors were resolved by consensus (k = 0.87 for inappropriate statistical tests). Appropriate method of descriptive statistics of ratio and interval data following the normal distribution is mean (SD) or mean ± SD. For ordinal data and for ratio and interval data not following the normal distribution, appropriate descriptive statistics is median and interquarantile range and for nominal data, frequency and percentage are appropriate. Appropriate statistical tests are selected on the basis of aim of the study and types of data. Once the statistical test is selected, all the assumptions for that particular statistical test should be checked before applying that statistical test.

Statistics

Values are described as frequencies, percentages, and 95% confidence interval around percentages.


   Results Top


Total 196 articles from various areas of research were downloaded from the journal sites. Major areas of research were diabetes (39 (19.8%) studies), central nervous system (17 (8.6%)), hepatoprotection (18 (9.1%)), and cardiovascular (17 (8.6%)). Other areas were inflammation (11 (5.6%)), antioxidants (9 (4.5%)), pain (7 (3.5%)), gastrointestinal (6 (3%)), and immunomodulation (4 (2%)). Most of the articles were dealing with animal studies (83% vs. 17%).

Descriptive statistics

Out of these 196 articles, information related to descriptive statistics was missing in four articles. Out of remaining 192 articles, inappropriate descriptive statistics was reported in 150 (78.1%, 95% CI 71.7-83.3%) articles. Out of these 150 studies 106/129 (82.1%) were from IJP and 44/63 (69.8%) from IJPP.

Most common reason for inappropriate reporting of descriptive statistics was the use of mean ± SEM at the place of "mean (SD)" or "mean ± SD" [Table 1].
Table 1: Inappropriate descriptive statistics in articles published in two Indian pharmacology journals (n = no. of articles)

Click here to view


Inferential statistics

Statistical methods

Most common type of statistical method used in the articles of both pharmacology journals was "one-way analysis of variance (ANOVA)" [Table 2]. Out of 214 statistical methods only 10.7% were nonparametric methods.
Table 2: Statistical methods used in articles published in two Indian pharmacology journals

Click here to view


Assumptions of statistical tests

Information related to fulfillment of assumptions of statistical tests was mentioned in only two articles. In one article, normal distribution was checked by Komolgorov-Smirnov test.

Inappropriate statistical tests

Out of 196 articles from both journals, information related to statistical test was missing in four articles. Out of remaining 192 articles inappropriate statistical tests were found in 61 (31.7%, 95% CI 25.6-38.6%) articles. Most common reason for inappropriate statistical test was use of two group test for analysis of three or more than three groups (22.9%) [Table 3].
Table 3: Inappropriate statistical tests in articles published in two Indian pharmacology journals (n = no. of articles)

Click here to view



   Discussion Top


Main findings of this study are as follows: majority of articles published in two Indian pharmacology journals have inappropriate reporting of descriptive statistics, assumption of statistical tests were checked in only two article, and inappropriate statistical tests was used to analyzed data in 31.7% of articles.

One major finding was inappropriate use of "mean ± SEM" for description of data. The ideal method of reporting of these kinds of data is "mean (SD)" or "mean ± SD." Although SD and SEM look similar, they give different information. [14] Standard deviation (SD) shows variability around the mean within the sample and standard error of mean (SEM) shows probability of proximity of sample mean around the population mean. [15] Readers and researchers are interested in knowing variability within the sample not the proximity of mean to the population mean. The value of SEM is always less than SD so when it is used as descriptive statistics readers may falsely conclude that variability of sample is small. To prevent confusion with CI in the place of "mean ± SEM" reporting as mean (SD) is a better method. [16] Similar findings were also observed in other studies done for western and Indian journals. In a study done by Negele (2001) for the articles published in four anesthesia journals, it was observed that inappropriate use of SEM was present in 23% articles. [17] In a similar study done for four Indian medical Journals by Saurabh et al. (2010), it was observed that inappropriate reporting of SEM was common in articles published in basic science journals but this inappropriateness was negligible in journals related to clinical practices. [18] In spite of highlighting this issue in various surveys, this practice of reporting the variability as SEM is common and is a matter of concern. [16],[19],[20] Ordinal data like scores or scales are sometimes described as "mean ± SEM" which is wrong as they should be reported as median (range). [16] This error was not much observed in this study as majority of data were in ratio scale but in some other studies this error found to be much more. [21]

In this study, majority of statistical tests were parametric tests. Nonparametric tests were used less frequently (10.7%). It has been observed that the use of nonparametric statistics is increasing regularly in articles published in medical journals. [22] Low proportion of nonparametric statistics may be because of ignoring of assumptions underlying parametric statistics by authors. [23] Most of the articles in this study were animal experiments where usually many groups are used for comparison; hence one-way ANOVA was most frequently used statistical method whereas in studies done for articles published in clinical journals student t test seems to be the most common method. [24] Most of the statistical methods were simple methods and sophisticated methods like survival analysis, multiple regressions were not observed. In this study, it is found that three statistical tests - one-way ANOVA, unpaired t test, and paired t test - cover about 82% of all statistical methods, so these are the most frequently used tests and interpretation of these tests should be taught in detail to postgraduate students and young researchers.

In this study, it was observed that fulfilling of assumptions of statistical tests was not reported in almost all the studies. One reason may be underreporting and second reason may be ignorance of researcher. Each statistical test has some assumptions and these assumptions need to be fulfilled before application of that statistical test. Information regarding fulfilling of these assumptions should be included in the manuscript. Similar observation was made in other studies. [25]

About 32% articles have at least one inappropriate statistical test and most frequent mistake was the use of two group test for comparison of three or more groups like use of unpaired t test for comparison of three unpaired groups. This problem was observed in other studies done for statistical reporting in western journals. [23],[25],[26] Frequency of statistical errors varies from journal to journal like for Chinese journals it is 46%, [23] for surgical journal it is 64%, [27] and for urology journals it is 28%. [28] Most common problem was the use of multiple unpaired t tests at the place of one-way ANOVA. Despite repeated recommendations, unpaired t test still continues to be used at the place of ANOVA, [29] which is a matter of concern. Another mistake observed was the use of parametric statistical tests for ordinal data like scores or scales. It is very important to understand that ordinal data do not follow the normal distribution. Hence the use of parametric tests for these kinds of data is not justifiable. [30] In a study, it was found that ordinal data were used in about one-third of articles and these data are appropriately presented and analyzed in 50% articles. [31] In this study, most of the articles were dealing with continuous variables so this finding is not as prominent as observed in other journals.

There may be various reasons for finding these kinds of statistical errors in the published articles like insufficient knowledge of statistics and research methodology in researcher, [32],[33] insufficient ethical review of protocol submitted for permission from institutional ethics committee, insufficient peer review of submitted manuscript, and less knowledge of statistics in journal editors. It is observed that in ethics committee statistical issues are not discussed in detail as members of ethics committee usually focus their attention on informed consent, etc. It is important to understand that poor-quality research is also unethical. So ethics committee should also have a qualified medical statistician who can give advice regarding the methodological and statistical aspects of the protocol. [34] Every article submitted to the journal should also be sent for statistical review and journals should have statistical advisors in their editorial board. It is observed that many journals do not have statistical advisors. [35] Postgraduate students and young researchers should be trained in research methodology and biostatistics. Research methodology should be incorporated in the curriculum of postgraduate course.

This study has some limitations. One of the major limitations is that focus of this study is very narrow. Only few but very important statistical parameters were observed. Parameters like post hoc power, adjustment of multiple endpoints, sample size calculation, confidence interval, use of exact P value etc. were not taken into consideration. Second limitation is only two pharmacology journals were considered for evaluation. As far as our perception goes, this is the first study done for articles published in Indian pharmacology journals and may be at international level.

This study shows that inappropriate statistics is very common in the articles published in Indian pharmacology journals. Measures should be taken by journal editors, ethics committee, and researchers to prevent these errors.

 
   References Top

1.Altman DG. Statistics and ethics in medical research: Misuse of statistics is unethical. BMJ 1980;281:1267-9.  Back to cited text no. 1
    
2.Halpern S, Karlawish JH, Berlin JA. The continuing unethical conduct of underpowered clinical trials. JAMA 2002;288:358-62.  Back to cited text no. 2
    
3.Altman DG: Statistics and ethics in medical research: III. How large a sample? BMJ 1980;281:1336-8.  Back to cited text no. 3
    
4.Newell DJ. Type II errors and ethics. BMJ 1978;23:1789.  Back to cited text no. 4
    
5.Altman DG. The scandal of poor medical research. BMJ 1994;308:283-4,   Back to cited text no. 5
    
6.Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research. PLoS Biol 2010;8: e1000412.  Back to cited text no. 6
    
7.Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, et al. The revised CONSORT statement for reporting randomized trials: Explanation and elaboration. Ann Intern Med 2001;134:663-94.  Back to cited text no. 7
    
8.Altman DG. Statistics in medical journals: Developments in the 1980s. Stat Med 1991;10:1897-913.  Back to cited text no. 8
    
9.Gore SM, Jones IG, Rytter EC. Misuse of statistical methods: Critical assessment of articles in the BMJ from January to March 1976. BMJ 1977;1:85-7.  Back to cited text no. 9
    
10.Pocock SJ, Hughes MD, Lee RJ. Statistical problems in the reporting of clinical trials. A survey of three medical journals. N Engl J Med 1987;317:426-32.  Back to cited text no. 10
    
11.MacArthur RD, Jackson GG. An evaluation of the use of statistical methodology in the Journal of Infectious Diseases. J Infect Dis 1984;149:349-54.  Back to cited text no. 11
    
12.Karan J, Goyal JP, Bhardwaj P, Yadav P. Statistical Reporting in Indian Pediatrics. Indian Pediatr 2009;46:811-2.  Back to cited text no. 12
    
13.Jaykaran, Kantharia N, Yadav P, Bharddwaj P. Reporting statistics in clinical trials published in Indian Journals: A survey. Pak J Med Sci 2010;26:212-6.  Back to cited text no. 13
    
14.Carlin JB, Doyle LW. Basic concepts of statistical reasoning: Standard errors and confidence intervals. J Paediatr Child Health 2000;36:502-5.  Back to cited text no. 14
    
15.Curran-Everett D. Explorations in statistics: Standard deviations and standard errors. Adv Physiol Educ 2008;32:203-8.  Back to cited text no. 15
    
16.Jaykaran. "Mean ± SEM" or "Mean (SD)"?. Indian J Pharmacol 2010;42:329.  Back to cited text no. 16
    
17.Nagele P. Misuse of standard error of the mean (SEM) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. Br J Anaesth 2003;90:514-6.  Back to cited text no. 17
    
18.Saurabh MK, Jaykaran N, Chavda P, Yadav ND, Kantharia N, Kumar L. Misuse of standard error of mean (SEM) when reporting variability of a sample: A critical appraisal of four Indian journals. J Pharm Res 2010;3:277-9.  Back to cited text no. 18
    
19.Streiner DL. Maintaining standards: Differences between the standard deviation and standard error, and when to use each. Can J Psychiatry 1996;41:498-502.  Back to cited text no. 19
    
20.Altman DG, Gore SM, Gardner MJ, Pocock SJ. Statistical guidelines for contributors to medical journals. Br Med J 1983;286:1489-93.  Back to cited text no. 20
    
21.Avram MJ, Shanks CA, Dykes MH, Ronai AK, Stiers WM. Statistical methods in anesthesia articles: An evaluation of two American journals during two six-month periods. Anesth Analg 1985;64:607-11.  Back to cited text no. 21
    
22.Kurichi JE, Sonnad SS. Statistical methods in the surgical literature. J Am Coll Surg 2006;202:476-84.  Back to cited text no. 22
    
23.Qian Wang MS, Boheng Zhang MS. Research Design and Statistical Methods in Chinese Medical Journals. JAMA 1998;280:283-5.  Back to cited text no. 23
    
24.Al-Benna S, Al-Ajam Y, Way B, Steinstraesser L. Descriptive and inferential statistical methods used in burns research. Burns 2010;36:343-6.  Back to cited text no. 24
    
25.Rushton L. Reporting of occupational and environmental research: Use and misuse of statistical and epidemiological methods. Occup Environ Med 2000;57:1-9.  Back to cited text no. 25
    
26.Simundic AM, Nikolac N. Statistical errors in manuscripts submitted to Biochemia Medica Journal. Biochemia Medica 2009;19:294-300.  Back to cited text no. 26
    
27.Morris RW. A statistical study of papers in the journal of bone and joint surgery (BR) 1984. J Bone Joint Surg Br 1988;70:242-6.  Back to cited text no. 27
    
28.Scales CD Jr, Norris RD, Peterson BL, Preminger GM, Dahm P. Clinical research and statistical methods in the urology literature. J Urol 2005;174:1374-9.  Back to cited text no. 28
    
29.Spina D. Statistics in Pharmacology. Br J Pharmacol 2007;152:291-3.  Back to cited text no. 29
    
30.LaValley MP, Felson DT. Statistical Presentation and Analysis of Ordered Categorical Outcome Data in Rheumatology Journals. Arthritis Rheum 2002;47:255-9.  Back to cited text no. 30
    
31.Jakobsson U. Statistical presentation and analysis of ordinal data in nursing research. Scand J Caring Sci 2004;18:437-40.  Back to cited text no. 31
    
32.Altman DG, Goodman SN, Schroter S. How statistical expertise is used in medical research. JAMA 2002;287:2817-20.  Back to cited text no. 32
    
33.Chanter DO. Maintaining the integrity of the scientific record: New policy is unlikely to give investigators more control over studies. BMJ 2002;324:169.  Back to cited text no. 33
    
34.Emanuel EJ, Wendler D, Grady C. What makes clinical research ethical? JAMA 2000;283:2701-11.  Back to cited text no. 34
    
35.Goodman SN, Altman DG, George SL. Statistical reviewing policies of medical journals: Caveat lector? J Gen Intern Med 1998;13:753-6.  Back to cited text no. 35
    



 
 
    Tables

  [Table 1], [Table 2], [Table 3]


This article has been cited by
1 Reporting statistics in biomedical research literature: The numbers say it all
Amir Maroof Khan,Siddarth Ramji
Indian Pediatrics. 2016; 53(9): 811
[Pubmed] | [DOI]
2 Research Design and Statistical Methods in Indian Medical Journals: A Retrospective Survey
Shabbeer Hassan,Rajashree Yellur,Pooventhan Subramani,Poornima Adiga,Manoj Gokhale,Manasa S. Iyer,Shreemathi S. Mayya,Daniele Marinazzo
PLOS ONE. 2015; 10(4): e0121268
[Pubmed] | [DOI]
3 A Review of the Study Designs and Statistical Methods Used in the Determination of Predictors of All-Cause Mortality in HIV-Infected Cohorts: 20022011
Kennedy N. Otwombe,Max Petzold,Neil Martinson,Tobias Chirwa,Barbara Ensoli
PLoS ONE. 2014; 9(2): e87356
[Pubmed] | [DOI]
4 Inappropriate use of standard error of the mean when reporting variability of study samples: A critical evaluation of four selected journals of obstetrics and gynecology
Wen-Ru Ko,Wei-Te Hung,Hui-Chin Chang,Long-Yau Lin
Taiwanese Journal of Obstetrics and Gynecology. 2014; 53(1): 26
[Pubmed] | [DOI]
5 Policy of reviewing statistics in Indian medical and surgical journals
Kannan, S. and Deshpande, S.P. and Gogtay, N.J. and Thatte, U.M.
Journal of Pharmacology and Pharmacotherapeutics. 2013; 4(2): 139-140
[Pubmed]
6 Reporting quality of multivariable logistic regression in selected Indian medical journals
Kumar, R. and Indrayan, A. and Chhabra, P.
Journal of Postgraduate Medicine. 2012; 58(2): 123-126
[Pubmed]
7 Beware statistics may deceive you
Jaykaran C
Int J Med Sci Public Health. 2012; 1(1): 10-13
[Pubmed] | [DOI]
8 Quality of Antidepressant Drugs Research Articles Published in Indian Medical Journals
Charan J, Saxena D, Yadav P, Kantharia ND.
Indian J Psychol Med. 2011; 33(2): 141-144
[Pubmed] | [DOI]



 

Top
  
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
    Introduction
    Materials and Me...
    Results
    Discussion
    References
    Article Tables

 Article Access Statistics
    Viewed3017    
    Printed216    
    Emailed0    
    PDF Downloaded490    
    Comments [Add]    
    Cited by others 8    

Recommend this journal