Table of Contents  
Year : 2015  |  Volume : 6  |  Issue : 2  |  Page : 91-93  

Comparison of agreement and rational uses of the WHO and Naranjo adverse event causality assessment tools

Department of Pharmacology, Postgraduate Institute of Medical Sciences, Rohtak, Haryana, India

Date of Web Publication20-Apr-2015

Correspondence Address:
Niti Mittal
Department of Pharmacology, Postgraduate Institute of Medical Sciences, Rohtak, Haryana - 124 001
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/0976-500X.155486

Rights and Permissions

How to cite this article:
Mittal N, Gupta MC. Comparison of agreement and rational uses of the WHO and Naranjo adverse event causality assessment tools. J Pharmacol Pharmacother 2015;6:91-3

How to cite this URL:
Mittal N, Gupta MC. Comparison of agreement and rational uses of the WHO and Naranjo adverse event causality assessment tools. J Pharmacol Pharmacother [serial online] 2015 [cited 2022 Jan 20];6:91-3. Available from:


Causality assessment of adverse events (AEs) is the standardized and detailed assessment of individual case safety reports for the likelihood of involvement of the suspected drug/s in causing the particular AE. The basic knowledge of causality assessment is indispensable for healthcare professionals as uncertainty of the potential causal relationship between drug and AE remains one of the major reasons of under-reporting in pharmacovigilance. [1] Methods for causality assessment of AEs can be broadly categorized as expert judgment/global introspection (GI), Bayesian methods and algorithms. [2] Despite the availability of a number of methods, no true gold standard exists. The World Health Organization (WHO) GI method [3] and the Naranjo adverse drug reaction (ADR) Probability Scale, [4] although the two most widely used and accepted causality assessment methods in both clinical and experimental settings, have not been validated so far. Hence, it becomes very important to explore the extent to which various methods agree with each other. The comparison of agreement between various methods of causality assessment has been reported by few researchers from the west. [5],[6] In a previous study, the agreement between various algorithms and the WHO GI method was reported as 21-56%. [7] However, due to the fact that extent of agreement between two methods may vary in different settings owing to the understanding, judgment and interpretation by experts and personnel assessing the causality, the present study aimed to compare the agreement between the WHO and Naranjo methods in an Indian setting, with a focus to identify the reasons for their mutual disagreement and address peculiar issues related to their practical applicability. To the best of our knowledge, no such comparison has been performed previously in India.

The study was conducted at the Pandit Bhagwat Dayal Sharma Postgraduate Institute of Medical Sciences (PGIMS), Rohtak, Haryana, India, which is a regional pharmacovigilance center under the Pharmacovigilance Programme of India (PvPI). We randomly selected 200 forms from all the ADR proformas collected within the period June 2012-June 2013. Causality assessment was performed by two well-trained independent clinical pharmacologists by applying the two methods - WHO and Naranjo - on each ADR proforma, after which they discussed the causality with each other and discrepancies, if any, were solved. Agreement between the two algorithms was compared using the Cohen's weighted kappa statistic.

For the present study, 200 ADR forms were included. The cases represented a wide spectrum of manifestations, the most common being cutaneous (28%) and gastrointestinal (22%) [Figure 1]. The mean age of the studied population was 35 ± 16 years, with more than 65% males (male/female: 133/67). The number of different branded/generic drugs suspected for causing ADRs was 173. A total of 34 (17%) ADRs were labeled as serious according to the WHO criteria. The use of concomitant medications was present in 108 (61%) cases. All the 200 AEs were probably or possibly caused by the suspected drugs. Causality was probable in 134 cases and possible in 42 cases with both methods. On the other hand, 24 cases were labeled as possible according to the WHO and probable according to the Naranjo algorithm. None of the cases was labeled as certainly/definitely or unlikely to be caused by the suspected drug/s. Full agreement between the two methods was seen in 88% (134 probable + 42 possible) cases while 12% (24) cases had partial agreement [Table 1]. Kappa analysis demonstrated a moderate to good agreement between the two scales (value of kappa coefficient = 0.701).
Figure 1: Distribution of various adverse events. Miscellaneous included psychosis, renal and metabolic effects

Click here to view
Table 1: Comparison of agreement between the WHO and Naranjo methods for causality assessment of adverse events

Click here to view

In the 24 discordant cases, positive dechallenge criteria was not met for the following reasons: Dechallenge not performed or information regarding dechallenge lacking or a negative dechallenge, i.e. AE not subsided on drug withdrawal. Hence, according to the WHO, causality was labeled as possible in these cases. But, due to fulfillment of few other criteria, e.g., previous conclusive reports on the reaction, change in reaction severity with dose alteration, presence of an objective evidence and toxic drug concentrations in the blood (e.g., antiepileptics), these were able to attain a score compatible with probable association by the Naranjo algorithm.

Hence, more number of cases were labeled as probable by the Naranjo algorithm compared with the WHO criteria (158 vs 134); a similar observation was previously reported (391 by Naranjo vs. 196 by WHO). [8] Few reasons for disagreement between the two scales as observed can be mentioned. As per the WHO, ADR is labeled as probably caused by the suspected drug if all these criteria are met, i.e., reasonable time sequence to administration of the drug, unlikely to be attributed to concurrent disease or other drugs or chemicals and presence of a clinically reasonable response on withdrawal, i.e., a positive dechallenge. Therefore, in the absence of any criteria, a lower level of causality (possible/unlikely etc.) is labeled. On the other hand, the Naranjo algorithm assigns causality on the basis of total score obtained after evaluating the 10 predefined questions, e.g. if the total score calculated is 5-8, then ADR is probably due to suspected drug. Hence, even if all the criteria as per the WHO are not fulfilled, the causality can be labeled as probable if there are favorable responses to other criteria. A similar scenario can be expected where an AE scores 1-4 on the Naranjo scale and, hence, is labeled as possibly caused by the particular drug. But, due to the absence of mandatory criteria for possible as per the WHO, i.e. reasonable time relationship, causality with this method becomes conditional/unlikely due to drug.

The WHO and Naranjo causality assessment methods are generic in nature and their applicability in different situations has been time tested. There are certain issues in which each method has its own distinct identity. The WHO method takes into account the clinical-pharmacological aspects of the case history and the quality of the documentation of the observation, with a less prominent role of previous knowledge and statistical chance. The Naranjo scale is particularly popular among clinicians because of its simplicity.

Both the WHO and the Naranjo methods are semi-quantitative as they do not exactly measure the likelihood of a relationship. An explicit outcome (unrelated or certain) is given in a minority of cases. Another major issue is a high threshold in assigning a "definite/certain" ADR, which is of paramount importance from a public safety and regulatory point of view. Besides, none of the methods addresses the issue of lack of efficacy/unexpected therapeutic failure, which is also categorized as an ADR (type F). [9] The knowledge of dechallenge and rechallenge holds a high importance when assigning causality by both methods. However, few peculiar areas pertaining to these concepts exist. For example, the concept of dechallenge may not be applicable where the drug is a one-dose treatment (e.g., vaccine), reaction resulting in death and reaction occurring after discontinuation of drug. Also, in certain cases, the evaluation may be tedious, like irreversible or long-lasting reaction, e.g., congenital anomaly, hepatotoxicity, bone marrow suppression with cytotoxic drugs. Moreover, dechallenge cannot be addressed in cases of adverse reactions showing spontaneous recovery despite continuation of therapy. A rechallenge may range from a similar episode in the past to a true planned prospective reexposure. Because of ethical and medical concerns, a true rechallenge is a rarity, particularly in serious reactions. Also, in a subjective reaction, more than one rechallenge may be required to achieve a convincing result. Moreover, the situation during rechallenge (dose or duration of treatment) may not be exactly identical to the original episode.

The Naranjo scale carries a low rate of inter-observer agreement; full agreement between two assessors was observed in only 35% of 106 case reports. [10] One reason for this disagreement might be subjective assessment in a few areas. For example, question no. 1 says: Are there previous conclusive reports on this reaction? This usually means "previous bibliographic description" and is based on judgment and needs clarification if there are few published case reports providing inconclusive evidence. Question no. 7 asks: Was the drug detected in the blood (or other fluids) in a concentration known to be toxic? Although this point is expected to give reproducible results with dose-related (type A) ADRs, the same does not stand true for type B reactions, where, in fact, it is inapplicable. The scale assesses the likelihood of ADR due to a single drug. In cases of multiple drug exposure, it is recommended to apply the scale to each of the possible causes, the most likely being the drug with highest score. However, the scale cannot be applied in AEs arising due to potential drug interaction/s between two or more drugs.

Keeping in mind these practical issues, some areas need to be revised in the causality assessment methods. Because of the practical nonfeasibilty and ethical concerns, should rechallenge should be considered a major criterion for labeling the causality as certain/definite. Certain areas should be addressed, like type B (nondose related) ADRs, lack of efficacy/therapeutic failure as an AE and AEs arising as a result of drug interactions. Few suggestions on rational usage and interpretation of the WHO and Naranjo methods can be made. The presence of a convincing evidence in the report, even in the lack of information on dechallenge or rechallenge with satisfactory outcome, suffices to decide the causal association, e.g., avascular necrosis with corticosteroids, grey baby syndrome with chloramphenicol, etc. Establishing the causal role of a drug and ruling out alternate causes can be strengthened by appropriate laboratory findings and clinical features. Besides, drug interaction-induced AEs demand knowledge of either drugs involved and the basic mechanisms of interactions. Nevertheless, the whole process of assessment and interpretation of causality is quite tedious in the presence of complex real clinical situations.

   References Top

Palleria C, Leporini C, Chimirri S, Marrazzo G, Sacchetta S, Bruno L, et al. Limitations and obstacles of the spontaneous adverse drugs reactions reporting: Two "challenging" case reports. J Pharmacol Pharmacother 2013;4 Supp 1:S66-72.  Back to cited text no. 1
Agbabiaka TB, Savovic J, Ernst E. Methods for causality assessment of adverse drug reactions: A systematic review. Drug Saf 2008;31:21-37.  Back to cited text no. 2
The use of the WHO-UMC system for standardised case causality assessment. Accessed from: [Last accessed on 2013 July 13].  Back to cited text no. 3
Naranjo CA, Busto U, Sellers EM, Sandor P, Ruiz I, Roberts EA, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther 1981;30:239-45.  Back to cited text no. 4
Théophile H, Arimone Y, Miremont-Salamé G, Moore N, Fourrier-Réglat A, et al. Comparison of three methods (consensual expert judgement, algorithmic and probabilistic approaches) of causality assessment of adverse drug reactions: An assessment using reports made to a French pharmacovigilance centre. Drug Saf 2010;33:1045-54.  Back to cited text no. 5
Macedo AF, Marques FB, Ribeiro CF, Teixeira F. Causality assessment of adverse drug reactions: Comparison of the results obtained from published decisional algorithms and from the evaluations of an expert panel, according to different levels of imputability. J Clin Pharm Ther 2003;28:137-43.  Back to cited text no. 6
Macedo AF, Marques FB, Ribeiro CF, Teixeira F. Causality assessment of adverse drug reactions: Comparison of the results obtained from published decisional algorithms and from the evaluations of an expert panel. Pharmacoepidemiol Drug Saf 2005;14:885-90.  Back to cited text no. 7
Koh Y, Li SC. A New Algorithm to Identify the Causality of Adverse Drug Reactions. Drug Saf 2005;28:1159-61.  Back to cited text no. 8
Edwards IR, Aronson JK. Adverse drug reactions: Definitions, diagnosis, and management. Lancet 2000;356:1255-9.  Back to cited text no. 9
Lanctot KL, Naranjo CA. Comparison of the Bayesian approach and a simple algorithm for assessment of adverse drug events. Clin Pharmacol Ther 1995;58:692-8.  Back to cited text no. 10


  [Figure 1]

  [Table 1]

This article has been cited by
1 Validation of a novel causality assessment scale for adverse events in non-small cell lung carcinoma patients treated with platinum and pemetrexed doublet chemotherapy
Jyoti Bhagatram Sharma, Manjunath Nookala Krishnamurthy, Ankita Awase, Amit Joshi, Vijay Patil, Vanita Noronha, Kumar Prabhash, Vikram Gota
Therapeutic Advances in Drug Safety. 2021; 12: 2042098621
[Pubmed] | [DOI]
2 Agreement Among Different Scales for Causality Assessment in Drug-Induced Stevens-Johnson Syndrome and Toxic Epidermal Necrolysis
Kiruthika Sivagourounadin, Priyadharsini Rajendran , Sandhiya Selvarajan, Mahalakshmi Ganesapandian
Current Drug Safety. 2021; 16
[Pubmed] | [DOI]
3 Agreement between WHO-UMC causality scale and the Naranjo algorithm for causality assessment of adverse drug reactions
AjayK Shukla, Ratinder Jhaj, Saurav Misra, ShahN Ahmed, Malaya Nanda, Deepa Chaudhary
Journal of Family Medicine and Primary Care. 2021; 10(9): 3303
[Pubmed] | [DOI]
4 Italian Emergency Department Visits and Hospitalizations for Outpatients’ Adverse Drug Events: 12-Year Active Pharmacovigilance Surveillance (The MEREAFaPS Study)
Niccolò Lombardi, Giada Crescioli, Alessandra Bettiol, Marco Tuccori, Annalisa Capuano, Roberto Bonaiuti, Alessandro Mugelli, Mauro Venegoni, Giuseppe Danilo Vighi, Alfredo Vannacci
Frontiers in Pharmacology. 2020; 11
[Pubmed] | [DOI]
5 Comparison of different methods for causality assessment of adverse drug reactions
Sapan Kumar Behera,Saibal Das,Alphienes Stanley Xavier,Srinivas Velupula,Selvarajan Sandhiya
International Journal of Clinical Pharmacy. 2018;
[Pubmed] | [DOI]
6 Use and Safety of Anthroposophic Medicinal Products: An Analysis of 44,662 Patients from the EvaMed Pharmacovigilance Network
Harald Johan Hamre,Anja Glockmann,Kirsten Heckenbach,Harald Matthes
Drugs - Real World Outcomes. 2017;
[Pubmed] | [DOI]
7 Addressing DRESS (drug reaction with eosinophilia and systemic symptoms)
Catherine M. McGrath,Karamjit Khangura
Adverse Drug Reaction Bulletin. 2015; 295(1): 1139
[Pubmed] | [DOI]


    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

  In this article
    Article Figures
    Article Tables

 Article Access Statistics
    PDF Downloaded541    
    Comments [Add]    
    Cited by others 7    

Recommend this journal