Utility of a Computerized ICD-10 Algorithm to Identify Idiosyncratic Drug-Induced Liver Injury Cases in the Electronic Medical Record | SpringerLink

Utility of a Computerized ICD-10 Algorithm to Identify Idiosyncratic Drug-Induced Liver Injury Cases in the Electronic Medical Record



Idiosyncratic drug-induced liver injury (DILI) is an important cause of liver injury that is difficult to diagnose and identify in the electronic medical record (EMR).


Our objective was to develop a computerized algorithm that can reliably identify DILI cases from the EMR.


The EMR was searched for all encounters with an International Classification of Diseases, Tenth Revision (ICD-10) T code for drug toxicity and a K-71 code for toxic liver injury between 1 October 2015 and 30 September 2018. Clinically significant liver injury was defined using predetermined laboratory values. An expert opinion causality score (1–3 = probable DILI, 4/5 = non-DILI), Roussel Uclaf Causality Assessment Method (RUCAM) score, and severity score was assigned to each case.


Among the 1,211,787 encounters searched, 517 had both an ICD-10 T code and a K-71 code, with 257 patients meeting the laboratory criteria. After excluding 75 cases of acetaminophen hepatotoxicity, the final study sample included 182 cases of potential DILI, with antineoplastics and antibiotics being the most frequently implicated agents. Causality assessment identified probable DILI in 121 patients (66.5%), whereas 61 (33.5%) had an alternative cause of liver injury. Although age, sex, race, and suspect drugs were similar, the probable DILI cases were more likely to present with a hepatocellular injury profile and have more severe liver injury than the non-DILI cases (p < 0.05).


A computerized algorithm based on a combination of ICD-10 codes identified 182 potential DILI cases with 121 true positives, 61 false positives, and a positive predictive value of 66.5%. Future studies incorporating natural language processing may further improve the utility of this algorithm in identifying high-causality idiosyncratic DILI cases.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3


  1. 1.

    Chalasani N, Bonkovsky HL, Fontana RJ, Lee W, Stolz A, Talwalkar J, et al. Features and outcomes of 899 patients with drug-induced liver injury: the DILIN prospective study. Gastroenterology. 2015;148:1340–52.

    Article  Google Scholar 

  2. 2.

    Hoofnagle JH, Bjornsson ES. Drug-induced liver injury- types and phenotypes. N Engl J Med. 2019;381:264–73.

    CAS  Article  Google Scholar 

  3. 3.

    Fontana RJ, Seeff LB, Andrade RJ, Bjornsson E, Day CP, Serrano J, Hoofnagle JH. Standardization of nomenclature and causality assessment in drug-induced liver injury: summary of a clinical research workshop. Hepatology. 2010;52:730–42.

    Article  Google Scholar 

  4. 4.

    Nourjah P, Ahmad SR, Karwoski C, Willy M. Estimates of acetaminophen (Paracetamol)-associated overdoses in the United States. Pharmacoepidemiol Drug Saf. 2006;15:398–405.

    Article  Google Scholar 

  5. 5.

    Reuben A, Koch DG, Lee WM. Drug-induced acute liver failure: results of a U.S. multicenter, prospective study. Hepatology. 2010;52:2065–76.

    Article  Google Scholar 

  6. 6.

    Chalasani NP, Hayashi PH, Bonkovsky HL, Navarro VJ, Lee WM, Fontana RJ. ACG clinical guideline: the diagnosis and management of idiosyncratic drug-induced liver injury. Am J Gastroenterol. 2014;109:950–66.

    Article  Google Scholar 

  7. 7.

    Fontana RJ, Hayashi PH, Gu J, Reddy KR, Barnhart H, Watkins PB, et al. Idiosyncratic drug-induced liver injury is associated with substantial morbidity and mortality within 6 months from onset. Gastroenterology. 2014;147:96–108.

    CAS  Article  Google Scholar 

  8. 8.

    Hayashi PH, Fontana RJ. Clinical features, diagnosis, and natural history of drug induced liver injury. Semin Liver Dis. 2014;34:134–44.

    Article  Google Scholar 

  9. 9.

    Ostapowicz G, Fontana RJ, Schiodt FV, Larson A, Davem TJ, Han SH, et al. Results of a prospective study of acute liver failure at 17 tertiary care centers in the United States. Ann Intern Med. 2002;137:947–54.

    Article  Google Scholar 

  10. 10.

    Lee WM, Senior JR. Recognizing drug-induced liver injury: current problems, possible solutions. Toxicol Pathol. 2005;33:155–64.

    CAS  Article  Google Scholar 

  11. 11.

    Jinjuvadia K, Kwan W, Fontana RJ. Searching for a needles in a haystack: use of ICD-9-CM codes in drug-induced liver injury. Am J Gastroenterol. 2007;102:2437–43.

    Article  Google Scholar 

  12. 12.

    Udo R, Maitlandvan AH, Egberts TCG, den Breeijen JH, Leufkens HGM, van Solinge WW, De Bruin ML. Validity of diagnostic codes and laboratory measurements to identify patients with idiopathic acute liver injury in hospital database. Pharmacoepidemiol Drug Saf. 2016;25:21–8.

    CAS  Article  Google Scholar 

  13. 13.

    Tan EH, Low EXS, Dan YY, Tai BC. Systemic review and meta-analysis of algorithms used to identify drug-induced liver injury (DILI) in health record databases. Liver Int. 2018;38:742–53.

    Article  Google Scholar 

  14. 14.

    ICD-10 Version 2016 WHO. https://apps.who.int/classifications/icd10/browse/2016/en. Accessed 24 June 2019.

  15. 15.

    Hohl CM, Karpov A, Reddekopp L, Doyle-Waters M, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc. 2014;21:547–57.

    Article  Google Scholar 

  16. 16.

    Stausberg J, Hasford J. Identification of adverse drug events: the use of ICD-10 coded diagnoses in routine hospital data. Dtsch Arztebl Int. 2010;107:23–9.

    PubMed  PubMed Central  Google Scholar 

  17. 17.

    Forns J, Cainzos-Achirica M, Hellfritzsch M, Morros R, Poblador-Plou B, Hallas J, et al. Validity of ICD-9 and ICD-10 codes used to identify acute liver injury: a study in 3 European data sources. Pharmacoepidemiol Drug Saf. 2019;28:965–75.

    CAS  Article  Google Scholar 

  18. 18.

    Timmer A, Di Sordi D, Kappen S, Kohse KP, Schink T, Perez-Gutthann S, Jacquot E, et al. Validity of hospital ICD-10-GM codes to identify acute liver injury in Germany. Pharmacoepidemiol Drug Saf. 2019;28:1344–52.

    Article  Google Scholar 

  19. 19.

    Sobhonslidsuk A, Poovorawan K, Soonthornworasiri N, Pan-ngum W, Phaosawasdi K. The incidence, presentation and outcomes, risk of mortality and economic data of drug-induced liver injury for a national database in Thailand: a population-based study. BMC Gastroenterol. 2016;16:135.

    Article  Google Scholar 

  20. 20.

    Hanauer DA, Mei Q, Law J, Khanna R, Zheng K. Supporting information retrieval from electronic health records: a report of University of Michigan’s nine-year experience in developing and using the Electronic Medical Record Search Engine (EMERSE). J Biomed Inform. 2015;55:290–300.

    Article  Google Scholar 

  21. 21.

    Fontana RJ, Watkins PB, Bonkovsky HL, Chalasani N, Davern T, Serrano J, et al. Drug-induced liver injury network (DILIN) prospective study: rationale, design and conduct. Drug Saf. 2009;32:55–68.

    CAS  Article  Google Scholar 

  22. 22.

    Danan G, Benichou C. Causality assessment of adverse reactions to drugs: a novel method based on the conclusions of international consensus meetings: application to drug-induced liver injuries. J Clin Epidemiol. 1993;46:1323–30.

    CAS  Article  Google Scholar 

  23. 23.

    Bell LN, Chalasani N. Epidemiology of idiosyncratic drug-induced liver injury. Semin Liver Dis. 2009;29:337–47.

    CAS  Article  Google Scholar 

  24. 24.

    Barnado A, Carroll R, Denny JC, Crofford L. Using ICD-10-CM codes to identify patients with systemic lupus erythematosus in the electronic health record [abstract]. Arthritis Rheumatol. 2018;70(suppl 10):1692.

    Google Scholar 

  25. 25.

    Mapakshi S, Kramer JR, Richardson P, El-Serag HB, Kanwal F. Positive predictive value of international classification of diseases, 10th revision, codes for cirrhosis and its related complications. Clin Gastro Hepatol. 2018;16:1677–8.

    Article  Google Scholar 

  26. 26.

    Teschke R, Wolff A, Frenzel C, Schwarzenboeck A, Schulze J, Eickhoff A. Drug and herb induced liver injury: Council for International Organizations of Medical Sciences scale for causality assessment. World J Hepatol. 2014;6:17–32.

    Article  Google Scholar 

  27. 27.

    Aithal GP, Rawlins MD, Day CP. Accuracy of hepatic adverse drug reaction reporting in one English health region. BMJ. 1999;319:1541.

    CAS  Article  Google Scholar 

  28. 28.

    Rockey DC, Seeff LB, Rochon J, Freston J, Chalasani N, Bonacini M, et al. Causality assessment in drug-induced liver injury using a structured expert opinion process: comparison to the RUCAM method. Hepatology. 2010;51:2117–266.

    Article  Google Scholar 

  29. 29.

    Heidemann L, Law J, Fontana RJ. A text searching tool to identify patients with idiosyncratic drug-induced liver injury. Dig Dis Sci. 2017;62:615–25.

    CAS  Article  Google Scholar 

  30. 30.

    Overby CL, Pathak J, Gottesman O, Haerian K, Perotte A, Murphy S, et al. A collaborative approach to developing an electronic health record phenotyping algorithm for DILI. J Am Med Inform Assoc. 2013;20:243–52.

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Robert J. Fontana.

Ethics declarations


No sources of funding were used to assist in the preparation of this study.

Conflict of interest

Robert J. Fontana has received research funding from Bristol-Myers Squibb, Gilead, and Abbvie and consults for Sanofi. Amoah Yeboah-Korang, Jeremy Louissaint, Irene Tsung, and Sharmila Prabhu have no conflicts of interest relevant to the content of this study.

Data sharing

Not available.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file 1 (PDF 110 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yeboah-Korang, A., Louissaint, J., Tsung, I. et al. Utility of a Computerized ICD-10 Algorithm to Identify Idiosyncratic Drug-Induced Liver Injury Cases in the Electronic Medical Record. Drug Saf 43, 371–377 (2020). https://doi.org/10.1007/s40264-019-00903-5

Download citation