Interobserver Agreement for Contrast-Enhanced Ultrasound (CEUS)-Based Standardized Algorithms for the Diagnosis of Hepatocellular Carcinoma in High-Risk Patients

Abstract

Objectives This pilot study aimed at assessing interobserver agreement with two contrast-enhanced ultrasound (CEUS) algorithms for the diagnosis of hepatocellular carcinoma (HCC) in high-risk patients.

Methods Focal liver lesions in 55 high-risk patients were assessed independently by three blinded observers with two standardized CEUS algorithms: ESCULAP (Erlanger Synopsis of Contrast-Enhanced Ultrasound for Liver Lesion Assessment in Patients at risk) and ACR-CEUS-LI-RADSv.2016 (American College of Radiology CEUS-Liver Imaging Reporting and Data System). Lesions were categorized according to size and ultrasound contrast enhancement in the arterial, portal-venous and late phase. Interobserver agreement for assessment of enhancement pattern and categorization was compared between both CEUS algorithms. Additionally, diagnostic accuracy for the definitive diagnosis of HCC was compared. Histology and/or CE-MRI and follow-up served as reference standards.

Results 55 patients were included in the study (male/female, 44/ 11; mean age: 65.9 years). 90.9 % had cirrhosis. Histological findings were available in 39/55 lesions (70.9 %). Reference standard of the 55 lesions revealed 48 HCCs, 2 intrahepatic cholangiocellular carcinomas (ICCs), and 5 non-HCC-non-ICC lesions. Interobserver agreement was moderate to substantial for arterial phase hyperenhancement (ĸ = 0.53 – 0.67), and fair to moderate for contrast washout in the portal-venous or late phase (ĸ = 0.33 – 0.53). Concerning the CEUS-based algorithms, the interreader agreement was substantial for the ESCULAP category (ĸ = 0.64 – 0.68) and fair for the CEUS-LI-RADS^® category (ĸ = 0.3 – 0.39). Disagreement between observers was mostly due to different perception of washout.

Conclusion Interobserver agreement is better for ESCULAP than for CEUS-LI-RADS^®. This is mostly due to the fact that perception of contrast washout varies between different observers. However, interobserver agreement is good for arterial phase hyperenhancement, which is the key diagnostic feature for the diagnosis of HCC with CEUS in the cirrhotic liver.

Zusammenfassung

Ziele Untersuchung der Interobserver-Variabilität zweier Kontrastmittelultraschall (CEUS) -basierter Diagnosealgorithmen für die Diagnostik des hepatozellulären Karzinoms (HCC) bei Hochrisikopatienten.

Methoden Fokale Leberläsionen bei 55 Hochrisikopatienten wurden unabhängig voneinander von 3 verblindeten Untersuchern anhand beider CEUS-Algorithmen klassifiziert (ESCULAP = Erlanger Synopsis of Contrast-enhanced Ultrasound for Liver lesion Assessment in Patients at risk; ACR-CEUS-LI-RADS^®v.2016 = American College of Radiology CEUS-Liver Imaging Reporting and Data System). Die Kategorisierung erfolgte nach Größe und Ultraschallkontrastmittelverhalten in arterieller, portalvenöser und Spätphase. Die Interobserver-Variabilität für Kontrastmittelverhalten und Kategorisierung sowie die diagnostische Genauigkeit für die definitive HCC-Diagnose wurden für beide Algorithmen verglichen. Histologie und/oder CE-MRT und Follow-Up dienten als Referenzstandard.

Ergebnisse 55 Patienten wurden eingeschlossen (männlich/weiblich = 44/11; 90,9 % Leberzirrhose). Histologische Befunde waren für 39/55 Läsionen verfügbar (70,9 %). Der Referenzstandard ergab 48 HCCs, 2 intrahepatische cholangiozelluläre Karzinome (ICCs), und 5 Non-HCC-non-ICC-Läsionen. Die Interobserver-Übereinstimmung war mäßig bis beachtlich für arterielles Hyperenhancement (ĸ = 0,53 – 0,67) und ausreichend bis mäßig für Washout (ĸ = 0,33 – 0,53). Die Interobserver-Variabilität war beachtlich für die ESCULAP-Kategorie (ĸ = 0,64 – 0,68) und ausreichend für die CEUS-LI-RADS^®-Kategorie (ĸ = 0,3 – 0,39). Diskrepanzen resultierten hauptsächlich aus unterschiedlicher Einschätzung des Kontrastmittel-Auswaschens.

Schlussfolgerungen Die Interobserver-Übereinstimmung für ESCULAP ist besser als für CEUS-LI-RADS^®. Hauptursache ist, dass die Wahrnehmung des Kontrastmittel-Auswaschens zwischen einzelnen Untersuchern stark variiert. Gute Übereinstimmung findet sich hingegen bei der Beurteilung des arteriellen Hyperenhancement, des Schlüsselkriteriums für die CEUS-basierte HCC-Diagnose bei Zirrhose-Patienten.

Key words

CEUS - LI-RADS - ESCULAP - HCC - interobserver agreement

• References

• 1 S3- Leitlinie Deutschland: AWMF-Leitlinie “Hepatozelluläres Karzinom”, Registernummer 032 – 053OL. Stand: 01.05.2013, gültig bis 30.04.2018; http://www.awmf.org/uploads/tx_szleitlinien/032-053OLl_S3_Hepatozellul%C3%A4res_Karzinom_Diagnostik_Therapie_2013-verlaengert.pdf
• 2 Leoni S, Piscaglia F, Granito A. et al. Characterization of primary and recurrent nodules in liver cirrhosis using contrast-enhanced ultrasound: which vascular criteria should be adopted?. Ultraschall in Med 2013; 34: 280-287
• 3 Strobel D, Bernatik T, Blank W. et al. Diagnostic accuracy of CEUS in the differential diagnosis of small (≤ 20  mm) and subcentimetric (≤ 10  mm) focal liver lesions in comparison with histology. Results of the DEGUM multicenter trial. Ultraschall in Med 2011; 32: 593-597
• 4 Strobel D, Seitz K, Blank W. et al. Tumor-specific vascularization pattern of liver metastasis, hepatocellular carcinoma, hemangioma and focal nodular hyperplasia in the differential diagnosis of 1349 liver lesions in contrast-enhanced ultrasound (CEUS). Ultraschall in Med 2009; 30: 376-382
• 5 Seitz K, Greis C, Schuler A. et al. Frequency of tumor entities among liver tumors of unclear etiology initially detected by sonography in the noncirrhotic or cirrhotic livers of 1349 patients. Results of the DEGUM multicenter study. Ultraschall in Med 2011; 32: 598-603
• 6 Boozari B, Soudah B, Rifai K. et al. Grading of hypervascular hepatocellular carcinoma using late phase of contrast enhanced sonography – a prospective study. Dig Liver Dis 2011; 43: 484-490
• 7 Kim TK, Jang HJ. Contrast-enhanced ultrasound in the diagnosis of nodules in liver cirrhosis. World J Gastroenterol 2014; 20: 3590-3596
• 8 D’Onofrio M, Crosara S, De Robertis R. et al. Contrast-Enhanced Ultrasound of Focal Liver Lesions. Am J Roentgenol 2015; 205: W56-W66
• 9 Shah S, Shukla A, Paunipagar B. Radiological features of hepatocellular carcinoma. J Clin Exp Hepatol 2014; 4 (Suppl. 03) S63-S66
• 10 Choi BI, Lee JM, Kim TK. et al. Diagnosing Borderline Hepatic Nodules in Hepatocarcinogenesis: Imaging Performance. Am J Roentgenol 2015; 205: 10-21
• 11 Roberts LR, Sirlin CB, Zaiem F. et al. Imaging for the Diagnosis of Hepatocellular Carcinoma: a Systematic Review and Meta-analysis. Hepatology 2018; 67: 401-421
• 12 Kim BR, Lee JM, Lee DH. et al. Diagnostic Performance of Gadoxetic Acid-enhanced Liver MR Imaging versus Multidetector CT in the Detection of Dysplastic Nodules and Early Hepatocellular Carcinoma. Radiology 2017; 285: 134-146
• 13 Bae JS, Kim JH, Yu MH. et al. Diagnostic accuracy of gadoxetic acid-enhanced MR for small hypervascular hepatocellular carcinoma and the concordance rate of Liver Imaging Reporting and Data System (LI-RADS). PLoS One 2017; 12: e0178495
• 14 Westwood M, Joore M, Grutters J. et al. Contrast-enhanced ultrasound using SonoVue® (sulphur hexafluoride microbubbles) compared with contrast-enhanced computed tomography and contrast-enhanced magnetic resonance imaging for the characterisation of focal liver lesions and detection of liver metastases: a systematic review and cost-effectiveness analysis. Health Technol Assess 2013; 17: 1-243
• 15 Claudon M, Dietrich CF, Choi BI. et al. Guidelines and good clinical practice recommendations for contrast enhanced ultrasound (CEUS) in the liver--update 2012: a WFUMB-EFSUMB initiative in cooperation with representatives of AFSUMB, AIUM, ASUM, FLAUS and ICUS. Ultraschall in Med 2013; 34: 11-29
• 16 Bota S, Piscaglia F, Marinelli S. et al. Comparison of international guidelines for noninvasive diagnosis of hepatocellular carcinoma. Liver Cancer 2012; 1: 190-200
• 17 Friedrich-Rust M, Klopffleisch T, Nierhoff J. et al. Contrast-Enhanced Ultrasound for the differentiation of benign and malignant focal liver lesions: a meta-analysis. Liver Int 2013; 33: 739-755
• 18 Wildner D, Bernatik T, Greis C. et al. CEUS in Hepatocellular Carcinoma and Intrahepatic Cholangiocellular Carcinoma in 320 Patients – Early or Late Washout Matters: A Subanalysis of the DEGUM Multicenter Trial. Ultraschall in Med 2015; 36: 132-139
• 19 McEvoy SH, McCarthy CJ, Lavelle LP. et al. Hepatocellular carcinoma: illustrated guide to systematic radiologic diagnosis and staging according to guidelines of the American Association for the Study of Liver Diseases. Radiographics 2013; 33: 1653-1668
• 20 Giorgio A, Montesarchio L, Gatti P. et al. Contrast-Enhanced Ultrasound: a Simple and Effective Tool in Defining a Rapid Diagnostic Work-up for Small Nodules Detected in Cirrhotic Patients during Surveillance. J Gastrointestin Liver Dis 2016; 25: 205-211
• 21 Schellhaas B, Wildner D, Pfeifer L. et al. LI-RADS-CEUS – Proposal for a Contrast-Enhanced Ultrasound Algorithm for the Diagnosis of Hepatocellular Carcinoma in High-Risk Populations. Ultraschall in Med 2016; 37: 627-634
• 22 Schellhaas B, Görtz RS, Pfeifer L. et al. Diagnostic accuracy of contrast-enhanced ultrasound for the differential diagnosis of hepatocellular carcinoma: ESCULAP versus CEUS-LI-RADS. Eur J Gastroenterol Hepatol 2017; 29: 1036-1044
• 23 American College of Radiology. Liver Imaging Reporting and Data System version 2014. Accessed 04/2017, from http://www.acr.org/Quality-Safety/Resources/LIRADS
• 24 Mitchell DG, Bruix J, Sherman M. et al. LI-RADS (Liver Imaging Reporting and Data System): summary, discussion, and consensus of the LI-RADS Management Working Group and future directions. Hepatology 2015; 61: 1056-1065
• 25 Tang A, Valasek MA, Sirlin CB. Update on the Liver Imaging Reporting and Data System: What the Pathologist Needs to Know. Adv Anat Pathol 2015; 22: 314-322
• 26 Elsayes KM, Kielar AZ, Agrons MM. et al. Liver Imaging Reporting and Data System: an expert consensus statement. J Hepatocell Carcinoma 2017; 4: 29-39
• 27 Piscaglia F, Wilson SR, Lyshchik A. et al. American College of Radiology Contrast Enhanced Ultrasound Liver Imaging Reporting and Data System (CEUS LI-RADS) for the diagnosis of Hepatocellular Carcinoma: a pictorial essay. Ultraschall in Med 2017;
• 28 Karlas T. LI-RADS-CEUS for the classification of HCC risk in liver lesions. Z Gastroenterol 2017;
• 29 McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012; 22: 276-282
• 30 Forner A, Vilana R, Bianchi L. et al. Lack of arterial hypervascularity at contrast-enhanced ultrasound should not define the priority for diagnostic work-up of nodules <2 cm. J Hepatol 2015; 62: 150-155
• 31 Bashir MR, Huang R, Mayes N. et al. Concordance of hypervascular liver nodule characterization between the organ procurement and transplant network and liver imaging reporting and data system classifications. J Magn Reson Imaging 2015; 42: 305-314
• 32 Ehman EC, Behr SC, Umetsu SE. et al. Rate of observation and inter-observer agreement for LI-RADS major features at CT and MRI in 184 pathology proven hepatocellular carcinomas. Abdom Radiol (NY) 2016; 41: 963-969
• 33 Barth BK, Donati OF, Fischer MA. et al. Reliability, Validity, and Reader Acceptance of LI-RADS-An In-depth Analysis. Acad Radiol 2016; 23: 1145-1153
• 34 Davenport MS, Khalatbari S, Liu PS. et al. Repeatability of diagnostic features and scoring systems for hepatocellular carcinoma by using MR imaging. Radiology 2014; 272: 132-142
• 35 Zhang YD, Zhu FP, Xu X. et al. Classifying CT/MR findings in patients with suspicion of hepatocellular carcinoma: Comparison of liver imaging reporting and data system and criteria-free Likert scale reporting models. J Magn Reson Imaging 2016; 43: 373-383
• 36 Beyer LP, Wassermann F, Pregler B. et al. Characterization of Focal Liver Lesions using CEUS and MRI with Liver-Specific Contrast Media: Experience of a Single Radiologic Center. Ultraschall in Med 2017; 38: 619-625