Interobserver Variability and Diagnostic Performance in US Assessment of Thyroid Nodule According to Size

Abstract

Purpose: To evaluate the interobserver variability for US assessments of thyroid nodules and analyze the diagnostic performances of US assessments in thyroid nodules according to nodule size.

Materials and Methods: This was an IRB-approved retrospective study with waiver of informed consent. A total of 400 surgically-confirmed thyroid nodules were included. Nodules were divided into 4 groups by size; group 1 (nodule size < 5 mm), group 2 (5 mm ≤ nodule size < 10 mm), group 3 (10 mm ≤ nodule size < 20 mm), and group 4 (nodule size ≥ 20 mm). Three experienced (7 – 10 years) radiologists retrospectively reviewed the US images. Agreement of each US descriptor and final US assessment, and diagnostic performances were calculated in each group and compared.

Results: Composition represented substantial or good agreement (k = 0.719 – 0.89). Margin showed the lowest agreement (k = 0.322 – 0.365). Individual kappa values for final assessment according to nodule size were as follows: group 1 (k = 0.674), group 2 (k = 0.596), group 3 (k = 0.674), and group 4 (k = 0.673). Specificity, PPV, and accuracy were significantly different among the groups with different size (p value < 0.05) and lowest in group 1. NPV, specificity, PPV and accuracy except PPV of observer 3 increased with nodule size (p < 0.05).

Conclusion: Interobserver agreements were relatively good (k = 0.637) in final US assessment regardless of nodule size in experienced radiologists. High false-positive rate was observed in US assessment in nodules less than 5 mm in maximum diameter.

Zusammenfassung

Ziel: Die Bewertung der Interobserver-Variabilität und die Analyse der diagnostischen Leistungsfähigkeit bei der US-Beurteilung von Schilddrüsenknoten entsprechend der Größe.

Material und Methoden: Es handelt sich um eine vom IRB genehmigte, retrospektive Studie mit Verzicht auf Einwilligung nach Aufklärung. Eingeschlossen wurden insgesamt 400 operativ bestätigte Schilddrüsenknoten. Die Knoten wurden je nach Größe in 4 Gruppen unterteilt: Gruppe 1 (Knotengröße < 5 mm), Gruppe 2 (Knotengröße ≥ 5 mm bis < 10 mm), Gruppe 3 (Knotengröße ≥ 10 mm bis < 20 mm) und Gruppe 4 (Knotengröße ≥ 20 mm). Drei Radiologen mit 7- bis 10-jähriger Erfahrung bewerteten retrospektiv die US-Bilder. Die Übereinstimmung jedes US-Deskriptors, die US-Endbewertung sowie die diagnostische Leistungsfähigkeit wurden jeder Gruppe berechnet und miteinander verglichen.

Ergebnisse: Die Struktur zeigte eine beachtliche bis gute Übereinstimmung (κ = 0,719 – 0,89). Die Begrenzung zeigte die niedrigste Übereinstimmung (κ = 0,322 – 0,365). Die individuellen Kappa-Werte der Endbewertung nach Knotengröße betrugen: In Gruppe 1 (κ = 0,674), in Gruppe 2 (κ = 0,596), in Gruppe 3 (κ = 0,674) und in Gruppe 4 (κ = 0,673). Spezifität, PVW und Genauigkeit zeigten signifikante Unterschiede zwischen den Gruppen unterschiedlicher Größe (p-Wert < 0,05) und waren jeweils in Gruppe 1 am niedrigsten. Abgesehen vom PVW bei Beobachter 3 stiegen NVW, Spezifität, PVW und Genauigkeit mit zunehmender Knotengröße (p < 0,05) an.

Schlussfolgerung: Die Interobserver-Übereinstimmungen erfahrener Radiologen waren unabhängig von der Knotengröße in der Endbewertung relativ gut (κ = 0,637). Eine hohe Rate an falsch positiven Ergebnissen wurde bei der sonografischen Beurteilung von Knoten unter 5 mm maximalem Durchmesser beobachtet.

Key words

head/neck - tumor - ultrasound

• References

• 1 American Association of Clinical Endocrinologists and Associazione Medici Endocrinologi medical guidelines for clinical practice for the diagnosis and management of thyroid nodules. Endocr Pract 2006; 12: 63-102
• 2 Choi SH, Kim EK, Kwak JY et al. Inter/Intraobserver Variation in Ultrasound Assessment of Thyroid Nodules. Thyroid 2010; 20: 167-172
• 3 Kim EK, Park CS, Chung WY et al. New sonographic criteria for recommending fine-needle aspiration biopsy of nonpalpable solid nodules of the thyroid. Am J Roentgenol 2002; 178: 687-691
• 4 Kwak JY, Koo H, Youk JH et al. Value of US correlation of a thyroid nodule with initially benign cytologic results. Radiology 2010; 254: 292-300
• 5 Cappelli C, Castellano M, Pirola I et al. The predictive value of ultrasound findings in the management of thyroid nodules. Qjm 2007; 100: 29-35
• 6 Nam-Goong IS, Kim HY, Gong G et al. Ultrasonography-guided fine-needle aspiration of thyroid incidentaloma: correlation with pathological findings. Clin Endocrinol 2004; 60: 21-28
• 7 Frates MC, Benson CB, Charboneau JW et al. Management of thyroid nodules detected at US: Society of Radiologists in Ultrasound consensus conference statement. Radiology 2005; 237: 794-800
• 8 Cooper DS, Doherty GM, Haugen BR et al. Revised American Thyroid Association management guidelines for patients with thyroid nodules and differentiated thyroid cancer. Thyroid 2009; 19: 1167-1214
• 9 Moon HJ, Son E, Kim EK et al. The Diagnostic Values of Ultrasound and Ultrasound-Guided Fine Needle Aspiration in Subcentimeter-Sized Thyroid Nodules. Ann Surg Oncol 2012; 19: 52-59
• 10 Mazzaferri EL, Sipos J. Should all patients with subcentimeter thyroid nodules undergo fine-needle aspiration biopsy and preoperative neck ultrasonography to define the extent of tumor invasion?. Thyroid 2008; 18: 597-602
• 11 Yoon JH, Kim EK, Son EJ et al. Diffuse microcalcifications only of the thyroid gland seen on ultrasound: clinical implication and diagnostic approach. Ann Surg Oncol 2011; 18: 2899-2906
• 12 Triggiani V, Guastamacchia E, Licchelli B et al. Microcalcifications and psammoma bodies in thyroid tumors. Thyroid 2008; 18: 1017-1018
• 13 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174
• 14 Cappelli C, Pirola I, Cumetti D et al. Is the anteroposterior and transverse diameter ratio of nonpalpable thyroid nodules a sonographic criteria for recommending fine-needle aspiration cytology?. Clin Endocrinol 2005; 63: 689-693
• 15 Chan BK, Desser TS, McDougall IR et al. Common and uncommon sonographic features of papillary thyroid carcinoma. J Ultrasound Med 2003; 22: 1083-1090
• 16 Gharib H, Papini E, Paschke R et al. American Association of Clinical Endocrinologists, Associazione Medici Endocrinologi, and EuropeanThyroid Association Medical Guidelines for Clinical Practice for the Diagnosis and Management of Thyroid Nodules. Endocr Pract 2010; 16: 1-43
• 17 Iannuccilli JD, Cronan JJ, Monchik JM. Risk for malignancy of thyroid nodules as assessed by sonographic criteria: the need for biopsy. J Ultrasound Med 2004; 23: 1455-1464
• 18 Kang HW, No JH, Chung JH et al. Prevalence, clinical and ultrasonographic characteristics of thyroid incidentalomas. Thyroid 2004; 14: 29-33
• 19 Kwak JY, Han KH, Yoon JH et al. Thyroid Imaging Reporting and Data System for US Features of Nodules: A Step in Establishing Better Stratification of Cancer Risk. Radiology 2011; 260: 892-899
• 20 Koike E, Noguchi S, Yamashita H et al. Ultrasonographic characteristics of thyroid nodules: prediction of malignancy. Arch Surg 2001; 136: 334-337
• 21 Moon HG, Jung EJ, Park ST et al. Role of ultrasonography in predicting malignancy in patients with thyroid nodules. World J Surg 2007; 31: 1410-1416
• 22 Moon HJ, Kwak JY, Kim MJ et al. Can vascularity at power Doppler US help predict thyroid malignancy?. Radiology 2010; 255: 260-269
• 23 Seiberling KA, Dutra JC, Grant T et al. Role of intrathyroidal calcifications detected on ultrasound as a marker of malignancy. Laryngoscope 2004; 114: 1753-1757
• 24 Park JY, Lee HJ, Jang HW et al. A proposal for a thyroid imaging reporting and data system for ultrasound features of thyroid carcinoma. Thyroid 2009; 19: 1257-1264
• 25 Choi SH, Kim EK, Kwak JY et al. Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid 2010; 20: 167-172
• 26 Wienke JR, Chong WK, Fielding JR et al. Sonographic features of benign thyroid nodules: interobserver reliability and overlap with malignancy. J Ultrasound Med 2003; 22: 1027-1031
• 27 Kim SH, Park CS, Jung SL et al. Observer Variability and the Performance between Faculties and Residents: US Criteria for Benign and Malignant Thyroid Nodules. Korean J Radiol 2010; 11: 149-155
• 28 Park CS, Kim SH, Jung SL et al. Observer variability in the sonographic evaluation of thyroid nodules. J Clin Ultrasound 2010; 38: 287-293
• 29 Moon WJ, Jung SL, Lee JH et al. Benign and malignant thyroid nodules: US differentiation – multicenter retrospective study. Radiology 2008; 247: 762-770
• 30 Popowicz B, Klencki M, Lewinski A et al. The usefulness of sonographic features in selection of thyroid nodules for biopsy in relation to the nodule's size. Eur J Endocrinol 2009; 161: 103-111