Content-based Image Retrieval in Medical Applications

T. M. Lehmann; M. O. Güld; C. Thies; B. Fischer; K. Spitzer; D. Keysers; H. Ney; M. Kohnen; H. Schubert; B. B. Wein

doi:10.1055/s-0038-1633877

Methods of Information in Medicine, Table of Contents

Methods Inf Med 2004; 43(04): 354-361
DOI: 10.1055/s-0038-1633877

Original Article

Schattauer GmbH

Content-based Image Retrieval in Medical Applications

T. M. Lehmann

¹Department of Medical Informatics, Aachen University of Technology (RWTH), Aachen, Germany

,

M. O. Güld

¹Department of Medical Informatics, Aachen University of Technology (RWTH), Aachen, Germany

,

C. Thies

¹Department of Medical Informatics, Aachen University of Technology (RWTH), Aachen, Germany

,

B. Fischer

¹Department of Medical Informatics, Aachen University of Technology (RWTH), Aachen, Germany

,

K. Spitzer

¹Department of Medical Informatics, Aachen University of Technology (RWTH), Aachen, Germany

,

D. Keysers

²Chair of Computer Science VI, Aachen University of Technology (RWTH), Aachen, Germany

,

H. Ney

²Chair of Computer Science VI, Aachen University of Technology (RWTH), Aachen, Germany

,

M. Kohnen

³Department of Diagnostic Radiology, Aachen University of Technology (RWTH), Aachen, Germany

,

H. Schubert

³Department of Diagnostic Radiology, Aachen University of Technology (RWTH), Aachen, Germany

,

B. B. Wein

³Department of Diagnostic Radiology, Aachen University of Technology (RWTH), Aachen, Germany

› Author Affiliations

Abstract

Summary

Objectives: To develop a general structure for semantic image analysis that is suitable for content-based image retrieval in medical applications and an architecture for its efficient implementation.

Methods: Stepwise content analysis of medical images results in six layers of information modeling incorporating medical expert knowledge (raw data layer, registered data layer, feature layer, scheme layer, object layer, knowledge layer). A reference database with 10,000 images categorized according to the image modality, orientation, body region, and biological system is used. By means of prototypes in each category, identification of objects and their geometrical or temporal relationships are handled in the object and the knowledge layer, respectively. A distributed system designed with only three core elements is implemented: (i) the central database holds program sources, processing scheme descriptions, images, features, and administrative information about the workstation cluster; (ii) the scheduler balances distributed computing; and (iii) the web server provides graphical user interfaces for data entry and retrieval, which can be easily adapted to a variety of applications for content-based image retrieval in medicine.

Results: Leaving-one-out experiments were distributed by the scheduler and controlled via corresponding job lists offering transparency regarding the viewpoints of a distributed system and the user. The proposed architecture is suitable for content-based image retrieval in medical applications. It improves current picture archiving and communication systems that still rely on alphanumerical descriptions, which are insufficient for image retrieval of high recall and precision.

Keywords

Information storage and retrieval - information management - pattern recognition - computer-assisted image processing

Full Text

References

References
1 Niblack W, Barber R, Equitz W, Flickner M, Yanker P, Ashley J. The QBIC-project – Querying images by content using color, texture and shape. Proceedings SPIE 1993; 1908: 173-87.
2 Wu KJ, Narasimhalu AD, Mehtre BM, Lam CP, Gao YJ. CORE – A content-based retrieval engine for multimedia information systems. Multimedia Systems 1995; 3: 25-41.
3 Carson C, Belongie S, Greenspan H, Malik J. Blobworld – Image segmentation using expectation-maximization and its application to image querying. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002; 24 (08) 1026-38.
4 Nappi M, Polese G, Tortora G. FIRST – Fractal indexing and retrieval system for image databases. Image and Vision Computing 1998; 16: 1019-31.
5 Pentland A, Picard RW, Scarloff S. Photobook – Content-based manipulation of image databases. Proceedings SPIE 1994; 2185: 34-47.
6 Korn P, Sidiropoulos N, Faloutsos C, Siegel E, Protopapas Z. Fast and effective retrieval of medical tumor shapes. IEEE Transactions on Knowledge and Data Engineering 1998; 10 (06) 889-904.
7 Shyu CR, Brodley CE, Kak AC, Kosaka A, Aisen AM, Broderick LS. ASSERT – A physician-in-the-loop content-based retrieval system for HRCT image databases. Computer Vision and Image Understanding 1999; 75 (1/2) 111-32.
8 Long LR, Antania S, Leeb DJ, Krainakc DM, Thomaa GR. Biomedical information from a national collection of spine x-rays – Film to content-based retrieval. Proceedings SPIE 2003; 5033: 70-84.
9 Chu WW, Hsu CC, Cárdenas AF, Tiara RK. Knowledge-based image retrieval with spatial and temporal constructs. IEEE Transactions on Knowledge and Data Engineering 1998; 10 (06) 872-88.
10 Tagare HD, Jaffe CC, Dungan J. Medical image databases: A content-based retrieval approach. Journal of the American Medical Informatics Association 1997; 4: 184-98.
11 Smeulders AWM, Worring M, Santint S, Gupta A, Jain R. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000; 22 (12) 1349-80.
12 Lehmann TM, Wein B, Dahmen J, Bredno J, Vogelsang F, Kohnen M. Content-based image retrieval in medical applications: A novel multistep approach. Proceedings SPIE 2000; 3972: 312-20.
13 Lehmann TM, Schubert H, Keysers D, Kohnen M, Wein BB. The IRMA code for unique classification of medical images. Proceedings SPIE 2003; 5033: 440-51.
14 Belongie S, Carson C, Greenspan H, Malik J. Color-and texture-based image segmentation using EM and its application to content-based image retrieval. Proceedings 6th International Conference on Computer Vision. 1998: 675-82.
15 Ngo CW, Pong TC, Chin RT. Exploiting image indexing techniques in DCT domain. IAPR International Workshop on Multimedia Information Analysis and Retrieval. 1998: 195-206.
16 Wang JZ, Wiederhold G, Firschein O, Wei SX. Content-based image indexing and searching using Daubechies’ wavelets. International Journal of Digital Libraries 1998; 1 (04) 311-28.
17 Castelli V, Kontoyiannis I, Robinson J, Bergman L, Li CS, Turek J. Progressive search and retrieval in large image archives. IBM Journal of Research and Development 1998; 42 (02) 253-68.
18 Chaudhuri BB, Sakar N. Texture segmentation using fractal dimension. IEEE Transactions on Pattern Analysis and Machine Intelligence 1995; 17: 72-6.
19 Zhou XS, Huang TS. Edge-based structural features for content-based image retrieval. Pattern Recognition Letters 2001; 22: 457-68.
20 Simard P, Le Cun Y, Denker J. Efficient pattern recognition using a new transformation distance. In Hanson S, Cowan J, Giles C. (eds) Advances in Neural Information Processing Systems 5. Morgan Kaufmann; San Mateo, CA: 1993: 50-8.
21 Thies C, Malik A, Keysers D, Kohnen M, Fischer B, Lehmann TM. Hierarchical feature clustering for content-based retrieval in medical image databases. Proceedings SPIE 2003; 5032: 598-608.
22 Melnik S, Garcia-Molina H, Rahm E. Similarity flooding – A versatile graph matching algorithm and its application to schema matching. Proceedings 18th ICDE Conference. 2002: 117-28.
23 Depiero F, Trivedi M, Serbin S. Graph matching using a direct classification of node attendance. Pattern Recognition 1996; 29 (06) 1031-48.
24 Schädler K, Wysotzki F. Comparing structures using a Hopfield-style neural network. Applied Intelligence 1999; 11: 15-30.
25 Lee EA, Parks TM. Dataflow process networks. Proceedings of the IEEE 1995; 83 (05) 773-99.
26 Güld MO, Thies C, Fischer B, Keysers D, Wein BB, Lehmann TM. A platform for distributed image processing and image retrieval. Proceedings SPIE 2003; 5150: 1109-20.
27 Keysers D, Dahmen J, Ney H, Wein BB, Lehmann TM. A statistical framework for modelbased image retrieval in medical applications. Journal of Electronic Imaging 2003; 12 (01) 59-68.
28 Paredes R, Keysers D, Lehmann TM, Wein BB, Ney H, Vidal E. Classification of medical images using local representations. Proceedings BVM 2002. Springer-Verlag, Berlin: 2002: 171-4.
29 Lehmann TM, Güld MO, Schubert H, Wenning A, Keysers D, Wein BB. Automatic detection of the view position of chest radiographs. Proceedings SPIE 2003; 5032: 1275-82.