Medical Multimedia Retrieval 2.0

Summary

Objectives The objective of this article is to provide an overview of current trends in medical multimedia retrieval. This state of the art is then compared with trends and techniques in the Web 2.0 field that addresses several of the problems of current multimedia retrieval and may help in solving them.

MethodsThe article uses a review of the current literature on medical imaging and medical multimedia retrieval from the Pubmed and Google scholar literature search engines. Other sources include popular Web 2.0 web pages and articles on Web 2.0 describing the main concepts of these technologies in the medical domain.

Results Web 2.0 has significantly changed the way we interact with the Internet and has created an important added-value by promoting activities of users and interactions between users. In the medical field, there are many examples of adoption of these methods and multimedia retrieval can play an important role in this field by supplying the right data at the right time.

Conclusions Medical multimedia retrieval still has a long way to go to become an everyday tool in the medical field and in clinical practice. Still, when reusing and linking much of the electronic patient record’s data, image retrieval can be a technology to help extract important knowledge and navigate through the large amounts of data. Key challenges for more widespread adoption of these technologies involve legal rules for secondary use of medical data because currently only very few image data sets with ground truth have been made available.

Keywords

Medical image retrieval - medical multimedia data - visual information search - Web 2.0

• References

• 1 Müller H, Michoux N, Bandon D, Geissbuhler A. A review of content-based image retrieval systems in medicine – clinical benefits and future directions. Int J Med Inform 2004; 73: 1-23.
• 2 Smeulders AWM, Worring M, Santini S, Gupta A, Jain R. Content-based image retrieval: The end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000; 22 (12) 1349-80.
• 3 Hirata K, Kato T. Query by visual example – Content-based image retrieval, International Conference on extending database technology. Springer Lecture Notes in Computer Science 1992; 580: 23-7.
• 4 Lowe H, Antipov I, Hersh W, Arnott CSmith. Towards Knowledge-Based Retrieval of Medical Images. The role of semantic indexing, Image Content Representation and Knowledge-Based Retrieval. Annual Symposium of the American Society for Medical Informatics (AMIA). 1998: 882-6.
• 5 Tagare HD, Jaffe CC, Duncan J. Medical Image Databases, a content-based retrieval approach. J Am Med InformAssoc 1997; 04: 184-98.
• 6 Nitrosi A, Borasi G, Nicoli F, Modigliani G, Botti A, Bertolini M, Notari P. A filmless radiology department in a full digital regional hospital: quantitative evaluation of the increased quality and efficiency. J Digital Imaging 2007; 20 (02) 140-8.
• 7 Aisen AM, Broderick LS, Winer-Muram H, Brodley CE, Kak AC, Pavlopoulou C. et al. Automated storage and retrieval of thin-section CT images to assist diagnosis: System description and preliminary assessment. Radiology 2003; 228: 265-70.
• 8 Lehmann TM, Plodowsky B, Spitzer K, Wein BB, Ney H, Seidl T. Extended Query refinement for content-based access to large medical image databases. SPIE Symposium on Medical Imaging 2004; 5371 (15) 90-8.
• 9 Müller H, Lovis C, Geissbuhler A. Medical Image retrieval – the MedGIFT project. Medical Imaging and telemedicine. 2005: 2-7.
• 10 Shyu CR, Broadley CE, Kak AC, Broderick LS. ASSERT: A physician-in-the-loop content-based retrieval system for HRCT Image Databases. Computer Vision and image Understanding 1999; 75 (1-2): 111-32.
• 11 Müller H, Boyer C, Gaudinat A, Hersh W, Geissbuhler A. Analyzing Web Log Files of the Health On the Net HONmedia Search Engine to Define Typical Image Search Tasks for Image Retrieval Evaluation. medinfo 2007, Brisbane, Australia: 2007
• 12 Müller H, Deselaers T, Lehmann T, Clough P, Kim E, Hersh W. Overview of the ImageCLEFmed 2006 Medical Retrieval and Medical Annotation Tasks. Springer Lecture Notes in Computer Science (LNCS);. 2007
• 13 Lessmann B, Nattkemper TW, Hans VH, Degenhard A. A method for linking computed image features to histological semantics in neuropathology. J Biomed Inform. 2007: 5.
• 14 Depeursinge A, Iavindrasana J, Hidki A, Cohen G, Geissbuhler A, Platon A. et al. A classification framework for lung tissue categorization. SPIE Medical Imaging. 2008
• 15 Müller H, Do Huang PA, Depeursinge A, Hoffmeyer P, Stern R, Lovis C, Geissbuhler A. Content-based image retrieval from a database of fracture images. SPIE Medical Imaging 2007. Volume 6516, San Diego, CA, USA: 2007
• 16 Haux R. Health information system: Past, present, and future. Int J Med Inform 2006; 75: 268-81.
• 17 Liu BJ, Law MY, Documet J, Gertych A. Imageassisted knowledge discovery and decision support in radiation therapy planning. Comput Med Imaging Graph 2007; 31 (4-5): 311-21.
• 18 Yamamoto S, Horiuchi J, Sekiguchi J, Wada S, Komizu M, Yamaguchi T. Design of a DICOM image-based program for estimating patient exposure dose in computed tomography. Technol Health Care 2007; 15 (02) 147-56.
• 19 Moreno RA, Furuie SS. A contextual medical image viewer. IEEE Trans Inf Technol Biomed 2007; 11 (05) 583-92.
• 20 AMIA, Secondary Uses and re-uses of healthcare data: Taxonomy for policy formulation and planning. internal publication of AMIA, available from http://www.amia.org/.
• 21 Safran C, Bloomrosen M, Hammond WE, Labkoff S, Markel-Fox S, Tang PC. et al. Toward a National Framework for the Secondary Use of Health Data: An American Medical Informatics Association White Paper. J Am Med Inform Assoc 2007; 14: 1-9.
• 22 Manos D. Secondary data use could cover technology costs. www.healthcarefinancenews.com 1.1.2007..
• 23 Lovis C, Spahni S, Cassoni N, Geissbuhler A. Comprehensive management of the access to the electronic patient record. Int J Med Inform 2007; 76 (5-6): 466-70.
• 24 O’Reilly T. What is Web 2.0? O Reilly Publishers; 2005
• 25 Giles J. Internet Encyclopedias go head to head. Nature 2005; 438 (7070): 900-1.
• 26 Boyer C, Baujard JRScherrer. HONcode: a standard to improve the quality of medical/health information on the Internet and HONs 5^th survey on the Use of the the Internet for Medical and Health Purposes. World Congress for Biomedical Sciences. 1999
• 27 Tang H, Hwee JKwoon Ng. Googling for a diagnosis – use of Google as a diagnostic aid: Internetbased study. Br Med J 2006; 333: 1143-5.
• 28 Giustini D. How Web 2.0 is changing medicine. Br Med J 2006; 333: 1283-4.
• 29 Basky E. Introducing Web 2.0: Weblogs and podcasting for health librarians. Journal of the Canadian Health Librarian Association 2006; 27: 33-4.
• 30 Charette RN. Visualizing electronic health records with Google-earth for the body. IEEE Spectrum. 2008 January..
• 31 Tommasi T, Orabona F, Caputo B. CLEF2007 Image Annotation Task: An SVM-based cue integration approach. CLEF working notes. 2007
• 32 Zhang J, Sun J, Stahl J. PACS and web-based image distribution and display. Comput Med Imaging Graph 2003; 27: 197-206.
• 33 Rosset A, Müller H, Martins M, Dfouni N, Vallée JP, Ratib O. Casimage project: a digital teaching file authoring environment. Journal of Thoracic Imaging 2004; 19: 103-8.
• 34 Vannier MW, Summers RM. Sharing Images. Radiology 2003; 228: 23-5.
• 35 Horsch A, Prinz M, Schneider S, Sipil O, Spinnler K, Vallée JP. et al. Establishing an Internation Reference Image Database for research and development in medical image processing. Methods Inf Med 2004; 43: 409-12.
• 36 Hersh W, Müller H, Jensen J, Yang J, Gorman P, Ruch P. Advancing Biomedical Image Retrieval: Development and Analysis of a Test Collection. J Am Med Inform Assoc (JAMIA). 2006 September/October: 488-96.
• 37 Kalpathy-Cramer J, Hersh W. Medical Image Retrieval and Automatic Annotation: OHSU at ImageCLEF 2007. CLEF working notes. 2007
• 38 Ruiz ME. Combining image features, case descriptions and UMLS concepts to improve retrieval of medical images. AMIA annual symposium. 2006: 674-8.
• 39 Demner-Fushman D, Antani S, Thoma G. Automatically finding images for clinical decision support. IEEE conference on Data Mining. 2007
• 40 Atmosukarto I. A unifying framework for combining content-based image retrieval with relational database queries for biomedical applications. Conference of the Society for Imaging Informatics in Medicine. Seattle: 2008
• 41 Deserno TM, Antani S, Long R. Exploring access to scientific literature using content-based image retrieval. SPIE Medical Imaging. 2007
• 42 Scott G, Shyu CR. Knowledge-driven multidimensional indexing structure for biomedical media database retrieval. IEEE Trans Inf Technol Biomed. 2007; May; 11 (03) 320-31.
• 43 Erdal S, Catalyurek UV, Payne PR, Saltz J, Kamal J, Gurcan MN. A knowledge-anchored integrative image search and retrieval system. J Digit Imaging. 2007
• 44 Felipe JC, Traina Jr C, Traina AJ. A new family of distance functions for perceptual similary retrieval of medical images. J Digit Imaging. 2008
• 45 Costa MOliveira, Cime W, Azevedo PMMarques. Towards applying content-based image retrieval in clinical routine. Future Generation computer systems 2007; 23 (03) 466-74.
• 46 Kalpathy-Cramer J, Hersh W. Automatic image modality classification and annotation to improve medical image retrieval. Medinfo 2007; 12 (02) 1334-8.
• 47 Kahn CE. Web 2.0 applications in Radiology: Feasibility of a dynamic, context-sensitive inline image gallery. Conference of the Society for Imaging Informatics in Medicine. Seattle: 2008
• 48 Grau V, Becher H, Noble JA. Registration of multiview real-time 3-D echocardiographic sequences. IEEE Trans Med Imaging 2007; 26 (09) 1154-65.
• 49 Reyes CCAldasoro, Bhalerao B. Volumetric texture segmentation by discriminant feature selection and multiresolution classification. IEEE Trans Med Imaging 2007; 26 (01) 1-14.
• 50 Lovo EE, Quintana JC, Puebla MC, Torrealba G, Santos JL, Lira IH. et al. A novel, inexpensive method of image coregistration for applications in image-guided surgery using augmented reality. Neurosurgery 2007; 60 (4 Suppl 2): 366-71 discussion 371-2..
• 51 Reinertsen I, Lindseth F, Unsgaard G, Collins DL. Clinical validation of vessel-based registration for correction of brain-shift. Med Image Anal. 2007: 30.
• 52 Curiel L, Chopra R, Hyninen K. Progress in Multimodality Imaging: Truly simultaneous Ultrasound and Magnetic Resonance Imaging. IEEE Trans Med Imaging 2007; 26 (12) 1740-6.
• 53 Saba L, Caddeo G, Mallarini G. Computer-aided detection of pulmonary nodules in computed tomography: analysis and review of the literature. Journal of Computer Assisted Tomography 2007; 31 (04) 611-9.
• 54 Friman O, Volosyak I, Graser A. Multiple channel detection of steady-state visual evoked potentials for brain-computer interfaces. IEEE Trans Biomed Eng 2007; 54 (04) 742-50.
• 55 Snoeren PR, Karssemeijer N. Gray-scale and geometric registration of full-field digital and filmscreen mammograms. Med Image Anal 2007; 11 (02) 146-56.