Toward a Formalization of the Process to Select IMIA Yearbook Best Papers

J. -B. Lamy; B. Séroussi; N. Griffon; G. Kerdelhué; M. -C. Jaulent; J. Bouaud

doi:10.3414/ME14-01-0031

Methods of Information in Medicine, Table of Contents

Methods Inf Med 2015; 54(02): 135-144
DOI: 10.3414/ME14-01-0031

Original Articles

Schattauer GmbH

Toward a Formalization of the Process to Select IMIA Yearbook Best Papers

J. -B. Lamy

¹Université Paris 13, Sorbonne Paris Cité, LIMICS, (UMR_S 1142), Bobigny, France

²INSERM, U1142, LIMICS, Paris, France

³Sorbonne Universités, UPMC Univ Paris 06, UMR_S 1142, LIMICS, Paris, France

,

B. Séroussi

¹Université Paris 13, Sorbonne Paris Cité, LIMICS, (UMR_S 1142), Bobigny, France

²INSERM, U1142, LIMICS, Paris, France

³Sorbonne Universités, UPMC Univ Paris 06, UMR_S 1142, LIMICS, Paris, France

⁴AP-HP, Hôpital Tenon, Département de Santé Publique, Paris, France

⁵APREC, Paris, France

,

N. Griffon

¹Université Paris 13, Sorbonne Paris Cité, LIMICS, (UMR_S 1142), Bobigny, France

²INSERM, U1142, LIMICS, Paris, France

³Sorbonne Universités, UPMC Univ Paris 06, UMR_S 1142, LIMICS, Paris, France

⁶CISMeF, LITIS EA 4108 – Université de Rouen, Rouen, France

,

G. Kerdelhué

⁶CISMeF, LITIS EA 4108 – Université de Rouen, Rouen, France

,

M. -C. Jaulent

¹Université Paris 13, Sorbonne Paris Cité, LIMICS, (UMR_S 1142), Bobigny, France

²INSERM, U1142, LIMICS, Paris, France

³Sorbonne Universités, UPMC Univ Paris 06, UMR_S 1142, LIMICS, Paris, France

,

J. Bouaud

¹Université Paris 13, Sorbonne Paris Cité, LIMICS, (UMR_S 1142), Bobigny, France

²INSERM, U1142, LIMICS, Paris, France

³Sorbonne Universités, UPMC Univ Paris 06, UMR_S 1142, LIMICS, Paris, France

⁷AP-HP, DRCD, Paris, France

› Author Affiliations

Abstract

Summary

Background: Each year, the International Medical Informatics Association Yearbook recognizes significant scientific papers, labelled as “best papers”, published the previous year in the subfields of biomedical informatics that correspond to the different section topics of the journal. For each section, about fifteen pre-selected “candidate” best papers are externally peer-reviewed to select the actual best papers. Although based on the available literature, little is known about the pre-selection process.

Objective: To move toward an explicit formalization of the candidate best papers selection process to reduce variability in the literature search across sections and over years.

Methods: A methodological framework is proposed to build for each section topic specific queries tailored to PubMed and Web of Science citation databases. The two sets of returned papers are merged and reviewed by two independent section editors and citations are tagged as “discarded”, “pending”, and “kept”. A protocolized consolidation step is then jointly conducted to resolve conflicts. A bibliographic software tool, BibReview, was developed to support the whole process.

Results: The proposed search strategy was fully applied to the Decision Support section of the 2013 edition of the Yearbook. For this section, 1124 references were returned (689 PubMed-specific, 254 WoS-specific, 181 common to both databases) among which the 15 candidate best papers were selected.

Conclusions: The search strategy for determining candidate best papers for an IMIA Yearbook’s section is now explicitly specified and allows for reproducibility. However, some aspects of the whole process remain reviewer-dependent, mostly because there is no characterization of a ”best paper“.

Keywords

Review literature as topic - publishing - medical informatics - IMIA Yearbook - decision support

Full Text

References

References
1 Georg G, Colombet I, Durieux P, Ménard J, Meneton P. A comparative analysis of four clinical guidelines for hypertension management. J Hum Hypertens 2008; 22 (12) 829-837. doi: 10.1038/ jhh.2008.99. Epub 2008 Aug 7.
2 Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol 2009; 104 (05) 1086-1092. DOI http://dx.doi.org/ 10.1038/ajg.2009.118.
3 Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. The Lancet 1999; 354 9193 1896-1900. DOI http://dx.doi.org/ 10.1046/j.1365-2168.2000.01610.x.
4 Higgins JPT, Green S. editors Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]. The Cochrane. collaboration 2011 DOI http://dx.doi.org/10.1002/ 9780470712184.
5 Hofdijk J, Weber P, Mantas J, Mihalas G, Masic I. A Short Factography About IMIA and EFMI. Acta Inform Med 2014; 22 (01) 71-78. doi: 10.5455/ aim.2014.22.7178. Epub 2014 Jan 25.
6 Kulikowski C, Ammenwerth E, Bohne A, Ganser K, Haux R, Knaup P, Maier C, Michel A, Singer R, Wolff AC. Medical Imaging Informatics and Medical Informatics: opportunities and constraints. Findings from the IMIA Yearbook of Medical Informatics 2002. Methods Inf Med 2002; 41 (02) 183-189.
7 Ammenwerth E, Haux R, Kulikowski C, Bohne A, Brandner R, Brigl B, Fischer G, Garde S, Knaup P, Ruderich F, Schubert R, Singer R, Wolff AC. Medical informatics and the quality of health: new approaches to support patient care - findings from the IMIA Yearbook of Medical Informatics 2003. Methods Inf Med 2003; 42 (02) 185-189.
8 Knaup P, Ammenwerth E, Brandner R, Brigl B, Fischer G, Garde S, Lang E, Pilgram R, Ruderich F, Singer R, Wolff AC, Haux R, Kulikowski C. Towards clinical bioinformatics: advancing genomic medicine with informatics methods and tools. Methods Inf Med 2004; 43 (03) 302-307.
9 Bott OJ, Ammenwerth E, Brigl B, Knaup P, Lang E, Pilgram R, Pfeifer B, Ruderich F, Wolff AC, Haux R, Kulikowski C. The challenge of ubiquitous computing in health care: technology, concepts and solutions. Findings from the IMIA Yearbook of Medical Informatics 2005. Methods Inf Med 2005; 44 (03) 473-479.
10 Haux R. Individualization, globalization and health - about sustainable information technologies and the aim of medical informatics. Int J Med Inform 2006; 75 (012) 795-808. Epub 2006 Jul 17.
11 Ammenwerth E, Wolff AC, Knaup P, Ulmer H, Skonetzki S, van Bemmel JH, McCray AT, Haux R, Kulikowski C. Developing and evaluating criteria to help reviewers of biomedical informatics manuscripts. J Am Med Inform Assoc 2003; 10 (05) 512-514. DOI http://dx.doi.org/10.1197/jamia. M1062.
12 Bornmann L. Does the journal peer review select the “best” from the work submitted? The state of empirical research. IETE Technical Review. 2010; 927 (02) 93-96. DOI http://dx.doi.org/ 10.4103/0256-4602.60162.
13 Lu Z. PubMed and beyond: a survey of web tools for searching biomedical literature. Database (Oxford): 2011 baq036. DOI http://dx.doi.org/ 10.1093/database/baq036.
14 Lu Z, Kim W, Wilbur WJ. Evaluation of query expansion using MeSH in PubMed. Inf Retr Boston. 2009; 12 (01) 69-80. DOI http://dx.doi.org/ 10.1007/s10791-008-9074-8.
15 Darmoni SJ, Soualmia LF, Letord C, Jaulent MC, Griffon N, Thirion B, Névéol A. Improving information retrieval using Medical Subject Headings Concepts: a test case on rare and chronic diseases. J Med Libr Assoc 2012; 100 (03) 176-183. DOI http://dx.doi.org/10.3163/1536-5050.100.3.007.
16 Darmoni SJ, Leroy JP, Baudic F, Douyère M, Piot J, Thirion B. CISMeF: a structured health resource guide. Methods Inf Med 2000; 39 (01) 30-35.
17 Soualmia LF, Sakji S, Letord C, Rollin L, Massari P, Darmoni SJ. Improving information retrieval with multiple health terminologies in a quality-controlled gateway. Health Information Science and Systems 2013; 1: 1-8. DOI http://dx.doi.org/ 10.1186/2047-2501-1-8.
18 Thirion B, Robu I, Darmoni SJ. Optimization of the PubMed automatic term mapping. Stud Health Technol Inform 2009; 150: 238-242. DOI http:// dx.doi.org/10.3233/978-1-60750-044-5-238.
19 Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005; 85 (03) 257-268.
20 Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol 1993; 46 (05) 423-429.
21 Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 1997; 126 (05) 376-80. DOI http://dx.doi.org/10.7326/0003-4819-126-5-199703010-00006.
22 Edwards P, Clarke M, DiGuiseppi C, Pratap S, Roberts I, Wentz R. Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Stat Med 2002; 21 (11) 1635-1640.
23 Mateen FJ, Oh J, Tergas AI, Bhayani NH, Kamdar BB. Titles versus titles and abstracts for initial screening of articles for systematic reviews. J Clin Epidemiol 2013; 6: 89-95. DOI http://dx.doi.org/ 10.2147/CLEP.S43118.
24 Wallace BC, Trikalinos TA, Lau J, Brodley C, Schmid CH. Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics 2010; 11: 55 DOI http://dx.doi. org/10.1186/1471-2105-11-55.
25 Spreckelsen C, Deserno TM, Spitzer K. The publication echo: effects of retrieving literature in PubMed by year of publication. Int J Med Inf 2010; 79 (04) 297-303. DOI http://dx.doi.org/10.1016/ j.ijmedinf.2010.01.007.
26 Rattinger GB, Mullins CD, Zuckerman IH, Onukwugha E, Walker LD, Gundlapalli A, Samore M, Delisle S. A sustainable strategy to prevent misuse of antibiotics for acute respiratory infections. PLoS One 2012; 7 (012) e51147 doi: 10.1371/journal.pone.0051147. Epub 2012 Dec 12.
27 Elkin PL, Brown SH, Wright G. Biomedical informatics: we are what we publish. Methods Inf Med 2013; 52 (06) 538-546. doi: 10.3414/ME13-01-0041. Epub 2013 Nov 19.
28 Iansavichus AV, Haynes RB, Lee CWC, Wilczynski NL, McKibbon A, Shariff SZ, Blake PG, Lindsay RM, Garg AX. Dialysis search filters for PubMed, Ovid MEDLINE, and Embase databases. Clin J Am Soc Nephrol 2012; 7 (10) 1624-1631. DOI http://dx.doi.org/10.2215/CJN.02360312.
29 Wilczynski NL, McKibbon KA, Walter SD, Garg AX, Haynes RB. MEDLINE clinical queries are robust when searching in recent publishing years. J Am Med Inform Assoc 2013; 20 (02) 363-368. DOI http://dx.doi.org/10.1136/amiajnl-2012-001075.
30 Falagas ME, Pitsouni I E, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB J 2008; 22 (02) 338-342. DOI http:// dx.doi.org/10.1096/fj.07-9492LSF.
31 Giustini D, Boulos MN. Google Scholar is not enough to be used alone for systematic reviews. Online J Public Health Inform 2013; 5 (02) 214 Print 2013.
32 Boeker M, Vach W, Motschall E. Google Scholar as replacement for systematic literature searches: good relative recall and precision are not enough. BMC Med Res Methodol 2013; 13: 131 doi: 10.1186/1471-2288-13-131.
33 Schuemie MJ, Talmon JL, Moorman PW, Kors JA. Mapping the domain of medical informatics. Methods Inf Med 2009; 48 (01) 76-83.
34 Bouaud J, Lamy JB. A medical informatics perspective on clinical decision support systems. Findings from the yearbook 2013 section on decision support. Yearb Med Inform 2013; 8 (01) 128-131.