Decay of References to Web sites in Articles Published in General Medical Journals: Mainstream vs Small Journals

Summary

Background: Over the last decade, Web sites (URLs) have been increasingly cited in scientific articles. However, the contents of the page of interest may change over the time.

Objective: To investigate the trend of citation to URLs in five general medical journals since January 2006 to June 2013 and to compare the trends in mainstream journals with small journals.

Methods: References of all original articles and review articles published between January 2006 and June 2013 in three regional journals – Archives of Iranian Medicine (AIM), Eastern Mediterranean Health Journal (EMHJ), and Journal of Postgraduate Medical Institute (JPMI) – and two mainstream journals – The Lancet and British Medical Journal (BMJ) – were reviewed. The references were checked to determine the frequency of citation to URLs as well as the rate of accessibility of the URLs cited.

Results: A total of 2822 articles was studied. Since January 2006 onward, the number of citations to URLs increased in the journals (doubling time ranged from 4.2 years in EMHJ to 13.9 years in AIM). Overall, the percentage of articles citing at least one URL has increased from 24% in 2006 to 48.5% in 2013. Accessibility to URLs decayed as the references got old (half life ranged from 2.2 years in EMHJ to 5.3 years in BMJ). The ratio of citation to URLs in the studied mainstream journals, as well as the ratio of URLs accessible were significantly (p<0.001) higher than the small medical journals.

Conclusion: URLs are increasingly cited, but their contents decay with time. The trend of citing and decaying URLs are different in mainstream journals compared to small medical journals. Decay of URL contents would jeopardize the accuracy of the references and thus, the body of evidence. One way to tackle this important obstacle is to archive URLs permanently.

Citation: Habibzadeh P. Decay of references to web sites in articles published in general medical journals: Mainstream vs small journals. Appl Clin Inf 2013; 4: 455–464

http://dx.doi.org/10.4338/ACI-2013-07-RA-0055

Keywords

Internet - scientometrics - citation analysis - uniform resource locator - URL

 

Conflicts of interest

The author has no conflicts of interest to declare.

• References

• 1 Brender J, Talmon J. On using references as evidence. Methods Inf Med 2009; 48: 503-507.
• 2 Browne RF, Logan PM, Lee MJ, Torreggiani WC. The accuracy of references in manuscripts submitted for publication. Can Assoc Radiol J 2004; 55: 170-173.
• 3 Siebers R, Holt S. Accuracy of references in five leading medical journals. Lancet 2000; 356: 1445.
• 4 Goldberg R, Newton E, Cameron J, Jacobson R, Chan L, Bukata WR. et al. Reference accuracy in the emergency medicine literature. Ann Emerg Med 1993; 22: 1450-1454.
• 5 Jackson K, Porrino Jr. JA, Tan V, Daluiski A. Reference accuracy in the Journal of Hand Surgery. J Hand Surg Am 2003; 28: 377-380.
• 6 Thorp AW, Schriger DL. Citations to Web pages in scientific articles: the permanence of archived references. Ann Emerg Med 2011; 57: 165-168.
• 7 Adelhard K, Obst O. Evaluation of medical internet sites. Methods Inf Med 1999; 38: 75-79.
• 8 Carnevale RJ, Aronsky D. The life and death of URLs in five biomedical informatics journals. Int J Med Inform 2007; 76: 269-273.
• 9 Wren JD. URL decay in MEDLINE–a 4-year follow-up study. Bioinformatics 2008; 24: 1381-1385.
• 10 Crichlow R, Winbush N, Davies S. Accessibility and accuracy of web page references in 5 major medical journals. JAMA 2004; 292: 2723-2724.
• 11 Wagner C, Gebremichael MD, Taylor MK, Soltys MJ. Disappearing act: decay of uniform resource locators in health care management journals. J Med Libr Assoc 2009; 97: 122-130.
• 12 Wren JD, Johnson KR, Crockett DM, Heilig LF, Schilling LM, Dellavalle RP. Uniform resource locator decay in dermatology journals: author attitudes and preservation practices. Arch Dermatol 2006; 142: 1147-1152.
• 13 Falagas ME, Karveli EA, Tritsaroli VI. The risk of using the Internet as reference resource: a comparative study. Int J Med Inform 2008; 77: 280-286.
• 14 Aronsky D, Madani S, Carnevale RJ, Duda S, Feyder MT. The prevalence and inaccessibility of Internet references in the biomedical literature at the time of publication. J Am Med Inform Assoc 2007; 14: 232-234.
• 15 Dellavalle RP, Hester EJ, Heilig LF, Drake AL, Kuntzman JW, Graber M. et al. Information science. Going, going, gone: lost Internet references. Science 2003; 302: 787-788.
• 16 Lawrence S, Pennock DM, Flake GW, Krovetz R, Coetzee FM, Glover E. et al. Persistence of Web References in Scientific Research. Computer 2001; 34: 26-31.
• 17 Internet users per 100 inhabitants, 2001–2011: ITU Statistics.. Available from: http://www.itu.int/ITUD/ict/statistics/material/excel/2011/Internet_users_01–11.xls and http://www.webcitation.org/6IV6Z70cn (Archived by WebCite). [accessed July 30, 2013]
• 18 Wren JD. 404 not found: the stability and persistence of URLs published in MEDLINE. Bioinformatics 2004; 20: 668-672.
• 19 Kahle B. Preserving the Internet 1997. Available from: http://web.archive.org/web 19970504212157/http://www.sciam.com/0397issue/0397kahle.html (Archived by Internet Archive). [accessed July 30, 2013]
• 20 Weiss R. On the Web, Research Work Proves Ephemeral 2003. Available from: http://stevereads.com cache/ephemeral_web_pages.html and http://www.webcitation.org/6IV8qT1QC (Archived by WebCite). [accessed July 30, 2013]
• 21 Ducut E, Liu F, Fontelo P. An update on Uniform Resource Locator (URL) decay in MEDLINE abstracts and measures for its mitigation. BMC Med Inform Decis Mak 2008; 8: 23.
• 22 Persistent Uniform Resource Locator (PURL).. Available from: http://purl.oclc.org/docs/index.html and http://www.webcitation.org/6IVBrfh3B (Archived by WebCite). [accessed july 30, 2013]
• 23 WebCite.. Available from: http://www.webcitation.org. [accessed July 30, 2013]
• 24 DiCarlo JV, Pastor X, Markovitz BP. The shadow uniform resource locator: standardizing citations of electronically published materials. J Am Med Inform Assoc 2000; 7: 149-151.