The Current Online Face of U.S. Academic Ophthalmology

Abstract

Purpose To evaluate the comprehensiveness and navigation ease of online content from Web sites of U.S. academic ophthalmology departments.

Design Cross-sectional analysis of 117 U.S. academic ophthalmology department Web sites from October 1 to October 31, 2017.

Methods Data were obtained on various categories, including: clinical access and subspecialty services, social media, patient support and accessibility, residency and fellowship details, and research and faculty characteristics. Percent of Web sites possessing each feature was calculated. In addition, a comparison of content completeness and navigation metrics of Web sites between the US News and World Report top 13 ranked departments and the remaining 96 that possess an ophthalmology residency was performed.

Results Greater than 80% of Web sites list a basic core of information online: address, contact information, resident and faculty characteristics, and clinical expertise. However, only 69.2% have capabilities to donate online and 59.8% supply educational material for common eye conditions. Less than half of institutions list emergency and trauma, oncology, and low-vision rehabilitation services; only 49.6% provide a social network platform. Strikingly, accessibility features are limited; scalable text, changeable color, and multilingual capabilities are available in 20.5, 4.3, and 8.5% of Web sites, respectively. In the comparison of top 13 departments with the remaining 96, the high-ranking departments possess more Web site features, but are just as efficient to navigate.

Conclusion U.S. academic ophthalmology departments in 2017 provide a basic foundation of online content, but few contain abundant features from the viewpoints of different users.

Keywords

academic ophthalmology - ophthalmology Web site - Internet Web site - Internet resource utilization - Web site accessibility

With the advent of the Internet and improvement in technology and infrastructure, users today have access to an abundance of information. A 2011 national phone survey[1] found that 80% of adult Internet users examine and review online content as a means of researching a specific disease or treatment. Thus, health content related to common conditions or surgical procedures plays an important role in patient education.[2] [3]

Several studies[4] [5] [6] [7] have investigated health literacy, reading levels, and quality of health material in different arenas of medicine. In ophthalmology, two studies[8] [9] have evaluated the readability of content from web searches of ophthalmologic diagnoses as well as published information from large organizations such as the American Academy of Ophthalmology. The scope of these studies is limited to patient education material. There is, however, other information that patients may seek: driving directions, a physician's credentials, or updates regarding medical research and clinical trials. In addition, an academic institution's Web site traffic is not entirely due to patients. Aspiring employees, scientists looking for collaboration, and entrepreneurs may wish to learn more about a specific institution in preparation for future contact. Consideration of the myriad of visitors and their needs may lead to a successful online presence.

Internet postings of various ophthalmology residency programs have previously been investigated.[10] While this work captured various aspects of ophthalmology department Web sites, its scope was mainly limited to residency, faculty, and medical student information. In contrast, our study performed a comprehensive analysis of Web sites belonging to all academic ophthalmology departments. A total of 117 academic ophthalmology departments were evaluated through the viewpoint of different users, capturing various key characteristics. The percent of Web sites possessing each feature was calculated. Additionally, the time and number of clicks required to get from the home page to a key feature were measured.

To ascertain if indeed these features were wanted queries for a myriad of online users, we hypothesized that “higher ranked” departments might contain a higher percentage of Web site characteristics. Thus, a scoring algorithm was created and scores were then compared between the U.S. News & World Report[11] (US News) top 13 ranked programs and the remaining 96 that possess an ophthalmology residency training program. US News ranked only the top 13; ranking was based solely on reputation stemming from survey responses of ophthalmologist gathered during the previous 3 years.

Methods

All academic medical programs from a comprehensive list supplied by the Association of American Medical Colleges (AAMC)[12] were included. From the 166 listed, only allopathic programs based in the United States were selected, limiting this study to 149 programs. A corresponding Web site for each program was identified; subsequently, all academic programs lacking departments of ophthalmology were filtered out, resulting in a total 117 programs (2 programs provided extremely limited Web site content, and thus were excluded from analysis). Ophthalmology departments that were listed under departments of surgery (total of four) were deemed acceptable in the inclusion criteria and were included in this analysis. Cross-checking of the 117 candidates with all residency-capable departments provided by SF Match[13] was completed.

Each academic department was evaluated and characterized from the unique viewpoints of patients, trainees (medical students, residents, fellows), physicians in all arenas (community, academia), and lastly scientists. All data were collected between October 1 and October 31, 2017. A thorough search was performed for each feature (discussed in the various sections below), noting if an academic Web site provided a link or access to such a feature. I fa Web site search function was available and a feature could not be located, the search function was used to ensure that indeed the feature did not exist. Of importance, the singular feature had to exclusively belong to the department of ophthalmology or the corresponding eye institute. For example, several departments provided an online donation link for the governing hospital but not specifically for the department of ophthalmology.

Patient's Perspective

Evaluating from a patient's viewpoint, Web sites were tracked for the following competencies: (1) access to clinical services: the department's address, education toward common conditions and surgeries (e.g., cataract, glaucoma), the ability to contact the department, to search and find a doctor, and to make an appointment online; (2) a comprehensive list of clinical services; and (3) logistical support: answering billing questions and providing driving directions, and accessibility: changes in text size, color, and language. In addition, an academic program's integration with media and public relations (M&PR) was measured by noting the presence of links to social media (e.g., Facebook, LinkedIn, Twitter, Google + ), online donations specifically for ophthalmology, a section containing news or events related to the department (News/Events), a description of that department's goal or mission, and photos of residents, fellows, and faculty. The date of the last update in the News/Events section was also noted to reflect the currency of highlighted material.

Trainee's Perspective

Evaluating a Web site from the perspective of trainees, the following information was collected: listing of current residents and presence of individual biographies (“biography” refers to anything in addition to educational background), a Frequently-Asked-Question (FAQ) section, published curriculum, salary, and recent graduates. In addition, evaluation criteria included faculty characteristics: listing of education and clinical expertise, a biography, their academic rank, and publications. Furthermore, a brief description of current research, listed clinical trials, and access (email or phone number) to the studies were noted.

Navigation Time and Clicks

To assess the efficiency of accessing key Web site features, measurements of time and number of clicks required to get from the homepage to the following desired page were obtained: “About Us”/mission statement, listed faculty, address, residency program, contact information (either phone or email), News/Events, “Find a Doctor,” and determining the department leader. If the above feature was listed on the homepage, then it counted as 0 clicks.

Comparison of the Top 13 Departments with Remaining 96

US News ranked only the top 13 departments in ophthalmology for the 2017–2018 year. To determine if a difference in the amount of Web site content existed between the top 13 and remaining 96, a scoring metric was created and employed for the four various categories: clinical access, advertised clinical subspecialties, M&PR, and patient support and accessibility. For each category, the total number of features was tallied. All features had an equal weight of 1 point. A score of the total points per category was obtained, and scores were normalized to the maximum points per given category.

Data Acquisition and Interrater Reliability

Each Web site was researched by two independent investigators, who were blinded to each other's results. The results of the qualitative characteristics were averaged between the two investigators. If there was any dispute in the dichotomous characteristics (i.e., “Yes” or “No”), a third investigator resolved the conflicting data. The two investigators agreed 98.1% of the time; in the rare circumstance of a disagreement, 95.5% were due to an investigator incorrectly omitting a Web site characteristic that indeed existed. A nonparametric U test (Mann–Whitney) was used for all interval variables. All data were analyzed using Graphpad Prism.[14] Results were considered significant with p < 0.05.

Results

Of the 149 possible academic departments, 117 met inclusion criteria and were evaluated. All (100%) had Web site access. In total, 109 (93.2%) listed a residency program, while 71 (60.7%) listed a fellowship program.

In the category of clinical access, 50 Web sites (42.7%) provided a way to schedule an appointment online, 70 (59.8%) supplied educational material for common eye conditions and procedures, and 83 (70.9%) incorporated a specific search function to locate a doctor in ophthalmology ([Fig. 1A]). All (100%) provided the department's address, while all but six (95%) provided contact information.

Fig. 1 Web site characteristics from a patient's viewpoint. Bar graphs display the percent of Web sites possessing different features: (A) clinical access, (B) clinical subspecialties, (C) media and public relations, and (D) patient support.

As far as posted subspecialty services ([Fig. 1B]), glaucoma was highest, displayed in 111 Web sites (94.9%), followed by cornea in 110 (94%), retina in 109 (93.2%), oculoplastics in 106 (90.6%), pediatric ophthalmology in 103 (88%), and neuro-ophthalmology in 99 (84.6%). The subsequent specialties were represented less than 80% of the time: low-vision support and rehabilitation (76 [65%]), immunology and uveitis (75 [64%]), and oncology (61 [52.1%]). Interestingly, emergency and trauma was least represented, with only 52 (44.4%) departments advertising such a clinical capability.

In terms of M&PR, 114 Web sites (97.4%) described themselves, by providing either an “About Us” page or mission statement ([Fig. 1C]). Resident and faculty photos were displayed in 81 (74.3%) and 108 (92.3%) departments, respectively. Interestingly, only 85 (72.6%) provided a News/Events section; analysis of these sections showed a mean time elapsed between the last update and date of evaluation of 28.7 weeks (median, 4 weeks; SD, 53.5 weeks) ([Supplementary Fig. S1]); 1 department was omitted, as dates could not be found. Thus, less than half (40.5%) of Web sites had updates within the last month, while 24 (28.57%) lagged greater than 6 months behind ([Supplementary Table S1]). Finally, 58 (49.6%) departments provided a connection with a social network (Facebook, Twitter, LinkedIn, Google + ), and 81 (69.2%) provided a charitable foundation webpage or capabilities to donate online.

Regarding an accessible patient-centered view, 85 (72.6%) ophthalmology departments provided driving directions ([Fig. 1D]), either displaying a map of the surrounding area or listing specific instructions on how to reach the campus from major highways. Only 39 (33.3%) offered a means to address financial (billing) support. Most surprising was the limited transformability of the department's Web sites: 24 (20.5%) allowed for scaling of text size, and only 5 (4.3%) granted the ability to modify the text color. Finally, only 10 Web sites (8.5%) had multilingual capabilities.

In terms of clinical faculty and research content displayed ([Fig. 2A]), all but one department (99.1%) displayed the highest leadership position (chair, chief, or director), and 107 (91.5%) provided a biography for each provider, supplying information on their education, clinical expertise, and academic rank (89.7, 84.6, and 80.3%, respectively). However, only about half (48.7%) of ophthalmology departments included an individual faculty member's publications. Instead, most Web sites (84.6%) listed a link to an overview of their department's current research ([Fig. 2C]). Although 56 departments (47.9%) listed their current clinical trials, only 33 (28.2%) offered access to trial enrollment.

Fig. 2 Web site characteristics from trainee and faculty viewpoints. Bar graphs display the percent of Web sites possessing different features: (A) faculty, (B) residency, and (C) research.

Results from each residency program evaluation show limited consistency across ophthalmology departments ([Fig. 2B]). Three commonly provided features were listing of current residents and fellows (86.2 and 64.8%, respectively), and an educational curriculum (79.8%). Thus, 13.8% of programs provided no information about current residents, while 35.2% did not display their current fellows. In addition, less than half of ophthalmology residency programs included material on recent graduates (44.0%), resident biographies (35.8%), an FAQ page (33.9%), or yearly salary (30.3%). The total number of residents and fellows at each program was tallied, resulting in a mean of 12.8 (5 SD) and 6.9 (5.8 SD), respectively.

In the assessment of time required to reach a particular web page destination, all features except “Find a Doc” and determining the department leader could be reached under 10 seconds from the home page ([Fig. 3A]). The shortest time was “About Us”/mission statement requiring 4.91 seconds (5.02 SD), followed by faculty, address, “Contact Us,” residency program, News/Events, and “Find a Doc” (5.93 [5.82 SD], 6.76 [6.65 SD], 7.39 [7.06 SD], 7.49 [6.28 SD], 7.69 [15.5 SD], and 10.26 seconds [12.7 SD], respectively). Interestingly, the longest time was spent determining the leader of the department (16.9 seconds [26.70 SD]). The number of clicks needed to reach each specific destination was also measured ([Fig. 3B]); address used the fewest clicks, 0.55 (0.79 SD), followed by “About Us”/mission statement with 0.66 (0.61 SD), and then “Contact Us,” News/Events, faculty, “Find a Doctor,” and residency program (0.69 [0.58 SD], 0.75 [0.87 SD], 1.14 [1.34 SD], 1.22 [1.00 SD], and 1.28 clicks [0.72 SD], respectively). The highest amount of clicks was determining the department's leader (1.46 [2.13 SD]).

Fig. 3 Navigational ease of Web site content. Average time (A) and average clicks (B) required to locate a distinct page, starting from the department's home page. Error bars represent standard deviation.

Upon comparison to the rest of U.S. ophthalmology residencies, the top 13 had significantly higher normalized scores in each evaluated competency ([Fig. 4A]–[D]; p = 0.0042, p < 0.0001, p = 0.0029, p = 0.0204, respectively). Surprisingly, the top 13 were as fast and required no more clicks than the rest of academia to navigate to multiple Web site feature destinations (p > 0.05), except address and residency program ([Fig. 4E], [F]; p = 0.0038 and 0.0476, respectively).

Fig. 4 Comparison of the US News top 13 ranked departments and remaining 96 that possess an ophthalmology residency. Bar graphs display a comparison of the normalized scores between the top 13 (T13) and remaining 96 (R96) in the following Web site features: (A) clinical access, (B) clinical subspecialties, (C) media and public relations, and (D) patient support. A comparison of (E) time and (F) number of clicks between the top 13 and the remaining 96. Error bars represent standard deviation. Statistics, Mann–Whitney U test for all panels.

Discussion

The Internet is vital in many professional arenas, including medicine and health; indeed, studies have used Internet-based interventions to affect smoking cessation,[14] behavioral health,[15] and improvements in glycemic control in diabetes management.[16] In 2003, Mayo and coworkers[10] found that approximately 17% of academic ophthalmology departments did not provide online access; of the departments that contained Internet postings, a minority provided postings in all categories. While today all U.S. academic ophthalmology departments can be accessed via the Internet, few provide content from the viewpoints of numerous users. There may be value in designing online content in a comprehensive and accessible manner.

According to the research herein, there are several areas that academic ophthalmology Web sites might target for improvement. It is possible that increasing educational health content and capabilities to schedule appointments online might lead to improved patient access. In addition, portraying all clinical capabilities could be useful to various users (e.g., patients, trainees, or industry contacts). Indeed, the top 13 departments consistently provide a list of all major subspecialty services.

Currently, academic ophthalmology Web sites lack up-to-date capabilities to interact with large social networks. Furthermore, only about 40% have updated their News/Events content in the last month, while about 30% have an elapsed time of half a year or more. News and event communications—similar to that of health content and guidelines[17]—could provide institutional benefit if regularly updated by possibly promoting interest and engagement with patients, trainees, alumni, scientists, and industry partners. In addition, donations are an integral part of fundraising, yet less than 70% of departments provide capabilities for online charitable donations. Byrne and colleagues[18] found that scientists competing for crowdfunding had higher levels of funding if they built a large audience and actively engaged with that audience. Thus, ophthalmology departments may achieve higher levels of funding by possibly increasing interactions with different users and social networking capabilities, updating news and events content, or implementing a capacity for online donation. Further work is needed to evaluate if indeed a larger online presence with active engagement can increase department donations and user interaction.

Accessibility and assistive tools (multilingual capabilities, text size, and color change) are extremely limited for most Web sites, despite the fact that ophthalmology departments cater to an older patient population, which may benefit from using the assistance of such technology. At this time, the United States has the highest immigrant population[19] in its history, underscoring the importance of multilingual education material. In addition, the ophthalmology patient population suffers from significant visual impairment,[20] establishing a need for such adaptable tools.

Various studies[21] [22] have used surveys to understand which criteria are important and helpful in terms of online content for medical trainees, although none were surveyed within ophthalmology. In line with other work,[23] [24] [25] academic ophthalmology departments do not seem to provide an abundance of residency or fellowship online content. As recent trends show an increase in residency applicants, comprehensive but pertinent online content could benefit applicants by helping them decide which programs to pursue.

A motivation of the current study was to determine the online representation of academic U.S. ophthalmology by measuring the extent of a departments' online content. Different users, however, might inquire into different Web site features. To our knowledge, no studies exist that characterize the merits or value of one Web site characteristic over another. Comparing Web site features of the top 13 departments with the remaining 96 serves as one potential validation approach. Indeed, our research shows that the top 13 academic programs have comprehensively constructed their Web sites, encompassing abundant yet pertinent information for different users. Interestingly, though, the top 13 are no more cumbersome to navigate through than the remaining 96, with similar measured time and number of clicks. Thus, a thoroughly designed Web site may be important for attracting prospective residents, visiting professors, and industry partners. However, financial costs could be prohibitive in comprehensive Web site construction and maintenance, especially for small departments with limited budgets.

There are several limitations in this study. First, there may be certain Web site features that are present but difficult to locate that it appears as if they do not exist. This potential caveat was minimized by having multiple investigators analyze each Web site and searching for the feature if a search function was available. Second, each characteristic had an equal weight in our analysis. However, for some users, assistive tools could be significantly more important than the campus address (as an example). To avoid any potential bias by attributing different values to different features, each feature was weighed equally. Thus, this study provides insight into a large variety of online content from current academic departments; individual program leaders can themselves choose whichever characteristics they deem most important.

Another limitation centers on news and event content updates; it is possible that since the evaluation in October 2017, a department has modified and updated key information. Finally, there was no evaluation of any Canadian departments or nonacademic ophthalmology groups; there could be a tremendous learning opportunity by comparing U.S. academic ophthalmology Web site design and online content with that of a similar advanced nation or perhaps with private-practice ophthalmology. Despite the acknowledged limitations, this work should be highly reproducible by most individuals, as Web site analysis does not require significant technical or clinical expertise.

This study is the first to investigate and summarize the level of comprehensiveness and navigational ease of U.S. academic ophthalmology Web sites from the viewpoints of multiple users. The end points herein are novel, with one validation approach incorporating a comparison of “higher ranked” departments with the rest of the field. Additional work is warranted to evaluate if the possession of certain Web site characteristics leads to better patient care or if a comprehensively constructed Web site (designed with multiple users in mind) improves the user's experience.

It is anticipated that the findings herein may be used by academic ophthalmology departments to improve their Web site design and online content. Although the majority of Web sites provide a strong basic foundation, further work may be necessary to enhance content as well as target accessibility. Accordingly, Web site design may be modeled after the top 13 ranked ophthalmology departments, although other academic programs may provide broad but pertinent online content.

Conflict of Interest

None declared.

Acknowledgments

We would like to thank Krasimir Tolev, Lauren Lombardi, Nisarg Chhaya, Lotem Nativ, and Yen Nguyen for help with data acquisition.

This paper was presented, in part, at the ARVO annual meeting in Hawaii, 2018.

• References

• 1 Hampton K, Goulet LS, Rainie L, Purcell K. Social networking sites and our lives [Technical Report]. Pew Internet & American Life Project. Washington, DC; 2011. Available at: http://www.pewinternet.org/2011/06/16/social-networking-sites-and-our-lives/
• 2 Tonsaker T, Bartlett G, Trpkov C. Health information on the Internet: gold mine or minefield?. Can Fam Physician 2014; 60 (05) 407-408
• 3 Hartzband P, Groopman J. Untangling the Web—patients, doctors, and the Internet. N Engl J Med 2010; 362 (12) 1063-1066
• 4 Rayess H, Zuliani GF, Gupta A. , et al. Critical analysis of the quality, readability, and technical aspects of online information provided for neck-lifts. JAMA Facial Plast Surg 2017; 19 (02) 115-120
• 5 Alsaiari A, Joury A, Aljuaid M, Wazzan M, Pines JM. The content and quality of health information on the internet for patients and families on adult kidney cancer. J Cancer Educ 2017; 32 (04) 878-884
• 6 Nghiem AZ, Mahmoud Y, Som R. Evaluating the quality of internet information for breast cancer. Breast 2016; 25: 34-37
• 7 Fisher JH, O'Connor D, Flexman AM, Shapera S, Ryerson CJ. Accuracy and reliability of internet resources for information on idiopathic pulmonary fibrosis. Am J Respir Crit Care Med 2016; 194 (02) 218-225
• 8 Huang G, Fang CH, Agarwal N, Bhagat N, Eloy JA, Langer PD. Assessment of online patient education materials from major ophthalmologic associations. JAMA Ophthalmol 2015; 133 (04) 449-454
• 9 Edmunds MR, Barry RJ, Denniston AK. Readability assessment of online ophthalmic patient information. JAMA Ophthalmol 2013; 131 (12) 1610-1616
• 10 Mayo GL, Lindhorst GC, Rosende C. American ophthalmology graduate medical education and the web: current state of internet resource utilization. Am J Ophthalmol 2003; 135 (05) 708-709
• 11 U.S. News & World Report. Best Hospitals for Ophthalmology [Report]. 2018. Available at: https://health.usnews.com/best-hospitals/rankings/ophthalmology . Accessed August 8, 2017
• 12 AAMC. All Medical School Members [Report]. Available at: https://members.aamc.org/eweb/DynamicPage.aspx?webcode=AAMCOrgSearchResult&orgtype=Medical%20School . Accessed November 2017
• 13 San Francisco Match. 2017 Ophthalmology Residencies [Accessed through applicant portal]. Available at: https://sfmatch.org/SpecialtyInsideAll.aspx?id=2&typ=1&name=Ophthalmology . Accessed November 2017
• 14 Graham AL, Carpenter KM, Cha S. , et al. Systematic review and meta-analysis of Internet interventions for smoking cessation among adults. Subst Abuse Rehabil 2016; 7: 55-69
• 15 Abraham AA, Chow WC, So HK. , et al. Lifestyle intervention using an internet-based curriculum with cell phone reminders for obese Chinese teens: a randomized controlled study. PLoS One 2015; 10 (05) e0125673
• 16 Cotter AP, Durant N, Agne AA, Cherrington AL. Internet interventions to support lifestyle modification for diabetes management: a systematic review of the evidence. J Diabetes Complications 2014; 28 (02) 243-251
• 17 Shekelle P, Eccles MP, Grimshaw JM, Woolf SH. When should clinical guidelines be updated?. BMJ 2001; 323 (7305): 155-157
• 18 Byrnes JE, Ranganathan J, Walker BL, Faulkes Z. To crowdfund research, scientists must build an audience for their work. PLoS One 2014; 9 (12) e110329
• 19 Camarota SA, Zeigler K. Immigrants in the United States: A Profile of the Foreign-Born Using 2014 and 2015 Census Bureau Data. Center for Immigration Studies; October 2016
• 20 Varma R, Vajaranant TS, Burkemper B. , et al. Visual impairment and blindness in adults in the United States: demographic and geographic variations from 2015 to 2050. JAMA Ophthalmol 2016; 134 (07) 802-809
• 21 Embi PJ, Desai S, Cooney TG. Use and utility of Web-based residency program information: a survey of residency applicants. J Med Internet Res 2003; 5 (03) e22
• 22 Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ. Aspects of residency programs' web sites important to student applicants. Acad Emerg Med 2005; 12 (01) 89-92
• 23 Oladeji LO, Yu JC, Oladeji AK, Ponce BA. How useful are orthopedic surgery residency web pages?. J Surg Educ 2015; 72 (06) 1185-1189
• 24 Homer N, Yoon MK. Evaluation of the American Society of Ophthalmic Plastic and Reconstructive Surgery (ASOPRS) Fellowship Program Website Content and Quality. Ophthal Plast Reconstr Surg 2017; 33 (06) 471-473
• 25 Silvestre J, Guzman JZ, Abbatematteo JM, Chang B, Levin LS. Evaluation of content and accessibility of hand fellowship websites. Hand (NY) 2015; 10 (03) 516-521