Keywords
academic ophthalmology - ophthalmology Web site - Internet Web site - Internet resource
utilization - Web site accessibility
With the advent of the Internet and improvement in technology and infrastructure,
users today have access to an abundance of information. A 2011 national phone survey[1] found that 80% of adult Internet users examine and review online content as a means
of researching a specific disease or treatment. Thus, health content related to common
conditions or surgical procedures plays an important role in patient education.[2]
[3]
Several studies[4]
[5]
[6]
[7] have investigated health literacy, reading levels, and quality of health material
in different arenas of medicine. In ophthalmology, two studies[8]
[9] have evaluated the readability of content from web searches of ophthalmologic diagnoses
as well as published information from large organizations such as the American Academy
of Ophthalmology. The scope of these studies is limited to patient education material.
There is, however, other information that patients may seek: driving directions, a
physician's credentials, or updates regarding medical research and clinical trials.
In addition, an academic institution's Web site traffic is not entirely due to patients.
Aspiring employees, scientists looking for collaboration, and entrepreneurs may wish
to learn more about a specific institution in preparation for future contact. Consideration
of the myriad of visitors and their needs may lead to a successful online presence.
Internet postings of various ophthalmology residency programs have previously been
investigated.[10] While this work captured various aspects of ophthalmology department Web sites,
its scope was mainly limited to residency, faculty, and medical student information.
In contrast, our study performed a comprehensive analysis of Web sites belonging to
all academic ophthalmology departments. A total of 117 academic ophthalmology departments
were evaluated through the viewpoint of different users, capturing various key characteristics.
The percent of Web sites possessing each feature was calculated. Additionally, the
time and number of clicks required to get from the home page to a key feature were
measured.
To ascertain if indeed these features were wanted queries for a myriad of online users,
we hypothesized that “higher ranked” departments might contain a higher percentage
of Web site characteristics. Thus, a scoring algorithm was created and scores were
then compared between the U.S. News & World Report[11] (US News) top 13 ranked programs and the remaining 96 that possess an ophthalmology
residency training program. US News ranked only the top 13; ranking was based solely
on reputation stemming from survey responses of ophthalmologist gathered during the
previous 3 years.
Methods
All academic medical programs from a comprehensive list supplied by the Association
of American Medical Colleges (AAMC)[12] were included. From the 166 listed, only allopathic programs based in the United
States were selected, limiting this study to 149 programs. A corresponding Web site
for each program was identified; subsequently, all academic programs lacking departments
of ophthalmology were filtered out, resulting in a total 117 programs (2 programs
provided extremely limited Web site content, and thus were excluded from analysis).
Ophthalmology departments that were listed under departments of surgery (total of
four) were deemed acceptable in the inclusion criteria and were included in this analysis.
Cross-checking of the 117 candidates with all residency-capable departments provided
by SF Match[13] was completed.
Each academic department was evaluated and characterized from the unique viewpoints
of patients, trainees (medical students, residents, fellows), physicians in all arenas
(community, academia), and lastly scientists. All data were collected between October
1 and October 31, 2017. A thorough search was performed for each feature (discussed
in the various sections below), noting if an academic Web site provided a link or
access to such a feature. I fa Web site search function was available and a feature
could not be located, the search function was used to ensure that indeed the feature
did not exist. Of importance, the singular feature had to exclusively belong to the
department of ophthalmology or the corresponding eye institute. For example, several
departments provided an online donation link for the governing hospital but not specifically
for the department of ophthalmology.
Patient's Perspective
Evaluating from a patient's viewpoint, Web sites were tracked for the following competencies:
(1) access to clinical services: the department's address, education toward common
conditions and surgeries (e.g., cataract, glaucoma), the ability to contact the department,
to search and find a doctor, and to make an appointment online; (2) a comprehensive
list of clinical services; and (3) logistical support: answering billing questions
and providing driving directions, and accessibility: changes in text size, color,
and language. In addition, an academic program's integration with media and public
relations (M&PR) was measured by noting the presence of links to social media (e.g.,
Facebook, LinkedIn, Twitter, Google + ), online donations specifically for ophthalmology,
a section containing news or events related to the department (News/Events), a description
of that department's goal or mission, and photos of residents, fellows, and faculty.
The date of the last update in the News/Events section was also noted to reflect the
currency of highlighted material.
Trainee's Perspective
Evaluating a Web site from the perspective of trainees, the following information
was collected: listing of current residents and presence of individual biographies
(“biography” refers to anything in addition to educational background), a Frequently-Asked-Question
(FAQ) section, published curriculum, salary, and recent graduates. In addition, evaluation
criteria included faculty characteristics: listing of education and clinical expertise,
a biography, their academic rank, and publications. Furthermore, a brief description
of current research, listed clinical trials, and access (email or phone number) to
the studies were noted.
Navigation Time and Clicks
To assess the efficiency of accessing key Web site features, measurements of time
and number of clicks required to get from the homepage to the following desired page
were obtained: “About Us”/mission statement, listed faculty, address, residency program,
contact information (either phone or email), News/Events, “Find a Doctor,” and determining
the department leader. If the above feature was listed on the homepage, then it counted
as 0 clicks.
Comparison of the Top 13 Departments with Remaining 96
US News ranked only the top 13 departments in ophthalmology for the 2017–2018 year.
To determine if a difference in the amount of Web site content existed between the
top 13 and remaining 96, a scoring metric was created and employed for the four various
categories: clinical access, advertised clinical subspecialties, M&PR, and patient
support and accessibility. For each category, the total number of features was tallied.
All features had an equal weight of 1 point. A score of the total points per category
was obtained, and scores were normalized to the maximum points per given category.
Data Acquisition and Interrater Reliability
Each Web site was researched by two independent investigators, who were blinded to
each other's results. The results of the qualitative characteristics were averaged
between the two investigators. If there was any dispute in the dichotomous characteristics
(i.e., “Yes” or “No”), a third investigator resolved the conflicting data. The two
investigators agreed 98.1% of the time; in the rare circumstance of a disagreement,
95.5% were due to an investigator incorrectly omitting a Web site characteristic that
indeed existed. A nonparametric U test (Mann–Whitney) was used for all interval variables. All data were analyzed using
Graphpad Prism.[14] Results were considered significant with p < 0.05.
Results
Of the 149 possible academic departments, 117 met inclusion criteria and were evaluated.
All (100%) had Web site access. In total, 109 (93.2%) listed a residency program,
while 71 (60.7%) listed a fellowship program.
In the category of clinical access, 50 Web sites (42.7%) provided a way to schedule
an appointment online, 70 (59.8%) supplied educational material for common eye conditions
and procedures, and 83 (70.9%) incorporated a specific search function to locate a
doctor in ophthalmology ([Fig. 1A]). All (100%) provided the department's address, while all but six (95%) provided
contact information.
Fig. 1 Web site characteristics from a patient's viewpoint. Bar graphs display the percent
of Web sites possessing different features: (A) clinical access, (B) clinical subspecialties, (C) media and public relations, and (D) patient support.
As far as posted subspecialty services ([Fig. 1B]), glaucoma was highest, displayed in 111 Web sites (94.9%), followed by cornea in
110 (94%), retina in 109 (93.2%), oculoplastics in 106 (90.6%), pediatric ophthalmology
in 103 (88%), and neuro-ophthalmology in 99 (84.6%). The subsequent specialties were
represented less than 80% of the time: low-vision support and rehabilitation (76 [65%]),
immunology and uveitis (75 [64%]), and oncology (61 [52.1%]). Interestingly, emergency
and trauma was least represented, with only 52 (44.4%) departments advertising such
a clinical capability.
In terms of M&PR, 114 Web sites (97.4%) described themselves, by providing either
an “About Us” page or mission statement ([Fig. 1C]). Resident and faculty photos were displayed in 81 (74.3%) and 108 (92.3%) departments,
respectively. Interestingly, only 85 (72.6%) provided a News/Events section; analysis
of these sections showed a mean time elapsed between the last update and date of evaluation
of 28.7 weeks (median, 4 weeks; SD, 53.5 weeks) ([Supplementary Fig. S1]); 1 department was omitted, as dates could not be found. Thus, less than half (40.5%)
of Web sites had updates within the last month, while 24 (28.57%) lagged greater than
6 months behind ([Supplementary Table S1]). Finally, 58 (49.6%) departments provided a connection with a social network (Facebook,
Twitter, LinkedIn, Google + ), and 81 (69.2%) provided a charitable foundation webpage
or capabilities to donate online.
Regarding an accessible patient-centered view, 85 (72.6%) ophthalmology departments
provided driving directions ([Fig. 1D]), either displaying a map of the surrounding area or listing specific instructions
on how to reach the campus from major highways. Only 39 (33.3%) offered a means to
address financial (billing) support. Most surprising was the limited transformability
of the department's Web sites: 24 (20.5%) allowed for scaling of text size, and only
5 (4.3%) granted the ability to modify the text color. Finally, only 10 Web sites
(8.5%) had multilingual capabilities.
In terms of clinical faculty and research content displayed ([Fig. 2A]), all but one department (99.1%) displayed the highest leadership position (chair,
chief, or director), and 107 (91.5%) provided a biography for each provider, supplying
information on their education, clinical expertise, and academic rank (89.7, 84.6,
and 80.3%, respectively). However, only about half (48.7%) of ophthalmology departments
included an individual faculty member's publications. Instead, most Web sites (84.6%)
listed a link to an overview of their department's current research ([Fig. 2C]). Although 56 departments (47.9%) listed their current clinical trials, only 33
(28.2%) offered access to trial enrollment.
Fig. 2 Web site characteristics from trainee and faculty viewpoints. Bar graphs display
the percent of Web sites possessing different features: (A) faculty, (B) residency, and (C) research.
Results from each residency program evaluation show limited consistency across ophthalmology
departments ([Fig. 2B]). Three commonly provided features were listing of current residents and fellows
(86.2 and 64.8%, respectively), and an educational curriculum (79.8%). Thus, 13.8%
of programs provided no information about current residents, while 35.2% did not display
their current fellows. In addition, less than half of ophthalmology residency programs
included material on recent graduates (44.0%), resident biographies (35.8%), an FAQ
page (33.9%), or yearly salary (30.3%). The total number of residents and fellows
at each program was tallied, resulting in a mean of 12.8 (5 SD) and 6.9 (5.8 SD),
respectively.
In the assessment of time required to reach a particular web page destination, all
features except “Find a Doc” and determining the department leader could be reached
under 10 seconds from the home page ([Fig. 3A]). The shortest time was “About Us”/mission statement requiring 4.91 seconds (5.02
SD), followed by faculty, address, “Contact Us,” residency program, News/Events, and
“Find a Doc” (5.93 [5.82 SD], 6.76 [6.65 SD], 7.39 [7.06 SD], 7.49 [6.28 SD], 7.69
[15.5 SD], and 10.26 seconds [12.7 SD], respectively). Interestingly, the longest
time was spent determining the leader of the department (16.9 seconds [26.70 SD]).
The number of clicks needed to reach each specific destination was also measured ([Fig. 3B]); address used the fewest clicks, 0.55 (0.79 SD), followed by “About Us”/mission
statement with 0.66 (0.61 SD), and then “Contact Us,” News/Events, faculty, “Find
a Doctor,” and residency program (0.69 [0.58 SD], 0.75 [0.87 SD], 1.14 [1.34 SD],
1.22 [1.00 SD], and 1.28 clicks [0.72 SD], respectively). The highest amount of clicks
was determining the department's leader (1.46 [2.13 SD]).
Fig. 3 Navigational ease of Web site content. Average time (A) and average clicks (B) required to locate a distinct page, starting from the department's home page. Error
bars represent standard deviation.
Upon comparison to the rest of U.S. ophthalmology residencies, the top 13 had significantly
higher normalized scores in each evaluated competency ([Fig. 4A]–[D]; p = 0.0042, p < 0.0001, p = 0.0029, p = 0.0204, respectively). Surprisingly, the top 13 were as fast and required no more
clicks than the rest of academia to navigate to multiple Web site feature destinations
(p > 0.05), except address and residency program ([Fig. 4E], [F]; p = 0.0038 and 0.0476, respectively).
Fig. 4 Comparison of the US News top 13 ranked departments and remaining 96 that possess
an ophthalmology residency. Bar graphs display a comparison of the normalized scores
between the top 13 (T13) and remaining 96 (R96) in the following Web site features:
(A) clinical access, (B) clinical subspecialties, (C) media and public relations, and (D) patient support. A comparison of (E) time and (F) number of clicks between the top 13 and the remaining 96. Error bars represent standard
deviation. Statistics, Mann–Whitney U test for all panels.
Discussion
The Internet is vital in many professional arenas, including medicine and health;
indeed, studies have used Internet-based interventions to affect smoking cessation,[14] behavioral health,[15] and improvements in glycemic control in diabetes management.[16] In 2003, Mayo and coworkers[10] found that approximately 17% of academic ophthalmology departments did not provide
online access; of the departments that contained Internet postings, a minority provided
postings in all categories. While today all U.S. academic ophthalmology departments
can be accessed via the Internet, few provide content from the viewpoints of numerous
users. There may be value in designing online content in a comprehensive and accessible
manner.
According to the research herein, there are several areas that academic ophthalmology
Web sites might target for improvement. It is possible that increasing educational
health content and capabilities to schedule appointments online might lead to improved
patient access. In addition, portraying all clinical capabilities could be useful
to various users (e.g., patients, trainees, or industry contacts). Indeed, the top
13 departments consistently provide a list of all major subspecialty services.
Currently, academic ophthalmology Web sites lack up-to-date capabilities to interact
with large social networks. Furthermore, only about 40% have updated their News/Events
content in the last month, while about 30% have an elapsed time of half a year or
more. News and event communications—similar to that of health content and guidelines[17]—could provide institutional benefit if regularly updated by possibly promoting interest
and engagement with patients, trainees, alumni, scientists, and industry partners.
In addition, donations are an integral part of fundraising, yet less than 70% of departments
provide capabilities for online charitable donations. Byrne and colleagues[18] found that scientists competing for crowdfunding had higher levels of funding if
they built a large audience and actively engaged with that audience. Thus, ophthalmology
departments may achieve higher levels of funding by possibly increasing interactions
with different users and social networking capabilities, updating news and events
content, or implementing a capacity for online donation. Further work is needed to
evaluate if indeed a larger online presence with active engagement can increase department
donations and user interaction.
Accessibility and assistive tools (multilingual capabilities, text size, and color
change) are extremely limited for most Web sites, despite the fact that ophthalmology
departments cater to an older patient population, which may benefit from using the
assistance of such technology. At this time, the United States has the highest immigrant
population[19] in its history, underscoring the importance of multilingual education material.
In addition, the ophthalmology patient population suffers from significant visual
impairment,[20] establishing a need for such adaptable tools.
Various studies[21]
[22] have used surveys to understand which criteria are important and helpful in terms
of online content for medical trainees, although none were surveyed within ophthalmology.
In line with other work,[23]
[24]
[25] academic ophthalmology departments do not seem to provide an abundance of residency
or fellowship online content. As recent trends show an increase in residency applicants,
comprehensive but pertinent online content could benefit applicants by helping them
decide which programs to pursue.
A motivation of the current study was to determine the online representation of academic
U.S. ophthalmology by measuring the extent of a departments' online content. Different
users, however, might inquire into different Web site features. To our knowledge,
no studies exist that characterize the merits or value of one Web site characteristic
over another. Comparing Web site features of the top 13 departments with the remaining
96 serves as one potential validation approach. Indeed, our research shows that the
top 13 academic programs have comprehensively constructed their Web sites, encompassing
abundant yet pertinent information for different users. Interestingly, though, the
top 13 are no more cumbersome to navigate through than the remaining 96, with similar
measured time and number of clicks. Thus, a thoroughly designed Web site may be important
for attracting prospective residents, visiting professors, and industry partners.
However, financial costs could be prohibitive in comprehensive Web site construction
and maintenance, especially for small departments with limited budgets.
There are several limitations in this study. First, there may be certain Web site
features that are present but difficult to locate that it appears as if they do not
exist. This potential caveat was minimized by having multiple investigators analyze
each Web site and searching for the feature if a search function was available. Second,
each characteristic had an equal weight in our analysis. However, for some users,
assistive tools could be significantly more important than the campus address (as
an example). To avoid any potential bias by attributing different values to different
features, each feature was weighed equally. Thus, this study provides insight into
a large variety of online content from current academic departments; individual program
leaders can themselves choose whichever characteristics they deem most important.
Another limitation centers on news and event content updates; it is possible that
since the evaluation in October 2017, a department has modified and updated key information.
Finally, there was no evaluation of any Canadian departments or nonacademic ophthalmology
groups; there could be a tremendous learning opportunity by comparing U.S. academic
ophthalmology Web site design and online content with that of a similar advanced nation
or perhaps with private-practice ophthalmology. Despite the acknowledged limitations,
this work should be highly reproducible by most individuals, as Web site analysis
does not require significant technical or clinical expertise.
This study is the first to investigate and summarize the level of comprehensiveness
and navigational ease of U.S. academic ophthalmology Web sites from the viewpoints
of multiple users. The end points herein are novel, with one validation approach incorporating
a comparison of “higher ranked” departments with the rest of the field. Additional
work is warranted to evaluate if the possession of certain Web site characteristics
leads to better patient care or if a comprehensively constructed Web site (designed
with multiple users in mind) improves the user's experience.
It is anticipated that the findings herein may be used by academic ophthalmology departments
to improve their Web site design and online content. Although the majority of Web
sites provide a strong basic foundation, further work may be necessary to enhance
content as well as target accessibility. Accordingly, Web site design may be modeled
after the top 13 ranked ophthalmology departments, although other academic programs
may provide broad but pertinent online content.
