Examining One-Star Reviews in Orthopaedic Hand Surgeons in Large U.S. Cities

Abstract

Introduction Physician-review websites are a commonly used resource by patients when choosing a surgeon. While data exist regarding some surgical specialties, no study has examined negative one-star reviews for orthopaedic-trained hand surgeons. The goal of this study was to investigate one-star reviews regarding orthopaedic-trained hand and upper extremity surgeons in the 10 largest cities in the United States to determine the associated factors behind unsatisfied patients to improve patient care.

Materials and Methods Patient reviews and narratives of orthopaedic-trained hand surgeons were collected from the 10 largest cities in the United States using Vitals.com. One-star reviews (out of a five-star maximum) with comments were identified and classified as operative or nonoperative. These reviews were further subclassified based on the nature of the comment.

Results A total of 830 one-star reviews with 1,662 complaints were included in this study. Of these complaints, 557 (33.5%) were from patients who received operative care and 1,105 (66.5%) were from nonoperative care patients. Nonoperative patient one-star reviews had a significantly higher proportion of complaints related to bedside manners (37.6 to 19.6%, p < 0.001), not enough time spent with the provider (18.1 to 4.5%, p < 0.001), and wait time (13.3 to 3.2%, p < 0.001) as compared with operative patient one-star reviews. Operative patient one-star reviews had a higher proportion of complaints related to disagreement with the physician's decision or plan (15.6 to 10.2%, p = 0.002); uncontrolled pain (14.4 to 7.9%, p < 0.001); and medical staff or institution (17.2 to 12.9%, p = 0.018) as compared with nonoperative patients.

Discussion Most one-star reviews regarding orthopaedic-trained hand surgeons referenced nonclinical components—bedside manner was the most common complaint. It was determined that surgical patients were less likely to leave a one-star review; however, if they did, the most common complaint was in reference to a disagreement with the physician's decision or uncontrolled pain postoperatively.

Type of Study Outcomes 2c.

Keywords

physician review websites - negative reviews - patients - orthopaedic surgery

Physician review websites (PRWs) are becoming an increasingly popular way for prospective patients to find a physician.[1] Despite the literature showing no correlation between the numerical rating of physicians on PRWs and mortality rates, a recent study showed that 37% of patients who visit PRWs avoid physicians with bad ratings.[2] [3] Furthermore, 68% of patients utilize PRWs when selecting a physician, regardless of specialty.[4] Recently, a retrospective study revealed that physicians with lower ratings were more likely to score lower in areas regarding their communication and interpersonal skills.[5] [6] Despite the lack of evidence between negative reviews on PRWs and associated poor outcomes, recent research has shown that 78% of physicians across various specialties have reported that their ratings on PRWs have led to an increase in job-associated stress.[7]

There are multiple studies that have examined the factors that lead to one-star reviews within orthopaedic surgery, to the best of the author's knowledge, there has been no research focusing on the components of one-star reviews on PRWs for orthopaedists subspecializing in hand and upper extremity surgery.[8] [9] [10] [11] [12] Previous literature has characterized the factors comprising one-star reviews of orthopaedic-trained surgeons within the subspecialties of trauma, arthroplasty, sports, and spine.[8] [9] [11] [12] [13] [14] [15] Vitals.com is a popular PRW and has been used in previous research assessing the characteristics of one-star reviews of orthopaedic-trained surgeons trained in other subspecialties.[8] [9] [12] While there has been research done regarding PRWs and hand surgeons,[16] [17] [18] none has focused primarily on negative reviews and factors patients consider when writing a one-star review. The purpose of this study was to characterize the factors that contribute to one-star reviews of orthopaedic-trained hand surgeons on PRWs to identify which factors lead to very low patient satisfaction.

Materials and Methods

Study Set Up

The current study was a retrospective study analyzing one-star reviews and related patient complaints on Vitals.com for orthopaedic-trained hand surgeons in the top 10 largest cities in the United States based on population data from 2023. Vitals.com was used to look for one-star reviews and patient complaints on the one-star reviews. Search criteria on Vitals.com was “Hand Surgery—Orthopedic Surgery” search term, physicians in the top 10 largest cities in the United States, and physicians within a 10-mile radius of the city on Vitals.com. Top 10 largest cities in the United States by population were found on the following websites: https://www.macrotrends.net/cities/largest-cities-by-population, which has been used previously in research involving one-star reviews for orthopaedic-trained trauma surgeons.[12] Data collection on Vitals.com was started on April 21st, 2023 and ended on April 28th, 2023. All one-star reviews from the inception of Vitals.com until the date the physician's profile was accessed were evaluated to ensure they met inclusion criteria.

Inclusion and Exclusion Criteria

Data for this study included one-star reviews by physicians in the top 10 largest cities (by population) in the United States with at least one one-star review with at least one comment. Physicians were excluded if they did not have any one-star reviews or if they had a one-star review without any comments as this study aims to capture both qualitative and quantitative data on negative reports for physician care. Any complaints or reviews that were not relevant to hand surgery were excluded. Physician data were excluded due to incomplete data, error on the Vitals.com website, chief complaint unrelated to hand and upper extremity surgery (e.g., hip replacement), clearly inaccurate one-star rating (very positive comment listed as one-star review), non-English reviews, and clearly duplicate comments.

Data Extraction

Data collected included specialty (orthopaedics), verified subspeciality (hand orthopaedic surgery), number of total reviews, number of one-star reviews, number of operative one-star reviews, number of nonoperative one-star reviews, and number of complaints from operative or nonoperative patients in the following categories: not enough time with provider, wait time, bedside manner, surgical complications/outcomes, disagreements with decisions or plans, and medical staff/institution complaint, which included billing issues. These categories were determined prior to data collection and were derived from Richman et al.[12] This study chose to use the categorizes of “operative” and “nonoperative” to stratify reviews in an attempt to reduce interreviewer variance during the data collection. To ensure consistency during the data extraction phase, reviews that mentioned in-office procedures were characterized as operative reviews.

Study Definitions

For the purposes of this study, a review refers to the rating on Vitals.com given to any given physician based on the number of stars (one, two, three, four, or five stars). A comment refers to the written information (positive or negative) provided by the patient on the review of the physician. A complaint refers to the negative written information provided by the patient on the one-star review of the physician. Each one-star comment had at least one complaint; however, it was possible for a comment to have multiple complaints. For example, if a comment mentioned both the physician's bedside manner and the amount of time they spent with the physician, this would constitute one comment with two complaints. For the purposes of this study, an operative patient is a patient who underwent some type of surgical intervention on the corresponding review on the basis of the narrative described. A nonoperative patient is a patient who did not receive any surgical intervention on the corresponding review, and this was also determined by analyzing the complaint and/or comment associated with that review. A similar study design has been used previously when evaluating one-star comments of orthopaedic-trained trauma surgeons on PRWs.[12]

Statistical Analysis

Statistical analysis was completed using SPSS version 29.0 (Armonk, NY; IBM Corp) for the statistical software. Frequency counts and descriptive data were used for this study. Chi-square tests were used to compare nominal variables between two groups to determine significance with α set at 0.05.

Results

Initial Search Results

There were 703 total orthopaedic-trained hand surgeons identified across the 10 most populated cities in the United States. All included surgeons were verified using Vitals.com to be board-certified orthopaedic-trained hand surgeons. The 10 most populous cities in the United States are as follows: New York City, Los Angeles, Chicago, Houston, Dallas, Miami, Atlanta, Philadelphia, Washington, DC, and Phoenix, as determined by the previously cited reference. Of the 703 physicians generated, 476 physicians (67.7%) had at least one review with one written comment. Of the 476 physicians with at least one review with one written comment, 430 physicians (90.3%) had complete data and were included in the final analysis. See [Fig. 1] below for more information on the included data.

Fig. 1 Flow chart for inclusion of orthopaedic-trained hand surgeons, total reviews, one-star review, and one-star complaints for the current study. Note that there may be multiple complaints in a single review.

One-Star Reviews

Among the 430 included physicians, there were a total of 7,641 reviews (one, two, three, four, and five stars) with 830 one-star reviews (10.9%). Of the 830 one-star reviews, 182 reviews (21.9%) were classified as operative one-star reviews and 646 reviews (77.8%) were classified as nonoperative one-star reviews based on patient type. See [Fig. 1] below for more information on the included data.

One-Star Complaints

From the 830 one-star reviews, there were 1,662 complaints associated with those one-star reviews for orthopaedic-trained hand surgeons. Of those 1,662 complaints, 557 complaints (33.5%) were from patients who received operative treatment and 1,105 (66.5%) were from patients who received only nonoperative treatment. For operatively treated patients, there were 25 complaints related to “not enough time spent with physician,” 18 complaints regarding “wait time,” 109 related to “bedside manner,” 87 complaints mentioned “disagreement with physician decision or plan,” 80 mentioned “uncontrolled pain,” 96 referenced the “medical staff or institution,” and 142 reviews cited surgical complications or outcomes. Conversely, in the nonoperative group, there were 200 complaints that referred to “not enough time spent with the physician,” 147 mentioned “wait time,” 416 reviews referenced “bedside manner,” 113 regarding a “disagreement with physician decision or plan,” 87 mentioned “uncontrolled pain,” and 142 cited the “medical staff or institution” in their one-star review. Infection was only listed in a one-star complaint seven times (one time for seven physicians). Bedside manner was the most common complaint for both operative and nonoperative patients with 525 total one-star complaints (31.6%). Surgical complications and outcomes were only responsible for 142 one-star complaints (25.5%) out of a total of 557 complaints from operative patients. The breakdown for total complaints for each category was 13.5% one-star complaints for “not enough time spent with the physician,” 9.9% one-star complaints for “wait time,” 31.6% one-star complaints for “bedside manner,” 12.0% one-star complaints for “disagreement with physician decision or plan,” 10.0% one-star complaints for “uncontrolled pain,” and 14.3% for “medical staff or institution.” See [Table 1] below for information on the type of one-star complaints, the number of operative and nonoperative complaints, and the total complaints for each type.

Table 1 Information on the one-star complaints included in the current study. One-star complaints are stratified by type, operative or nonoperative patients, and total percentage of one-star complaints. There were statistically significant differences in the number of complaints regarding wait time, time spent with physician, bedside manner, disagreements with the physician's plan, and uncontrolled pain when comparing operative patients to nonoperative

Type of one-star complaint–	Operative complaints	Nonoperative complaints	Total complaints	Percent of total complaints (%)
Not enough time with physician	25	200	225	13.5
Wait time	18	147	165	9.9
Bedside manner	109	416	525	31.6
Surgical complications/outcomes	142	–	142	8.5
Disagree with physician decision or plan	87	113	200	12.0
Uncontrolled pain	80	87	167	10.0
Medical staff/institution related complaint	96	142	238	14.3
Total number of complaints	557	1,105	1,662	100.0

Operative versus Nonoperative One-Star Complaints

Nonoperative patient complaints of “not enough time with physician” had a significantly higher proportion among total one-star complaints as compared with operative patient complaints (18.1 to 4.5%, p < 0.001). Nonoperative patient complaints of “wait time” had a significantly higher proportion among total one-star complaints as compared with operative patient complaints (13.3 to 3.2%, p < 0.001). Nonoperative patient complaints of “bedside manner” had a significantly higher proportion among total one-star complaints as compared with operative patient complaints (37.6 to 19.6%, p < 0.001). Operative patient complaints of “disagreement with physician decision or plan” had a significantly higher proportion among total one-star complaints as compared with nonoperative patient complaints (15.6 to 10.2%, p = 0.002). Operative patient complaints of “uncontrolled pain” had a significantly higher proportion among total one-star complaints as compared with nonoperative patient complaints (14.4 to 7.9%, p < 0.001). Operative patient complaints of “medical staff or institution” had a significantly higher proportion among total one-star complaints as compared with nonoperative patient complaints (17.2 to 12.9%, p = 0.018). See [Table 2] below for more information on the included data. Overall, nonoperative patient complaints had a significantly higher proportion of complaints related to not enough time spent with the physician, wait time, and bedside manner. Operative patient complaints had a significantly higher proportion of complaints related to disagreement with the physician's decision or plan, uncontrolled pain, and medical staff or institution as compared with nonoperative patient complaints.

Table 2 Information on the proportions of complaints of surgical and nonsurgical groups as compared with the total number of complaints

Type of one-star complaint	Operative group complaint proportions (%)	Nonoperative group complaint proportions (%)	Between group p-Value
Not enough time with physician	4.5	18.1	p < 0.001
Wait time	3.2	13.3	p < 0.001
Bedside manner and patient experience	19.6	37.6	p < 0.001
Disagree with physician decision or plan	15.6	10.2	p = 0.002
Uncontrolled pain	14.4	7.9	p < 0.001
Medical staff/institutional related complaint	17.2	12.9	p = 0.018

Discussion

More patients are relying on PRWs when choosing a physician than ever before, which represents a continuing shift toward consumer focus in the market.[4] [19] More than one-third of patients will avoid a physician with negative reviews. Thus, it becomes imperative for physicians to understand what components contribute to these negative reviews.[2] [3] Our results indicate that most one-star reviews of orthopaedic-trained hand surgeons written on Vitals.com tend to occur primarily with patients who did not undergo surgical intervention. One-star complaints, among patients in this cohort, were significantly more likely to mention time spent with the physician, wait time, and bedside manner when compared with those who underwent any type of procedure. Alternatively, one-star reviews written about orthopaedic-trained hand surgeons in patients who did undergo a surgical intervention were more likely to mention disagreement with the physician's plan and uncontrolled pain when compared with nonoperative one-star reviews. This concurs with previous literature describing a correlation between negative reviews and patients who were not offered surgery.[8] [13] [14]

PRWs are growing in popularity, but studies have shown that while patients are avoiding physicians with negative reviews, these reviews do not represent the technical shortcomings of the physician.[2] [19] Our study found that 67% of complaints were related to nonoperative visits, which has been supported by previous literature.[13] [14] A study by Trehan et al[15] showed that there was no significant difference in the rate of postoperative complications between orthopaedic-trained arthroplasty surgeons with high and low ratings on multiple PRWs. This further suggests that negative reviews do not adequately depict a physician's technical ability, but rather focus on their interpersonal skills, bedside manner, and office staff.[2] [13] [15] This information may seem paradoxical to the practicing surgeon—despite low rates of infection and postoperative complications, a surgeon may have a preponderance of negative online ratings.[13]

It is important to note that the majority of comments on PRWs are positive. This finding has been seen in other previous studies,[11] as well as in our research. Pollocket al found that more than 90% of reviews on PRWs were positive.[13] Although only a small percentage of reviews on PRWs are comprised of negative reviews, physicians can continue to grow their practice by learning from their weaknesses pointed out on PRWs.[20] While orthopaedic surgeons may be frustrated that the primary complaint on PRWs does not center on their technical ability, negative reviews can be avoided with an emphasis on bedside manner, time with the patient, and wait time reduction.[8] [9] [13]

This study has multiple limitations that may impact the interpretation of our results. First, there are several different PRWs available for patients to leave reviews on, and this study only included reviews posted on Vitals.com. While Vitals.com is a popular PRW, all reviews are posted anonymously, which may lead to reviews being published under inappropriate circumstances or pretenses. This website has been used in other orthopaedic studies examining negative one-star reviews, but may still fail to capture a full understanding of the associated factors behind negative reviews.[8] [9] [12] Additionally, not all physician profiles had been set up appropriately and some could not be accessed or had limited data. It is unclear whether this is secondary to the underlying coding issue with Vitals.com, but clicking on some physician profiles resulted in an error message. Another limitation has to do with the possibility of misclassification bias during the data collection of this study. For example, if a patient received operative care, but did not mention their surgery or related care in their comment, the review and comment would have been classified as belonging to a nonoperative patient. Due to this, the number of complaints that were ultimately classified as nonoperative may have been falsely inflated. Additionally, the number of nonoperative complaints may have been greater solely for the reason that once a patient had a negative experience with a physician in a preoperative visit, they were less likely to pursue operative care from the same physician. Finally, our study focused on the 10 largest cities in the United States, which can skew results due to selection bias, as perceptions could be different in a rural setting versus urban environments. Overall, more research is needed to determine the frequency and impact of physician one-star reviews for orthopaedic-trained hand surgeons on patient choice of physician and patient care and outcomes.

In summary, this study determined that one-star negative reviews of orthopaedic-trained hand surgeons on Vitals.com were more likely to be nonoperative in nature, with bedside manners being the most common complaint. Furthermore, surgical patients were more likely to write a negative review in reference to complications or unsatisfactory outcomes following their procedure. Nonoperative patient complaints for orthopaedic-trained hand surgeons had a significantly higher proportion of complaints related to not enough time spent with the physician, wait time, and bedside manner as compared with operative patient complaints. Meanwhile, operative patient complaints for orthopaedic-trained hand surgeons had a significantly higher proportion of complaints related to disagreement with the physician's decision or plan, uncontrolled pain, and medical staff or institution as compared with nonoperative patient complaints. More research is needed to determine the impact of one-star reviews for orthopaedic-trained hand surgeons. Additionally, further research should be performed to determine if anonymity on PRWs impacts the content of patient reviews.

Conflict of Interest

None declared.

• References

• 1 Gao GG, McCullough JS, Agarwal R, Jha AK. A changing landscape of physician quality reporting: analysis of patients' online ratings of their physicians over a 5-year period. J Med Internet Res 2012; 14 (01) e38
• 2 Okike K, Peter-Bibb TK, Xie KC, Okike ON. Association between physician online rating and quality of care. J Med Internet Res 2016; 18 (12) e324
• 3 Rastegar-Mojarad M, Ye Z, Wall D, Murali N, Lin S. Collecting and analyzing patient experiences of health care from social media. JMIR Res Protoc 2015; 4 (03) e78
• 4 Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public awareness, perception, and use of online physician rating sites. JAMA 2014; 311 (07) 734-735
• 5 Widmer RJ, Maurer MJ, Nayar VR. et al. Online physician reviews do not reflect patient satisfaction survey responses. Mayo Clin Proc 2018; 93 (04) 453-457
• 6 López A, Detz A, Ratanawongsa N, Sarkar U. What patients say about their doctors online: a qualitative content analysis. J Gen Intern Med 2012; 27 (06) 685-692
• 7 Holliday AM, Kachalia A, Meyer GS, Sequist TD. Physician and patient views on public physician rating websites: a cross-sectional study. J Gen Intern Med 2017; 32 (06) 626-631
• 8 Zhang J, Omar A, Mesfin A. Online ratings of spine surgeons: analysis of 208 surgeons. Spine 2018; 43 (12) E722-E726
• 9 Bakhsh W, Mesfin A. Online ratings of orthopedic surgeons: analysis of 2185 reviews. Am J Orthop 2014; 43 (08) 359-363
• 10 Langerhuizen DWG, Brown LE, Doornberg JN, Ring D, Kerkhoffs GMMJ, Janssen SJ. Analysis of online reviews of orthopaedic surgeons and orthopaedic practices using natural language processing. J Am Acad Orthop Surg 2021; 29 (08) 337-344
• 11 Bernstein DN, Mesfin A. Physician-review websites in orthopaedic surgery. JBJS Rev 2020; 8 (03) e0158
• 12 Richman EH, Kuttner NP, Foster LO. et al. Characterizing single-star negative online reviews of orthopaedic trauma association members. J Am Acad Orthop Surg 2023; 31 (08) 397-404
• 13 Pollock JR, Arthur JR, Smith JF. et al. The majority of complaints about orthopedic sports surgeons on yelp are nonclinical. Arthrosc Sports Med Rehabil 2021; 3 (05) e1465-e1472
• 14 Arthur JR, Etzioni D, Schwartz AJ. Characterizing extremely negative reviews of total joint arthroplasty practices and surgeons on yelp.com. Arthroplast Today 2019; 5 (02) 216-220
• 15 Trehan SK, Nguyen JT, Marx R. et al. Online patient ratings are not correlated with total knee replacement surgeon-specific outcomes. HSS J 2018; 14 (02) 177-180
• 16 Kirkpatrick W, Abboudi J, Kim N. et al. An assessment of online reviews of hand surgeons. Arch Bone Jt Surg 2017; 5 (03) 139-144
• 17 Garofolo G, Akinleye SD, Golan EJ, Choueka J. Utilization and impact of social media in hand surgeon practices. Hand (N Y) 2020; 15 (01) 75-80
• 18 Trehan SK, DeFrancesco CJ, Nguyen JT, Charalel RA, Daluiski A. Online patient ratings of hand surgeons. J Hand Surg Am 2016; 41 (01) 98-103
• 19 Zeckhauser R, Sommers B. Consumerism in health care: challenges and opportunities. Virtual Mentor 2013; 15 (11) 988-992
• 20 Lee V. Transparency and trust—online patient reviews of physicians. N Engl J Med 2017; 376 (03) 197-199

Publication History

Received: 03 November 2023

Accepted: 09 January 2024

Article published online:
31 January 2024