Keywords
classification - radius fractures - reproducibility of results
Introduction
Distal radius fractures are extremely prevalent, accounting for 16% of body and 74% of forearm fractures. They present a bimodal distribution, affecting adolescents/young adults (high-energy trauma) and the elderly (low-energy trauma). The most common mechanism of injury is a fall to the ground with the wrist in hyperextension.[1]
[2]
[3]
[4]
Despite the high prevalence, there has never been much consensus in the literature regarding the best classification for distal radius fractures. The first concepts began before the advent of radiography, with the description from Colles for fractures with dorsal displacement in 1814. In 1951, Gartland and Werley proposed the first classification for fractures of the distal radius, followed by Frykman in 1967, the AO group's from Muller in 1986, Fernandez's in 1991, the Universal one from Cooney in 1993, and, most recently, the IDEAL classification from the Division of Hand Surgery from the Universidade Federal de São Paulo (UNIFESP), in 2013.[1]
[3]
[5]
The IDEAL classification relies on five parameters (two epidemiological and three radiographic), namely: age (younger or older than 60), energy of the trauma resulting in the fracture, fragment displacement (presence or absence), joint incongruity (incongruence or separation > 2 millimeters), and associated injuries (presence or absence). Each parameter scores zero or one point and their sum gives the fracture classification, with types I (0–1 points), II (2–3), or III (4–5). Each type suggests a treatment and prognosis of the injury.[1]
Previous studies show low to moderate levels of intra- and interobserver agreement for the oldest classifications available in the literature, such as the Frykman, Fernandez, and AO. The Universal and IDEAL classifications presented better results compared to the previous ones.[1]
[2] Those with more subtypes and divisions presented lower interobserver agreement, which may raise issues concerning intraobserver agreement due to the longer time needed to get used to the instrument.[1]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
Because there are several classifications for fractures of the distal end of the radius, it is essential to define the best one in studies such as this one, assessing their reproducibility and reliability. This study aims to analyze the reproducibility and intra- and interobserver agreement of the IDEAL classification for distal radius fractures, and to determine the influence of the observers' training level.
Materials and Methods
This qualitative, analytical, retrospective, and direct documentation study evaluated radiographs of patients with distal radius fractures by observers with different experience levels in traumatology. The research occurred at a University Hospital, which provided the radiographs and allowed interviews for data collection from November to December 2022.
The Giraudeau and Mary method[13] for sample determination, per the expected level of agreement, the number of evaluators, and the confidence interval (CI) estimated several minimum samples. [Table 1] shows that an expected Kappa of 0.70 and a 90% confidence interval requires a minimum of 41 samples.[14] We obtained 50 pairs of radiographs (anteroposterior and lateral views) showing distal radius fractures from the electronic medical records of patients treated at this university hospital from 2019 to 2022.
Table 1
|
Number of participants required for a 95% CI at three confidence levels
|
|
Number of observers
|
Expected ICC
|
± 0.05
|
± 0.10
|
± 0.15
|
|
2
|
0.9
|
56
|
14
|
4
|
|
0.8
|
200
|
50
|
13
|
|
0.7
|
400
|
100
|
25
|
|
0.6
|
630
|
158
|
40
|
|
0.5
|
865
|
217
|
55
|
|
4
|
0.9
|
36
|
9
|
3
|
|
0.8
|
119
|
30
|
8
|
|
0.7
|
222
|
56
|
14
|
|
0.6
|
322
|
81
|
21
|
|
0.5
|
401
|
101
|
26
|
|
6
|
0.9
|
31
|
8
|
2
|
|
0.8
|
103
|
26
|
7
|
|
0.7
|
187
|
47
|
12
|
|
0.6
|
263
|
66
|
17
|
|
0.5
|
314
|
79
|
20
|
|
10+
|
0.9
|
29
|
8
|
2
|
|
0.8
|
92
|
23
|
6
|
|
0.7
|
164
|
41
|
11
|
|
0.6
|
224
|
56
|
14
|
|
0.5
|
259
|
65
|
17
|
The inclusion criteria were patients whose medical records had the International Classification of Diseases (ICD) for distal radius fractures (S52.5) and who received treatment at the University Hospital. The exclusion criteria were patients who had undergone any type of treatment, surgical or otherwise, for a distal radius fracture before the radiograph, and with no imaging of distal radius fracture available in their medical records.
There were three orthopedic specialists in hand surgery, four general orthopedic surgeons from the orthopedic service of the University Hospital, and three orthopedic residents, one from each year, from the University Hospital, who participated in the study as observers. They evaluated the radiographs to classify them according to the IDEAL method. The evaluation occurred three times, with an interval of 15.3 ± 4.34 days.
We tabulated the results from the observers' assessments in Microsoft Excel 2019 (Microsoft Corp., Redmond, WA, USA) and performed the Cohen and Fleiss Kappa tests for intra- and interobserver assessment, respectively, using the Statistical Package Social Sciences (SPSS, IBM Corp., Armonk, NY, USA), version 26.0, for statistical analysis.[15] Interobserver agreement assessment tables showed the Kappa index measurement for each observer class (residents, general orthopedists, and hand surgeons) for the three separate assessments. Intraobserver agreement assessment tables compared each assessment with the other two and determined the presence of upper and lower limit values for a 90% confidence interval (CI).
Kappa values with p < 0.1 were considered significant. The interpretation of results used the method proposed by Landis and Koch, in which values lower than and up to zero indicate poor agreement, with little agreement from 0 to 0.2, reasonable from 0.2 to 0.4, moderate from 0.4 to 0.6, substantial from 0.6 to 0.8, and excellent or virtually perfect agreement from 0.8 to 1.[16]
The Research Ethics Committee approved this research under the CAAE number 63490322.8.0000.8050 and opinion number 5,726,415.
Results
The Cohen Kappa indexes for intraobserver agreement ([Table 2]) were 0.259 (poor agreement) in one instance (CM1 T1 x T2), with statistical significance (p = 0.021), and 0.140 or lower (poor agreement) in all others, with no statistical significance in any case (p > 0.1).
Table 2
|
T1 x T2
|
T2 x T3
|
T1 x T3
|
|
κ
|
p
|
κ
|
p
|
κ
|
p
|
|
R1
|
0.091
|
0.384
|
0.028
|
0.786
|
0.049
|
0.660
|
|
R2
|
0.140
|
0.174
|
-0.078
|
0.419
|
-0.026
|
0.805
|
|
R3
|
-0.151
|
0.139
|
-0.043
|
0.673
|
0.015
|
0.885
|
|
GO1
|
-0.022
|
0.838
|
-0.053
|
0.624
|
-0.017
|
0.872
|
|
GO2
|
0.009
|
0.940
|
-0.177
|
0.159
|
0.006
|
0.963
|
|
GO3
|
-0.121
|
0.255
|
0.054
|
0.646
|
-0.069
|
0.570
|
|
GO4
|
0.108
|
0.352
|
-0.032
|
0.779
|
-0.029
|
0.797
|
|
HS1
|
0.259
|
0.021
|
-0.078
|
0.463
|
-0.009
|
0.933
|
|
HS2
|
0.138
|
0.178
|
0.028
|
0.791
|
0.042
|
0.683
|
|
HS3
|
0.028
|
0.791
|
-0.053
|
0.646
|
0.006
|
0.956
|
[Table 3] shows the Fleiss Kappa indexes for interobserver agreement in the resident group ranged from 0.277 to 0.383 in the three assessments, with statistical significance. The CIs do not contain the parameter value and p ≤ 0.008.
Table 3
|
κ index
|
p
|
90% CI
lower limit
|
90% CI
upper limit
|
|
κ T1
|
0.305
|
< 0.001
|
0.206
|
0.404
|
|
Type 1
|
0.349
|
< 0.001
|
0.214
|
0.483
|
|
Type 2
|
0.215
|
0.008
|
0.081
|
0.350
|
|
Type 3
|
0.386
|
< 0.001
|
0.252
|
0.521
|
|
κ T2
|
0.383
|
< 0.001
|
0.282
|
0.483
|
|
Type 1
|
0.435
|
< 0.001
|
0.301
|
0.569
|
|
Type 2
|
0.302
|
< 0.001
|
0.167
|
0.436
|
|
Type 3
|
0.452
|
< 0.001
|
0.317
|
0.586
|
|
κ T3
|
0.277
|
< 0.001
|
0.175
|
0.378
|
|
Type 1
|
0.308
|
< 0.001
|
0.173
|
0.442
|
|
Type 2
|
0.250
|
0.002
|
0.116
|
0.384
|
|
Type 3
|
0.292
|
< 0.001
|
0.157
|
0.426
|
[Table 4] shows the Fleiss Kappa indexes for the general orthopedists ranged from 0.114 to 0.225, with statistical significance. The CIs do not contain the parameter value and p ≤ 0.008.
Table 4
|
κ index
|
p
|
90% CI
lower limit
|
90% CI
upper limit
|
|
κ T1
|
0.186
|
< 0.001
|
0.115
|
0.258
|
|
Type 1
|
0.472
|
< 0.001
|
0.377
|
0.567
|
|
Type 2
|
0.112
|
0.053
|
0.017
|
0.207
|
|
Type 3
|
0.065
|
0.261
|
-0.030
|
0.160
|
|
κ T2
|
0.114
|
0.008
|
0.043
|
0.184
|
|
Type 1
|
0.330
|
< 0.001
|
0.235
|
0.425
|
|
Type 2
|
0.011
|
0.849
|
-0.084
|
0.106
|
|
Type 3
|
0.090
|
0.119
|
-0.005
|
0.185
|
|
κ T3
|
0.225
|
< 0.001
|
0.154
|
0.295
|
|
Type 1
|
0.359
|
< 0.001
|
0.335
|
0.454
|
|
Type 2
|
0.148
|
< 0.001
|
0.361
|
0.243
|
|
Type 3
|
0.223
|
< 0.001
|
0.485
|
0.318
|
[Table 5] shows the Fleiss Kappa indexes for the hand surgeons ranged from 0.449 to 0.553, all with statistical significance. Results for each classification type were higher for type III in all evaluations when compared with types I and II, but all assessments had excellent significance levels. The CIs do not contain the parameter value and p < 0.001).
Table 5
|
κ index
|
p
|
90% CI
lower limit
|
90% CI
upper limit
|
|
Fleiss' κ T1
|
0.533
|
< 0.001
|
0.430
|
0.637
|
|
Type 1
|
0.469
|
< 0.001
|
0.335
|
0.604
|
|
Type 2
|
0.495
|
< 0.001
|
0.361
|
0.629
|
|
Type 3
|
0.620
|
< 0.001
|
0.485
|
0.754
|
|
Fleiss' κ T2
|
0.449
|
< 0.001
|
0.347
|
0.550
|
|
Type 1
|
0.365
|
< 0.001
|
0.231
|
0.500
|
|
Type 2
|
0.430
|
< 0.001
|
0.296
|
0.564
|
|
Type 3
|
0.525
|
< 0.001
|
0.391
|
0.659
|
|
Fleiss' κ T3
|
0.531
|
< 0.001
|
0.430
|
0.631
|
|
Type 1
|
0.627
|
< 0.001
|
0.493
|
0.761
|
|
Type 2
|
0.470
|
< 0.001
|
0.336
|
0.604
|
|
Type 3
|
0.542
|
< 0.001
|
0.407
|
0.676
|
Discussion
Distal radius fractures are prevalent, requiring a complete understanding of the potential fracture patterns' complexity and broadening its scope to other factors impacting their prognosis.[10]
[12] The IDEAL classification meets these requirements, as it includes age and trauma energy in its parameters.
The limitations of this study included the low number of observers for each category and the absence of hand surgery residents.
We observed a tendency towards little or no agreement, or even disagreement, in the intraobserver evaluation, with most cases not presenting considerable statistical significance. This finding is inconsistent with the literature, in which most studies detected moderate to high agreements.[1]
[3]
[4]
[5]
The degree of interobserver agreement measured by the Fleiss Kappa index had more statistical solidity than the Cohen one in analyzing intraobserver agreement. Observers had difficulty agreeing on the intermediate type of classification, and it was easier on the extremes. Given the low or even absent agreement between orthopedists and residents, we can infer that the difference in training level apparently did not imply a higher level of agreement for the group with more experience in orthopedics, as we observed an even greater agreement among residents. Andersen et al.[17] and Belloti et al.[18] reported no influence from the observers' experience level, which is consistent with our findings since our less experienced observers had higher agreement levels and better statistical significance when compared with more experienced ones.
In contrast, hand surgeons obtained the best levels of interobserver agreement among the three observer groups, with moderate levels in all three assessments. Results from hand surgeons allow us to infer that the additional specific training enabled them to obtain more concordant results than the other groups. In this scenario, differing from Illarramendi et al.,[8] Andersen et al.,[17] Jayakumar et al.,[3] and Belloti et al.,[18] the hand surgeons' additional experience was the main factor for the best interobserver agreement levels in this study.
The general objective of classifications is to provide a tool to accurately classify a fracture into a type to guide its treatment and define a prognosis. They also allow effective communication between professionals from different backgrounds.[4] Although we did not detect high levels (> 0.8) of intra- and interobserver agreement in this or other studies in the literature,[1]
[3]
[4]
[5]
[6]
[7]
[8] it seems that fulfilling this objective is possible.
Conclusion
This study found interobserver agreement levels ranging from poor to moderate, demonstrating that the training level only influenced the results from hand surgeons, with no significant difference between residents and orthopedists. In conclusion, the classification proved to be, to a certain extent, irreproducible and inconsistent.
Nevertheless, it is essential to perform further studies of this type, either with this or other classifications, to provide increasingly solid scientific evidence and allow the choice of the best classification.
