Running Performance and Position is Not Related to Decision-Making Accuracy in Referees

Abstract

We aimed to assess if running performance is related to decision-making accuracy in referees. We also investigated the relationships between movement, position, decision-making and time of infringement. The study included 347 free kicks and penalties awarded during all 15 home field matches for one team in the Norwegian top league. Movement and position were measured by a radio-based positioning system, and two external experts analysed the video clips. The referees made the correct decision in 98% of the assessed foul situations. There were no associations between the correctness and running speed at the time of the infringements or total distance accumulated during the 30 s prior to the infringement (p>0.08). Decision-making accuracy was not associated with the referees’ position in relation to the infringements (p=0.82) or the area where the infringements occurred (p=0.28). Most foul play incidents were sanctioned in the central area of the field (84%) where the distance to the infringements was shorter (13.4 m, 95% CI 12.8–14.0) than in the lateral areas (21.9 m, 95% CI 20.4–23.4). Norwegian top referees were practically always right when they interfered, possess the necessary fitness to avoid fatigue that would influence decision-making accuracy, and maintain adequate positional ability throughout the matches.

Key words

match analysis - soccer - football

Introduction

The main duty of soccer referees is to enforce the laws of the game and sanction any infringements of these laws [1]. Decision-making is therefore an essential component of a soccer referee’s match performance, because each decision may have a direct or indirect impact on the match result. A referee makes about 140 observable decisions during a soccer match [2] and calls 25–41 foul play infringements [2] [3] [4]. Expert panels assessing the infringements retrospectively by video analysis find that the proportion of incorrect decisions ranges from 14–36% [4] [5] [6]. The referees’ level of expertise influences offside decision-making skills and recall accuracy [4] [5] [7] [8]. In addition, stake size, attendance crowd, and the crowd’s distance to the field may also affect the referees’ decision-making processes [9].

In addition to decision-making skills, a high level of physical fitness is required for referees to be able to keep up with the play and get an unobstructed view of potential infringements. During top-level national and international matches, the field referees usually cover 10–12 km, with high-intensity running (HIR) accounting for 600–1 200 m [10] [11] [12]. The referees must position themselves based on where infringements might occur while maintaining an overview of the players and the play, but without interfering with the players or ball. Thus, the field referees cannot be too close or too far away from the infringements. The distance from the referee to the infringement may be related to decision-making accuracy [13] and the lowest proportion of decision-making errors is achieved when the fouls are called from between 11 and 15 m [4]. If the referee becomes fatigued during the match, the distance to infringements may increase because the referee cannot keep up with the play [11]. Also, if a referee realises he is positioned too far from potential infringements, he is likely to move faster for a better view rather than simply adjust his position if he is closer to the action. To our knowledge, however, the association between position and running speed in relation to match period has not been assessed in soccer referees.

The relationship between decision-making and physical exertion has been investigated in football players [14] as well as assistant referees [15] [16]. Catteeuw et al. [16] found no evidence that fatigue affects decision-making in assistant referees, because the offside-decision accuracy remained consistent throughout the match [16]. In contrast, Oudejans et al. [15] found that assistant referees made more offside-decision errors when running compared to when they were standing still, walking, or jogging. More recently, a study investigating Australian football field officials concluded that 300 m running time did not affect performance in a video-based decision-making test performed immediately after running [17]. However, higher relative running speed immediately prior to a decision increased the amount of decision-making errors made by Australian football field officials during match play [18].

The majority of studies investigating the relationship between fatigue and decision-making in sport officials are performed on assistant football referees and on Australian football field officials. Thus, how factors related to physical performance influence decision-making in field referees is unknown. Therefore, the primary aim of the present study is to assess if running performance prior to a called foul or positioning affects decision-making accuracy in soccer field referees. Secondly, we aim to assess if the referees’ movements, positioning, and time of the infringements are associated.

Materials and Methods

Participants

Eleven male field referees weighing 74.9±6.5 kg, 182.5±7.3 cm tall, and aged 35.4±7.0 years with 9.8±5.2 years of experience as top level referees in Norway participated in the study. Data were collected during all 15 top league matches (four referees officiated two matches) on one arena during the 2016 season. The Norwegian Data Protection Official for Research (NSD) approved the study and it meets the ethical standards of the International Journal of Sports Medicine [19]. All referees were informed orally and in writing of the procedures and provided their written consent before they were included in the study.

Video analysis

Videos of the matches were produced for commercial broadcasting by Norwegian TV2 (Bergen, Norway) and accessed via the online publishing tool, Mediabanken (www.mediabank.me). A qualified referee viewed video footage of all 15 matches and created separate MP4 video files of 10–20 s’ duration of all foul play infringements (including replay when available) where the referee awarded a free kick, penalty, yellow or red card.

An expert panel consisting of two active referees (one national and former FIFA international, and one regional) independently assessed each infringement. Fifty-five infringements were reported where at least one of the expert referees classified the decision as “not entirely correct” or “difficult to assess”. Subsequently, the two experts met to reach a conclusion regarding the 55 infringements. The infringements were classified according to the following pre-defined categories: 1) correct decision; 2) correctly awarded penalty/free kick, but a card should also been awarded; 3) correctly awarded penalty/free kick, but the card should not have been awarded; 4) incorrect decision, no foul play was committed; 5) incorrect decision, the other team committed foul play; 6) unknown, due to poor video quality; 7) unknown, due to the complexity in the situation. The expert panel could replay each infringement as many times as necessary. When decision-making accuracy was analysed, categories 1 through 3 were combined into “right decisions” and compared to the combined “incorrect decision” categories 4 and 5. To increase the number of cases, a secondary analysis compared category 1 to the “somewhat incorrect” categories 2 through 5.

Movement analysis

The movements were captured by a fully automatic tracking system based on radio waves (ZXY Technology Ecosystem, Chyronhego, Trondheim, Norway) previously described in detail [20]. In brief, the referees’ positions were monitored by transponders, placed in lightweight belts worn around the hip, which sent radio waves (20 Hz) to four calibrated, stationary sensors, each mounted in one of the light masts surrounding the football field (ZXY RadioEyeTM Positioning Sensor, Chyronhego). By integrating information from the four receivers in an advanced vector-based process, the referees’ positions and movements were determined. The data were compressed and filtrated by a Linux server using Ubuntu 14.04 and stored in a structured query language (SQL) database before being transferred to a software program (Microsoft Excel, 2013). The test-retest reliability of the system had been assessed in a previous study reporting an interclass correlation (ICC) coefficient of 1.0, 0.999, and 0.999 (P=0.001) for x- and y- and total distance [21]. All matches were monitored live and controlled by two researchers using the ZXY Sport Tracking software program.

Distance covered by the referees was classified into total distance (TD), running distance (RD), and HIR distance in accordance with previous, similar studies [21]. Running and HIR were defined as movement at a speed>14.4 and≤19.8 km·h⁻¹ and>19.8 km· h⁻¹, respectively. The TD, RD, and HIR distances accumulated during the 10, 30, and 60 s prior to the infringements were defined as TD10, TD30, and TD60; RD10, RD30, and RD60; and HIR10, HIR30, and HIR60.

Combined analysis

The home team players wore identical transponders as the referees, enabling measurement of the distance between the referee and the infringements using a semiautomatic feature in ZXY Sport Tracking. At the time of the infringement, the home team player involved in the infringement and the referee were identified and the software computed the distance between them. This distance was defined as the referee’s distance to the infringement. This distance was classified into different categories (0–5, 6–10, 11–15, 16–20, 21–25, and>25 m) similar to previous studies [4] [13]. Simultaneously with the distances, the running speed of the referee was recorded with ZXY Sport Tracking. The infringements were classified according to the area where they were awarded and in accordance with the recommendations of the FIFA Referee Department and a previous study [4]. Lateral areas were defined as the areas between the sideline and an imaginary line from the intersection between the sideline and the middle line and the middle point of the goalmouth, representing the influence zone of each assistant referee. The remaining area of the field was defined as the central area.

Statistical analysis

Normal distribution was visually inspected, and normally distributed data (referee’s distance to infringement, running speed of the referee at the time of the infringement, TD30, TD60) are presented as mean plus standard deviation (SD) or 95% confidence interval (95% CI) when appropriate. Not normally distributed data (TD10, RD10, RD30, RD60, HIR10, HIR30, HIR60) are presented as median with interquartile range (IQR). The associations between binary and continuous variables were analysed using binary logistic regression with decision-making accuracy or field area as the dependent variables. The independent variables were the running speed of the referee at the time of the infringement, TD30, distance from the field referee to the foul play infringement, and match time of the infringement. The correlations between normally distributed variables were assessed by the Pearson product-moment correlation (r), and Spearman’s rho test was used if one or more variables were not normally distributed. The difference between infringements in the central and lateral zone was analysed by a one-sample binomial test. To investigate the effect of match period on the referee’s distance to the infringement and running speed of the referee at the time of the infringement, we used a one-way analysis of variance (ANOVA). The distribution of infringements over the match periods was assessed by one-sample chi-square test.

Results

The present study consists of 347 called foul play infringements (15 matches) with a mean of 23.1 (±3.6) infringements per match. The referees made the correct decision in 98% of all assessed fouls ([Table 1]).

Table 1 Video-based classification of the correctness of the decision was penalties or free kicks awarded.

	n	%
Correct decision	328	94.5
Correct decision , but a card should have also been awarded	6	1.7
Correct decision , but the card should not have been awarded	3	0.9
Incorrect decision, no foul play was committed	5	1.4
Incorrect decision, the other team committed foul play	1	0.3
Unknown, due to poor video	3	0.9
Unknown, due to the complexity of the situation	1	0.3

Associations between movement/position and decision-making

There were no associations between the correctness of called fouls and running speed at the time of the infringement (β=0.19, p=0.09); total distance in the last 30 s (β=0.01, p=0.55); distance to the infringement (β=0.02, p=0.82); time of infringement (β<0.00, p=0.92); or field zone of infringement (chi square=1.17 p=0.28). None of the incorrect decisions were made for infringements in lateral zones, and 2.1% of the calls made on infringements in the central zone were incorrect. The same analyses were performed including situations with incorrectly awarded cards in the incorrect decision category, and the results were similar (p>0.08). [Figure 1a–d] is stratified by decision correctness and displays running speed at the time of the infringement, total distance in the last 30 s, distance from the field referee to the foul play infringement, and time of infringement.

Fig. 1 The field referee's; speed of movement when he calls fouls a, distance from the infringements when he calls fouls b, total distance accumulated during the 30 seconds prior to the called foul c, the match time of the infringements d stratified by correctness of the decision for all infringements in one area over the entire season. Data are presented as mean with 95% confidence intervals.

Associations between movement, position, and time of infringement

The running speed of the referee and the referees’ distance to the infringement were weakly correlated (r=0.31, p<0.001). The correlation between distance covered prior to the infringement and running speed at the time of the infringement or the referee’s distance from the infringement is presented in [Table 2].

Table 2 Correlation (Spearman rho) between referee running speed at the time of the infringement or the referee’s distance from the infringement, and distance covered prior to the infringement assessed over a period of 10, 30, and 60 s.

		TD10s	TD30s	TD60s	RD10s	RD30s	RD60s	HIR10s	HIR30s	HIR60s
Speed at the time of infringement	Rho	0.159	0.135	0.096	0.097	0.092	0.044	−0.044	−0.011	−0.011
	p	0.004	0.017	0.088	0.086	0.103	0.432	0.432	0.849	0.847
Distance from infringement	Rho	0.472	0.308	0.286	0.436	0.268	0.247	0.211	0.114	0.102
	p	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001	0.039	0.067

TD, total distance; RD, running distance (accumulated distance at speeds>14.4 and ≤19.8 km · h⁻¹); HIR, high-intensity running (accumulated distance at speeds>19.8 km · h⁻¹).

Eighty-four percent of the infringements (292) were committed in the central area and 16% (55) in the lateral areas (p<0.001). The distances between the referee and the infringements were 13.4 m (95% CI 12.8–14.0) and 21.9 m (95% CI 20.4–23.4) in the central and lateral areas, respectively (β=0.77, p<0.001).

The number of infringements was unevenly distributed (p<0.001) in the different distance categories, with most infringements being called from a distance between 10 and 14.9 m. There were no differences in number of infringements (p=0.94), distance from infringement (p=0.16), referee running speed (p=0.78), or total distance in the last 30 s (p=0.10) between the 15-minute match periods.

Discussion

Neither the field referees’ movements prior to infringements, running speed at the time of the infringements, nor distance to the infringement affected decision-making accuracy. However, there was no statistically significant tendency towards more decision-making errors in the central area of the field. The distance between the referee and the called infringements was also greater in the lateral areas of the field. The running speed of the referee increased with increasing distance to the infringement, and the running speed at the time of the infringement was also positively associated with the distance covered prior to the infringement.

In the present study, only six free kicks/penalties (1.7%) were awarded to players not performing foul play. This is substantially lower than what was previously reported from international matches (14.2%) [4] and national matches in New Zealand (19.7%) [5]. Even when we included incorrectly awarded yellow or red cards and failure to award cards when warranted, the error percentage remained very low (4%). The Norwegian top referees were practically always right when they called a foul play. The explanation for this low error rate remains unclear. In the present study and other studies investigating decision-making accuracy in soccer referees [4] [5], situations in which the referees called fouls were analysed. From a comparable decision-making accuracy study in Australian football officials, unwarranted sanctioned foul play was shown in 3.5% of the decisions and missed foul play in 9.5% of the decisions [18]. Based on the results from Australian football, one might speculate that the proportion of incorrect decisions would increase if situations not sanctioned had been included in the present findings. In the best leagues [22] and at international matches [23], the locomotion is greater, and the crowd size and stake size are larger. These factors may all contribute to decision-making errors [9] and possibly explain the low proportion of incorrect decisions in the present study compared to the other studies also investigating called fouls only [4] [5]. Even though the error percentage was low in the present study, the number of infringements per match was comparable to previous studies [4] [5] or lower [2] [3].

Associations between movement/position and decision-making

The present study suggests that neither the distance covered during the 30 s prior to the infringement nor running speed at the time of the infringement affects decision-making accuracy. This confirms the findings from previous studies on soccer field referees during national matches in New Zealand [5] and Australian football field officials [18], but it contradicts findings in offside judgements by assistant soccer referees [15]. Offside judgments may be more sensitive to the movements of the referee compared to free-kick judgments, because the assistant referees must focus on the position of the attacking and defending players in relation to each other when the ball is played. An assistant referee seeks to be in line with the last defender, who may be moving at high speed. In contrast to the assistant referees, a field referee does not have to be in line with the last defender and may run where he chooses. This enables the field referee to focus on the acts committed by the players and on the interpretation of the laws of the game. The field referee may also face forward, whereas the assistant referee must often shuttle sideways. Thus, free-kick decision-making by the field referee may be less sensitive to referee movements compared to offside decisions made by the assistant referee [18].

The distance between the referee and the infringements did not affect decision-making accuracy. This supports the findings from international soccer matches [4] and national Australian football matches [18]. This indicates that referees in general are able to maintain an appropriate distance to the play. In the present study, the mean distance was slightly shorter (14.8 m) compared to the study by Mallo et al. [4] (16.7 m). Furthermore, Krustrup et al. [11] reported the distance to the infringements to be 12 m in the central zone and 16 m in the attacking zones of the field. Fixed situations such as throw-ins and free kicks where the attacking team plays the ball towards players deep into the attacking zone often result in physical contact between many players within a small space. In these situations foul play may occur, but it is demanding for the referee because the view may be blocked by the concentration of players on a small part of the field. Thus the probability of incorrect decisions may increase in such situations even if the referee does not have to move rapidly ahead of the situation or have time to choose a position ahead of the throw-in/free-kick. Therefore, fixed offensive situations may confound a possible relationship between running speed or distance to infringements and decision-making accuracy.

The larger distance between the referees and the infringement in the lateral areas compared to the central areas of the field confirms the results from previous studies [3] [4]. In the present study, there was no difference in decision-making errors between infringements in the central and lateral areas. This contradicts the findings of Mallo et al. [4], who reported a larger proportion of incorrect decisions for infringements in the lateral areas of the field, even though Mallo et al. [4] report distances to the infringements in lateral and central areas of the field similar to the present study. It seems plausible that the field referees make fewer incorrect calls in the lateral areas, where the assistant referees are supposed to assist the field referees in calling foul plays and are able to view potential infringements from another perspective. The lack of association between decision-making accuracy and any of our possible proxies for fatigue (total distance prior to infringement, speed at infringement, time of infringement, distance to infringement) indicates that Norwegian top referees possess the necessary fitness to keep up with the play and not call unwarranted fouls.

Associations between movement, position and time of infringement

The running speed of the referee was positively related to the distance to the infringements. This may indicate that the referee anticipates possible infringements and seeks to be closer to the situation by increasing running speed when the distances are large. In the present study and the study by Elsworthy et al. [18], most infringements are called when the distance to the infringement is between 11 and 15 m, a distance previously reported to be the distance with the lowest proportion of decision-making errors [4]. However, in contrast to the present study, Mallo et al. [4] report that most incidents are called from a distance of more than 25 m.

The number of infringements, distance from the infringement, running speed at the time of the infringement, and total distance in the last 30 s before the infringement were evenly distributed between the 15-minute periods. This may indicate that the referees are able to maintain their running and decision-making performance throughout the match. This builds on the evidence from a previous study reporting consistent amounts of high-intensity running and accelerations in Norwegian soccer referees throughout the matches [20]. Our findings are also partly in agreement with two studies by Krustrup et al. [11] [24] which found increased distance between the referee and infringements in the attacking zones at the end of the matches, whereas the distance remained stable in the central areas of the field.

Strengths and limitations

The present study is strengthened by high-quality data from radio-based tracking of the referees and infringements filmed by high-quality video produced for commercial broadcast. A relatively large sample size and the assessments of the decisions by external referees, a method which previously demonstrated good validity within referees [4], between referees [3] [4] and between referees and coaches [25], also strengthen the study. The major limitation of the study is low statistical power for detecting differences between incorrect and correct decisions due to the low number of incorrect decisions. However, post hoc analysis including incorrect card decisions in the incorrect decision category and between the 55 situations where at least one expert referee initially classified the decision as not entirely correct or difficult to assess against the other situations did not find any difference between the groups either. The assistant referees are supposed to assist the field referees’ decision-making, especially in the lateral areas of the field, and lack of data from the communication between the referees can be regarded as a limitation when assessing factors related to decision-making errors. Another weakness is that we assessed only situations where the referee sanctioned foul play, whereas potential situations where the referee failed to sanction foul play were not included in the present study. The occurrence of such decision-making errors is unknown, yet in New Zealand soccer it has been reported to be as high as 41% of all decision-making errors [5]. However, the proportion of unwarranted calls of foul play was almost 10 times greater in the latter study compared to the present study. Nevertheless, field referees are reported to make about 140 observable decisions during a soccer match [2], thus the decisions investigated in the present study make up only a small proportion of the decisions the referees made during the matches. The referees in the present study seldom called unwarranted fouls and the amount of missed fouls was not investigated.

Conclusion

Based on the present study, it seems that the Norwegian field referees have the necessary physical fitness and positional ability to maintain their decision-making ability and to keep up with play throughout the match. The Norwegian referees are practically always correct when they call a foul. Future research within this field should focus on infringements of the laws of the game that are not sanctioned by the referees and factors related to these “missed” infringements.

Conflict of Interest

The authors declare that they have no conflict of interest.

Acknowledgements

We would like to thank, Yngve Håkonsen, Kjell August Ytre-Eide and Sogndal Fotball for their help and support in the project. We sincerely thank the referees for participating and the head of referees in the Norwegian football association Terje Hauge for endorsing the project.

• References

• 1 FIFA. Laws of the Game 2015/16. 2016 [cited 21.09.2016]
• 2 Helsen W, Bultynck JB. Physical and perceptual-cognitive demands of top-class refereeing in association football. J Sports Sci 2004; 22: 179-189
• 3 Mallo J. et al. Activity profile of top-class association football referees in relation to fitness-test performance and match standard. J Sports Sci 2009; 27: 9-17
• 4 Mallo J. et al. Effect of positioning on the accuracy of decision making of association football top-class referees and assistant referees during competitive matches. J Sports Sci 2012; 30: 1437-1445
• 5 Mascarenhas P. et al. Physical performance and decision making in association football referees: A naturalistic study. Open Sports Sci J 2009; 2: 1-9
• 6 Gilis B. et al. Interpretation and application of the laws of the game in football incidents leading to player injuries. Int J Sport Psychol 2006; 37: 121-138
• 7 Gilis B. et al. Offside decisions by expert assistant referees in association football: Perception and recall of spatial positions in complex dynamic events. J Exp Psychol Appl 2008; 14: 21-35
• 8 Gilis B. et al. Interpretation and application of the offside law by expert assistant referees: Perception of spatial positions in complex dynamic events on and off the field. J Sports Sci 2009; 27: 551-563
• 9 Dohmen T, Sauermann J. Referee bias. J Econ Surv 2016; 30: 679-695
• 10 Weston M. et al. Analysis of physical match performance in English Premier League soccer referees with particular reference to first half and player work rates. J Sci Med Sport 2007; 10: 390-397
• 11 Krustrup P. et al. Activity profile and physical demands of football referees and assistant referees in international games. J Sports Sci 2009; 27: 1167-1176
• 12 Barbero-Alvarez J. et al. Physical and physiological demands of field and assistant soccer referees during America's cup. J Strength Cond Res 2012; 26: 1383-1388
• 13 Mallo J. et al. Activity profile of top-class association football referees in relation to performance in selected physical tests. J Sports Sci 2007; 25: 805-813
• 14 Mohr M, Krustrup P, Bangsbo J. Fatigue in soccer: A brief review. J Sports Sci 2005; 23: 593-599
• 15 Oudejans RRD. et al. How position and motion of expert assistant referees in soccer relate to the quality of their offside judgements during actual match play. Int J Sport Psychol 2005; 36: 3-21
• 16 Catteeuw P. et al. Offside decision making of assistant referees in the English Premier League: Impact of physical and perceptual-cognitive factors on match performance. J Sports Sci 2010; 28: 471-481
• 17 Paradis K, Larkin P, O’Connor D. The effects of physical exertion on decision-making performance of Australian football umpires. J Sports Sci 2016; 34: 1535-1541
• 18 Elsworthy N, Burke D, Dascombe B. Factors relating to the decision-making performance of Australian football officials. Int J Perf Anal Spor 2014; 14: 401-410
• 19 Harriss DJ, Atkinson G. Standards for ethics in sport and exercise science research: 2014 update. Int J Sports Med 2013; 34: 1025-1028
• 20 Riiser A. et al. Accelerations and high intensity running in field and assistant football referees during match play. Sci & Med Football 2017; 1: 280-287
• 21 Ingebrigtsen J. et al. Acceleration and sprint profiles of a professional elite football team in match play. Eur J Sport Sci 2015; 15: 101-110
• 22 Bush M. et al. Evolution of match performance parameters for various playing positions in the English Premier League. Hum Mov Sci 2015; 39: 1-11
• 23 Stolen T. et al. Physiology of soccer: An update. Sports Med 2005; 35: 501-536
• 24 Krustrup P, Bangsbo J. Physiological demands of top-class soccer refereeing in relation to physical capacity: effect of intense intermittent exercise training. J Sports Sci 2001; 19: 881-891
• 25 Coleclough J. Soccer coaches’ and referees’ perceptions of tackle incidents with respect to the laws of the game. Int J Perf Anal Spor 2013; 13: 553-566