Neurosurgery Simulators Developed for Neurosurgical Training in Brazil: A Systematic Review

Abstract

Introduction Simulation in neurosurgery is a growing trend in medical residency programs around the world due to the concerns there are about patient safety and the advancement of surgical technology. Simulation training can improve motor skills in a safe environment before the actual setting is initiated in the operating room. The aim of this review is to identify articles that describe Brazilian simulators, their validation status and the level of evidence (LoE).

Methodology This study was conducted using the Preferred Reported Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. A search was performed in the Medline, Scielo, and Cochrane Library databases. The studies were evaluated according to the Medical Education Research Quality Instrument (MERSQI), and the LoE of the study was established according to the classification system of the Oxford Centre for Evidence-Based Medicine (OCEBM), which has been adapted by the European Association of Endoscopic Surgery.

Results Of all the studies included in this review, seven referred to validated simulators. These 7 studies were assigned an average MERSQI score of 8.57 from 18 possible points. None of the studies was randomized or conducted in a high-fidelity environment. The best evidence was provided by the studies with the human placenta model, which received a score of 2b and a degree of recommendation of 3.

Conclusion Brazilian simulators can be reproduced in the different laboratories that are available in the country. The average MERSQI score of Brazilian studies is similar to the international average score. New studies should be undertaken to seek greater validation of the simulators and carry out randomized controlled trials.

Resumo

Introdução A simulação em neurocirurgia é uma tendência crescente em programas de residência médica em todo o mundo devido às preocupações que existem sobre a segurança do paciente e o avanço da tecnologia cirúrgica. O treinamento com simulação permite aprimorar as habilidades motoras em um ambiente seguro antes de partir para o cenário real na sala de cirurgia. O objetivo desta revisão é identificar artigos que descrevam simuladores brasileiros, determinar o status de validação e nível de evidência (LoE).

Metodologia Esse estudo foi realizado utilizando o Preferred Reported Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. Foi realizado uma busca nas bases de dados Medline, Scielo e Cochrane. Os estudos foram avaliados de acordo com o Medical Education Research Quality Instrument (MERSQI) e o LoE foi estabelecido de acordo com o Oxford Centre for Evidence-Based Medicine (OCEBM) adaptado pela Associação Européia de Cirurgia Endoscópica.

Resultados De todos os estudos incluídos nessa revisão, sete se referiam a simuladores validados. Estes receberam uma pontuação MERSQI média de 8,57 de 18 pontos possíveis. Nenhum dos estudos foi randomizado ou conduzido em ambiente de alta fidelidade. A melhor evidência foi fornecida pelos estudos com a placenta humana que recebeu uma pontuação 2b e um grau de recomendação de 3.

Conclusão Os simuladores brasileiros podem ser reproduzidos nos diferentes laboratórios disponíveis no país. O escore médio do MERSQI de estudos brasileiros é semelhante a pontuação média de estudos internacionais. Novos estudos devem buscar maior validação dos simuladores e maior nível de evidência com ensaios clínicos randomizados.

Keywords

simulation training - neurosurgery - spine - education - Brazil

Palavras-chave

treinamento de simulação - neurocirurgia - espinha - educação - Brasil

Introduction

The reduction of the working hours of residents in the United States and Europe has made simulation training a reality in all surgical specialties.[1] Simulation allows residents to acquire a skill quicker and more safely before they go into the real scenario of the operating room.[2] A recent meta-analysis has shown the benefits of simulation for motivated individuals who receive feedback for their performance.[3]

While simulation training has just started in Brazil and is taking its first steps, simulators in other countries are developing as new technologies are now emerging. Simulations are being done on synthetic models, using virtual reality and 3D printing,[4] [5] [6] rather than human cadavers and live animals. There is now the possibility of pathology-specific training with an educational purpose and preoperative planning.[7] [8]

Simulation training in neurosurgery has become even more important because one error can lead to devastating consequences for the patient. Kirkman et al demonstrated the benefits of simulation in the first systematic review of simulation in neurosurgery.[9] However, most of the simulators demonstrated were expensive and so would be costly if they were used in Brazilian neurosurgery simulation laboratories.

The objectives of this systematic review are: 1. To identify studies that describe simulation methods developed by Brazilian neurosurgeons. 2. To determine the quality of the study, the validation status, the level of evidence (LoE) and the degree of recommendation.

Material and Methods

This study was conducted by using the approved guidelines of the Preferred Reported Items for Systematic Reviews and Meta-Analysis (PRISMA).[10]

Inclusion and Exclusion Criteria

Articles describing validated and non-validated simulators for neurosurgical training were included in this review, while studies describing simulators for lumbar puncture, central venous access and rhizotomy procedures were excluded. Articles that were not written in either English or Portuguese were also excluded.

Information Sources and Search

A search in the databases at Medline, Scielo and Cochrane Library was performed and the studies took place between January 1, 1998, and September 29, 2018. The search terms used were “neurosurgery,” “spine surgery” and “simulation training” as these were found to provide the largest number of articles. A more specific search was then performed afterwards using the terms “skill transfer,” “skill retention,” “motor performance” and “haptics.” This allowed the researcher to find other supplementary studies.

Studies Selection and Data Collection

Articles approved in the inclusion criteria were submitted to evaluation of their abstracts, according to the PRISMA protocol. Duplicate papers, conference publications and articles that were not related to neurosurgery or surgery simulation were all excluded. The selected studies were submitted to a full-text evaluation. Articles that did not describe or validate simulators were also excluded. Studies with patient-specific planning simulators were considered simulators. Only studies by the main Brazilian authors were selected. The relevance tests were performed by two authors, and the study inclusion was done when consent had been obtained from both of them. When there was any disagreement, a third author defined the selection.

Collected Data

The data extracted from each study was categorized according to the type of simulator, the neurosurgical subspecialty addressed, the type of procedure, the validation of the simulator, the Medical Education Research Study Quality Instrument (MERSQI)[11] [12] ( [Table 1] ), and the (LoE of the study , according to the classification system of the Oxford Centre for Evidence-Based Medicine (OCEBM), adapted by the European Association of Endoscopic Surgery[13] [14] ( [Table 2] and [3]).

Table 1 Medical Education Research Quality Instrument score for validated simulators

Items of the scale (possible points)	Sub items of the scale (points if present)	Number of studies (%)	Mean
Study design (3)	Single group cross-sectional or single group posttest only (1)	4 (57)	1.28
	Single group pretest and posttest (1.5)	2 (28)
	Nonrandomized, 2 groups (2)	1 (14)
	Randomized controlled trial (3)	0.00
Sampling (3)	Number of institutions studied:
	1 (0.5)	4 (57)	0.5
	2 (1)	2 (28)
	3 (1.5)	1 (14)
	Response rate, %:
	Not applicable
	< 50 or not reported (0.5)	7 (100)
	50–74 (1)	0.00
	> 75 (1.5)	0.00
Type of data (3)	Assessment by study participant (1)	4 (57)	1.85
	Objective measurement (3)	3 (42)
Validity of evaluation instrument (3)	Internal structure:
	Not applicable
	Not reported (0)	3 (42)	1.14
	Reported (1)	4 (57)
	Content:
	Not applicable
	Not reported (0)	3 (42)
	Reported (1)	4 (57)
	Relationships to other variables:
	Not applicable
	Not reported (0)	3 (42)
	Reported (1)	0.00
Data analysis (3)	Appropriateness of analysis:
	Data analysis inappropriate for study design or type of data (0)	3 (42)	2.57
	Data analysis appropriate for study design or type of data (1)	4 (57)
	Complexity of analysis:
	Descriptive analysis only (1)	3 (42)
	Beyond descriptive analysis (2)	4 (57)
Outcomes (3)	Satisfaction, attitudes, perceptions, opinions, general facts (1)	4 (57)	1.21
	Knowledge, skills (1,5)	3 (42)
	Behaviors (2)	0.00
	Patient/health care outcome (3)	0.00
Total score			8.57

Table 2 Modified levels of evidence classification for validation studies, adapted from the Oxford Centre for Evidence-Based Medicine classification by the European Association of Endoscopic Surgeons (Carter et al, 2005)

Level of evidence	Criteria
1a	Systematic reviews (meta-analysis) containing at least some trial of level 1b evidence, in which results of separate, independently controlled trials are consistent
1b	Randomized controlled trial of good quality and of adequate sample size (power calculations)
2a	Randomized trials of reasonable quality and/or of inadequate sample size
2b	Nonrandomized trials, comparative research (parallel cohort)
2c	Nonrandomized trials, comparative research (historical cohort, literature controls)
3	Nonrandomized, non-comparative trials, descriptive research
4	Expert opinions, including the opinion of Work Group members

Table 3 Levels of recommendation for training models, adapted from the Oxford Centre for Evidence-Based Medicine Classification by the European Association of Endoscopic Surgeons (Carter et al, 2005)

Level of evidence	Criteria
1	Based on one systematic review (1a) or at least two independently conducted research projects classified as 1b
2	Based on at least two independently conducted research projects classified as level 2a or 2b, within concordance
3	Based on one independently conducted research project level 2b, or at least two trials of level 3, within concordance
4	Based on one trial at level 3 or multiple expert opinions, including the opinions of Work Group members (e.g., level 4)

The simulators were categorized according to their neurosurgical subspecialty: vascular, functional, pediatric, spine, skull base, oncology, trauma, and basic neurosurgery. The results were then tabulated, and the simulators from each neurosurgical field grouped. The type of validation of the simulator was classified, according to the definitions of McDougall and Van Nortwick et al[1] ([Fig. 1]). All the studies were evaluated for the LoE, and the studies' quantitative analysis also received a MERSQI score.

Fig. 1 Types of validity. Definitions from McDougall et al; van Nortwick et al.[30] [31] Translated by Aydin et al.[1] Current Status of Simulation and Training Models in Urological Surgery: A Systematic Review. DOI: 10.1016/j.juro.2016.01.131.

Results

Selected Articles

Our search strategy found 512 articles. After the duplicates and articles that were not published in either English or Portuguese had been excluded, 494 remained for the screening of the title. After this stage, 191 articles were submitted to the abstract evaluation. Two authors agreed on the selection of 19 papers for a review of the full text version, out of which 15 were selected for inclusion in this systematic review ([Fig. 2]).

Fig. 2 Preferred Reported Items for Systematic Reviews and Meta-Analysis flow diagram.

Characteristics of the Selected Studies

The neurosurgery fields that had the largest number of simulation studies were the vascular and pediatric, with 4 (26.66%) studies each. The most described simulator type was the human placenta (33.33%), followed by 3D printed simulators (26.66%), and the synthetic simulators (20%). Only one study (6.66%) reported the use of virtual or mixed reality. Eleven studies (73.33%) had the resident's skill training as their main purpose and 4 (26.66%) had patient-specific simulation. The most simulated neurosurgical procedure was ventricular neuroendoscopy (20%). [Table 4] shows the relationship between the type of simulator that was used for each simulated procedure.

Table 4 Type of simulator used for training in each neurosurgical procedure

Subspeciality	Procedure	Human Placenta	Synthetic Model	3D Printing	Mixed Reality	Corpse	Fruit
Neurovascular	Vascular microsurgery	x
	Intracranial bypass	x
	Angiography/ Endovascular	x
	Microsurgery aneurysm	x
Neuropediatric	Neuroendoscopy		x
	Craniosynostosis		x	x
	Meningocele			x
Neuroncology	Microsurgery for tumor	x
	Neuroendoscopy		x
Spine	Arthrodesis				x
	Corpectomy			x
Basic Neurosurgery	Craniotomy						x
	Anatomy			x
	External ventricular drain/neuroendoscopy					x

Study Quality and Level of Evidence

Of all the studies included in this review, 7 (46.66%) referred to simulators that were validated. These studies were evaluated according to the MERSQI score and they presented an average score of 8.57 from 18 possible points ([Table 1]). The studies with the highest scores were the models in the human placenta (MERSQI 12 to 8.5). No study was randomized to the control group. No studies were conducted in a high-fidelity environment. Only one study demonstrated the skill transfer from the simulator to the surgical center.[15] The studies with the best evidence were the models in the human placenta, which received a score of 2b and a degree of recommendation of 3 ([Table 5]).

Table 5 Level of evidence of validated studies

Author	MERSQI	LoE	LoR	Simulated Procedure	Type of Validation
Oliveira M. M. R. 2018	12	2b	3	Aneurysm clipping	Predictive and competition
Oliveira M. M. R. 2018	8.5	2b	3	Bypass	Competition and construct
Oliveira M. M. R. 2015	9.5	3	4	Endovascular	Construct, content and face
Oliveira M. M. R. 2014	9.5	2b	3	Microsurgery tumor	Construct, content and face
Grillo F. W., 2018	5.5	4	4	Craniotomy	Content
Filho F. V. 2011	9.5	2b	3	Neuroendoscopy	Construct and face
Coelho, G. 2018	5.5	3	4	Arthrodesis	Construct and face

Abbreviations: LoE, level of evidence; LoR, level of recommendation; MERSQI, Medical Education Research Study Quality Instrument.

Data Synthesis

Validated Simulation Models

Of the 15 studies included in this systematic review, 7 (46.66%) had at least 1 type of validation. The most used of these were construct and face validity, which occurred in 4 studies (57.14%). Three studies (42.85%) showed content validity, 2 (28.57%) concurrent validity, and 1 (14.28%) presented predictive validity. Vascular neurosurgery was the area that had the highest number of validated studies.

Non-Validated Simulation Models

Eight studies (53.33%) were not validated. Four of them (50%) were related to patient-specific simulation. A descriptive study using the placenta was subsequently validated in further studies. The area of pediatric neurosurgery was the one that presented the most non-validated studies ([Table 6]).

Table 6 Level of evidence of non-validated studies

Author	LoE	LoR	Simulated procedure
Coelho, G. 2014	3	4	Neuroendoscopy
Ghizoni E. 2017	3	4	Craniosynostosis
Oliveira M. M. R. 2014	2b	3	Vascular microsurgery
Paiva W. S. 2007	3	4	Spine tumor
Coelho G., 2014	3	4	Craniosynostosis
Ferreira C. D., 2014	3	4	Neuroendoscopic
Coelho, G. 2017	3	4	Meningocele
Drummond-Braga B. 2016	3	4	Craniotomy

Abbreviations: LoE, level of evidence; LoR, level of recommendation.

Vascular Neurosurgery

Four studies were presented, with three of them being validated and one descriptive. The human placenta was the only type of simulator that was described for vascular neurosurgery simulation. It was used in the four mentioned studies. These studies received the highest scientific evidence and the highest MERSQI score. The descriptive study demonstrated the potential of the placenta for simulation in vascular neurosurgery[16] (LoE—3, level of recommendation [LoR]—4). The study with the highest MERSQI score (12) used the placenta to demonstrate the transfer of skills that are acquired in the simulator to help with the neurosurgical procedure[15] (LoE 2b, LoR 3). The validated intracranial-intracranial (IC-IC) bypass model presented the possibility of performing several different bypass techniques[17] (MERSQI 8.5, LoE 2b, LoR 3). The placenta was also used to demonstrate how useful it is in simulating endovascular procedures[18] (MERSQI 9.5, LoE 3, LoR 4).

Pediatric Neurosurgery

Four studies were presented for simulation in pediatric neurosurgery, but none of them were validated. Two studies were used for patient-specific simulation. Ghizoni et al demonstrated the use of 3D printing in both planning and preoperative simulation in three cases of craniosynostosis.[19] Coelho et al performed the patient-specific simulation for an encephalocele on the face using a multimaterial 3D print. This was later replicated in surgery.[20]

Coelho et al presented two studies with the same simulator (ASPEN) that was developed for the simulation of craniosynostosis and ventricular neuroendoscopy.[21] [22] The studies of pediatric neurosurgery were evaluated with a LoE of 3 e a LoR of 4.

Neurosurgical Oncology

Two studies were included with simulators to remove brain tumors. Both of these simulators were validated. Oliveira et al described the use of the human placenta to simulate microsurgery for an intracranial tumor.[23] (MERSQI 9.5, LoE 2b, LoR 3). Filho et al described and validated a synthetic simulator called Sinus Model Oto-Rhino Neuro Trainer (S.I.M.O.N.T.)[24] for the simulation of neuroendoscopy for the resection of a ventricular tumor (MERSQI 9.5, LoE 2b, LoR 3) and access to the base of the skull.

Spine

Two simulators were described for the spine. One simulator underwent validation and the other was a patient-specific simulation description. Coelho et al described and validated a simulator to perform lumbar spine procedures, such as arthrodesis and laminectomy.[25] They used the mixed reality that combines virtual reality with the synthetic model (MERSQI 5.5, LoE 3, LoR 4). Paiva et al described the patient-specific simulation with a 3D printed model for surgical planning of a corpectomy and removal of a complex tumor in the cervical spine[26] (LoE 3, LoR 4).

Basic Neurosurgery

There was no validation for any of the three studies that were related to the basic procedures of neurosurgery. Drummond-Braga et al described the use of a coconut fruit as a simulator for cerebrospinal fluid leak avoidance during craniotomy for residents of the first year.[27] Ferreira et al described a method that is used for dilation of the ventricular system in cadavers to simulate ventricular endoscopy.[28] Grillo et al presented the creation of a phantom used for neurosurgical training and preoperative planning.[29] All the studies in this area were classified according to their LoE, which was 3, and the LoR, which was 4. [Table 7] shows a summary of each study's main characteristics.

Table 7 Summary of the main characteristics of each study

Author	Validity	Type of validation	Neurosurgery field	Procedure	Simulator type	Function	Participants	Objective	Main results	Limitations
Grillo F. W., 2018	Yes	Contend	Basics	Craniotomy	3D print	Patient-specific simulator	(n = 17) Neurosurgeons	Creation of a realistic phantom for reproduction of the cerebral cortical morphology, cerebrospinal fluid, meninges and scalp.	Most neurosurgeons rated the model as very good for realism and educational purpose.	It is not a simulator created for a specific neurosurgical technique.
Oliveira M. M. R., 2018	Yes	Predictive and concurrent	Vascular	Aneurysm microsurgery	Human placenta	Skill training	(n = 12) Residents and neurosurgeons	This study assessed concurrent and predictive validity of brain aneurysm surgery simulation in a human placenta model compared with a “live” human brain cadaveric model.	Residents trained in the human placenta simulator consistently had the highest overall performance scores when compared with those who had trained in the cadaver model and those who had simply watched operative videos (p < 0.001).	The fresh human placenta can be used for only 1 to 2 weeks. After this time, the vessel walls become weaker, more difficult to dissect and suture.
Filho F. V., 2011	Yes	Construct and face	Oncology	Neuroendoscopy	Synthetic	Skill training	(n = 22) Residents and neurosurgeons	Assess both the quality of the model and the development of surgical skills by trainees.	The experts considered the simulator capable of reproducing surgical situations as if they were real and presenting great similarity with the human brain.	Expensive, not largely available, poor haptic feedback.
Oliveira M. M. R., 2016	Yes	Construct, Contend and face	Oncology	Tumor microsurgery	Human placenta	Skill training	(n = 16) Residents and neurosurgeons	To describe and asses face, contend and construct validity of the model	The human placental brain tumor microsurgical resection model is a high-fidelity training model that may have significant potential in the evaluation and training of neurosurgical residents.	The model does not replicate all aspects of tumor resection.
Oliveira M. M. R., 2018	Yes	Concurrent and construct	Vascular	Intracranial-intracranial bypass	Human placenta	Skill training	(n = 9) Residents	To describe the human placenta vascular anatomy to guide IC-IC bypasses apprenticeship.	An ex vivo bypass model offers great similarity to main brain vessels with the possibility to practice a variety of IC-IC bypass techniques in a single simulator. Placenta vascular anatomy knowledge can improve laboratory microsurgical training.	Biological risks, patient consent issues, placenta need to be fresh.
Coelho, G., 2018	Yes	Contend and face	Spine	Arthrodesis	Mixed reality	Skill training	(n = 16) Spinal surgeons	To propose and validate a new tool for neurosurgical education, associating virtual and realistic simulation (mixed reality), for spine surgery	The surgery team considered that this virtual simulation provides a highly effective training environment, and it significantly enhances teaching of surgical anatomy and operative strategies.	Expensive, not largely available.
Oliveira M. M. R., 2015	Yes	Construct, contend and face	Vascular	Neurointerventional	Human placenta	Skill training	(n = 12) Residents and neurosurgeons	To describe and asses face, contend and construct validity of the model	Excellent haptics, low startup costs, and ready availability for any institution with interventional capabilities.	Biological risks, patient consent issues.
Ghizoni E., 2018	No	N/A	Pediatrics	Craniosynostosis	3D print	Patient- specific simulator	(n = 2) Residents and neurosurgeons	Describe the use of three models for craniosynostosis practice made with a 3D printer	Single sagittal stenosis, bilateral coronal stenosis and complex stenosis of multiple sutures were simulated prior to the surgical procedure and anatomical variations were observed by surgeons	Descriptive only, with two participants.
Drummond-Braga B., 2016	No	N/A	Basics	Craniotomy	Fruit- Coconut	Skill training	N/A	To describe an original model for learning craniotomy first steps with CSF leak avoidance using a coconut.	Its main advantages are that coconuts are affordable and widely available and simulate CSF leaks. It has a potential pedagogic neurosurgical application for freshman residents, and further validity is necessary to confirm this hypothesis.	Complete absence of anatomic landmarks and the consistency of the coconut layers.
Paiva W. S., 2007	No	N/A	Spine	Corpectomy	3d print	Patient- specific simulator	N/A	The purpose of this work is to demonstrate the practical use of the stereolithography an auxiliary method for training and surgical simulation	It is easier for the surgeon to understand the complexity of the case and plan the approach before any surgical procedure. Careful planning and previous rehearsal reduce the risk of surprises during an operation.	High cost involved in prototyping process.
Coelho G., 2014	No	N/A	Pediatrics	Craniosynostosis	Synthetic	Skill training	N/A	To describe a synthetic simulator for the practice of craniosynostosis	This training model may represent a very useful method to simulate the steps of surgery for scaphocephaly. This training provides an alternative to the use of human cadavers and animal models.	High cost for maintenance.
Ferreira C. D., 2014	No	N/A	Basics	Neuroendoscopy	Cadaver	Patient- specific simulator	N/A	Create an anatomical model that simulates hydrocephalus and can be used in training in neuroendoscopy techniques.	The adequate use of the anomalous chemical–physical characteristics of the water molecule may provide a good mechanism to expand the ventricular cavity, to create an experimental model of hydrocephalus.	Biological risk. Difficulty of availability of cadavers.
Coelho, G. 2014	No	N/A	Pediatrics	Neuroendoscopy	Synthetic	Skill training	N/A	In this study, we present a new pediatric neuroendoscopic simulator that facilitates training. Description	Description of the simulator This realistic simulator was built with a synthetic thermo-retractile and thermo-sensible rubber called Neoderma®	High cost for maintenance.
Coelho, G. 2017	No	N/A	Pediatrics	Encephalocele	3d print	Patient- specific simulator	N/A	To describe a case of frontoethmoidal encephalocele, (nasofrontal subtype) of a 19-month-old girl, whose surgical correction was planned using 3D printing modeling.	The 3D model allowed predicting with millimetric precision the bilateral orbitotomy measurements, reducing the time of surgery, and precontouring the osteosynthesis material.	Demand computer design expert, high cost.
Oliveira, M. M. R. 2014	No	N/A	Vascular	Vascular microsurgery	Human placenta	Skill training	N/A	To describe an aneurysm surgical model that recreates the acquisition and maintenance of the specific microneurosurgical skills used in clipping cerebral aneurysms	The human placenta provides an inexpensive, convenient biological model for modeling cerebral aneurysms with high fidelity to neural tissue. In addition, it can be used to create aneurysms of various morphologies.	The fresh human placenta can be used for only 1 to 2 weeks. After this time, the vessel walls become weaker, more difficult to dissect and suture.

Abbreviations: CSF, cerebrospinal fluid; IC, intracranial.

Discussion

The Brazilian studies that underwent validation presented an average score of 8.56 points in the MERSQI. This value approximates the score described in the systematic review by Kirkman et al in 2014 (9.21 points).[8] The worst scores were received by validation topics that related to study designs and the number of institutions involved.

Few of the studies on simulation were able to demonstrate the transfer of skills from the simulator to the surgical room.[2] However, in 2018, Oliveira et al were able to demonstrate the transfer of an acquired ability in simulations of vascular microsurgery in the placenta to the real scenario in aneurysm surgeries. It was the only Brazilian study that described predictive validity.[15]

The highest LoE found was 2b and the best grade of recommendation was 3. This demonstrates the lack of a randomized blind study in Brazilian simulators. When analyzed in conjunction with the MERSQI score, the indication is that further Brazilian studies in simulation should be performed that search for randomization with control groups and multicentric studies.

The majority of the studies were of vascular and pediatric subspecialties. The most simulated procedures had the greatest demand for manual skills, such as vascular microsurgery and recent technological evolution, such as ventricular endoscopy. The least frequently simulated were functional neurosurgery and neurotrauma.

Among the non-validated simulators, the use of patient-specific or pathology-specific simulators is noteworthy. These simulators were used to provide education about specific conditions found in neurosurgery, as well as preoperative planning and rehearsal in complex cases. Three-dimensional printing was the most described type of simulation used for this purpose. Craniosynostosis surgery was the most simulated patient-specific procedure described.

This systematic review had its limitations. The main ones being the different forms of methodology of each study, and the different groups of evaluated participants and heterogeneous simulators. Perhaps the greatest limitation was the difference in the quality of the studies and the fact that most simulators lack a validation instrument.

Future research should focus on the creation of high-fidelity simulators that are accessible to the resident physician and could be introduced to neurosurgical training laboratories throughout Brazil. Further validated and randomized studies should be performed to define the ideal simulators that could truly fit in every level of Brazil's residency skill training program.

Conclusion

The MERSQI score of the Brazilian studies resembles the international average. The LoE and the degree of recommendation of most of the published articles is still low. New studies should pursue a further validation of the simulators and hold randomized trials with a control group. There is a lot of creativity, simplicity and technology involved in Brazilian simulators. Most of them can be reproduced at the skill training laboratories that are available in the country.

Conflict of Interest

None declared.

• References

• 1 Aydin A, Shafi AM, Shamim Khan M, Dasgupta P, Ahmed K. Current Status of Simulation and Training Models in Urological Surgery: A Systematic Review. J Urol 2016; 196 (02) 312-320
• 2 Rehder R, Abd-El-Barr M, Hooten K, Weinstock P, Madsen JR, Cohen AR. The role of simulation in neurosurgery. Childs Nerv Syst 2016; 32 (01) 43-54
• 3 McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med 2011; 86 (06) 706-711
• 4 Loh CYY, Wang AYL, Tiong VTY. , et al. Animal models in plastic and reconstructive surgery simulation-a review. J Surg Res 2018; 221: 232-245
• 5 Pfandler M, Lazarovici M, Stefan P, Wucherer P, Weigl M. Virtual reality-based simulators for spine surgery: a systematic review. Spine J 2017; 17 (09) 1352-1363
• 6 Vakharia VN, Vakharia NN, Hill CS. Review of 3-Dimensional Printing on Cranial Neurosurgery Simulation Training. World Neurosurg 2016; 88: 188-198
• 7 Ryu WHA, Dharampal N, Mostafa AE. , et al. Systematic Review of Patient-Specific Surgical Simulation: Toward Advancing Medical Education. J Surg Educ 2017; 74 (06) 1028-1038
• 8 Dawe SR, Pena GN, Windsor JA. , et al. Systematic review of skills transfer after surgical simulation-based training. Br J Surg 2014; 101 (09) 1063-1076
• 9 Kirkman MA, Ahmed M, Albert AF, Wilson MH, Nandi D, Sevdalis N. The use of simulation in neurosurgical education and training. A systematic review. J Neurosurg 2014; 121 (02) 228-246
• 10 Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. ; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6 (07) e1000097
• 11 Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA 2007; 298 (09) 1002-1009
• 12 Cook DA, Reed DA. Appraising the quality of medical education research methods: the Medical Education Research Study Quality Instrument and the Newcastle-Ottawa Scale-Education. Acad Med 2015; 90 (08) 1067-1076
• 13 Carter FJ, Schijven MP, Aggarwal R. , et al. Consensus guidelines for validation of virtual reality surgical simulators. Simul Healthc 2006; 1 (03) 171-179
• 14 Morgan M, Aydin A, Salih A, Robati S, Ahmed K. Current Status of Simulation-based Training Tools in Orthopedic Surgery: A Systematic Review. J Surg Educ 2017; 74 (04) 698-716
• 15 de Oliveira MMR, Ferrarez CE, Ramos TM. , et al. Learning brain aneurysm microsurgical skills in a human placenta model: predictive validity. J Neurosurg 2018; 128 (03) 846-852
• 16 Oliveira Magaldi M, Nicolato A, Godinho JV. , et al. Human placenta aneurysm model for training neurosurgeons in vascular microsurgery. Neurosurgery 2014; 10 (Suppl. 04) 592-600 , discussion 600–601
• 17 Oliveira MM, Wendling L, Malheiros JA. , et al. Human Placenta Simulator for Intracranial-Intracranial Bypass: Vascular Anatomy and 5 Bypass Techniques. World Neurosurg 2018; 119: e694-e702
• 18 Ribeiro de Oliveira MM, Nicolato A, Santos M. , et al. Face, content, and construct validity of human placenta as a haptic training tool in neurointerventional surgery. J Neurosurg 2016; 124 (05) 1238-1244
• 19 Ghizoni E, de Souza JPSAS, Raposo-Amaral CE. , et al. 3D-Printed Craniosynostosis Model: New Simulation Surgical Tool. World Neurosurg 2018; 109: 356-361
• 20 Coelho G, Chaves TMF, Goes AF, Del Massa EC, Moraes O, Yoshida M. Multimaterial 3D printing preoperative planning for frontoethmoidal meningoencephalocele surgery. Childs Nerv Syst 2018; 34 (04) 749-756
• 21 Coelho G, Zymberg S, Lyra M, Zanon N, Warf B. New anatomical simulator for pediatric neuroendoscopic practice. Childs Nerv Syst 2015; 31 (02) 213-219
• 22 Coelho G, Warf B, Lyra M, Zanon N. Anatomical pediatric model for craniosynostosis surgical training. Childs Nerv Syst 2014; 30 (12) 2009-2014
• 23 Oliveira MM, Araujo AB, Nicolato A. , et al. Face, Content, and Construct Validity of Brain Tumor Microsurgery Simulation Using a Human Placenta Model. Oper Neurosurg (Hagerstown) 2016; 12 (01) 61-67
• 24 Filho FV, Coelho G, Cavalheiro S, Lyra M, Zymberg ST. Quality assessment of a new surgical simulator for neuroendoscopic training. Neurosurg Focus 2011; 30 (04) E17
• 25 Coelho G, Defino HLA. The Role of Mixed Reality Simulation for Surgical Training in Spine: Phase 1 Validation. Spine 2018; 43 (22) 1609-1616
• 26 Paiva WS, Amorim R, Bezerra DA, Masini M. Aplication of the stereolithography technique in complex spine surgery. Arq Neuropsiquiatr 2007; 65 (2B): 443-445
• 27 Drummond-Braga B, Peleja SB, Macedo G. , et al. Coconut Model for Learning First Steps of Craniotomy Techniques and Cerebrospinal Fluid Leak Avoidance. World Neurosurg 2016; 96: 191-194
• 28 Ferreira CD, Matushita H, Silva BR. , et al. Proposal of a new method to induce ventricular system dilation to simulate the features of hydrocephalus and provide an anatomical model for neuroendoscopy training. Childs Nerv Syst 2014; 30 (07) 1209-1215
• 29 Grillo FW, Souza VH, Matsuda RH. , et al. Patient-specific neurosurgical phantom: assessment of visual quality, accuracy, and scaling effects. 3D Print Med 2018; 4 (01) 3
• 30 McDougall EM. Validation of surgical simulators. J Endourol 2007; 21 (03) 244-247
• 31 Van Nortwick SS, Lendvay TS, Jensen AR. , et al. Methodologies for establishing validity in surgical simulation studies. Surgery 2010; 147 (05) 622-630