Research Article
No access
Published Online: 12 August 2021

Validation of a Novel Simulation-Based Test in Robot-Assisted Radical Prostatectomy

Publication: Journal of Endourology
Volume 35, Issue Number 8

Abstract

Purpose: To investigate validity evidence for a simulator-based test in robot-assisted radical prostatectomy (RARP).
Materials and Methods: The test consisted of three modules on the RobotiX Mentor VR-simulator: Bladder Neck Dissection, Neurovascular Bundle Dissection, and Ureterovesical Anastomosis. Validity evidence was investigated by using Messick's framework by including doctors with different RARP experience: novices (who had assisted for RARP), intermediates (robotic surgeons, but not RARP surgeons), or experienced (RARP surgeons). The simulator metrics were analyzed, and Cronbach's alpha and generalizability theory were used to explore reliability. Intergroup comparisons were done with mixed-model, repeated measurement analysis of variance and the correlation between the number of robotic procedures and the mean test score were examined. A pass/fail score was established by using the contrasting groups' method.
Results: Ten novices, 11 intermediates, and 6 experienced RARP surgeons were included. Six metrics could discriminate between groups and showed acceptable internal consistency reliability, Cronbach's alpha = 0.49, p < 0.001. Test–retest reliability was 0.75, 0.85, and 0.90 for one, two, and three repetitions of tests, respectively. Six metrics were combined into a simulator score that could discriminate between all three groups, p = 0.002, p < 0.001, and p = 0.029 for novices vs intermediates, novices vs experienced, and intermediates vs experienced, respectively. Total number of robotic operations and the mean score of the three repetitions were significantly correlated, Pearson's r = 0.74, p < 0.001.
Conclusion: This study provides validity evidence for a simulator-based test in RARP. We determined a pass/fail level that can be used to ensure competency before proceeding to supervised clinical training.

Get full access to this article

View all available purchase options and get full access to this article.

References

1. Rawla P. Epidemiology of prostate cancer. Rev World J Oncol 2019;10:63–89.
2. Goldenberg MG, Lee JY, Kwong JCC, et al. Implementing assessments of robotic-assisted technical skill in urologic education: A systematic review and synthesis of the validity evidence. BJU Int 2018;122:501–519.
3. Morris C, Hoogenes J, Shayegan B, et al. Towards development and validation of an intraoperative assessment tool for robot-assisted radical prostatectomy training: Results of a Delphi study. Int Braz J Urol 2017;43:661–670.
4. Danish Prostate Cancer Group DaProCa: Clinical guideline for radical prostatectomy in prostate cancer [In Danish]. 2019.
5. Ficarra V, Wiklund PN, Rochat CH, et al. The European Association of Urology Robotic Urology Section (ERUS) survey of robot-assisted radical prostatectomy (RARP). BJU Int 2013;111:596–603.
6. MacCraith E, Forde JC, Davis NF. Robotic simulation training for urological trainees: A comprehensive review on cost, merits and challenges. J Robot Surg 2019;13:371–377.
7. Brewin J, Ahmed K, Challacombe B. An update and review of simulation in urological training. Int J Surg 2014;12:103–108.
8. Abboudi H, Khan MS, Guru KA, et al. Learning curves for urological procedures: A systematic review. BJU Int 2014;114:617–629.
9. Mcgaghie WC, Issenberg SB, Barsuk JH, et al. A critical review of simulation-based mastery learning with translational outcomes. Med Educ 2014;48:375–385.
10. Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Mastery learning for health professionals using technology-enhanced simulation: A systematic review and meta-analysis. Acad Med 2013;88:1178–1186.
11. Bjerrum F, Thomsen ASS, Nayahangan LJ, et al. Surgical simulation: Current practices and future perspectives for technical skills training. Med Teach 2018;40:668–675.
12. Ghaderi I, Manji F, Soo Park Y, et al. Technical skills assessment toolbox a review using the unitary framework of validity. Ann Surg 2015;261:251–262.
13. Jørgensen M, Konge L, Subhi Y. Contrasting groups' standard setting for consequences analysis in validity studies: Reporting considerations. Adv Simul 2018;3:1–7.
14. Downing SM. Reliability: On the reproducibility of assessment data. Med Educ 2004;38:1006–1012.
15. Mazzon G, Sridhar A, Busuttil G, et al. Learning curves for robotic surgery: A review of the recent literature. Curr Urol Rep 2017;18:89.
16. Alemozaffar M, Narayanan R, Percy AA, et al. Validation of a novel, tissue-based simulator for robot-assisted radical prostatectomy. J Endourol 2014;28:995–1000.
17. Whittaker G, Aydin A, Raison N, et al. Validation of the RobotiX mentor robotic surgery simulator. J Endourol 2016;30:338–346.
18. Harrison P, Raison N, Abe T, et al. The validation of a novel robot-assisted radical prostatectomy virtual reality module. J Surg Educ 2018;75:758–766.
19. Sethi AS, Peine WJ, Mohammadi Y, et al. Validation of a novel virtual reality robotic simulator. J Endourol 2009;23:503–508.
20. Bjerrum F, Sorensen JL, Konge L, et al. Randomized trial to examine procedure-to-procedure transfer in laparoscopic simulator training. Br J Surg 2016;103:44–50.
21. Maruthappu M, Gilbert BJ, El-Harasis MA, et al. The influence of volume and experience on individual surgical performance: A systematic review. Ann Surg 2015;261:642–647.
22. Downing SM, Yudkowsk R. Validity and its threats. In: Yudkowsky R, Downing SM, eds. Assessment in Health Professions Education. London: Routledge Taylor & Francis Group, 2009, pp. 21–55.
23. Borgersen NJ, Naur TMH, Sørensen SMD, et al. Gathering validity evidence for surgical simulation: A systematic review. Ann Surg 2018;267:1063–1068.
24. Havemann MC, Dalsgaard T, Sørensen JL, et al. Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills. J Robot Surg 2019;13:99–106.
25. Goldenberg M, Lee JY. Surgical education, simulation, and simulators—Updating the concept of validity. Curr Urol Rep 2018;19:52.
26. Downing SM. Validity: On the meaningful interpretation of assessment data. Med Educ 2003;37:830–837.
27. Prebay ZJ, Peabody JO, Miller DC, et al. Video review for measuring and improving skill in urological surgery. Nat Rev Urol 2019;16:261–267.
28. White LW, Truong M, Lendvay TS, et al. Crowd-sourced assessment of technical skills: differentiating animate surgical skill through the wisdom of crowds. J Endourol 2015;29:1183–1188.
29. Kwong JC, Lee JY, Goldenberg MG. Understanding and assessing nontechnical skills in robotic urological surgery: A systematic review and synthesis of the validity evidence. J Surg Educ 2019;76:193–200.

Information & Authors

Information

Published In

cover image Journal of Endourology
Journal of Endourology
Volume 35Issue Number 8August 2021
Pages: 1265 - 1272
PubMed: 33530867

History

Published online: 12 August 2021
Published in print: August 2021
Published ahead of print: 10 March 2021
Published ahead of production: 2 February 2021

Permissions

Request permissions for this article.

Topics

Authors

Affiliations

Rikke Groth Olsen [email protected]
Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark.
Flemming Bjerrum
Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark.
Department of Surgery, Herlev/Gentofte Hospital, Herlev, Denmark.
Lars Konge
Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark.
Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.
Jan Viberg Jepsen
Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark.
Department of Urology, Herlev/Gentofte Hospital, Herlev, Denmark.
Nessn H. Azawi
Department of Surgery, Herlev/Gentofte Hospital, Herlev, Denmark.
Department of Urology, Zealand University Hospital, Roskilde, Denmark.
Sarah Hjartbro Bube
Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark.
Department of Urology, Zealand University Hospital, Roskilde, Denmark.

Notes

Address correspondence to: Rikke Groth Olsen, MD, Copenhagen Academy for Medical Education and Simulation (CAMES), Teilum, Blegdamsvej 9, Copenhagen 2100, Denmark [email protected]

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

Metrics & Citations

Metrics

Citations

Export citation

Select the format you want to export the citations of this publication.

View Options

Get Access

Access content

To read the fulltext, please use one of the options below to sign in or purchase access.

Society Access

If you are a member of a society that has access to this content please log in via your society website and then return to this publication.

Restore your content access

Enter your email address to restore your content access:

Note: This functionality works only for purchases done as a guest. If you already have an account, log in to access the content to which you are entitled.

View options

PDF/EPUB

View PDF/ePub

Full Text

View Full Text

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share on social media

Back to Top