AutoSlide Reviews

Curva de aprendizado na cirurgia de catarata do residente de oftalmologia.

INTRODUCTION Cataract surgery is one of the most frequently performed surgical procedures in the world; therefore, competence in this operation is a public health necessity. It is estimated that 50 million Americans will develop cataracts by 2050, and while the demand for competent surgeons intensifies, the quality of training has stagnated and been put at risk by reduced training opportunities and diminishing financial resources.1-3 Concerning reports have drawn attention to the fact that some graduating ophthalmology residents are not ready to operate safely and independently, and almost two-thirds of residents have indicated that they would have benefited from additional surgical training.4-7 Moreover, a recent survey of program directors found that almost 1 in 10 residents were designated as having significant problems in developing their surgical skills.8 Evidently, there is room for improvement, and one avenue for enhancing surgical skills training involves developing a standardized set of metrics that can accurately and precisely quantify skill level to ensure the highest level of patient safety and quality of care. While several tools have been developed to evaluate cataract surgery competency in the form of rating scales and checklists, many rely on subjective rater opinions and require considerable time to complete, limiting their incorporation within training curricula.9,10 The objective of this study was to track the operative phases of cataract surgery performed by a resident throughout their training, with the goal of quantitatively measuring competence and skill progression through time. This significantly augments understanding of the early cataract surgery learning curve and establishes a learning curve beyond the typical case volumes for US programs. 82 © 2023 ELSEVIER INC. ALL RIGHTS RESERVED. 0002-9394/$36.00 https://doi.org/10.1016/j.ajo.2022.12.006 METHODS • DATA SOURCE: This study retrospectively collected 100 cataract surgery video recordings performed by a single resident (n = 1) throughout their first year of surgical training from October 5, 2021 to November 3, 2022. These videos were captured in operating rooms at the Kensington Eye Institute, University of Toronto, Ontario, Canada. In total, 28 different faculty surgeons supervised the resident’s surgeries across all recordings. Only full-length recordings of adequate video quality (ie, centering on the eye, no motion artifacts, reasonable lighting conditions, no obscured view by the surgeon’s hand or other instruments) and without supervisor intervention were included. Each video was manually trimmed to begin prior to the first surgical action and end immediately after the last action. Overall, 20.7 hours of video (4.5 million frames) with a resolution of 1080 × 1920 pixels were collected. This study was approved by the Kensington Eye Institute, Trillium Health Partners, and University of Toronto Institutional Review Boards(RIS protocol number: 41734). These videos were originally captured for training purposes and did not reveal identifying patient features; therefore, informed consent was not necessary. • VIDEO ACTION ANNOTATION: Eighteen distinct operative phases of cataract surgery were explored in this study, in addition to an idle phase representing periods of inactivity between other actions: (1) side port incision; (2) intracameral anesthetic injection; (3) ophthalmic viscosurgical device (OVD) injection; (4) trypan blue stain; (5) main incision;(6) capsulorrhexis(cystotome);(7) capsulorrhexis (forceps); (8) OVD burp; (9) hydrodissection; (10) phacoemulsification; (11) cortical removal; (12) intraocular lens (IOL) insertion; (13) IOL repositioning; (14) OVD removal; (15) hydrated wound closure; (16) basic salt solution (BSS) irrigation; (17) weck-cel drying; (18) intraocular pressure (IOP) eye-tapping assessment; and (19) idle. A trained annotator (M.B.) manually labelled the start and end times of each of the phases in each video using ELAN version 6.1, a well-established, professional annotation software tool that has been used to create other benchmark annotated video datasets.11,12 For most actions, start and end times were determined by the entrance and exit of the surgical instrument into the eye, respectively. Actions that do not involve access within the eye (including OVD burp, weck-cel drying, and IOP eye-tapping assessments) were instead annotated while the action was being performed. For example, video frames were only labeled with the “weck-cel” phase while it was actively drying the eye, and not if it was idle on the screen. All attempted actions were annotated, whether successful or not, as distinct phases. If there was a >1 second difference between 2 action phases, an “idle” phase would be inserted between them to represent a state of inactivity; otherwise, the 2 action annotations were merged. • ANNOTATION VALIDATION: To ensure that all videos were manually annotated correctly and accurately, software was developed to automatically flag potentially erroneous annotations, either due to a suspicious order of actions (eg, a main incision following phacoemulsification) or atypical action times (eg, a 2-minute side incision). All flagged actions across all videos were then independently verified by the annotator (M.B.) in addition to 2 other study investigators (J.M.K. and A.M.). The rules governing this validation software are available in Appendix 1. • STATISTICAL ANALYSIS: Action phase times were stratified by video number, case number, and date. Operative times were reported as averages, ranges, and interquartile ranges (IQR). The timings for actions performed more than once in each case were summed together for a total time per video. Scatterplots were generated for each action with a nonlinear trend line using the non-parametric locally estimated scatterplot smoothing (LOESS) regression method.13 All analyses were conducted using R version 3.5.0 (R Foundation for Statistical Computing) with an a priori specified significance level of P = .05 (two-tailed). To detect whether there were any trends in operative timing for any of the action phases throughout the resident’s training, the Mann-Kendall trend test was used. This nonparametric test is a version of the Kendall rank correlation coefficient that statistically assesses for consistently increasing or decreasing (monotonic) trends that may be linear or nonlinear.14,15 To measure the magnitude of any trends, the slope for operative timing along with corresponding confidence intervals were computed using the Theil-Sen estimator.16,17 Also referred to as Sen’s Slope, it is one of the most well-established non-parametric methods for estimating a linear trend by generating lines through all pairs of points and choosing the median of their slopes. Unlike the least squares estimate, Sen’s Slope is insensitive to outliers and robust to values that do not fit a linear trend. To develop a better understanding of trends in operative timing throughout training, it was intended to identify at which stage(s) in the timeseries there were abrupt or significant changes in these trends (referred to as changepoints). These change-points may signal transitions that occurred between states of operative competence. Pettitt’s method was used to detect the point of maximal change for each of the operative actions. It is a rank-based nonparametric method that uses the Mann-Whitney statistic to test whether data before and after a single point come from the same distribution and chooses the change-point that maximizes the statistic over the entire timeseries.18 Since there were multiple comparisons being applied (1 for each of the action phases), the Bonferroni correction was applied to all statistical tests. VOL. 249 THE CATARACT SURGERY LEARNING CURVE 83 FIGURE 1. Total operative time for each of the 100 cataract surgeries included in this dataset by the resident throughout their first year of training, with corresponding case number and date. The locally estimated scatterplot smoothing (LOESS) curve with surrounding 95% confidence intervals (the shaded region surrounding the curve) depicts this resident’s overall learning curve. RESULTS The final dataset comprised 100 video recordings that were spread throughout the resident’s sixth and 760th cases that were performed completely independently in their first year of training. The median total time for each video was 11.6 minutes (IQR 10.1-14.4 minutes), with the shortest taking 7.4 minutes and the longest taking 26.0 minutes. Using the Theil-Sen estimator, the overall operative speed increased at a rate of 43.4 seconds for every 10 sequential videos(95% CI 35.1, 52.7 seconds), representing a 7.2 minute reduction in time from the trainee’s first included cases to their last (Figure 1). Nine of the 19 labelled action annotations demonstrated statistically significant downward trends in operative time across the resident’s training after Bonferroni correction using both the Mann-Kendall test and Theil-Sen estimator (Figure 2). The main incision (8.2%), phacoemulsification (7.8%), and hydrodissection (6.0%) phases had the greatest increases in speed throughout training relative to their average procedural time (Table 1). Phases that did not exhibit a significant reduction in timing were those that were typically performed quickly and/or many times throughout each case, such as hydrated wound closures, intraocular pressure eye-tapping assessments, or weck-cel drying. Of the 9 significant action trends, 8 had change-points that reached statistical significance using Pettitt’s test after Bonferroni correction (Supplemental Table 1). These maximal change-points may represent periods in the resident’s training where they manifest breakthrough advances in surgical proficiency. A timeline of when these significant change-points occur is shown in Figure 3, where the resident had performed between 100 to 350 cases. Of note, the maximal change-point in overall operative timing did not occur until the resident’s 300th case. There were an average 26.9 distinct operative actions (excluding idle periods) in each video throughout training (range 20-50). The number of distinct action phases appeared to steadily decrease with every quarter of training for most actions and corresponded with operative timing (Table 2). Capsulorrhexis (forceps), IOL insertion, and the main incision all began with multiple attempts to achieve success until the final quarter of training (starting with case number 416), at which point the procedures were consistently performed successfully in a single attempt. Moreover, by the final quarter of training the resident was comfort84 AMERICAN JOURNAL OF OPHTHALMOLOGY MAY 2023 TABLE 1. Learning Curve Trends Action Frequency Average Time (s) Time Range (s) Theil-Sen Estimator Slope (per 10 videos) Theil-Sen 95% CI Mann-Kendall P Value∗ Overall N/A 745.3 441.7-1562.3 –43.4 –52.7 to –35.1 .00 Anesthetic 88% 6.65 1.80-22.13 –0.23 –0.46 to 0.00 .90 BSS irrigation 27% 14.96 1.46-55.99 –3.48 –6.16 to 0.84 1.00 Capsulorrhexis (cystotome) 93% 19.2 9.33-76.53 –1.15 –1.53 to –0.78 .00 Capsulorrhexis (forceps) 100% 50.25 27.34-161.72 –2.5 –3.33 to –1.70 .00 Cortical removal 100% 88.46 31.46-395.00 –3.7 –6.13 to –1.61 .01 Hydrodissection 100% 27.81 9.34-88.00 –1.67 –2.41 to –1.01 .00 Idle 100% 212.04 119.19-382.22 –10.4 –13.25 to –7.49 .00 IOL insertion 100% 17.42 8.06-40.53 –0.27 –0.53 to 0.00 .97 IOL repositioning 99% 17.88 3.40-109.00 –0.84 –1.37 to –0.29 .06 IOP eye-tapping assessment 100% 7.66 1.47-37.67 –0.13 –0.43 to 0.10 1.00 Main incision 100% 9.42 4.20-25.65 –0.77 –0.90 to –0.66 .00 OVD burp 76% 2.97 0.60-12.54 –0.13 –0.28 to 0.00 .69 OVD injection 100% 24.68 15.26-58.74 –0.67 –1.06 to –0.32 .00 OVD removal 100% 30.92 17.00-82.26 0.11 –0.48 to 0.70 1.00 Phacoemulsification 100% 203.23 75.34-459.40 –15.87 –20.01 to –11.94 .00 Side port incision 100% 4.72 2.60-20.80 –0.21 –0.27 to –0.14 .00 Trypan blue stain 13% 6.96 2.73-24.80 –5.09 –10.83 to –1.97 .08 Weck-cel drying 73% 6.67 0.67-44.54 –0.24 –0.54 to 0.03 1.00 Wound closure 100% 15.18 3.94-35.75 0.2 –0.13 to 0.51 1.00 Actions with statistically significant trends are bolded. BSS = basic salt solution; IOL = intraocular lens; OVD = ophthalmic viscosurgical device. ∗ Bonferroni-adjusted P-values TABLE 2. Operative Actions Stratified by Training Quarter Average Number of Distinct Actions Phases (SD) Action Videos: 1 to 25 Videos: 26 to 50 Videos: 51 to 75 Videos: 76 to 100 Overall 55.0 (9.5) 55.0 (12.3) 51.5 (6.2) 50.3 (9.4) Anesthetic 1.2 (0.6) 1.0 (0.2) 0.7 (0.5) 0.9 (0.3) BSS irrigation 0.8 (0.9) 0.4 (1.0) 0.3 (0.6) 0.3 (0.7) Capsulorrhexis (cystotome) 1.0 (0.0) 1.0 (0.3) 1.1 (0.3) 0.8 (0.4) Capsulorrhexis (forceps) 1.6 (1.6) 1.3 (0.9) 1.3 (0.7) 1.0 (0.2) Cortical removal 1.3 (0.6) 1.3 (0.7) 1.3 (0.8) 1.2 (0.4) Hydrodissection 2.8 (1.3) 2.6 (0.8) 2.2 (0.9) 1.7 (0.8) Idle 27.2 (4.7) 27.0 (5.9) 25.2 (3.1) 24.7 (4.7) IOL insertion 1.7 (0.5) 1.1 (0.4) 1.1 (0.3) 1.0 (0.0) IOL repositioning 1.7 (0.9) 1.7 (0.8) 1.4 (0.7) 1.6 (0.8) IOP eye-tapping assessment 2.3 (1.4) 2.9 (2.5) 2.4 (1.5) 2.9 (1.7) Main incision 2.0 (0.8) 1.9 (1.0) 1.1 (0.3) 1.0 (0.2) OVD burp 0.7 (0.8) 0.8 (0.4) 1.0 (0.2) 0.8 (0.4) OVD injection 2.5 (0.8) 2.5 (0.7) 2.1 (0.3) 2.2 (0.6) OVD removal 1.0 (0.2) 1.0 (0.0) 1.1 (0.3) 1.2 (0.5) Phacoemulsification 1.1 (0.3) 1.2 (0.4) 1.1 (0.4) 1.2 (0.5) Side incision 1.2 (0.6) 1.0 (0.2) 1.0 (0.2) 1.0 (0.2) Trypan blue stain 0.4 (0.6) 0.1 (0.3) 0.1 (0.3) 0.0 (0.2) Weck-cel drying 0.9 (0.9) 1.5 (1.6) 1.0 (0.8) 1.5 (1.4) Wound closure 4.2 (1.4) 4.6 (1.8) 5.9 (1.7) 5.2 (1.8) BSS = basic salt solution; IOL = intraocular lens; OVD = ophthalmic viscosurgical device. VOL. 249 THE CATARACT SURGERY LEARNING CURVE 85 FIGURE 2. Cumulative operative time per case for each of the action phases that demonstrated statistically significant changes in timing across all 100 cataract surgeries included in this dataset. The locally estimated scatterplot smoothing (LOESS) curves with surrounding 95% confidence intervals (the shaded regions surrounding the curves) illustrates the learning curve for each of these actions. able enough with capsulorrhexis that the cystotome was used less than once per video, on average (ie, in some instances they were only using forceps). Other actions like hydrated wound closure, OVD removal, and weck-cel drying were used more often as training progressed. Scatterplots with associated trend lines are shown for all operative action phases in Appendix 2, along with summary statistics and results of the statistical tests (without Bonferroni correction). DISCUSSION The concept of a learning curve was first described in 1936, when aeronautical engineer T. P. Wright hypothesized that as the experience of the manufacturing workforce would grow, so too would the efficiency of airplane component production, driving down the costs.19 This idea has since been adopted in medicine and surgery – among other fields – with operative performance being a function of experience and exhibiting an initial period of difficulty followed by rapid improvement and eventual stabilization.20-22 The current study presented the change in timing for each operative action across training in the form of learning curves because they are simple and intuitive representations of skill progression; they can help derive a threshold at which competency is achieved; and they can function as comparators between individuals or between procedural/educational interventions in future studies. This study found that operative timing for many procedural actions, in addition to overall operative time, signif86 AMERICAN JOURNAL OF OPHTHALMOLOGY MAY 2023 FIGURE 3. Timeline of statistically significant maximal change-points using Pettitt’s method across the resident’s first year of training, stratified by month, video range, and case range. The maximal change-points suggest that the resident’s greatest leap in surgical proficiency occurred halfway through their training, at approximately the 300-case mark. icantly improved across a resident’s first year of independently performed cataract surgeries. The resident’s learning curve has yet to completely reach a plateau, indicating that there is further quantifiable room for improvement beyond 1 year of training and 760 cataract cases when measuring operative timing. The action phases of cataract surgery that demonstrated statistically significant changes in time throughout training for this resident included: side port incision, OVD injection, main incision, capsulorrhexis (cystotome), capsulorrhexis (forceps), hydrodissection, phacoemulsification, cortical removal, and idle time. Operative timing decreased for all actions over the course of the resident’s training. Interestingly, the shapes and trajectories of the learning curves differed depending on the operative action. For example, phacoemulsification demonstrated an exponential decrease in time until cases 80 to 100, at which point it leveled off and eventually plateaued by case 350. This is in keeping with a previous study by Randleman and associates23 that measured a phacoemulsification learning curve and found that phacoemulsification efficiency (total time multiplied by energy) significantly improved after 80 cases and continued to decrease throughout residency training. Conversely, the total idle time per surgery in the current study declined gradually, with significant variability between cases until case 350, at which point it precipitously began to drop. These different inflection points may signal different kinds of advances in skill, such as achieving basic competency after 80 cases and improving surgical efficiency after 350 cases. Phacoemulsification and capsulorrhexis have been rated by trainees as the most difficult actions to perform in cataract surgery24, yet they both demonstrated an inflection point after approximately 80 cases in the current learning curve, providing further support for the Accreditation Council for Graduate Medical Education (ACGME)-mandated minimum of 86 cataract procedures performed during residency.25 Of the remaining action phases that did not show a statistically significant change, none increased in operative timing by the end of training. The IOL insertion and repositioning, for example, remained relatively constant throughout. Other actions—like anesthetic administration, OVD removal, and hydrated wound closure— depicted sinusoidal-shaped learning curves whereby speed improved for the first 80 cases, then declined back to original levels until approximately case 350, where they improved again. BSS irrigation and trypan blue staining were not performed frequently enough to derive any meaningful conclusions, although both seemed to also show some improvement in timing after the first 80 cases. Previous related studies have generated learning curves for the phacoemulsification step or for cataract surgery as a whole. Many of these studies derived curves using intraoperative complication rates and found competency thresholds ranging from 60 to 100 cases, similar to the 86-case minimum mandated by the ACGME.23,26-30 Liebman and associates31 noted a progressive decrease in total operative time for the first 40 cases, followed by a gradual decrease until case 150, after which there were no statistical improvements in operative time. Wiggins and Warner32 and Taravella and associates33 found significant decreases in total time by the 86th and 75th cases, respectively, followed by smaller decreases after that, in keeping with the aforementioned competency thresholds. Although operative timing from these studies may be inaccurate due to a reliance on chart reviews or inclusion of collagen shield placement and eye patching in timing data, the current study reported simiVOL. 249 THE CATARACT SURGERY LEARNING CURVE 87 lar results, with operative timing inflection points at around 80 to 100 cases for phacoemulsification and overall cataract surgery. However, as previously discussed, many other actions do not follow this pattern and have distinct learning curves with inflection points at different stages of training. Moreover, Pettitt’s test for maximal change-point detection revealed that the shift in central tendency of the timeseries occurred halfway through the first year of training, at approximately 300 cases. Taken together, these results indicate that a basic level of surgical competency was likely achieved after 80 to 100 cases, although there were also important change-points occurring later in training after 300 cases that represented even greater improvements in skill and signaled the beginning of surgical finesse and efficiency. It is believed that this is the first study to derive cataract surgery learning curves across all relevant operative actions. This annotated dataset is the most comprehensive in existing literature, encompassing 19 distinct operative action phases labeled at the frame-level across 100 full-length video recordings. Furthermore, validation software was developed to flag potentially incorrect annotations and ensure the dataset is completely accurate and reliable. Future work will involve additional residents to compare learning curves on an action-by-action basis, especially given that previous studies have noted heterogeneity in the progression of individual residents’ operative times.34 Although video recordings of cataract surgery are routinely captured to help with training, it is unfeasible to manually derive learning curves for each trainee as a component of surgical evaluation in real-world practice. As such, it is also planned to develop deep learning models that can automatically segment videos into their constituent surgical phases, significantly reducing the intense time requirements of manual annotation. This is a novel field that has been rapidly gaining interest, and although recent studies have shown promising results, their applicability remains limited for now.35-38 This study had several limitations, the most notable being that operative timing and action frequencies alone are insufficient to fully characterize surgical skill. A host of other factors will be important to consider in future studies, both from the surgical standpoint – such as intraoperative judgment and instrument motion paths – as well as the patient safety standpoint, including outcomes like perioperative complications. However, previous studies have shown that speed does correlate to competence, and that compared with novices, expert surgeons perform significantly fewer surgical movements in less time and with a shorter path length in cataract and other intraocular procedures.39,40 Another issue with this study was that patient characteristics—such as age, nuclear density, past ocular history, or history of previous intraocular surgery—were unavailable, and thus a marker of case complexity could not be defined. With increasing experience, the resident likely operated on more difficult cases, which may have affected the trajectory of the learning curve. However, as with any training program, there are higher expectations with higher skill levels, and so the learning curve in this study represents the real-world progression of both a resident’s surgical proficiency and responsibility. Finally, the data in this study were restricted to a single resident at a single residency program. As such, the program’s pedagogical approach may have partially influenced the learning curve trajectory, and the resident’s particular skillset and rate of learning may not generalize to others. However, there were 28 different faculty surgeons supervising the resident-led surgeries, representing almost a third of all cases in this dataset, and so it is unlikely that supervisor procedural preferences or teaching styles significantly confounded results. Nonetheless, there is likely to be significant variability among trainee learning curves, and the results presented herein may not apply to others. Future studies with larger and more diverse samples will be needed to generalize results and explore possible archetypes of learning. To conclude, this is the first study to derive cataract surgery learning curves for all relevant operative actions. The procedural speeds, action frequencies, and learning curve inflection points discovered in this work may be used to inform trainees and their supervisors of the different types and rates of surgical skill development. The results show support for the notion that basic cataract surgery competency is achieved after approximately 80 cases, while also indicating that the start of surgical finesse and efficiency begins later in training after 300 cases. Future work will incorporate additional trainees and involve the development of algorithms that can automate the action phase annotation process used in this study.