Procedure-Specific Resident Objective Modular Training Evaluation

• Purpose:

The purpose is to investigate whether procedure-specific skills acquired in a surgical lab result in improved OR surgical skills; to examine related outcomes of intra-operative events and resident satisfaction; to examine the feasibility of implementation of modular training in a gynecology residency program.

• Hypothesis: That the surgical training modules proposed will improve resident surgical skills in the operating room.
• Justification:

Increasing constraints are placed on the educational experience of young surgical trainees. Specifically in the area of Obstetrics and Gynecology (ObGyn), residents are expected to become skilled surgeons in addition to acquiring vast knowledge on a myriad of women's health topics. Operating room (OR) time is scarce. Medicolegal concerns can be a limitation to resident involvement. Financial incentives to support surgical mentorship are dwindling. With the rapid acceptance and advancement of minimally invasive surgical techniques in gynecology, there is a learning curve for the attending surgeons who must familiarize themselves with the techniques first and thus give fewer practice opportunities to residents.

Traditionally, acquisition of surgical technical skills happens through repetition in the OR, under the supervision of an experienced surgeon. Because of the above named constraints, surgical education of residents has been referred to as "education by random opportunity". Surgical educators have thus attempted to teach in other settings, remote from the OR. Options have included the use of suturing stations, animal and cadaver models, videotapes of procedures and laparoscopic simulators. Although the efficacy of these options is recognized, few ObGyn programs provide formal surgical curricula and technical skills are still taught mostly in the OR and through lectures. A survey of all the 266 ObGyn residency programs in the United States, published more than 10 years ago, showed that surgical teaching curricula were only used by 29% of the programs. A more recent survey conducted in Canada showed that, 10 years later, only 40% of Canadian ObGyn programs compared to 76% of other surgical residencies used a program-specific surgical training curriculum. Most Canadian residency program directors who did not have access to a standard training curriculum wished to have one implemented. Results of this survey are consistent with other Canadian studies showing variability in training and assessment of surgical skill across Canada. Without a formal curriculum, residents may graduate with widely varying surgical skills, currently not objectively evaluated as a requirement for graduation.

There are obvious advantages to a surgical training curriculum remote from the OR. Residents can practice basic and advanced skills as many times as necessary to achieve competency prior to performing surgeries on real patients. Patient safety is thus not compromised. Dedicated teaching faculty can provide immediate feedback. If training objectives are not met, mandatory retraining targeted to specific deficient areas can be undertaken on the same modules.

Many tools are available for designing an appropriate specialty-specific curriculum. A review of these tools concluded that based on the available evidence a successful curriculum would include didactic training and surgical anatomy, the practice of surgical skills on box trainers and animal models followed by consolidation of skills in the real OR). A Canadian curriculum with surgical training in various core surgical skills did not result in improvement in performance of actual surgical procedures on a cadaver assessment model. On the other hand, a laparoscopic training module in basic laparoscopic tasks has been shown to impart skills transferable to real OR General Surgery procedures. In ObGyn, transferability of skills to the OR has been demonstrated after instruction via procedure-specific modules for laparoscopic tubal ligation and episiotomy repair.

Our primary aim in the current study is to investigate whether instruction via procedure-specific surgical modules results in transferability of resident surgical skill to real OR urogynecologic procedures. Secondary aims include impact of this instruction on patient outcomes, resident case volume, need for remedial instruction, cost and feasibility as well as resident satisfaction and self-confidence. Our ultimate long-term goal is to disseminate our findings across Canada and hopefully encourage all ObGyn programs to adopt an evidence-based, competency-based, modular, standard training approach to training gynecologic surgeons.

• Objectives:

Primary: To investigate whether procedure-specific skills acquired in a surgical lab result in improved OR surgical skills as tested via a global rating scale on each of several index procedures.

Secondary:

• To investigate whether procedure-specific skills acquired in a surgical lab result in improved knowledge of procedural steps on each of several index procedures
• To investigate whether procedure-specific skills acquired in a surgical lab result in less OR time, fewer intraoperative complications and less blood loss on each of several index procedures
• To examine how the implementation of modular training affects resident case volume in the first two months after implementation.
• To determine the proportion of residents needing remedial training after module completion and examine the feasibility of retraining given constraints of time and cost
• To examine the cost of implementation of modular training in urogynecology procedures in a gynecology residency program.
• To examine resident satisfaction and self-confidence with operative skills after modular training
• Research Method:

Design This study is a randomized controlled trial of an intervention group, consisting of surgical training in specific urogynecologic procedures via modules versus a control group, consisting of usual training. Thirty ObGyn junior residents having performed less than five of each index procedure independently will be recruited to participate. Baseline demographics and number of index procedures performed independently prior to enrollment will be recorded. A baseline knowledge pretest consisting of multiple choice and short answer questions on technical aspects of the index procedures will be administered to all residents. The residents in the intervention group will then receive individual training. Each index procedure will be taught using a module. Each procedure-specific module consists of a didactic component, practice on suturing, knot tying and technical aspects of the procedure with individual feedback, and evaluation. Evaluation consists of a knowledge posttest and a practical examination of acquired surgical skills on procedure-specif practical models. If the resident fails the written or practical examinations, he/she will not be allowed to perform surgery in the real OR but will undergo mandatory retraining using the same modules. When considered competent by passing the written and practical examinations, the resident will then be allowed to independently perform corresponding index procedures in the real OR and will be evaluated via a validated global rating scale of surgical skill. The residents in the control group will undergo training as usual by individual reading and observation and practice in the OR. When allowed to independently perform one of the index procedures, their performance will be evaluated via a validated global rating scale of surgical skill. After the data collection is complete on the last index procedure, residents in the control group will be offered the same surgical training given to the residents in the intervention group. This will be done to ensure equal training opportunities for all residents.

Sample Size: We used the GRS developed by Reznick et al11 and previously used to compare gynaecology residents trained on an episiotomy repair module versus controls. Banks et al established a 21% significant difference in scores for the episiotomy repair module versus controls.17 As our index procedures are more advanced then episiotomy repair, we determined that 50 residents (25 per group) were required to find a 20 point overall difference (out of 100) in mean global rating scale (GRS) score between intervention and control residents, with a standard deviation of 25 points (to account for the variation in surgical skill of residents), 80% power and a significance level of 0.05.

• Statistical Analysis:

Statistical analysis

• Primary outcome: Student's t-test for Global Rating Scale score average on each index procedure in the OR - intervention residents versus controls
• Secondary outcomes: Student's t-test for procedural steps scale score average - intervention residents versus controls; Mann-Whitney test for average OR time (expressed in minutes) and estimated blood loss (expressed in milliliters) - intervention residents versus controls (results likely not normally distributed); descriptive statistics for intraoperative complications - intervention residents versus controls; student's t-test for number of cases performed in the OR in the 2 months following intervention - intervention residents versus controls; proportion of intervention residents requiring remedial training and cost thereof; cost of implementation of modular training; student's t-test for Self Confidence Scale score average and satisfaction score average on each index procedure in the OR - intervention residents versus controls; chi square test for differences in outcomes based on categorical variables of year of training and number of previously performed index procedures

Inclusion criteria

• An Obstetrics and Gynaecology residents/medical students/ fellow/junior staff in the UBC program General surgery residents Willing to participate Previous performance of less than 5 of each of the index procedures independently in the OR

Exclusion criteria

• Previous performance of 5 or more of each of the index procedures independently in the OR

For the cutoff score participants there is no exclusion criteria.