Development and Assessment of a Simulation Model of Focused and Ongoing Professional Practice Evaluations

Since 2007, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) has required evaluation of practicing attending physicians by management to ensure that procedural skills and medical care are meeting the accepted standards of care as assessed by each individual institution. This assessment is known as the ongoing professional practice evaluation (OPPE). There are several methods by which this can be accomplished, including through chart review of clinician practice and by verbally assessing knowledge of the steps required to do a particular procedure. However, for infrequently performed or complicated procedures, these methods may not allow objective evaluation on a regular basis. Simulation using task trainers or manikin models offers an alternative method of objective evaluation in a standardized setting.

The goal of this study is to develop two simulated scenarios to assess physician skill in relatively uncommon procedures and compare the data obtained against the verbal assessment and chart review model previously used at our institution. Two raters will assess each practitioner's verbal and simulated procedures. They will also assess globally their confidence that the physician is capable of performing the procedure safely and correctly based on the verbal or simulated trial.

The investigators anticipate that using a simulated experience for assessment will increase the ability of raters to assess proficiency for the purposes of an OPPE, specifically by increasing the number of critical procedural components that can be objectively evaluated. The investigators hypothesize that there will not be a strong relationship between the scores obtained on the verbal assessment and the scores obtained on the task trainer assessment. The investigators anticipate that there will be a portion of study participants that do not meet a minimum passing standard and may require additional deliberate practice and further testing.

Completed

Looking for future studies?

Notify Me

Key information

Conditions

Competence Disease Attributes Emergencies Pathologic Processes Pathological Conditions, Signs and Symptoms

Sex eligibility

All sexes

Study type

Observational

About this study

Physician training has developed over recent years to include simulation-based learning as a method of both formative and summative assessment. Simulation based medical education (SBME) with deliberate practice has been shown to be superior to traditional clinical education with respect to procedural skills in physicians in training (McGaghie,W.C. 2011), and a similar concept with Mastery Learning has shown to increase resident skill in lumbar puncture placement. In addition, it has been shown that Simulation Based Mastery Learning (SBML) in central line placement reduces hospital wide rates of central line associated bacterial infections.

When used in continuing medical education (CME), simulation has been shown to be feasible, and to increase clinician confidence in patient management. A recent study of nephrology fellows and attendings in practice showed that both groups scored similarly on a pre-test examination, and that all fellows who completed mastery learning subsequently passed the checklist assessment at the competency level and rated the training session highly.

In other work, simulation has been used for ongoing professional practice assessments (OPPE), for example in psychiatry using online simulated patients and assessments as a way of identifying a smaller number of attendings requiring focused assessment in person). Using electronically collected clinical data, anesthesia cases were reviewed for deviation from the standard of care on several key pieces of documentation common to many anesthesia procedures. This method of direct chart assessment for OPPE may be possible in anesthesia, a field where many cases have similarities in procedure, but would not work for a field such as Emergency Medicine, where the type and acuity of patient encounters varies greatly from day to day.

The ability of Simulation to present a standardized, specific procedural need offers an opportunity to assess Attending Emergency Physicians on a variety of critical procedures in a concise way. The investigators' research aims to present one possible approach to this assessment.

Who can participate

Healthy volunteers accepted: Yes

Only the study team can determine whether someone qualifies for participation.

Inclusion criteria

Emergency Medicine physician

Exclusion criteria

Not Emergency Medicine physician

Treatment and study plan

Verbal versus procedural assessment

Other

Faculty were assessed on both verbal and simulated performance of procedures.

Primary outcomes

Performance on Procedural Checklist

Time frame: Immediate

A 29-item procedural checklist for Central Line and a 21-item procedural checklist for Lumbar puncture will be used to assess performance by clinicians on Central Venous Catheter insertion and Lumbar Puncture performance.

Secondary outcomes

Acceptance of simulated training and assessment format

Time frame: Immediate

A survey tool will be used to query participants on their preferred way of both practicing and being assessed on procedural skill. It will also assess whether participants feel inclined to complete a future simulated procedural training session.

Sponsors and collaborators

Lead sponsor

Rush University Medical Center

Other

Registry information

Acronym: OPPE16

Important dates

Study start: 2015
Primary completion: 2016
Study completion: 2016
First posted: Jul 25, 2017
Registry last updated: Jul 28, 2017

OpenTrials presents study information sourced from ClinicalTrials.gov. The official registry record should be consulted for the latest information.

View the official ClinicalTrials.gov record (opens in a new tab)

This listing is for discovery and informational purposes only. It is not medical advice, does not guarantee that a study is recruiting, and does not determine eligibility. Contact the study team and a qualified healthcare professional when considering participation.

Published trials that share one or more normalized conditions with this study.

Completed