Use of Objective Structured Clinical

Examination in a Senior

Baccalaureate Nursing Course for

Assessment of End of Program

Outcomes

Diane Marcyjanik, MSN, EdS, RN

Nita Johnson, MSN, RN

Presentation Objectives

1. Apply the use of OSCE

in an undergraduate

curriculum.

2. Identify the potential

use of OSCE for end

of program outcome

assessment.

Definition of OSCE

An observed examination of clinical skills

in a controlled simulated environment,

with the utilization of structured

checklists, simulators and standardized

patients.

(Jones, Pegram & Fordham, 2010; Meechan,

Jones, & Valler, 2011; Paul, 2010)

Basic Principles• Objective structured

assessment of clinical

skills

• A series of stations

• Predefined competencies

• Use of simulators or

standardized patients

• Students move between

stations at timed

intervals

Role of the ExaminerOSCE examiner (faculty) is an observer.

• Examiner does not ask questions

• Only observe performance behaviors and mark

the standardized checklist accordingly

• Some stations may require students to answer

written questions without observer present

Why OSCE?

• Healthcare industry acknowledged

students not ready for clinical practice (Kardong-Edgren, 2013)

• Highly adaptable to a variety of

competencies (Bartfay, 2004)

• Needed objective method to assess

clinical skills prior to graduation

• Clinical skills not objectively assessed

after first semester junior year

SPRING ‘13 FALL 2013 SPRING 2014 FALL 2014 SPRING 2015 FALL 2015Proposal to UGCC

Determine station numbertypes & timing

Pilot 1 held Pilot 2 held Pilot 3 cancelled

OSCE added to curriculum

Determine assessment areas

Prepare rubrics & debriefing

Debriefparticipants

Debriefparticipants

Begin logistical implementation

Debriefparticipants

Determine related clinical skills

Prepare directions for participants

Revise stations Decision to cancel Pilot 3

Select models & examiners

IRB Approval

Begin logistical implementation

Revise directions & debriefing

Revise stations

Select cohort, models & examiners

Develop electronic rubric

Revise rubrics,debriefing &

Logistical implementation

Revise directions forparticipants

Select cohort, models & examiners

Pilot 1

Pilot 2

Pilot 3

Final OSCE

Pilot Study

• Purpose: Two pilot studies were

conducted to implement OSCE in second

semester senior year.

• Methodology: Qualitative descriptive

• Framework: Experiential Learning Theory

(Kolb,1984)

• Scenario: Holistic and cultural diversity

Description• Orientation

• Station 1- Knowledge: Ten point quiz on wound healing

• Station 2- Psychomotor skill: Perform dressing change

– (mannequin- with voice over- standard set of responses)

• Station 3- Patient teaching: Homecare instructions for dressing change (standardized patient)

• Debriefing

Criteria For Passing OSCE

• Station 1 = 10 % of Total score

• Station 2 = 70 % of Total score

• Station 3 = 20 % of Total score

Must meet all Critical Behaviors

& achieve > 85 % of total points

Implementation of OSCE

• Rubrics developed for stations

– Adapted from previous courses

– Expert review

– Pilot 1: paper rubrics

– Pilot 2: electronic rubrics

• Faculty orientation

• Student preparation

– Pilot 1 & 2: list of content areas for OSCE

– Pilot 2: added orientation power point

Data Collection

• Student Scores

– Individual stations and overall score

• Survey

– Faculty and students completed online evaluation survey immediately following the OSCE

• Focus groups

– Faculty and students participated in discussion around OSCE

Combined Pilot Results

Station 1-passed writen

exam

Station 2-passed perf.

skills

Station 3-passed pt.teaching

Passed OSCE

Points 9 6 7 0

Critical Points 2 12

0

2

4

6

8

10

12

14

16

(N=17)

na

Survey Results: Students

Did you prepare for the OSCE?N = 16 YES NO A LITTLE

3 8 5

How did you feel before starting

the OSCE?N = 16 NERVOUS CALM & CONFIDENT

12 4

Survey Results: Students

How do you feel now that the OSCE is over?

N = 16 BETTER WORSE NERVOUS SAME

11 3 1 1

Do you feel that an OSCE would be a

beneficial method of testing skills

competency in the undergraduate nursing

program?

N = 16 YES MIXED FEELINGS

13 3

Survey Results: Faculty OSCE Support• “I am in full support of adding an OSCE to

every semester of the nursing program.”

• “I approve of beginning with an end-of-program OSCE and moving down through Senior 1 and Junior 2 semesters.”

• “…identify the best course in which to situate the OSCE each semester.”

Survey Results: Faculty Faculty Concerns regarding OSCE• “I'm not sure we have the resources to give

students opportunity to re-take OSCE's until they pass them.”

Electronic OSCE Rubrics• Easy to use

• Liked ability to write comments

• Reset automatically

Recommendations

• OSCE in week 1-2 of second semester senior year

• Conduct OSCE over 1-2 days (maximum)

• Student orientation

• Faculty orientation

• Use of electronic directions, rubrics, and evaluation tools

• Score 15 % of course grade

• Remediation plan

Remediation Plan

• Develop individualized remediation plan with OSCE team

Level 1

1 missed Critical Behavior and > 80 % of total points

• Practice and review specific content

• Retake station within 1 week

Level 2

>1 missed Critical Behavior and/or < 80 % of total points

• Practice and review entire content

• Retake entire OSCE within 6 weeks

ReferencesAdamson, K. A., & Kardong-Edgren, S. (2012). A method and resources for assessing the reliability of simulation evaluation instruments.

Nursing Education Perspectives, 33(5), 334-339.

Bartfay, W. J., Rombough, R., Howse, E., Leblanc, R. (2004). The OSCE approach in nursing education. Canadian Nurse, 100 (3), 18-23.

Bensfield, L. A.,Olech, M. J., & Horsley, T. L. (2012). Simulation for high-stakes evaluation in nursing. Nurse Educator, 37(2), 71-74.

doi: 10.1097/NNE.0b013e3182461b8c

Bultas, M. W., Hassler, M., Ercole, P. M., & Rea, G. (2014). Effectiveness of high-fidelity simulation for pediatric staff nurse education. Pediatric Nursing, 40(1), 27-42.

Jones, A., Pegram, A., & Fordham-Clark, C. (2010). Developing and examining and objective structured clinical examination. Nurse Education Today, 30, 137-141. doi:10.1016/j.nedt.2009.06.014

Kardong-Edgren, S. (2013). A wake up call…with an objective structured clinical examination. Clinical Simulation in Nursing,

9 (1), e3-e4. Retrieved from www.nursingsimulation.org/article/S1876-1399 (12)00356-8

Kirkman, T. R. (2013). High fidelity simulation effectiveness in nursing students' transfer of learning. International Journal of Nursing Education Scholarship, 10(1), 1-6. doi:10.1515/ijnes-2012-0009

Kirton, S., & Kravitz, L. (2011). Objective structured clinical examinations (OSCEs) compared with traditional assessment methods. American Journal of Pharmaceutical Education, 75(6), 1-7.

Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall.

McWilliam, P. L., & Botwinski, C. A. (2012). Identifying strengths and weaknesses in the utilization of objective structured clinical examination (OSCE) in a nursing program. Nursing Education Perspectives, 33(1), 35-39. doi:10.5480/1536-5026-33.1.35

Meechan, R., Jones, H., & Valler-Jones, T. (2011). Students' perspectives on their skills acquisition and confidence. British Journal of Nursing, 20(7), 445-450.

Paul, F. (2010). An exploration of student nurses’ thoughts and experiences of using a video-recording to assess their performance of cardiopulmonary resuscitation (CPR) during a mock objective structured clinical examination (OSCE). Nurse Education in Practice, 10, 285-290. doi:10.1016/j.nepr.2010.01.004

Diane Marcyjanik, MSN, EdS, RN

Clinical Assistant Professor

marcyjdl@uwec.edu

715-836-4012

Nita Johnson, MSN, RN

Skills/Simulation Lab Coordinator, Clinical Instructor

johnsnit @uwec.edu

715-836-5714