Using Embedded Assessments to Measure Student Learning
OutcomesHarper College
Outcomes Assessment
IOAC Assessment Conference and Share Fair
March 8, 2013
Previous Workshops Why do we assess, and how do student learning outcomes help us assess?
How do we begin to develop student learning outcomes?
What are the characteristics of well-written outcomes?
Don’t worry if you don’t know the answers to these questions, just contact me after this session for more information.
Learning OutcomesUpon completion of this workshop, participants will be able to: Explain how embedded assessment can be beneficial when measuring student learning outcomes.
Identify what course activities might be used to measure specific student learning outcomes.
(Be able to) create an assessment plan using embedded assessments.
Today’s Agenda Background: Where are you on your assessment journey?
What are embedded assessments, and what are the advantages of using them?
Which course activities should be used for embedded assessment?
How are course activities linked to student learning outcomes?
(Create an assessment plan using embedded assessment techniques.)
What are embedded assessments?Embedded assessments… Measure outcomes through work your students already do in your course
May be a project, a set of questions included in a final exam, a final paper, or anything that is regularly used to evaluate the students in the class
What are the advantages of using embedded assessments? Uses items you are already asking your students to complete
Lessens faculty resistance; promotes positive dialogue and shared responsibility
Can be graded separately from the assessment evaluation itself, if faculty desire
Students generally understand the value (even if only to their own grades) of embedded assignments and take them seriously
What are the advantages of using embedded assessments? Impact on work loads minimized
Saves time because assignments will be (or are already) required for the course
Usually easier to coordinate and score than are new/outside assessments
What are the disadvantages of using embedded assessments? Coordination may be difficult, especially for larger departments, departments with large numbers of adjuncts, or departments that do not usually work together on individual courses.
May be seen as limiting faculty autonomy within the course.
Note: May still be easier and less intrusive than other forms of assessment.
Which of my course activities can be used for embedded assessment? Course-level
Questions embedded on the final exam Final essays/papers scored using a common rubric Projects scored using a common rubric Existing assignments or projects that all students in
the course complete Program-level
Capstone experience Internship evaluations Licensure exams Surveys or interviews (graduate, employer) Portfolio analysis Existing assignments or projects that all students in
the program complete …others you are thinking about?
Which of my course activities can be used for embedded assessment?
Nearly anything your students do in your course can be used for assessment For course-level outcomes, use an item near the end of the course
For program-level outcomes, use an item from a course that is near the end of the program (or after the program, i.e. externship evaluations)
Which of my course activities can be used for embedded assessment? If you are coordinating assessment across several sections of a course: Start by asking if there is an assignment or exam that all (or nearly all) instructors use
Even if the items are graded differently or have different questions, the department can find or create common pieces to use in assessment• Embed common exam questions• Use a rubric for assessment but allow instructors to continue grading using their own system, if desired
Where do I start? How do I link these assignments to the course outcomes?Please call the Assessment Office at any time you would like additional help, advice, etc.1. Get as many faculty involved in this
process as possible. 2. Review the course (or program) learning
outcomes (and ensure all faculty teaching the course have the same version of those outcomes)
3. Discuss what common items the faculty teaching the course/program already have students complete (final exam, paper, project). They do not have to be completely common. You
may simply find that all instructors do a paper at the end of the course, but each paper is different.
Where do I start? How do I link these assignments to the course outcomes?4. Determine which outcomes might be measured
using those common items. Do you have an outcome related to
communication? Perhaps you also have an assignment, presentation, or test question that can measure written or oral communication through a common rubric.
Note: if a common rubric is used and questions are similar, the prompts would not necessarily have to be the same for this item.
Do you have an outcome related to content knowledge? Embedding a set of questions on the final exam may work well, even if individual instructors maintain the majority of their own exam questions.
Where do I start? How do I link these assignments to the course outcomes?4. Determine which outcomes might be measured
using those common items. Do you have an outcome related to critical
thinking? Again, a rubric used to measure a common assignment or essay question may work well here.
In this case, the prompts may have to be the same, or at least share similar qualities, to elicit similar responses from the students.
Wouldn’t it be wonderful to measure more than one outcome through one item? You could use a single rubric to measure communication, critical thinking, and content knowledge, if you had an appropriate assignment or test question embedded in the course.
Once you have aligned assignments/test questions/performances with course outcomes
5. Revise prompts and questions as needed. Depending on the outcome and assessment
method, some items may need to be adjusted in order to ensure students in each section are receiving similar instructions.
Item should also count toward a similar percentage of the students’ grades (from section to section).
6. Develop a rubric or other common scoring mechanism. It will be helpful to train faculty members
on the use of the rubric. Remember, faculty members do not have to use the assessment rubric for their own grading.
Discuss the possibility of sharing the rubric with students. This improves their buy-in as well as their understanding of how the assignment is being assessed.
Once you have aligned assignments/test questions/performances with course outcomes
7. Schedule the assessment, with plenty of time to allow faculty to adjust their syllabi, assignments, exams, etc. where needed.
8. Assess, collect and record results, and discuss use of results.
9. Start the assessment cycle again, incorporating what you learned.
Please call the Assessment Office at any time you would like additional help, advice, etc.
Examples of Embedded Assessments at Harper Speech (Course-Level)
Did you see Margaret Bilos’s poster? Assessing persuasive speeches (already done in every section of Speech 101)
Take video recordings of students Faculty members randomly assess the speeches using a common rubric (with at least three evaluators per speech)
Examples of Embedded Assessments at Harper Art (Course-Level)
Faculty members are developing an assessment of multiple learning outcomes in a single course, based on an assignment they were already using
The assignment and rubric are being adjusted, but it is an joint process among faculty members who teach that class
The instructor for the course will assess using the rubric, then the assignment will go to a second reader (and a third, if necessary)
Examples of Embedded Assessments at Harper Math (Course-Level)
Embedding common questions on Math 103 final exam
Questions will be written to target learning outcomes, but the exam itself is an embedded part of every Math 103 course
Examples of Embedded Assessments at Harper Certified Nurse Assistant (Program-Level) Use results of competency exams that are already required by the state
Many other career programs use similar structures, when accreditors, licensing bodies, etc. require exams
Can delineate different areas that need additional support or attention, based on the results of the exams
Examples of Embedded Assessments at Harper Law Enforcement and Justice Administration (Program-Level) Use internship evaluations completed by local agencies
Good for increasing understanding of students’ capabilities outside the classroom
External evaluators may be more objective in their appraisal of student abilities and achievements
Examples of Embedded Assessments at Harper Fashion (Program-Level)
Evaluate capstone projects for both Design and Merchandising
Design students are evaluated through the Jury Show, which involves external evaluation
Can assess multiple outcomes through a single capstone project
Want to know more?
Full workshop will be offered through Professional Development (will include hands-on work with your course work and outcomes)
One-on-one, group, department, and division meetings are always available
Please direct all questions & comments toFaon GrandinettiManager, Outcomes Assessment(847) [email protected]