Data Analysis for Continuous Improvement Template - MDE

The continuous improvement process is guided by data available. The depth to

which the data is analyzed depends on where in the process work is being done.

Below is a table with an overview of the steps and how they are linked to the

Record of Continuous Improvement (RCI), and Implementation Science (IS). Below

the table is further information on the process.

Coaching notes for the process:

• Presentation

• Use the “Data Analysis & Use for Continuous Improvement Template”

Power Point, and insert identified data.

• Time needed

• Allow 3 hours for steps 1-4 on the first day, and another 3 hours for steps

5-7 on a second day.

• Data

• Be sure to have data (charts and tables) easily readable for all

participants. Ideally, each participant will have their own set of data charts

and tables on which to take notes.

• Materials Needed

• Data charts for participants

• 3 large pieces of poster paper

• Write “Questions” on one, “Concerns” on one, and “Feelings” on

one.

1

• 3 pieces of paper for each participant. Have them write the words from

above on each of their sheets

• Post-It notes

• Recommended process

• Complete all steps with School Leadership Team, possibly with a few

additional key teachers.

• Utilize information from Leadership Team process, and then repeat steps 4-

7 with larger teacher group or all teachers.

• Alternate process #1

• Complete steps 1-4 with School Leadership Team, possibly with a few


• Utilize information from Leadership Team process, and complete steps 5-7

with larger teacher group or all teachers.

• Alternate process #2

• Complete all steps with School Leadership Team, possibly with a few


1

• Using school-level proficiency data, create the SMART goal for math and

reading.

2

Overview

• Schools should have an “All Students” goal that focuses on increasing proficiency

schoolwide for students enrolled the full academic year in the school.

• Be sure to refer to the “Proficiency Rate Change Expectations” document when

setting SMART goals.

Report(s)

• Location

• MDE Data Center > Data Reports and Analytics > Secured Reports >

School Improvement Assessment Reports. Start by selecting your district

and school.

• Detailed Selections

• Report Type: Proficiency Trends

• Characteristics: Enrolled Oct 1

• Data View

• Proficiency Rate Trends for Both Mathematics and Reading

• Focus

• Set schoolwide achievement goals based on trends

3

Setting the SMART Goal

• Review the “SMART Goal Format” document

• Utilizing the “SMART Goal Setting” template, set the smart goals for math and

reading

• Only use the first sheet titled “All Student Goal Setting.”

• This serves as your school’s “Elevator Speech,” or the big picture goal

communicated with your parents and community.

• This needs to be done before moving ahead in the data mining/analysis process.

• This is a quick process (should not take more than 20-30 minutes).

4

• Insert the information for the school and subject

5

• It is important that school leaders know their student populations and population distributions prior to doing deeper analysis.

• Identify the largest student groups to help understand where to focus time and effort.

• Using School Improvement Enrollment Reports from the Secure Reports on the MDE website, identify the largest student groups.

6

http://w1.education.state.mn.us/MDEAnalytics/DataTopic.jsp?TOPICID=62


• Identify the largest student groups to help understand where to focus time and effort.

• Using School Improvement Enrollment Reports from the Secure Reports on the MDE website, identify the largest student groups.

7


Ethnicity TrendsOverview• Gain understanding Enrollment of Ethnicity over time.

Report(s) • Location

• MDE Data Center > Data Reports and Analytics > Secured Reports > School Improvement Enrollment Reports


• Report Type: Achievement Level Trends• Characteristics: Enrolled Oct 1

• Data View

• Enrollment Trends by Ethnicity• Focus

• Identify any changes in ethnic enrollment distribution• Q1: Has any ethnic student group increased significantly in the last 5

years?

8

Special Population TrendsOverview

Report(s) • Gain understanding Enrollment of Special Populations over time.




• Report Type: Special Population Trends• Data View

• Enrollment Trends by 3 Special Populations: Free/Reduced Lunch, English Learners, Special Education

• Focus• Identify any changes in special population enrollment distributions• Q1: Has any special population group increased significantly in the last 5

years?

9

Special Population Proportions within Ethnic Student GroupsOverview




• Report Type: Special Populations Proportions within Ethnic Student groups

• Data View

• student group Proportions within Each of 3 Special Populations: Free/Reduced Lunch, English Learners, Special Education

• Focus• Identify any changes in student group distributions within each special

population • Q1: Is any ethnic student group represented in a special population at a

rate greater than enrollment?

10

Special Population Proportions within Ethnic Student GroupsOverview




• Report Type: Special Populations Proportions within Ethnic Student groups

• Data View

• student group Proportions within Each of 3 Special Populations: Free/Reduced Lunch, English Learners, Special Education

• Focus• Identify any changes in student group distributions within each special

population • Q1: Is any ethnic student group represented in a special population at a

rate greater than enrollment?

11

GenderOverviewReview Enrollment by gender to see if there have been significant changes, or if a gap exists.




• Report Type: Gender• Data View

• Gender• Focus

• Has either gender increased significantly in the last 5 years?• Is there a significant gap between the gender counts?

12


• Gauging the Gap: Four Perspectives

• Simple Gap Narrowing• Have absolute gaps between groups decreased over time?

• Progress for All• Have all groups of students gained over time?

• Gap Size• What is current size of gap between groups?

• Group Comparisons Across Jurisdictions• How does each group of students compare with similar group in

other jurisdictions (or state)?• How are your gaps closing?

• Using the SMART Goal Setting Template• Enter information up to 2014• Answer questions on sheets to identify which groups are of highest

concern/need.• Write Student Group/Achievement Gap SMART Goals.

13

No Notes

14

Purpose• Find content relationships to determine improvement focus.• Reviewing trend data at the school level is part of the Comprehensive Needs

Assessment on the Record of Continuous Improvement.

Using School Improvement Assessment Reports, review proficiency and achievement level trend data for all. Next review proficiency and achievement level trend data by the student groups identified above in step #2.

15


Proficiency Trends: All Students


• MDE Data Center > Data Reports and Analytics > Secured Reports > School Improvement Assessment Reports


• Report Type: Proficiency Trends• Characteristics: Enrolled Oct 1 and Different Student groups Represented in

School• Data View

• Proficiency Rate Trends for Both Mathematics and Reading By Student groups• Focus

• Identify content needing improvement attention, Mathematics or Reading or both

• Q1: Is a content declining while other is increasing?• Q2: Is a content performance inconsistent, i.e. not increasing consistently over

time?• Q3: Is one content area increasing at a rate that makes it difficult to achieve

2017 target?

16

Proficiency Trends: Student Groups*Duplicate this slide for all Student Groups identified in step #2




• Report Type: Proficiency Trends• Characteristics: Enrolled Oct 1 and Different Student groups Represented in School

• Data View

• Proficiency Rate Trends for Both Mathematics and Reading By Student groups• Focus

• Identify content needing improvement attention, Mathematics or Reading or both• Q1: Is a content declining while other is increasing?• Q2: Is a content performance inconsistent, i.e. not increasing consistently over

time?• Q3: Is one content area increasing at a rate that makes it difficult to achieve 2017

target?

17

Achievement Trends: All Students




• Report Type: Achievement Level Trends• Characteristics: Enrolled Oct 1 and Different Student groups Represented in School

• Data View

• Achievement Level Distribution Trends for Both Mathematics and Reading• Focus

• Identify content needing improvement attention, Mathematics or Reading or both• Q4: Is a content showing increasing proportions of students earning a Does Not

Meet the Standards achievement level while proficiency rates stay the same or are growing?

• Q5: Is a content showing decreasing or inconsistent proportions of students in the Exceeds the Standards achievement level while the proficiency rates are staying the same or increasing?

18

Achievement Trends: Student Groups*Duplicate this slide for all Student Groups identified in step #2




• Report Type: Achievement Level Trends• Characteristics: Enrolled Oct 1 and Different Student groups Represented in School

• Data View

• Achievement Level Distribution Trends for Both Mathematics and Reading• Focus

• Identify content needing improvement attention, Mathematics or Reading or both• Q4: Is a content showing increasing proportions of students earning a Does Not

Meet the Standards achievement level while proficiency rates stay the same or are growing?

• Q5: Is a content showing decreasing or inconsistent proportions of students in the Exceeds the Standards achievement level while the proficiency rates are staying the same or increasing?

19



20

Proficiency Trends by Contrasting Student Groups: Math*Duplicate this slide for all Student Groups identified in step #2


• MDE Data Center > Secured Reports > School Improvement Assessment Reports


• Report Type: Proficiency Trends• Characteristics: Enrolled Oct 1• Contrasting Student groups: Select appropriate comparison for each

question• Q1: Contrasting Race/Ethnicities• Q2: Free/Red Price Lunch vs. Non-Free/Red Price Lunch• Q3: Special Ed vs. Non-Special Ed• Q4: English Learners vs. Non-English Learners

• Data View

• Proficiency Rate Trends for 1 Content (Mathematics or Reading) for student group and Appropriate Counter Group(s)

• Focus• Determine if school has student group achievement gap in proficiency

21

ratesWhat are trends in differences of proficiency rates between:

• Q1: white students and each ethnic/racial student group represented in school?

• Q2: students not eligible for free/reduced lunch and students eligible for free/reduced lunch?

• Q3: students not eligible for special education services and students eligible for special education services?

• Q4: students not eligible for English learner services and students eligible for English learner services?

• Q5: Which student group has the largest persistent achievement gap?• Q6: Which achievement gap persistently impacts greatest number of

students being low performing?

21

Proficiency Trends by Contrasting Student Groups: Reading*Duplicate this slide for all Student Groups identified in step #2




• Report Type: Proficiency Trends• Characteristics: Enrolled Oct 1• Contrasting Student groups: Select appropriate comparison for each

question• Q1: Contrasting Race/Ethnicities• Q2: Free/Red Price Lunch vs. Non-Free/Red Price Lunch• Q3: Special Ed vs. Non-Special Ed• Q4: English Learners vs. Non-English Learners

• Data View

• Proficiency Rate Trends for 1 Content (Mathematics or Reading) for student group and Appropriate Counter Group(s)

• Focus• Determine if school has student group achievement gap in proficiency

22

ratesWhat are trends in differences of proficiency rates between:

• Q1: white students and each ethnic/racial student group represented in school?

• Q2: students not eligible for free/reduced lunch and students eligible for free/reduced lunch?

• Q3: students not eligible for special education services and students eligible for special education services?

• Q4: students not eligible for English learner services and students eligible for English learner services?

• Q5: Which student group has the largest persistent achievement gap?• Q6: Which achievement gap persistently impacts greatest number of

students being low performing?

22

Achievement Level Trends for Contrasting Student Groups: Math*Duplicate this slide for all Student Groups identified in step #2


• MDE Data Center > Secured Reports > School Improvement Assessment Reports• Detailed Selections

• Report Type: Achievement Level Trends• Characteristics: Enrolled Oct 1• Contrasting Student groups: Select appropriate comparison for each question

• Q1: Contrasting Race/Ethnicities• Q2: Free/Red Price Lunch vs. Non-Free/Red Price Lunch• Q3: Special Ed vs. Non-Special Ed• Q4: English Learners vs. Non-English Learners

• Data View

• Achievement Level Trends for 1 Content (Mathematics or Reading) for student group and Appropriate Counter Group(s)

• Focus• Determine if changes in distribution of achievement levels is comparable across

compared student groups• Q1: Is proportion of students earning Does Not Meet the Standards achievement level

23

about same for each compared student group?• Q2: Is proportion of students earning Exceeds the Standards achievement

level about same for each compared student group?

23

Achievement Level Trends for Contrasting Student Groups: Reading*Duplicate this slide for all Student Groups identified in step #2


• MDE Data Center > Secured Reports > School Improvement Assessment Reports• Detailed Selections

• Report Type: Achievement Level Trends• Characteristics: Enrolled Oct 1• Contrasting Student groups: Select appropriate comparison for each question

• Q1: Contrasting Race/Ethnicities• Q2: Free/Red Price Lunch vs. Non-Free/Red Price Lunch• Q3: Special Ed vs. Non-Special Ed• Q4: English Learners vs. Non-English Learners

• Data View

• Achievement Level Trends for 1 Content (Mathematics or Reading) for student group and Appropriate Counter Group(s)

• Focus• Determine if changes in distribution of achievement levels is comparable across

compared student groups• Q1: Is proportion of students earning Does Not Meet the Standards achievement level

24

about same for each compared student group?• Q2: Is proportion of students earning Exceeds the Standards achievement

level about same for each compared student group?

24

• Using School Improvement Assessment Reports, review proficiency and achievement level trend data for all. Review the trend data at grade level by the student groups identified in step #2 by grade level.

• Compare grade levels• Focus on student groups identified in step #2

25


Compare Grades: Proficiency Trends




• Report Type: Proficiency Trends• Characteristics: Enrolled Oct 1

• Data View

• Proficiency Rate Trends for 1 Content (Mathematics or Reading) for Selected Grades

• Focus• Determine if proficiency rates are increasing effectively for each grade• Q1: Are all grades showing steady increases? Or is one or more grades

showing inconsistent performance?• Q2: Is there a cohort effect occurring in grade performance?• Q3: Is a subsequent grade able to maintain same or greater proficiency

rate for a cohort?

26

Compare Grades: Proficiency Trends




• Report Type: Proficiency Trends• Characteristics: Enrolled Oct 1

• Data View

• Proficiency Rate Trends for 1 Content (Mathematics or Reading) for Selected Grades

• Focus• Determine if proficiency rates are increasing effectively for each grade• Q1: Are all grades showing steady increases? Or is one or more grades

showing inconsistent performance?• Q2: Is there a cohort effect occurring in grade performance?• Q3: Is a subsequent grade able to maintain same or greater proficiency

rate for a cohort?

27

Maintenance of Proficiency


• MDE Data Center > Secured Reports > Growth Detail Download• Data View

• Follow instructions on Growth Detail: Learn Its Power Power Point presentation showing how to manipulate and use the growth detail download report to analyze data and interpret growth.

• Use Slides 1-7, 15-17• Shows percentage of students proficient in the current year who were and

were not proficient in the previous year.

• Focus• Are students staying proficient over time? • Are students moving from not proficient to proficient?

28



Using School Improvement Assessment Reports, review proficiency and achievement level trend data for all.

29


Compare Grades: Achievement Level Trends*Duplicate this slide for all grades as needed





• Data View

• Achievement Level Trends for 1 Content (Mathematics or Reading) for Selected Grades

• Focus• Determine if changes in distribution of achievement levels is comparable

in subsequent grade following year• Q1: Is the distribution of students in achievement levels maintained or

improved as students move to subsequent grade?• Q2: Is one grade struggling more than other grades to maintain

achievement level distributions?

30

Compare Grades: Achievement Level Trends*Duplicate this slide for all grades as needed





• Data View

• Achievement Level Trends for 1 Content (Mathematics or Reading) for Selected Grades

• Focus• Determine if changes in distribution of achievement levels is comparable

in subsequent grade following year• Q1: Is the distribution of students in achievement levels maintained or

improved as students move to subsequent grade?• Q2: Is one grade struggling more than other grades to maintain

achievement level distributions?

31

Maintenance of Achievement Levels: Math



• Follow instructions on Growth Detail: Learn Its Power Point presentation showing how to manipulate and use the growth detail download report to analyze data and interpret growth.

• Use Slides 1-14• Shows percentage of students proficient in the current year who were and


• Focus• Are students remaining at same achievement level?• Are students improving at same rate across grades?

32

Maintenance of Achievement Levels: Reading




• Use Slides 1-14• Shows percentage of students proficient in the current year who were and


• Focus• Are students remaining at same achievement level?• Are students improving at same rate across grades?

33



Using School Improvement Assessment Reports, review growth data for all. Next review proficiency and achievement level trend data by the student groups identified above in step #2.

34


• Explain chart

35

• Explain chart

36

• Explain what to look for• Happiest about positive growth for students with below proficient scores in prior

year.• Happy about positive growth for students with proficient scores in prior year.• Unhappy about negative growth for students with below proficient scores in

prior year.• Very unhappy about negative growth for students with proficient scores in prior

year.• When would I use it?

• After doing proficient and achievement data analysis

37

Prior Year Score vs Current Year Growth z-score*Duplicate this slide for all grades as needed



• Utilize Math and Reading Growth Scatter Plot Templates• Template is in: Continuous Improvement Tools in the Data Analysis folder

• Complete the templates as per the instructions• View graphs by all, and by student groups identified in step #2

• Filter for Oct 1• Focus

• Distribution: Where are the scores concentrated?

38








39








40








41

Average Growth over Time


• MDE Data Center > Data Reports and Analytics > Multiple Measurement District Download


• District: select the district• School: select the school• Subject: math or reading• Rating Year: select the desired year (*repeat the download for each year)

• Data View

• Go to the “Growth” sheet within the downloaded spreadsheet and enter the growth z-score average for the year into the table for the graph.

• Focus• Q1: What is the trend for growth?• Q2: Is a content declining while other is increasing?• Q3: Is a content performance inconsistent, i.e. not increasing consistently

over time?• TO ENTER INTO GRAPH

• Right+click on graph

42

• Select “Edit Data”• Place data in spreadsheet

42

Average Growth over Time: by grade


• MDE Data Center > Secure Reports > Growth Detail Download• Detailed Selections

• District: select the district• School: select the school• Subject: math or reading• Rating Year: select the desired year (*repeat the download for each year)

• Data View

• Make the data on the sheet a table• Filter

• Oct 1 in school: yes• Use In Growth: yes• Grade: each separately

• Calculate the average growth z-score for each grade and place into the date table for the chart

• Focus• Q1: What is the trend for growth?• Q2: Is a content declining while other is increasing?• Q3: Is a content performance inconsistent, i.e. not increasing consistently over time?

43

Average Growth over Time: by grade

Report(s)

• Location

• MDE Data Center > Secure Reports > Growth Detail Download


• District: select the district

• School: select the school

• Subject: math or reading

• Rating Year: select the desired year (*repeat the download for each year)

• Data View

• Make the data on the sheet a table

• Filter

• Oct 1 in school: yes

• Use In Growth: yes

• Grade: each separately

• Calculate the average growth z-score for each grade and

place into the date table for the chart

• Focus

• Q1: What is the trend for growth?

• Q2: Is a content declining while other is increasing?

• Q3: Is a content performance inconsistent, i.e. not increasing consistently

over time?

44

% of Students Making Expected Growth in Math & Reading




• Use Slides 1-8• Paste the Pivot table for math and reading into this slide.

• Focus• What % of students are making expected growth• What is the count of students for each?

45

% of Students Making Expected Growth in Math & Reading by grade level*Duplicate this slide for all grades as needed*Duplicate this slide for student groups as wanted




• Use Slides 1-8• Paste the Pivot table for math and reading into this slide.

• Focus• What % of students are making expected growth• What is the count of students for each?

46

Answer questions• Have participants post their questions• Go through questions and find answers (where possible)

Prioritize concerns• Have participants post their top 3 concerns• Review as a groups and prioritize top 3-5• What does the data suggest?

• Is student performance in one content area weaker than another? • Is achievement gap for a group and its counter group significantly larger than

another? • Is size of a group significantly larger than another, making it more important to

change?

47

No Notes

48

Description and/or Use• Using one of the tools, explore identified instructional problem(s) to determine cause.• A Root Cause is an early controllable factor in a chain of factors which impacts student

learning.• It identifies a relationship between a factor (the cause) and an outcome (the effect; i.e.,

student learning), where the outcome is understood as a consequence of the factor.

Tools [Choose the one(s) that will work best for the group]• Data Driven Dialogue: Inferences • Hypothesis Tool• Five Why’s• Fishbone

49

Data Analysis for Continuous Improvement Template - MDE

Documents

Transcript of Data Analysis for Continuous Improvement Template - MDE