Succeeding at the Gateway: Online Algebra Effectiveness

UNF—FLVS—NACOL 2004-2005

Succeeding at the Gateway:

Online Algebra Effectiveness

Cathy Cavanaugh, University of North Florida

Kathy Jo Gillan, Florida Virtual SchoolWith NACOL


Study Authors

Dr. Cathy Cavanaugh Assistant Professor, Curriculum and Instruction, University of North Florida

Dr. Kathy Jo GillanManager of Research and Grants, Florida Virtual School

Dr. Jan Bosnick Associate Professor, Mathematics Education, University of North Florida

Dr. Melinda Hess Director, Center for Research, Evaluation, Assessment, and Measurement, University of South Florida

Heather Scott Research Associate, Center for Research, Evaluation, Assessment, and Measurement, University of South Florida


OverviewThis study focused on the quality of the online

Algebra I course at the Florida Virtual School. 1. How does algebra achievement of virtual

school students compare to that of public classroom-based students in the same state?

2. How effective is an interactive module for helping students learn graphing linear equations? The performance of students using the module was compared to the performance of students who did not use the new tools.


The Setting for the Study

The Florida Virtual School (FLVS) the setting for this study because of its long and successful history of offering high school courses to students online.

The school educates thousands of students each year from a wide diversity of geographic locations, socioeconomic groups, and school backgrounds. The sample of students participating in the study was seen as representative of virtual school students nationwide.

Because each course offered by FLVS is developed by certified and qualified school staff, there was flexibility to adapt and evaluate course components.


The Setting (2)

The Algebra course aligns with state and national standards, a factor that makes it similar to algebra courses offered in other settings.

The Algebra course has high enrollment at FLVS, and the instructors of the course regularly collect data on students’ performance in each module of the course.

Quality is a prime goal of FLVS, and it is enhanced through frequent course reviews and outside evaluations. Therefore the school administrators and staff had a strong interest in acquiring knowledge via this study that would have the potential to improve the quality of the algebra course.


Part 1. The Effectiveness of the Online Algebra Course

The algebra achievement of two groups of algebra students was measured using an established, reliable algebra exam. The groups were:

1. Students enrolled in online algebra courses developed by the Florida Virtual School

2. Students enrolled in classroom-based algebra courses taught in a Florida public school

All students were taught by Florida certified mathematics teachers following the state standards, and using state adopted materials.


Part 1. The Effectiveness of the Online Algebra Course (2)

Algebra achievement was measured when students had reached 70% completion of the course

The Assessment of Algebraic Understanding (AAU) published by Educational Testing Service (ETS) is a 50-item, multiple-choice test that was designed “for determining the extent to which students have mastered algebraic concepts. The items are aligned with the Algebra Standards as described by the National Council of Teachers of Mathematics (NCTM)” (ETS, 2004). The reliability coefficient (KR-20) for the 2003 form is 0.8655.

The test can be administered according to any schedule.



The FLVS students who had reached the same point in the algebra course as face-to-face classroom students (70% completion) by March 15 received the exam, answer form, and consent form by mail.

Students enrolled in classroom-based algebra courses taught in a Florida public school in a district served by FLVS also received the tests and consent forms during the same time period.

Tests and answer forms were returned by mail and scored electronically by project staff.



Students from three types of schools, virtual school students, virtual franchise students, and public classroom-based students, participated in the study.

Virtual school students

Franchise students

Public classroom students

Received exams

123 16 98

Returned exams

11 1 97



The differences in the variability of the scores may be due to:

• the larger, more variable sample of the classroom group

• the assessment being administered in a group setting within the classroom and as individuals in the online course

• the voluntary participation in the assessment of the online course

• the fact that the assessment did not count toward the course grade

• differences between the virtual students who did complete the assessment and those who did not complete the assessment.



The online algebra course appears to be as effective as public school classroom-based courses.

But:• Larger samples of students should be tested in

more schools• Data should be gathered on students, including

grade, gender, ethnicity, first time course taker.• The assessments should be administered

electronically.


Part 2. Interactive Tools for Graphing Linear Equations

This part of the study evaluated the effectiveness of interactive tools to provide students the skills they need to succeed in graphing linear equations.

Several existing tools were examined, and it was decided that new tools would best meet the needs of the school and the students. – The tools were embedded into the course’s

management system– The tools were created to address the specific course

competencies, scope, and sequence


Part 2. Interactive Tools for Graphing Linear Equations (2)

The tools were designed to address these objectives:– Determine the impact of changing the slope value of a

linear equation graphed in a coordinate plane. – Determine the impact of changing the y-intercept value of

a linear equation graphed in a coordinate plane.– Calculate the slope of a line given only the graph of that

line. – Calculate the slope of a line given two ordered pairs that

are on the line. – Apply their knowledge of the concept of slope to

determine the line orientation and the degree of slope given the equations of the lines or the numerical value of the slope.

– Transform the equation of a line from standard form to slope intercept form.



Tool 1 allows students to enter the coordinates of two points on a line, and provides the calculated answer and a graph of the line that indicates the rise and fall. It is designed to enhance conceptual connection between the calculation, the points on the line and the meaning of slope.



Tool 2 takes students through a developmental series of screen shots designed to aid students in building their connections between the equations of lines, solution sets of the equations, and the graphs of the lines.



Tool 3 allows students to input the m and b values of a linear equation in y = mx+b slope intercept form, and the a, b and c values of an equation in ax + by = c standard form. Values for perpendicular and horizontal lines can also be entered. The tool provides the graphs of the lines.



The two versions of the course were reviewed and aligned by the FLVS math specialist and a math education specialist.

The alignment study evaluated the degree of alignment among the instructional objectives, the lesson materials, and the items on the student assessment of skill in this module.

Two external reviewers rated the alignment between the module expectations and the module assessment.

The degree of equivalency between the version of the Algebra 1 module with interactive tools for graphing equations and the version without the tools was also rated by the outside reviewers.



Pre- and posttests containing 20 items on graphing linear equations were used as the performance measures for determining whether there was a difference between the two groups of students.

Between February 15 and April 15, all students entering the Algebra course module for graphing linear equations were given pre-tests. If students scored above 80%, they moved on to the next course module. Students scoring below 80% participated in the module instruction.

At the end of the module students took the post-test. Students who began the module during the first month did not use the newly developed interactive tools. Students beginning the module during the second month used the new tools.



101214161820

Pre PostTest

Mea

n

Without Graphing Tool With Graphing Tool

The results of the repeated measures ANOVA for the between subjects effects of Group were not statistically significant, F(1, 45) = 0.32, p = .57 > α = .05. This indicates that the observed average difference between the group without the graphing tools and the group with the graphing tools is not different enough to conclude that a difference between these groups exists in the population.



The higher means in the group without the graphing tools may be due to the sample being half the size of the group with the graphing tools or because those students were somehow different from the group who received the graphing tools.

These students may be more academically motivated as well as naturally higher achieving than those in the other group.


Conclusions

The online algebra course appears to be as effective as the classroom-based course, and the interactive interventions resulted in insignificantly larger gains scores for graphing linear equations.

One should be cautious when interpreting the data derived from this study due to the small sample size in both parts of the study and due to the inconsistent numbers of participants in the groups.


Discussion

For future research in K-12 online learning to efficiently improve practice and outcomes:

• Researchers need access to online educators and learners

• Schools and researchers need time to develop a well-planned program of inquiry

• Access requires a trusting relationship between researchers and school professionals and an understanding of the technical and ethical limitations of access to student data.

• Time generally results from adequate funding. • Researchers studying online learning must recognize the

unique attributes of the online learning ecosystem, and they must account for as much of the complexity of that ecosystem as possible in the research design.


Discussion (2)

National and state-level education policymakers seek data on effectiveness of educational approaches in terms of numbers and groups of students served, student success rates, and resources needed to implement the approaches effectively.

This study indicates that online algebra courses and interactive tools have the potential to reach any student with access to the necessary technology, and that the courses with these tools can be effective on a large scale.


Discussion (3)

Educational leaders at national, state, district, and school levels need efficiency and effectiveness data to understand what works, but in order to select and implement programs successfully they need the detailed information about how online learning approaches work well.

This study indicates that online algebra learning can work, but it points out the need for greater participation of schools in extended in-depth research projects to identify specific factors that maximize online algebra learning.


Discussion (4)

Teachers need detail related to instructional variables that enhance effectiveness of education approaches, and the skills for interpreting that data in a way that will translate to improvement in their own situation. In return teachers should consider opening their “classrooms” to researchers in order to continue the cycle of learning about learning.

Teachers who teach conceptually can take away from this study the idea that interactive manipulatives are effective for helping students develop abstract concepts.


Discussion (5)

Students, parents, and local community members want to feel confident that the time and financial contributions they make to education result in an effective, equitable, accessible, and efficient system.

Online algebra as offered by Florida Virtual School appears to meet those criteria, but more information is needed. It will be possible to acquire that information in an education system in which all stakeholders work in trusting and collaborative relationships to move the cycle forward.


Other Presentations from FLVS Staff!Monday, October 24th

• 1:00 – 2:00 A Student Perspective – A Panel of Online High School Students; Moderator: Julie Young, President and CEO, Florida Virtual School

• 2:15 – 3:15 Project-based Online Learning: Maximum Flexibility Speaker: David Glick, Consultant and FLVS representative

• Effective Communication Styles in the Virtual Classroom; Speakers: Jamie Nagle and Anna Coppola, FLVS Science Instructors

Tuesday, October 25th

• 10:00 – 11:00 Keeping Pace With Online Learning , President and CEO, Julie Young, will serve on this panel discussion.

• 2:30 – 3:30 Connecting with Parents, Speaker: Matt Vangalis, FLVS Math Instructor

• Effective Online Teacher Training: How to Develop and Sharpen Online Teaching Skills, Shearon Arnott, FLVS Training Manager serve on this panel discussion.


Visit FLVS at Booth #115

Pick up a special offer just for Virtual School

Symposium Attendees!

Succeeding at the Gateway: Online Algebra Effectiveness

Documents

Transcript of Succeeding at the Gateway: Online Algebra Effectiveness