Action Research Report - THE EFFICACY OF COMPASS LEARNING AS A TUTORIAL PROGRAM FOR 6TH GRADE MATH STUDENTS

The efficacy of Compass Learning as a tutorial program for 6^th grade math students

An Action Research Project

Presented to

The Faculty of the College of Graduate Studies

Lamar University

In Partial Fulfillment

of the Requirements for the Degree

Master of Education in Education Technology Leadership

by

Yvonne Stephanie Carrasco

December 2013

ABSTRACT

 The efficacy of Compass Learning as a tutorial program for 6^th grade math students

by

Yvonne Stephanie Carrasco

65 students were invited to participate in an afterschool Learning Lab.  Invitations were based on performance on the 2012 5^th Grade Math State of Texas Assessment of Academic Readiness (STAAR.)  Students participating in the study scored 70% or less on the STAAR.  At the time that the cohort was selected, yearly passing standards for 2012 had not been established.  41 students participated in the Learning Lab utilizing Compass Learning as an intervention tool.  Students met twice weekly for an hour each day and completed Pre and Post Tests directly related to the 6^th grade math curriculum in preparation for the 2013 6^th Grade Math STAAR.  78% of the students involved in the study successfully passed the state test after intervention.  Results are inconclusive as 92% of the students in the cohort that did not receive interventions were also successful in passing the 2013 6^th Grade Math STAAR.  Additional attention to student progress could impact student success, because although the Compass Learning software program is self-directed, students will still get off task and need to be refocused.  This review has increased the possibilities for data-driven intervention strategies for student success.

 The efficacy of Compass Learning as a tutorial program for 6^th grade math students

            This action research was completed to investigate the effectiveness of utilizing a software program, Compass Learning, through an after school tutorial session, to enhance the performance of 6^th grade math students at John Drugan School in the Socorro Independent School District, in El Paso, Texas.  The primary researcher was an 8^th grade science teacher with an interest in the Compass Learning Software Program that had been introduced as a district initiative for student success.  Teachers in the school district received training on the program and were told that there had been success in utilizing the program in other districts.  Very little data was provided to support claims and teachers were reluctant to adopt the program.  After discussion with the campus administrator and a number of teachers, it was determined that the group that could benefit from increased support would be the year’s current 6^th grade students.  This cohort of students had done poorly in math on the previous year’s high stake test, the State of Texas Assessment of Academic Readiness or STAAR. To complete this study, a group of students had to be selected and invited to participate in the tutorial sessions.  One of the biggest concerns about the sessions was student attendance.  There was concern that even with a strategy for assistance, that students would not attend and therefore would not receive additional support.

Background

            Attendance was anticipated to be a problem, and for a few students, it was.  65 students were invited to participate, but all students did not accept the invitation.  Of the 65 students that were invited, 24 students did not participate.  While 63% of the students invited participated, it was again an attendance issue with only 41% of students attending  75% or more of the tutoring sessions.  Another issue encountered was monitoring student progress.  Students were assigned to tasks based on need and students accelerated at their own pace.  Working with and tracking 41 students did lead to difficulty in tracking due to limited time available.  As this was a new program, there was also a learning curve that had to be overcome by both students and monitoring interns.  Students needed additional instruction on the process of pre-testing, reviewing material related to the pre-test, and then post-testing concepts.  Students were anxious to complete work and either rushed through pre-tests and cycled through pre-test after pre-test, or students rushed through pre-tests and were faced with many learning strands.  As the program continued, students were encouraged to slow down on pre-tests with the explanation that if a student scored well on the pre-test, there would be fewer review assignments later.  As students came to understand this, the dynamic in the tutoring session changed with more students using scratch paper and working through problems thoughtfully.

Problem Statement

            The proposed question is, “How can a Compass Learning Computer Lab support teachers as they work with their lowest performing students, while maintaining a focus on historically low performing sub-groups for our campus?”

Significance of the Study

            The prime beneficiary in this project would be the students.  The students that were selected to participate in this research were students that were having difficulty in math.  The students had demonstrated difficulty within the 1^st quarter of the 2012-2013 school year and were to receive additional assistance as they prepared for the STAAR Test to be administered in May 2013.  The students worked with a computer program designed to support their learning using technology.  The format is one that is stimulating to these digital natives.  The teachers would also benefit from this research as it would add another dimension to student learning; teachers would be able to add another tool to their toolbox.  And the researcher will benefit from the experience in working with a wider range of students.  The researcher also worked with two other interns in a leadership capacity and the experience helped the researcher to develop organizational, communication, and planning skills.

Definitions

STAAR – State of Texas Assessment of Academic Readiness; state-mandated high stakes testing.  The Math test is administered 3^rd – 8^th grade.  Students that are in 5^th and 8^th grade must pass the Math STAAR to be promoted to the next grade level.

COMPASS LEARNING – computer program by The Learning Odyssey Software Company.  The Compass Learning program is a support program designed to assist student learning through a guided curriculum supplemented with multi-media clips, pre-tests and supportive learning tools.

DISTRICT INITIATIVE – a program or training that is provided by the district and is to be implemented in the classroom.

COHORT – group of students to be tracked.

LEARNING LAB – after school tutorial program held in a computer lab.

Review of the Literature 

There is a debate about the efficacy of Compass Learning tutoring program.  The Times Leader writer, Mark Guydish, reports in 2007, that Dallas School District  Board member Gary Mathers, voted “no, absolutely not” when voting on the purchase of the Compass Learning software program.  As stated in the Times Leader (Guydish, 2007) the district planned to spend $68,560 a year on a new computer tutoring program designed to boost state test score results.  And while Electronic Resources for Schools reviewed Compass Learning as highly recommended because it “enables improved instruction by integrating 21st-century learning skills with social media capabilities.” (2013); Mathers apparently did not agree.

Compass Learning is an online program that is designed to engage students by using technology (Reaching the digital natives, 2013) and it is the same program that was recently purchased by the Socorro Independent School District also with the intent of boosting state test scores.  Dallas Assistant Superintendent Paul Reinert agreed to forfeit future raises if student test results did not increase by 2010 (Guydish, 2007.)  These were lofty promises that were very similar to statements made by the Socorro Independent School District at the time of implementation of Compass Learning in the district.  There were questions about the effectiveness of the program from the faculty about Compass Learning and the action research project was intended to help answer those questions. 

There has been evidence that computerized tutoring can assist students in their learning.  Carnegie Learning developed a program that showed that high school students who used the tutoring package at home were better with balancing chemical equations than peers that did not use the tutoring program. (Viadero, 2007)  The researcher was hopeful that similar findings would occur with the use of Compass Learning by the John Drugan School tutoring group.

Summary

A review of current research does not clearly state whether computer assisted tutoring programs have been successful.  There are differing ideas with support coming from both sides; positive ideas that support computer assisted tutoring and commentary that strongly opposes computer assisted tutoring.  This research is hoping to provide a clearer answer on the effectiveness of implementing programs like Compass Learning to support student success.

Action Research Design

Participants

The initial number of participants was 65.  These students were 6^th grade math students during the 2012-2013 school year.  This group of participants was selected based on their performance on the 2011-2012 Math STAAR.  The sample was based on students that scored 70% or lower on the Math STAAR. Student data was collected for all students in the grade level and students that met the criteria were invited to participate in the study.  Of the 65 students that were invited, 63% or 41 students attended the After School Learning Lab (Learning Lab).  Students were invited to attend the Learning Lab where they would receive additional exposure to the 6^th grade Math curriculum that would be assessed by the 2013 Math STAAR.  At the time that students were selected for participation, the passing standard for the 2012 Math STAAR had not been set by the State of Texas.  As the year progressed, state standards were revealed and the group of participants was reviewed based on this new information. 

Procedures
Students that were identified to participate in this program had demonstrated that they were struggling with the curriculum.  With 65 students or 35% of the students at or below a score of 70% on the 5^th grade Math STAAR, 6^th grade math teachers felt that students were in need of additional assistance.  The Learning Lab was established with Compass Learning as the intervention tool.  Compass Learning is a software program created by MindTouch .  As stated by Compass Learning, on their website, “we combine colorful, dynamic, and age-appropriate graphics and audio with explicit instruction and examples to create an exciting and student-friendly learning environment. Quizzes and other assessments embedded in Compass Learning activities provide updates on student progress and drive changes to the prescribed learning path.” (Reaching the digital native, 2013).  Participants in the Learning Lab were assigned Pre-Tests that were created through The Learning Odyssey Portal for Compass Learning.  Each Pre-Test was created based on a Texas Essential Knowledge and Skill (TEKS) objective.  The Learning Odyssey program will assign tasks and review assignments for students after the Pre-Test is taken.  Assignments are based on student outcomes and target areas of need.  Students then complete tutorials and assignments to support their learning.  After assignments are completed, students complete a Post-Test.  All assessments were assigned and completed with a view to increasing student exposure to the math curriculum and supporting student learning.  The goal for the interventions was to increase assessment percentages and outcomes for students on the 6^th grade Math STAAR 2013.

This project was started in the 3^rd quarter of the 2012-2013 school year; it ran from Jan 6 – March 6, 2013.  Participants attended the Learning Lab from 3:30-4:30pm, Mondays and Wednesdays.  The Learning Lab did not meet on Monday, January 21, 2013. Oscar Torres and Sandra Burton assisted with students in the Learning Lab.  Mr. Torres was a graduate student working on his master’s in sociology.  Ms. Burton had completed her master’s program and was working to increase her knowledge of curriculum and technology. 

Data Collection

Data was collected using Eduphoria, a Student Information System.  Student scores were collected for STAAR 2012 Math and STAAR 2013 Math.  The data collected was compared to determine if there was a change in student outcomes after Learning Lab intervention.  Data was polled for all students; the participants were identified based on a score of 70% or lower on the 2012 Math STAAR.  Table A.

Findings 

The findings for this study detail attendance and performance for 6^th grade math students during the 2012-2013 school year.  There were 65 students invited to participate in the project.  There were 24 students of 65 or 37% that did not participate.  Of these students, 8% (2 of 24 students) did not meet the STAAR requirements for 2012.  63% or 41 of 65 students invited to participate did participate.  22% of these students (9 of 41 students) did not meet the STAAR requirements for 2012.  After administration of the 2013 Math STAAR, results were analyzed.  Students that did not have interventions were compared with students that received interventions.  Of the 24 students that did not participate in the Learning Lab, 8% or two students did not pass the 2013 6^th grade Math STAAR.  The two students that did not pass the 2012 5^th grade Math STAAR passed the 2013 6^th grade Math STAAR.  41 students participated in the Learning Lab.  9 or 21% of these students did not pass the 2012 5^th grade Math STAAR.  The students received interventions in the form of Compass Learning in the Learning Lab twice a week, for 1 hour each day, for 9 weeks.  44% (4 of 9) of the students that did not meet the STAAR standards in 2012 went on to meet the requirements for the 2013 6^th grade Math STAAR.  Overall, 37% of students involved in the Learning Lab increased their score from the previous year, while 50% of the students that did not receive intervention increased their score.  In comparing the scores of students that participated versus students that did not participate in the Learning Lab, the results were as follows:  92% of the students that did not participate in the Learning Lab met the standard for the 2013 6th Grade Math STAAR while 8% did not meet the standard as seen in Graph A; 78% of the students that did participate in the Learning Lab met the standard for the 2013 6th Grade Math STAAR while 22% did not meet the standard for the 2013 6th Grade Math STAAR. (Graph B.)

Summary

Based on the information provided by the district, the researcher was hopeful that the data would support the implementation of the District Initiative for Compass Learning.  The Socorro Independent School District had proposed the use of the program for use in increasing student learning as an intervention tool.  Teachers had been reluctant to create tests and assignments for students and the hope was that the activity would reinforce the confidence in the program.  At this time, it is difficult to conclude that the program has been successful as an intervention tool.  With 65 students selected to participate, almost a third opted out of the program.  This allowed the researcher to compare intervention groups and non-intervention groups.  Although not equally divided, it did allow the researcher to observe two distinct groups.  92% of students that were not involved in the Learning Lab met the standard for the 2013 6^th Grade Math STAAR.  78% of students that participated in the Learning Lab, based on Compass Learning, met the standard for the 2013 6^th Grade Math STAAR.  More positive results were anticipated and hoped for, but at this time, it would be hard to definitively state that the Compass Learning Software program is the solution for intervention.

Conclusions and Recommendations

A cohort of 6^th grade math students was invited to participate in a Learning Lab based on their performance on the 2012 5^th Grade Math STAAR.  Students that scored 70% or lower on the assessment were invited to participate.  Of the 65 students invited, 41 students participated in the Learning Lab to receive additional instruction in the math curriculum.  The study was completed with a desire to demonstrate the efficacy of the Compass Learning software program that was introduced to teachers as a district initiative.

Conclusions

At the conclusion of this project, the researcher did not feel that there was enough data to support the use of Compass Learning in an afterschool Learning Lab as the only intervention tool to improve performance on a high stakes test such as the State of Texas Assessment of Academic Readiness (STAAR).  While there were students that did improve through use of this intervention tool, it is not clear that this is the direct cause of their improvement.  As students were progressing through the school year, there were other opportunities for development and students may have increased their skills through classroom work with the Math curriculum.  Additional analysis of study participants would be necessary to identify factors that supported student improvements other than the Learning Lab.  This study could also be improved by evaluating a different subject area to review and study additional data. 

Reflection

            I was able to answer my research question by reviewing the data collected.  Student data for 2012 and 2013 school years was compared and the study produced usable information.  The design of the project was good as I was able to review information and detail results based on that information.  The information collected was standard information that is available through the Student Information System; teachers are encouraged to review the information so that classroom learning is data driven. I do feel that there is room for improvement in this project as there were other content areas of instruction that could be developed.

Recommendations

            The primary recommendation for further study would be to more actively follow student progress.  The software program is designed for self-directed student use.  In the Learning Lab, the lab was established with that idea in mind.  Students were given instructions on how to access the program and how to work through the assignments.  Students were also told that they needed to complete the Pre-Test so that the program could determine a course of study for the individual student.  If students focused on the test and tried to be successful, the student would have less work to review, and if the student was successful and passed the Pre-Test for the assigned Texas Essential Knowledge and Skills (TEKS) then students were able to move on to the Post-Test.  Although the program used as intervention was self-directed, students would benefit from additional supervision.  Students rushed through work and did not focus on assignments.  More progress may have been made if students were more focused on work and additional redirection was made for students off task.

Concluding Remarks

            Interventions are import to student success and identifying what works for students could be a good starting point.  In the video, “Bridging the digital divide” by Comcast Newsmakers (2012) the point is made that students learn in diverse ways.  The interventions provided by Compass Learning utilize these ideas.  Today’s students are digital natives and Compass Learning brings technology into student learning.  The Learning Lab allowed students to have additional exposure to the curriculum and supported student learning, but this study also showed that intervention using technology is not the only way for students to be successful.  In the non-intervention group, 100% of the students that had not met the 2012 5^th Grade Math STAAR standards did meet the standards for the 2013 6^th Grade Math STAAR.  It is difficult to draw conclusions from this data because the sample size was only two participants.  Overall, students did find success and it is partially due to the interventions, but it is also due to teacher support and student tenacity. 

 

 Table A.

	March 2012 STAAR Mathematics, Grade 5		April 2013 STAAR Mathematics, Grade 6
ID	% Score	Met Standard	Percent Score	Met / Satisfactory
1	56%	Yes	54%	Yes
2	70%	Yes	56%	Yes
3	62%	Yes	58%	Yes
4	70%	Yes	56%	Yes
5	70%	Yes	52%	Yes
6	70%	Yes	42%	Yes
7	44%	No	58%	Yes
8	60%	Yes	56%	Yes
9	68%	Yes	71%	Yes
10	58%	Yes	60%	Yes
11	48%	No	50%	Yes
12	66%	Yes	27%	No
13	64%	Yes	52%	Yes
14	66%	Yes	56%	Yes
15	50%	No	83%	Yes
16	70%	Yes	38%	No
17	64%	Yes	56%	Yes
18	52%	Yes	48%	Yes
19	58%	Yes	62%	Yes
20	70%	Yes	58%	Yes
21	48%	No	35%	No
22	62%	Yes	73%	Yes
23	60%	Yes	69%	Yes
24	68%	Yes	88%	Yes
25	64%	Yes	81%	Yes
26	38%	No	65%	Yes
27	68%	Yes	71%	Yes
28	56%	Yes	52%	Yes
29	50%	No	42%	Yes
30	68%	Yes	62%	Yes
31	70%	Yes	35%	No
32	64%	Yes	48%	Yes
33	70%	Yes	71%	Yes
34	66%	Yes	63%	Yes
35	70%	Yes	40%	No
36	60%	Yes	60%	Yes
37	70%	Yes	37%	No
38	62%	Yes	63%	Yes
39	62%	Yes	60%	Yes
40	42%	No	38%	No
41	66%	Yes	52%	Yes
42	58%	Yes	69%	Yes
43	70%	Yes	60%	Yes
44	46%	No	37%	No
45	64%	Yes	65%	Yes
46	70%	Yes	88%	Yes
47	70%	Yes	87%	Yes
48	56%	Yes	65%	Yes
49	54%	Yes	63%	Yes
50	70%	Yes	63%	Yes
51	60%	Yes	62%	Yes
52	42%	No	27%	No
53	52%	Yes	50%	Yes
54	50%	No	77%	Yes
55	68%	Yes	60%	Yes
56	70%	Yes	67%	Yes
57	64%	Yes	54%	Yes
58	54%	Yes	56%	Yes
59	68%	Yes	35%	No
60	62%	Yes	44%	Yes
61	64%	Yes	79%	Yes
62	66%	Yes	73%	Yes
63	52%	Yes	67%	Yes
64	70%	Yes	65%	Yes
65	44%	No	31%	No

	Did not participate in learning lab
	Participated in learning lab
	Participated in learning lab and passed 2013 STAAR
	Did not participate in lefarning lab and passed 2013 STAAR
	Did not pass STAAR in tested year

References

CoSN - Advancing K-12 Technology Leadership. (2012). Comcast Newsmakers: Bridging the digital divide [Video file]. Retrieved from http://www.schooltube.com/channel/cosn_edtech/

Electronic Resources for Schools. Internet@Schools [serial online]. March 2013;20(2):28. Available from: MasterFILE Premier, Ipswich, MA. Accessed July 10, 2013.

Mark Guydish m. Computer tutoring program gets Dallas school board ok. Times Leader, The (Wilkes Barre, PA) [serial online]. August 14, 2007:Available from: NewsBank, Ipswich, MA. Accessed July 10, 2013.

Reaching the digital natives. (2013). Retrieved July 3, 2013, from
 http://www.compasslearning.com/sample-activities

Viadero D. New breed of digital tutors yielding learning gains. Education Week [serial online]. April 4, 2007;26(31):9. Available from: Academic Search Complete, Ipswich, MA, Accessed July 10, 2013.