Classworks Efficacy: Reading and Mathematics with NWEA

ESSA Classification

An independent third-party review of the study design, population, methods, analysis, and results was conducted by SEG Measurement. The analysis concludes that this study meets the requirements for Level 2 evidence under the ESSA legislation. See the independent analysis and the full study.

About SEG Measurement

SEG Measurement provides educational publishers, technology providers, schools, and government education agencies with program evaluation and product effectiveness research services to support decision making. Backed by 30 years of research and evaluation experience, SEG helps organizations design and conduct scientifically-based research, from initial design to data analysis and reporting, and provides independent evaluations of research and efficacy studies. SEG provides advanced assessment solutions for K-12 and higher education, offering a range of assessment design, development, psychometric, and implementation services. SEG has delivered more than 100 million tests to tens of thousands of K-12 schools.

Type of Study

Quasi-experimental study consistent with the requirements for Level 2 evidence under the ESSA legislation

Measurement Data

Classworks Usage and NWEA Map Growth assessment data

Grade Levels

6 - 8

Study Size

1,840 students

Summary

Classworks is a supplemental, online instructional program that provides English language arts, reading, and mathematics instruction for students based on their NWEA MAPGrowth Reading and Mathematics assessment data. In addition, Classworks provides on-grade level, standards-based reading and mathematics instruction to support teachers in the classroom. While using Classworks, students engage with individualized content based on their assessment results. Assessments generated from Classworks measure student growth and progress, and teacher-facing reporting provide formative and longitudinal data, allowing teachers to make data-driven instructional decisions.Classworks provides instructional software to 30 NWEA school districts across the southeast.The current report explores performance trends from the 2018-2019 school year. The following evaluation questions are addressed in the present study:

● Do middle school students with exposure to Classworks instruction outperform students without exposure to Classworks instruction?

● What are the effects in reading versus mathematics?

● Do impacts on student outcomes vary by prior student achievement?

Results

The following results present an integration of all data collected regarding student achievement data and program usage data. We begin with the program usage data, showing how each grade interacted with the program. Then, we present the impact of the program on student achievement.

Reading results:

Across all grades combined, after controlling for the covariates, students in the treatment group exhibited more growth on the NWEA Reading exam than students in the comparison group. The size of this difference was a statistically significant effect size of .263 [F(1, 1836) = 50.53, p < .001]. Analyses also examined differences between the groups for each grade level individually. Here, statistically significant differences favoring the performance of the students in the treatment group were found in sixth, seventh, and eighth grade (see Table 3). These results are summarized as follows:

● In sixth grade, the adjusted mean NWEA Reading assessment score for the treatment group was 212.06 and for the comparison group was 209.44. This difference reflected a statistically significant effect size of .183 [F(1, 592) = 8.75, p < .01].

● In seventh grade, the adjusted mean NWEA Reading assessment score for the treatment group was 213.72 and for the comparison group was 208.51. This difference reflected a statistically significant effect size of .283 [F(1, 554) = 21.94, p < .001].

● Lastly, in eighth grade, the adjusted mean NWEA Reading assessment score for the treatment group was 215.39 and for the comparison group was 211.33. This difference reflected a statistically significant effect size of .233 [F(1, 685) = 18.45, p < .001].

In addition to exploring the impact of the Classworks program on overall student achievement, additional analyses were performed that investigated whether or not the program was correlated with any significant impact for select subgroups of students. Specifically, an analysis of covariance was used to examine whether the Classworks program fostered any differential impact students below and above the 50th percentile in baseline reading score.Based on the results of these analyses, the reading performance of five student subgroups were found to significantly differ between the treatment and comparison groups.Seventh and eighth grade students in the treatment group who scored below the 50th percentile at baseline had significantly higher post-test scores than those students in the comparison group. Also, sixth, seventh, and eighth grade students in the treatment group who scored above the 50th percentile at baseline had significantly higher post-test scores than those students in the comparison group. Table 4 below provides the adjusted mean scores for students in each of these subgroups.

Mathematics results:

Across all grades combined, after controlling for the covariates, students in the treatment group exhibited more growth on the NWEA Mathematics exam than students in the comparison group. The size of this difference was a statistically significant effect size of .198 [F(1, 1902) = 36.26, p < .001]. Analyses also examined differences between the groups for each grade level individually. Here, statistically significant differences favoring the performance of the students in the treatment group were found in sixth and eighth grade (seeTable 3). In seventh grade, significant differences were not found between the groups. These results are summarized as follows:

● In sixth grade, the adjusted mean NWEA Math assessment score for the treatment group was 219.97 and for the comparison group was 217.06. This difference reflected a statistically significant effect size of .215 [F(1, 680) = 13.47, p < .001].

● In seventh grade, the adjusted mean NWEA Math assessment score for the treatment group was 221.95 and for the comparison group was 220.19. This difference reflected anon-significant effect size of .098 [F(1, 592) = 2.83, p = n.s.].

● Lastly, in eighth grade, the adjusted mean NWEA Math assessment score for the treatment group was 226.17 and for the comparison group was 222.92. This difference reflected a statistically significant effect size of .227 [F(1, 625) = 16.16, p < .001].

Related Blog Posts

Better Together: Use Classworks and Your Assessment Data to Individualize! 🖥️

We streamlined how you individualize with Classworks using your assessment data. We're thrilled to hear how you're putting data to work more efficiently! Our video library and Resource Center contain several helpful videos in case you need a refresh.

The Secret to a Successful Middle School RTI Implementation

Implementing a Response to Intervention (RTI) program in middle school can come with unique challenges. Instructional technology can be extremely useful in helping middle schools effectively implement RTI. What should you look for?