Analysis of the National Assessment of Educational Progress (NAEP) Mathematics Process

We are examining the National Assessment of Education Progress (NAEP) process, outcome, and survey data to understand the test-taking behavior and mathematics performance of learners with and without disabilities. In partnership with Dr. Susu Zhang from the University of Illinois Urbana-Champaign, this project uses innovative statistical and machine learning techniques.

What is the NAEP process data?

The NAEP process records the series of clicks, entries, and timestamps during test takers’ interactions with the test items. By analyzing this data, our research team can examine the key components of test-taking behavior of learners with disabilities and uncover patterns that would not be apparent if using outcome or survey data alone. We have used innovative machine learning approaches coupled with expert review to:

• Extract meaningful patterns from process data to reflect key components of students’ mathematics test-taking behavior (i.e., math cognitive process, time on task, level of engagement, and accommodation usage)
• Examine how test-taking behavior differs by students’ disability status
• Identify patterns of test-taking behavior that lead to successful performance on the NAEP mathematics grade 8 assessment
• Investigate the association between test-taking behaviors and test performance for students with disabilities and students without disabilities

In addition, the research team uses structural equation modeling (SEM) and Latent Profile Analysis (LPA) to understand the degree to which the interrelationships between math instruction, performance, and key components of testing behavior (i.e., math cognitive process, time on task, level of engagement, and accommodation usage) differ by students’ disability status.

Project Findings

The findings of this study have provided important insights about the test-taking behavior of students with disabilities and their cognitive processes, time on task, level of engagement, and accommodation usage during their interactions with a digital assessment. Results have revealed helpful strategies that students with disabilities can use to solve math problems as well as features of digital assessments that increase engagement and accessibility and provide more equitable and accurate measures of the performance of students with different abilities.

Latest News

Upcoming Conference: Join us at SREE 2024! We’re excited to share our latest research and explore collaboration opportunities with fellow academics and practitioners. Let’s advance the field together!

Connect with our team for more information or to engage further with our research at xwei@digitalpromise.org. Follow our updates through Digital Promise’s LinkedIn.

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R324P230002 to Digital Promise. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.