Deriving Decisions from Disrupted Data

*** COLLEAGUE CORNER – MEMBER SPOTLIGHT ***

NATP member and Distinguished University Professor, Dr. Stephen Sireci recently released an important content piece on decision-making during a disrupted educational system. This is the very situation that testing professionals and school administrators are finding themselves at this very moment. Here we aim to draw from the research findings of our research peers. Here are the highlights of Dr. Sireci’s work:

• The COVID-19 pandemic negatively affected the quality of data from educational testing programs.
• These data were previously used for many important purposes ranging from placing students in instructional programs to school accountability.
• In this article, Sireci and co-author, Javier Suarez-Alvarez draw from the research design literature to point out the limitations inherent in “disrupted” educational testing data, suggest questions and criteria to be considered in evaluating the use of such data for decision making, and indicate how such data may be valid or invalid for specific purposes.
• Six criteria are proposed for evaluating the degree to which educational testing data are valid for specific decisions.

Key Findings

• These criteria suggest data from COVID-disrupted school years are not likely to be valid for accountability purposes but may be valuable for making decisions at the individual student level.

Key Takeaway

• Researchers and policy makers are encouraged to focus on how decisions derived from such disrupted data affect children.

Contact Dr. Sireci for a copy of the complete article: sireci@umass.edu

• About Stephen Sireci – Stephen G. Sireci, Ph.D. is Distinguished University Professor and Director of the Center for Educational Assessment in the College of Education at the University of Massachusetts Amherst. He is known for his research in evaluating test fairness, particularly issues related to content validity, test bias, cross-lingual assessment, standard setting, and computerized-adaptive testing. He has authored/coauthored over 130 publications and is the co-architect of the multistage-adaptive Massachusetts Adult Proficiency Tests. He is a Fellow of the American Educational Research Association, and of Division 5 of the American Psychological Association; Past-President of the National Council on Measurement in Education, and President-Elect of the International Test Commission.