Measurement Error in Progress Monitoring Data: Comparing Methods Necessary for High-Stakes Decisions

Support for the use of progress monitoring results for high-stakes decisions is emerging in the literature, but few studies support the reliability of the measures for this level of decision-making. What little research exists is limited to oral reading fluency measures, and their reliability for progress monitoring (PM) is not supported. This dissertation explored methods rarely applied in the literature for summarizing and analyzing progress monitoring results for medium- to high-stakes decisions. The study was conducted using extant data from 92 "low performing" third graders who were progress monitored using mathematics concept and application measures. The results for the participants in this study identified 1) the number of weeks needed to reliably assess growth on the measure; 2) if slopes differed when results were analyzed with parametric or nonparametric analyses; 3) the reliability of growth; and 4) the extent to which the group did or did not meet parametric assumptions inherent in the ordinary least square regression model. The results indicate reliable growth from static scores can be obtained in as few as 10 weeks of progress monitoring. It was also found that within this dataset, growth through parametric and nonparametric analyses was similar. These findings are limited to the dataset analyzed in this study but provide promising methods not widely known among practitioners and rarely applied in the PM literature.