ISTEP and testing FAQ's

A recent commenter posted several questions about how ISTEP scores are used in the schools. I will post some of the most frequently asked questions (FAQ's) that I hear about ISTEP and testing in the schools.

How can you compare ISTEP scores from several years ago to today’s students? Aren’t those different tests now? Aren’t the students different too?

Yes, and that can complicate matters but not near as much as conventional wisdom would have one believe when schools are trying to do program evaluation. Several years ago, each school “weighed-in” in their goal areas (math computation for example) and tried to get a well-rounded measure of how students were performing in that specific area. They collected many different assessment scores from a variety of assessments and a variety of grade levels in order to establish a baseline for math computation. They may have had 10-15 different measures or more, including ISTEP. In cases where the ISTEP test changed, the old test scores are replaced with the new so that “apples-to-apples” comparisons are made.

Then, each school put in place an improvement plan targeting that area and went to work over the years implementing improvement strategies. After lots of hard work the schools this year will "weigh-out" by repeating those 10-15 different measures spread over those different assessments and different grade levels and try to determine if the overall pattern of achievement in math computation among the new students is different. This method is used for trying to determine if the program changes the school has made is changing the patterns of student achievement.

Yes, these are different students. However, when sample sizes get large enough on each assessment (generally speaking, the research says 70-100 students per sample size) the statistical probability of the group of students being “out of norm” is a lot less. Then by spreading this risk over multiple assessments and multiple grade levels, we can generally use this information to make some basic decisions about our curriculum and methods.

Then the tough questions come up.

Are the new students so different that the results are not comparable? This can happen but it is not as likely as most believe.

Was it the interventions we used that made the biggest difference or was it something else? If so, which ones? Should the staff continue with the strategies they were using or will another one make a bigger difference for our students?

These are the questions that professionals must wrestle with. Data will NOT make the decisions for us, but I am proud to work in a district that will at least use some professional decision-making processes to guide our discussions.

Ultimately our decisions still end up being professional judgments, but at least we are using information and data to help inform our practice.