High-stakes testing doesn't create more crime

So, ditching the practice of social promotion has allowed serious crime to run rampant on the streets? Only if you believe an oversimplified interpretation of an incomplete study performed using Louisiana data.

Recently, researchers released a paper investigating whether preventing social promotion, or the practice of advancing children academically unprepared a grade level regardless, causes more adult crime. They hypothesized that holding back children between grade levels past research had identified as most crucial to lead to high school graduation, eighth and ninth, discouraged desire for schooling got those repeating eighth grade so that later in life they more likely turned to violent crime than did their peers.

Two decades ago, Louisiana halted its practice of social promotion and instituted the Louisiana Educational Assessment Program tests in English and mathematics, where students had to pass both to enter ninth grade. To come as close as possible to having experimental and control groups, these analysts reviewed years of data for students who passed the eighth grade standardized tests needed for advancement by one point to those who failed these by one point.

While the two groups showed no significant differences in amount of juvenile crime committed, nor any for most crimes as adults many years later, the group held back did commit significantly more violent crimes by age 25. Dropping out disproportionately when asked to repeat eighth grade, their lack of education led them to these crimes, the authors speculate, which encourages reviewing the prohibition of social promotion.

But before those vehemently against testing – reflecting an attitude that has put Louisiana students at the bottom of achievement that will take a generation or more to rectify – call for the end of testing as a means of deciding grade promotion, they must consider the study’s shortcomings and limitations. Two big questions stand out about its conclusions.

First, if in fact discouraged students drop out, miss an education, and resort to crime eventually, why do they seem more likely only to commit violent crimes? What’s the matter with dealing drugs, burglary, and dealing in stolen goods, all behaviors that could produce a good living? The authors did test for such matters but didn’t find a significant difference between the groups. It seems odd that the discouragement/dropout/limited prospects effect only would apply for violent crime, so something is missing here that casts doubt on the hypothesis.

Second, while the authors do their best to create a control group of those scoring just better than the experimental group, it’s not the same thing. Possibly the function describing the link between promotion (that is, test scores) and crime is nonlinear and not discontinuous at that precise point between groups. If so, there may not be a “break” requiring an explanation (discouragement) of a treatment effect (retention). Fortunately, the authors have the data to explore this possibility.

However, the effort has one flaw built into it. While it can simulate control and treatment groups for the impact of retention, it can’t at all test for the impact that testing and its consequences has on future academic progress. That is, the mere fact that knowing a test for promotion looms may make some students take their academic studies more seriously. In turn, they better prepare themselves to succeed in the years ahead, culminating in the very least with a high school diploma. Without testing well enough as an incentive, they may have dropped out over poor and uninspired performance or, if social promotion still reigned, received a diploma yet fare poorly in the world of work.

This group of discouraged individuals, following the authors’ hypothesis, would turn to crime. In fact, this group could outnumber those who would have been held back, and therefore more violent crime would result if a testing regime didn’t retain some students. The authors simply can’t evaluate this effect, and admit it: “Whether test-based promotion has other offsetting benefits that may make it cost-beneficial is beyond the scope of our analysis ….”

Some Louisiana districts have arrangements to reduce the potential negative impact of retention, a “transitional” Grade 9 that attempts to keep failing students in their peer group and gives them another year to catch up. That seems reasonable to offset deleterious spillover effects of merit promotion. But as a result of this paper to think those effects necessarily exceed the positive impact of testing relative to future criminal activity misreads it and misleads in policy-making.