As Louisiana progresses through
its second year of its statewide Student Scholarships for Education Excellence
program, data
produced still can’t reveal whether the program is improving significantly the
lot of children or what effect if any it may have, even as it succeeds on a
cost basis.
Some data came out this week from
the Department of Education, in the form of School Performance Scores for
students who accepted vouchers through the program. Children who once attended subpar
schools are eligible to receive state money to attend a nonpublic or higher-ranked
public school, if space is available. DOE for each school where adequate data
could be collected computed a ranking of that voucher cohort, as if it were its
own school, and released that data.
They showed a wide range of
success for the cohorts treated as schools, but overall most were not terribly
different from the underperforming environments that the students had left.
Almost half were scored at the D or F level, equivalent by definition to the
scores of their previous schools (students at C-ranked public schools also are
eligible for the program, but only if space is available after the pool of students
from the lower-ranked schools in an area is exhausted).
Unfortunately, this tells little
about the program’s impact, for a few reasons. First, because cohorts at some
schools were too small for reliable testing – qualifying schools have at least
40 students in testing grades or at least 10 students per grade – these had to
be excluded. Second, testing, which is a large component to SPS calculations
below high school, begins only in the third grade and many voucher students are
below that. For this reason, not much more than half of the students in the
program can provide data for these calculations.
Third, while for elementary
schools the SPS
basically is achievement made on tests, another small factor enters in to
scoring in middle schools and is makes up a minority of the score for high
schools. Thus, this enters into the measure components not directly tied to and/or
quantifiable of academics that detracts from telling if the program is doing what
it’s supposed to – improve academic achievement of students relative to that of
the public schools in which they had attended. Finally, given that students on which
data could be derived had spent several years at least in the public schools,
one year probably is not enough time to make a definitive judgment on the
program’s effects (more time enriching the data available also will take care
of the limited amount of data, as program officials estimate that within a few
years about 90 percent of students enrolled will have data available on their performances
conceptualized as a “school” within a school).
However, the largest impediment
for use of this data in trying to evaluate this objective is that there is no
comparative use of it. Alone, these data tell us nothing because of the
numerous confounding factors that only comparison can parse out.
As examples, a voucher cohort
could have come into a nonpublic school as the worst performers from a previous
school, meaning they start from the worst baseline with really low absolute
scores that nonetheless improved. Or, students from really poor schools by SPS
could end up in a cohort in a school that performs merely poorly, where any
improvement as a result of the program is masked because individual tracking
isn’t occurring. Also, we can’t entirely be sure that extraneous factors don’t
influence the results, such as more motivated parents disproportionately comprising
the family population using the vouchers, which then might explain any boost in
achievement, not the program.
Therefore, the only way for
policy-makers to know whether the program is providing better education than
without it is to create three groups of students and compare not their absolute
achievement, but relative growth year over year. One would be children of
families that did not try to use vouchers and were at the school (or feeder
school) the previous year. Another would be those who did use vouchers to go on
to another school. Lastly would be the group of families who applied for vouchers
but either did not choose to use them or who tried but failed to get their children
into another school because of space limitations.
For all three groups, test scores
for previous years would be compared to their current year. The comparison
between those who never pursued a voucher and those who did successfully tells
of the program’s impact combined with familial motivation, which is why the
last group also would have their scores compared, to factor out this cause and
leave a pure measure of change brought about by the program only.
Even this unadulterated measure
of change needs to be taken in context. For example, let’s say after analysis
of these individual (as opposed for the SPS aggregate) data there was no
significant change statistically speaking between the performances of children
who stayed in public schools and those using vouchers. But given that on
average the typical voucher recipient costs the state over three thousand dollars
fewer than for a public school child for an education quality roughly the same,
then the program still is worthwhile because it delivers results at least as
good for less money, thus more efficiently.
No comments:
Post a Comment